/instrument-data-to-allotrope

Field Classification Guide

This guide helps classify instrument data fields into the correct ASM document locations. Use this when mapping raw instrument output to Allotrope Simple Model structure.

ASM Document Hierarchy

<technique>-aggregate-document
├── device-system-document          # Instrument hardware info
├── data-system-document            # Software/conversion info
├── <technique>-document[]          # Per-run/sequence data
│   ├── analyst                     # Who performed the analysis
│   ├── measurement-aggregate-document
│   │   ├── measurement-time
│   │   ├── measurement-document[]  # Individual measurements
│   │   │   ├── sample-document
│   │   │   ├── device-control-aggregate-document
│   │   │   └── [measurement fields]
│   │   └── [aggregate-level metadata]
│   ├── processed-data-aggregate-document
│   │   └── processed-data-document[]
│   │       ├── data-processing-document
│   │       └── [processed results]
│   └── calculated-data-aggregate-document
│       └── calculated-data-document[]

Field Classification Categories

1. Device/Instrument Information → device-system-document

Hardware and firmware details about the physical instrument.

Rule: If the value describes the physical instrument and doesn't change between runs, it goes in device-system-document.

2. Software/Data System Information → data-system-document

Information about software used for acquisition, analysis, or conversion.

Rule: If the value describes software, file metadata, or data provenance, it goes in data-system-document.

3. Sample Information → sample-document

Metadata about the biological/chemical sample being analyzed.

Rule: If the value identifies or describes what was measured (not how), it goes in sample-document.

4. Device Control Settings → device-control-aggregate-document

Instrument settings and parameters used during measurement.

Rule: If the value is a configurable instrument parameter that affects measurement, it goes in device-control-aggregate-document.

5. Environmental Conditions → device-control-document or technique-specific

Ambient or controlled environmental parameters during measurement.

Rule: Environmental conditions that affect measurement quality go with device control or in technique-specific locations.

6. Raw Measurement Data → measurement-document

Direct instrument readings - the "ground truth" data.

Rule: If the value is a direct instrument reading that wasn't computed from other values in this analysis, it goes in measurement-document.

7. Calculated/Derived Data → calculated-data-aggregate-document

Values computed from raw measurements.

Calculated data document structure:

{
  "calculated-data-name": "viability",
  "calculated-result": {"value": 95.2, "unit": "%"},
  "calculation-description": "viable cells / total cells * 100"
}

Rule: If the value was computed from other measurements in this analysis, it goes in calculated-data-aggregate-document. Include calculation-description when possible.

8. Processed/Analyzed Data → processed-data-aggregate-document

Results from data processing algorithms (peak integration, cell classification, etc.).

Associated data-processing-document:

{
  "cell-type-processing-method": "trypan blue exclusion",
  "cell-density-dilution-factor": {"value": 2, "unit": "(unitless)"},
  "minimum-cell-diameter-setting": {"value": 5, "unit": "µm"},
  "maximum-cell-diameter-setting": {"value": 50, "unit": "µm"}
}

Rule: If the value results from an algorithm or processing method applied to raw data, it goes in processed-data-aggregate-document with its processing parameters in data-processing-document.

9. Timing/Timestamps → Various locations

Rule: Use ISO 8601 format: 2024-01-15T10:30:00Z

10. Analyst/Operator Information → <technique>-document

Rule: Analyst goes at the technique-document level, not in individual measurements.

Decision Tree

Is this field about...

THE INSTRUMENT ITSELF?
├── Hardware specs → device-system-document
└── Software/files → data-system-document

THE SAMPLE?
└── Sample ID, name, type, batch → sample-document

INSTRUMENT SETTINGS?
└── Configurable parameters → device-control-aggregate-document

ENVIRONMENTAL CONDITIONS?
└── Temp, humidity, etc. → device-control-document

A DIRECT READING?
└── Raw instrument output → measurement-document

A COMPUTED VALUE?
├── From other measurements → calculated-data-document
└── From processing algorithm → processed-data-document

TIMING?
├── When measured → measurement-document.measurement-time
└── When run started/ended → analysis-sequence-document

WHO DID IT?
└── Operator/analyst → <technique>-document.analyst

Common Instrument-to-ASM Mappings

Note: These mappings are derived from the Benchling allotropy library. For authoritative mappings, consult the parser source code for your specific instrument.

Cell Counter (Vi-CELL BLU)

Source: allotropy/parsers/beckman_vi_cell_blu/vi_cell_blu_structure.py

Spectrophotometer (NanoDrop)

Plate Reader

Chromatography (HPLC)

Unit Handling

Only use units explicitly present in source data. If a value has no unit specified:

• Use (unitless) as the unit value
• Do NOT infer units based on domain knowledge

Calculated Data Traceability

When creating calculated values, always link them to their source data using data-source-aggregate-document:

{
    "calculated-data-name": "DIN",
    "calculated-result": {"value": 5.8, "unit": "(unitless)"},
    "calculated-data-identifier": "TEST_ID_147",
    "data-source-aggregate-document": {
        "data-source-document": [{
            "data-source-identifier": "TEST_ID_145",
            "data-source-feature": "sample"
        }]
    }
}

This declares: "DIN 5.8 was calculated from the sample at TEST_ID_145."

Why this matters:

• Audits: Prove a value came from specific raw data
• Debugging: Trace unexpected results back to their source
• Reprocessing: Know which inputs to re-analyze if algorithms change

Assign unique IDs to:

• Measurements, peaks, regions, and calculated values
• Use a consistent naming pattern (e.g., INSTRUMENT_TYPE_TEST_ID_N)

This enables bidirectional traversal: trace from calculated → raw, or raw → all derived values.

Nested Document Structure (Critical)

A common mistake is "flattening" fields directly onto measurement documents when they should be wrapped in nested structures. This breaks schema compliance and loses semantic context.

Why Nesting Matters

ASM uses nested documents for semantic grouping:

Sample Document Fields

These fields MUST be inside sample document, not flattened on measurement:

// ❌ WRONG - Fields flattened on measurement
{
  "measurement identifier": "TEST_001",
  "sample identifier": "Sample_A",
  "location identifier": "A1",
  "absorbance": {"value": 0.5, "unit": "(unitless)"}
}

// ✅ CORRECT - Fields nested in sample document
{
  "measurement identifier": "TEST_001",
  "sample document": {
    "sample identifier": "Sample_A",
    "location identifier": "A1",
    "well plate identifier": "96WP001"
  },
  "absorbance": {"value": 0.5, "unit": "(unitless)"}
}

Fields belonging in sample document:

• sample identifier - Sample ID/name
• written name - Descriptive sample name
• batch identifier - Batch/lot number
• sample role type - Standard, blank, control, unknown
• location identifier - Well position (A1, B3, etc.)
• well plate identifier - Plate barcode
• description - Sample description

Device Control Document Fields

Instrument settings MUST be inside device control aggregate document:

// ❌ WRONG - Device settings flattened
{
  "measurement identifier": "TEST_001",
  "device identifier": "Pod1",
  "technique": "Custom",
  "volume": {"value": 26, "unit": "μL"}
}

// ✅ CORRECT - Settings nested in device control
{
  "measurement identifier": "TEST_001",
  "device control aggregate document": {
    "device control document": [{
      "device type": "liquid handler",
      "device identifier": "Pod1"
    }]
  },
  "aspiration volume": {"value": 26, "unit": "μL"}
}

Fields belonging in device control:

• device type - Type of device
• device identifier - Device ID
• detector wavelength setting - Wavelength for detection
• compartment temperature - Temperature setting
• sample volume setting - Volume setting
• flow rate - Flow rate setting

Custom Information Document

Vendor-specific fields that don't map to standard ASM terms go in custom information document:

"device control document": [{
  "device type": "liquid handler",
  "custom information document": {
    "probe": "2",
    "pod": "Pod1",
    "source labware name": "Inducer",
    "destination labware name": "GRP1"
  }
}]

Liquid Handler: Transfer Pairing

For liquid handlers, a measurement represents a complete transfer (aspirate + dispense), not separate operations:

// ❌ WRONG - Separate records for aspirate and dispense
[
  {"measurement identifier": "OP_001", "transfer type": "Aspirate", "volume": {"value": 26, "unit": "μL"}},
  {"measurement identifier": "OP_002", "transfer type": "Dispense", "volume": {"value": 26, "unit": "μL"}}
]

// ✅ CORRECT - Single record with source and destination
{
  "measurement identifier": "TRANSFER_001",
  "sample document": {
    "source well location identifier": "1",
    "destination well location identifier": "2",
    "source well plate identifier": "96WP001",
    "destination well plate identifier": "96WP002"
  },
  "aspiration volume": {"value": 26, "unit": "μL"},
  "transfer volume": {"value": 26, "unit": "μL"}
}

Pairing logic:

1. Match aspirate and dispense operations by probe number
2. Create one measurement per matched pair
3. Use source_* fields for aspirate location
4. Use destination_* fields for dispense location
5. Include both aspiration volume and transfer volume

Quick Reference: Nesting Decision

Is this field about...

THE SAMPLE BEING MEASURED?
├── Sample ID, name, batch → sample document
├── Well position → sample document.location identifier
├── Plate barcode → sample document.well plate identifier
└── Source/destination locations → sample document (with prefixes)

INSTRUMENT SETTINGS?
├── Standard settings → device control aggregate document
└── Vendor-specific → custom information document

A MEASUREMENT VALUE?
└── Direct on measurement document (e.g., absorbance, volume)

TRANSFER OPERATION TYPE?
└── DON'T use "transfer type" - pair into single measurement
    with source/destination fields instead

Validation

Use validate_asm.py to check for nesting issues:

python scripts/validate_asm.py output.json --reference known_good.json

The validator checks for:

• Fields incorrectly flattened on measurements
• Missing sample document wrapper
• Missing device control aggregate document wrapper
• Missing custom information document for vendor fields
• Liquid handler: separate transfer types instead of paired records

Sources

• Allotrope Simple Model Introduction
• Benchling allotropy library
• Allotrope Foundation ASM Overview