# Sun Photometer Text Exports

> **Status:** Supported (scoped) · **Vendor:** Solar Light / YES Inc. · **Extensions:** `.OUT`, `.TXT`, `.csv`, `.lev10`, `.lev15`, `.lev20` (text); `.nc` (NetCDF, behind `fmt-hdf5`)

Sun photometers measure spectral aerosol optical thickness and irradiance at a
handful of fixed filter wavelengths. These are not core NIR lab spectra, but
they appear in the sample corpus and exercise the same normalization contract:
channel wavelengths become the spectral axis and each observation row becomes
one `SpectralRecord`. nirs4all-formats reads the common ASCII exports (MFR, Microtops,
AERONET MAN); the ARM MFRSR and Microtops MAN NetCDF paths live in the NetCDF
reader and require the `fmt-hdf5` feature.

## Instruments & software

Solar Light Microtops II and MFR-7 rotating-shadowband radiometers, the DOE ARM
MFRSR datastream, and AERONET Maritime Aerosol Network (MAN) cruise exports.

## File structure

The ASCII reader recognises three text layouts by content:

- **MFR-7 `.OUT`** — a `MFR-7 Sun Photometer` title and site line, a `Record …`
  header, then fixed-width rows with `Channel_<nm>` columns.
- **Microtops `.TXT`/CSV** — a comma-separated header containing `AOT_1020` /
  `AOT_870`, then one row per observation with `AOT_<nm>` columns.
- **AERONET MAN ASCII** — a `Maritime Aerosol Network` preamble (version/level,
  campaign, data policy, PI line) and a `Date(dd:mm:yyyy)` header with
  `AOD_<nm>nm` (and optional `STD_<nm>nm`) columns.

The NetCDF path (ARM MFRSR b1 and Microtops MAN) is decoded through the
`fmt-hdf5` NetCDF reader.

## What nirs4all-formats extracts

- **MFR `.OUT`** — one record per row, signal `channels` (raw counts) at the
  filter wavelengths (e.g. 415, 500, 614, 673, 870, 940 nm). Record number,
  time and air mass are preserved per record; site/lat/lon/alt from the header.
- **Microtops `.TXT`/CSV** — one record per row, signal `aot` typed
  `aerosol_optical_thickness` at the AOT filter wavelengths (e.g. 1020, 870,
  675 nm). Location, pressure, solar geometry and water-column fields are
  preserved as metadata.
- **AERONET MAN ASCII** — one record per row, signal `aot`
  (`aerosol_optical_thickness`) over the valid 380–870 nm channels; a paired
  `aot_std` signal (typed `uncertainty`) when STD columns are present. Missing
  `-999` channels are omitted from the axis; campaign, level, aggregation, PI
  fields and row metadata are preserved. Records carry the
  `microtops_man_ascii_experimental` warning.
- **NetCDF** (via `fmt-hdf5`) — ARM MFRSR exposes 7-filter
  hemispheric/diffuse/direct irradiance plus voltage and direct/diffuse ratio
  signals, with ARM datastream metadata, filter centroids/FWHM, solar geometry
  and per-signal QC; Microtops MAN NetCDF exposes `aot` and `aot_std`.

## Variants & support status

| Variant | Status | Notes |
|---|---|---|
| MFR-7 `.OUT` text | Supported | Synthetic fixture validates the parser; a redistributable real `.OUT` is still wanted. |
| Microtops `.TXT`/CSV text | Partial | Synthetic CSV covered; no redistributable legacy `.TXT` field export found. |
| AERONET MAN ASCII (`.lev10/.lev15/.lev20`) | Supported (local-only) | All-points, daily and series aggregations validated on local Okeanos samples; not redistributable (MAN data policy). |
| Microtops MAN NetCDF | Supported | Real PANGAEA fixture; uses a contiguous-layout fallback decoder (see below). |
| ARM MFRSR NetCDF | Experimental (local-only) | One local b1 fixture; broader ARM/xarray conformance pending. |

## Limitations & known gaps

- No atmospheric correction or unit conversion is applied.
- The Microtops MAN NetCDF fixture is discovered as a Microtops `aot_<nm>`
  schema. When the high-level `hdf5-reader` 0.5 API fails to resolve this
  layout's shared attribute heap, the reader falls back to a generic
  contiguous-layout decoder (`DataLayout::Contiguous` blocks via fractal-heap
  link records and `get_or_parse_header(addr)`), emitting
  `microtops_man_netcdf_contiguous_layout_fallback` and, for byte-scanned
  global string attributes, `microtops_man_netcdf_global_attributes_byte_scan`.
  This fallback should disappear once `hdf5-reader` resolves NetCDF4 shared
  attribute heaps cleanly.
- AERONET MAN ASCII and ARM MFRSR NetCDF support are validated on local samples
  only and are not redistributed.
- Redistributable MFR-7 `.OUT` and legacy Microtops `.TXT` field exports, and a
  generic NetCDF path, are still needed.

## Reference readers

Ad-hoc parsers, SPECCHIO and `xarray` / ARM ACT (for the NetCDF datastreams) are
the reference candidates.

## Samples & validation

`samples/mfr/synthetic_mfr.OUT` (50 records) and
`samples/microtops/synthetic_microtops.TXT` (20 records) are golden-backed in
`crates/nirs4all-formats/tests/goldens/`, alongside the real
`samples/microtops/microtops_arc_msm114_2.nc` MAN NetCDF (378 records, PANGAEA
MSM114/2, CC-BY-4.0). Local-only fixtures cover the ARM MFRSR b1 NetCDF (4,320
records × 7 filters, with a QC YAML sidecar attached as a `qc_sidecar` source)
and the AERONET MAN ASCII Okeanos exports. Microtops/MAN `aot` arrays use the
`aerosol_optical_thickness` signal type; `aot_std` uses `uncertainty`. The
probes report `mfr-sun-photometer`, `microtops-sun-photometer` and
`microtops-man-ascii` at `Confidence::Definite`.