# MATLAB MAT / R RData Datasets

> **Status:** Supported (scoped) · **Vendor:** MATLAB / R ecosystem · **Extensions:** `.mat`, `.MAT`, `.RData`, `.rda` · **Feature flag:** `fmt-matlab`

MATLAB `.mat` files and R `.RData` workspaces are common carriers for NIRS
matrices, calibration datasets and Eigenvector-style data set objects. This
reader maps simple matrix-style layouts and selected academic structured
datasets in Rust.

## Instruments & software

Vendor-neutral research and ML tooling: MATLAB, the Eigenvector PLS_Toolbox data
sets, and the R `prospectr` package. Committed fixtures include synthetic
matrices, Eigenvector corn / NIR shootout / DSO datasets and the prospectr
`NIRsoil` workspace.

## File structure

The reader is gated behind the `fmt-matlab` feature, which itself requires
`fmt-hdf5` because MATLAB v7.3 is an HDF5 container. Dispatch is by extension
plus content:

- **MAT v5** (`.mat` / `.MAT`, probe `Definite`) — numeric arrays via the
  pure-Rust `matfile` crate, with a targeted native parser for Eigenvector-style
  object/struct datasets (numeric/char arrays, cells, structs/objects, zlib
  compression, little/big endian). Arrays are MATLAB column-major.
- **MAT v7.3** (`.mat` with HDF5 magic, probe `Likely`) — decoded via
  `hdf5-reader`; `X` is often stored `bands x samples`, and both orientations
  are detected.
- **R RData** (`.RData` / `.rda`) — RDX3/XDR streams (probe `Definite`) and
  XZ-compressed workspaces (probe `Likely`) via `rds2rust`, schema-mapped to the
  prospectr `NIRsoil` data.frame and its `spc` matrix.

Single-file `.mat` (v5 and v7.3) and `.RData` all decode in-memory through
`open_bytes` with no companion files. The exception is the Indian Pines cube,
which is sidecar-bearing: `open_path` reads the optional `indian_pines_gt.mat`
from disk, and `open_with_sidecars` lets a resolver serve that ground-truth file
(the GT filename is hard-coded). Plain `open_bytes` still succeeds without it,
dropping the target column.

The expected matrix layout is an `X` spectra matrix, a wavelength axis named
`wavelengths`, `wavelength`, `wavelength_nm` or `x`, and an optional numeric `y`
target vector.

## What nirs4all-formats extracts

- **Signals** — the spectra matrix, emitting one record per sample. Eigenvector
  datasets expose their named spectral blocks as separate signals (e.g.
  `m5spec`, `mp5spec`, `mp6spec`; `instrument_1`, `instrument_2`).
- **Axis** — the wavelength / wavenumber axis when present; otherwise a generated
  index axis (e.g. for the Indian Pines cube, which carries no calibration).
- **Targets** — the `y` vector or the dataset's labelled target columns
  (`moisture`, `oil`, `protein`, `starch`; `weight`, `hardness`, `assay`;
  `Nt`, `Ciso`, `CEC`; `land_cover_class`).
- **Metadata & provenance** — container type, axis/orientation hints, source
  file + SHA-256; a provenance warning when an axis is generated for lack of
  wavelength calibration.

## Variants & support status

| Variant | Status | Notes |
|---|---|---|
| MAT v5 numeric (`X` + axis + `y`) | Supported | Column-major arrays via `matfile`. |
| MAT v5 Eigenvector data set objects | Supported | Schema-mapped; multi-signal + labelled targets. |
| MAT v7.3 (HDF5) | Supported | Via `hdf5-reader`; both matrix orientations. |
| prospectr `NIRsoil.RData` | Supported | RDX3/XZ workspace, mapped from the data.frame. |
| Indian Pines cube + `_gt.mat` | Experimental (local-only) | One record per pixel, generated band index. |
| Arbitrary MATLAB structs / RData objects | Planned | Generic heterogeneous structures not yet mapped. |

## Limitations & known gaps

- Unknown MATLAB structs/objects are not treated as generic numeric arrays:
  their spectra, labels, axis scales and targets live inside nested objects
  rather than top-level `X`/`wavelengths` arrays.
- R workspace support is intentionally schema-mapped — the `.RData` path accepts
  the prospectr `NIRsoil` fixture and validates its expected columns.
- The Indian Pines MATLAB path is schema-mapped and local-only (academic-use
  source without a clear redistribution license); CI skips it when
  `samples_local/` is absent. The cube emits one record per pixel with a
  generated band-index axis and a warning because it carries no wavelength
  calibration.
- Generic MAT/RData structures, MAT v7.3 cubes and heterogeneous metadata/targets
  remain to be broadened.

## Reference readers

`scipy.io` and the `hdf5-reader` crate (MAT), R serialization and `prospectr`
(RData). nirs4all-formats adds axis detection, signal typing, target mapping and
provenance.

## Samples & validation

Fixtures under `samples/matlab/`: `synthetic_nirs_v5.mat` / `synthetic_nirs_v73.mat`
(50 records each, `absorbance`/`y`), `eigenvector_corn.mat` (80 records, three
spectral signals, four targets), `eigenvector_nir_shootout_2002.mat` (655
records, two instruments), `scpdata_dso.mat` (20 records, `cm-1` axis),
`scpdata_als2004dataset.MAT` (204 records, index axis) and
`prospectr_NIRsoil.RData` (825 records, `nm` axis). The local-only
`samples_local/hyperspectral_cubes/indian_pines_corrected.mat` (21,025 pixels x
200 bands, `land_cover_class` from the optional `_gt.mat`) is covered when
present. All committed fixtures are golden-backed in
`crates/nirs4all-formats/tests/goldens/`.