This page documents the application of the Complete Data Fusion (CDF) algorithm to the IASI / AERIS ozone profile product. It covers both the legacy FORLI v20151001 files and the newer CDR V1.2.1 release that has replaced the legacy archive on the AERIS portal. The two formats share the same underlying retrieval algorithm (FORLI-O3) but differ in variable naming, units, and metadata compliance.
1. Overview
| Provider | AERIS — French atmospheric data infrastructure (IASI Portal). Algorithm developed by ULB-LATMOS; data production by EUMETSAT; reconstruction and netCDF formatting by LATMOS. |
| Instrument | IASI (Infrared Atmospheric Sounding Interferometer), nadir sounding |
| Platforms | Metop-A, Metop-B, Metop-C |
| Retrieval algorithm | FORLI-O3 v20151001 (same algorithm for both legacy and CDR releases) |
| Measured constituents | O3 |
| File format | netCDF (two variants — see Section 3) |
| CDF status | Tested — CDF(2022) passes for both formats; CDF(2015) fails for both formats (see §7) |
2. FAIR References
The following references allow the dataset to be uniquely identified, accessed, and cited according to FAIR (Findable, Accessible, Interoperable, Reusable) principles. Verified on 2026-04-11 against the AERIS IASI portal and the global attributes of the CDR V1.2.1 test file.
| Product title (CDR) | IASI/METOPA ULB-LATMOS ozone (O3) L2 products (profiles of partial columns) |
| Product version (current) | CDR Release 1 of IASI 2b — 2nd extraction (software version 1.2, product ID O3M-521.x) |
| Legacy version | v20151001 — no longer distributed through the AERIS portal (superseded by the CDR). Test results reported here refer to a local copy of a 2008 Metop-A file. |
| Persistent identifier | No DOI assigned to the CDR ozone product at the portal level (verified 2026-04-11). The Metop-A L1C reprocessed input used by EUMETSAT has DOI 10.15770/EUM_SEC_CLM_0014. |
| Product landing page | https://iasi.aeris-data.fr/o3/ — general O3 page https://iasi.aeris-data.fr/ozos_iasi_b_arch/ — IASI/Metop-B O3 L2 CDR archive |
| Data access | IASI catalog — free and open |
| Algorithm reference | Hurtmans, D. et al., FORLI radiative transfer and retrieval code for IASI, J. Quant. Spectrosc. Ra., 113, 1391–1408, doi:10.1016/j.jqsrt.2012.02.036, 2012. (cited verbatim in the CDR file references global attribute) |
| Metadata conventions | CF-1.8 (CDR only); standard names vocabulary: NetCDF CF Metadata Convention v80 (2022-02-07) |
| License / data policy | Free of charge under the AERIS IASI data use policy. For substantial use, co-authorship must be offered to the Principal Investigators (Cathy Clerbaux — LATMOS, and Pierre-François Coheur — ULB). |
| Required acknowledgement | “IASI is a joint mission of EUMETSAT and the Centre National d’Etudes Spatiales (CNES, France). The authors acknowledge the AERIS data infrastructure for providing access to the IASI data in this study, ULB-LATMOS for the development of the retrieval algorithms, and EUMETSAT/AC SAF for CO/O3 data production.” |
3. Dataset Description
The IASI Portal (https://iasi.aeris-data.fr/), operated by the AERIS data infrastructure, provides free and open access to atmospheric composition Level 2 products retrieved from IASI/Metop observations. The ozone profile product is the result of an optimal-estimation retrieval on IASI L1C radiances carried out with the FORLI-O3 radiative transfer and retrieval code developed at ULB-LATMOS.
Two file formats have been tested in the framework of the PNRR project, both based on the same FORLI v20151001 retrieval:
| Format | Status | Main characteristics |
|---|---|---|
| Legacy FORLI v20151001 |
Superseded, no longer distributed via AERIS portal. | netCDF with lowercase ozone_* variables, pressure in hPa, absolute error profile, no CF conventions declared, single nobservations dimension. |
| CDR V1.2.1 Release 1, 2nd extraction |
Current release on the AERIS portal. Generated by a re-processing of the EUMETSAT operational L2 chain (adapted V6.6), with ERA5 auxiliary data, and re-formatted by LATMOS. | CF-1.8 compliant netCDF with O3_* variables, unlimited time dimension, pressure in Pascal, relative error profile, retrieval_quality_flag (0=bad, 1=ok, 2=best), averaging_kernel_matrix (singular). |
The availability of the dataset depends on the platform:
| Platform | Temporal coverage |
|---|---|
| Metop-A | 01/01/2008 – 15/11/2021 |
| Metop-B | 08/03/2013 – ongoing |
| Metop-C | 20/09/2020 – ongoing |
The nominal spatial resolution of IASI is 12 km at nadir. The retrieval is performed in principal-component space, but the product is represented entirely in profile space within both netCDF formats.
4. State Vector
The state vector is composed of 41 elements, corresponding to O3 partial columns (mol m-2) on a fixed vertical grid of 40 layers of about 1 km height between the Earth’s surface and 40 km, plus one additional layer from 40 km up to the top of the atmosphere (60 km). The structure of the vertical grid is identical in the legacy and CDR formats.
5. Test Files
5.1 Legacy format — IASI_FORLI_O3_metopa_20080130_v20151001.nc
A Metop-A netCDF file (≈ 1300 MB) was used as the reference for the legacy format. All 167 470 products of the file were considered (no quality filtering). The average distance between each pixel and its nearest neighbour is 22.23 km.
5.2 CDR V1.2.1 format — IASI_METOPA_L2_O3_PROFILE_20080131_ULB-LATMOS_CDR_V1.2.1.nc
The CDR test file covers Metop-A on 31 January 2008 (one day later than the legacy reference). Products are selected by retrieval_quality_flag ≥ 1 (i.e. “ok” or “best” quality only; “bad” pixels with flag 0 are discarded).
retrieval_quality_flag ≥ 1.
6. Completeness Test Results
All completeness tests are passed for both formats. The retrieval algorithm is the same FORLI-O3 algorithm used for the IASI/EUMETSAT product. The retrieval happens in principal-component space but the full state vector and its characterisation matrices are stored in profile space in the netCDF files.
| # | Requirement | Legacy | CDR V1.2.1 |
|---|---|---|---|
| 1 | Optimal estimation retrieval | ✓ | ✓ |
| 2 | Full state vector available | ✓ | ✓ |
| 3 | Vertical profiles in terms of concentrations or partial columns | ✓ | ✓ |
| 4 | Vertical grid availability | ✓ | ✓ |
| 5 | Full a priori state vector availability | ✓ | ✓ |
| 6a | Averaging kernel availability (full state vector) | ✓ | ✓ |
| 6b | Total errors covariance matrix availability | ✓* | ✓* |
| 6c | A priori covariance matrix availability | ✓† | ✓† |
* Neither format stores the full total error covariance matrix explicitly; it is reconstructed from the averaging kernel matrix and the (reconstructed) a priori covariance matrix inside the reader.
† The a priori covariance is provided externally as a relative correlation matrix Sa_rel (distributed as /IASI-AERIS/Sa.pkl) and then rescaled layer-by-layer by the a priori profile to obtain the absolute Sa. This is identical for both formats.
7. Auto-consistency Test Results
The auto-consistency test has been applied in the profile space, which is the native representation for both file formats (the state vector is fully stored in profile space in both legacy and CDR files).
7.1 Legacy FORLI v20151001
In the CDF(2015) formulation the pseudo-inversion of \mathbf{S}_{ni} considered only the eigenvalues larger than fifteen orders of magnitude below the largest one. CDF(2022) passes with negligible errors; CDF(2015) fails due to poor conditioning of \mathbf{S}_{n,\mathrm{FORLI}}.
| # | IASI-AERIS-O3 Auto-consistency test results | \max(|\Delta x|)\% | (|\Delta\mathrm{DOFs}|)\% |
|---|---|---|---|
| 1 | CDF(2022) total error VCM inversion | <0.000001 | 0.000001 |
| 2 | CDF(2015) noise error VCM inversion | 3523.164798 | 0.000002 |
7.2 CDR V1.2.1 format
CDF(2022) passes with negligible errors; CDF(2015) fails due to poor conditioning of \mathbf{S}_{n,\mathrm{FORLI}}, as for the legacy format, though the error is smaller (242% vs 3523%).
| # | IASI-AERIS-O3 CDR V1.2.1 Auto-consistency test results | \max(|\Delta x|)\% | (|\Delta\mathrm{DOFs}|)\% |
|---|---|---|---|
| 1 | CDF(2022) total error VCM inversion | <0.000001 | 0.000003 |
| 2 | CDF(2015) noise error VCM inversion | 242.599938 | 0.000198 |
Detailed results are shown in the figures below.
8. CDF Variables Mapping
The following table lists the quantities strictly required by the CDF algorithm, their common Python attribute inside the reader Product object, and the corresponding netCDF variable in each of the two formats. The readers used in the test scripts are:
- Legacy:
cdftools.readers.iasi_netcdf_aeris_reader.IASIAREISnetcdfReader(used by05_PNRR_filetest/08_IASI_AERIS.py) - CDR V1.2.1:
cdftools.readers.iasi_netcdf_aeris_new_reader.IASIAREISCDRnetcdfReader(used by05_PNRR_filetest/10_IASI_AERIS_new.py)
| CDF quantity | Symbol | Reader attribute | Legacy variable | CDR V1.2.1 variable |
|---|---|---|---|---|
| Retrieved state vector (41 O3 partial columns, mol m-2) |
\widehat{\mathbf{x}} | p.o3 |
ozone_partial_column_profile |
O3_partial_column_profile |
| A priori state vector | \mathbf{x}_{a} | p.o3_apriori |
ozone_apriori_partial_column_profile |
O3_apriori_partial_column_profile |
| A priori covariance matrix (computed as \mathrm{diag}(\mathbf{x}_a)\,\mathbf{S}_{a}^{\mathrm{rel}}\,\mathrm{diag}(\mathbf{x}_a)) |
\mathbf{S}_{a} | p.vcm_a |
Sa_rel loaded from /IASI-AERIS/Sa.pkl (external file, common to both formats) |
|
| Total error covariance matrix (reconstructed as (\mathbf{I}-\mathbf{A})\,\mathbf{S}_{a}) |
\mathbf{S}_{n} | p.vcm_total |
Reconstructed inside the reader from \mathbf{A} and \mathbf{S}_{a} for both formats. The CDR file also provides a relative error profile in O3_partial_column_profile_error. |
|
| Averaging kernel matrix | \mathbf{A} | p.ak |
averaging_kernels_matrix (plural) |
averaging_kernel_matrix (singular) |
| Vertical grid (pressure levels) | — | p.pressure |
atmosphere_pressure_grid (hPa) |
atmosphere_pressure_grid (Pa → hPa in reader) |
| Quality filtering | — | getGoodPixels() |
No explicit flag — all observations are used. | retrieval_quality_flag: 0=bad (rejected), 1=ok, 2=best. |
Note on Sa: neither netCDF file stores the a priori covariance matrix explicitly. Both readers load a pre-computed relative correlation matrix Sa_rel from an external pickle file (~/progetti/aurora/data/IASI-AERIS/Sa.pkl) and convert it to absolute units by multiplying layer-wise by the a priori profile. This step is identical in the legacy and CDR readers.
9. Notes and Open Issues
- Both IASI/AERIS formats share the same FORLI v20151001 retrieval — the CDR is a re-formatting (and re-processing of the L1C inputs) by LATMOS and EUMETSAT, not a new retrieval algorithm. The auto-consistency results are therefore expected to be very similar between the two formats, and indeed they are.
- The legacy FORLI files are no longer distributed through the AERIS portal; users must migrate to the CDR V1.2.1 release, whose main entry point for the Metop-B archive is https://iasi.aeris-data.fr/ozos_iasi_b_arch/. The CDF reader
IASIAREISCDRnetcdfReaderhas been introduced in order to support this migration without breaking the existing scripts. - The CDR files are CF-1.8 compliant, have standard names, and include a quality flag and a tracking ID — a clear improvement in terms of FAIRness with respect to the legacy format.
- An open item is the fusion of IASI-O3 products from different providers (IASI-EUMETSAT, IASI-AERIS FORLI, IASI-AERIS CDR, IASI+GOME2) — see the To Be Studied page.
Back to: Tested Datasets overview.