To apply the CDF to a generic level 2 atmospheric product (primarily vertical profiles of temperature or constituents), the product must have specific characteristics. These requirements ensure that the CDF algorithm can be correctly applied and that the results are physically meaningful.
Six Fundamental Requirements
- Optimal estimation technique — The product must be retrieved using the optimal estimation method.
- Full state vector — The complete state vector \widehat{\mathbf{x}}_{i} must be available.
- Profile definition — Profiles in the state vector must be defined in terms of concentrations or partial columns.
- Vertical grid — The vertical grid (pressure or height, levels or layers) on which the profiles are defined must be available.
- A priori state vector — The complete a priori state vector \mathbf{x}_{ai} must be available, with profile components defined on the same grid as the retrieved profiles.
- Characterisation matrices — At least two of the following three matrices must be available:
- The Averaging Kernels Matrix \mathbf{A}_{i} (for the full state vector)
- The Variance-Covariance Matrix of total errors \mathbf{S}_{i} (for the full state vector)
- The a priori Variance-Covariance Matrix \mathbf{S}_{ai} (for the full state vector)
If only two of the three matrices in requirement 6 are available, the third can be calculated using the relationships below.
Requirements 1, 5 and 6c presuppose an OE retrieval. Products retrieved with a Tikhonov constraint or with a Levenberg–Marquardt minimisation without an a priori covariance, and products whose file does not contain all the quantities listed above (AKM and covariance of the species block only, diagonal covariances, no reference profile, compressed AKM), are treated on the Non-OE and incomplete products page. The CDF uses of each product only the information of the measurement, which can be reconstructed from the AKM and the matching error covariance without \mathbf{S}_{ai}; which covariance matches which formulation, and which checks replace the auto-consistency test, is stated there.
Deriving Missing Characterisation Matrices
The three matrices in requirement 6 are related by the following equations. Use these to compute the missing matrix when only two are available.
Note on equation (P2) and (P3): Both inverses refer to matrices that are typically invertible. However, the noise covariance matrix \mathbf{S}_{ni}, if available instead of the total error covariance \mathbf{S}_{i}, is more problematic because the inverse of the averaging kernel matrix is often singular. The relationship \mathbf{S}_{ni} = \mathbf{A}_{i} \cdot \mathbf{S}_{i} can be safely calculated in the forward direction, but the inverse transformation is not generally possible.
Unlike (P1)–(P3), the relationship \mathbf{S}_{ni} = \mathbf{A}_{i} \cdot \mathbf{S}_{i} does not require an OE retrieval: it holds for any retrieval with a symmetric quadratic constraint (see the Non-OE and incomplete products page, Eq. N.3).
Special Case: Total Column Products
For products that provide total columns (integrated quantities) rather than vertical profiles, the same requirements apply with the following modifications:
- The covariance matrices reduce to scalar values (variances)
- The averaging kernel becomes a row vector (derivative of column with respect to true profile)
- The a priori information used in the total column retrieval must be available
Completeness Testing
Data completeness should be assessed using a systematic checklist. The table below summarizes the requirements to verify:
| Test | Requirement |
|---|---|
| 1 | Optimal estimation technique used |
| 2 | Full state vector available |
| 3 | Vertical profiles in terms of concentrations or partial columns |
| 4 | Vertical grid availability (pressure or height) |
| 5 | Full a priori state vector availability |
| 6a | Averaging kernel matrix available (full state vector) |
| 6b | Total error covariance matrix available (full state vector) |
| 6c | A priori covariance matrix available (full state vector) |
Table 1: Completeness test checklist. At least two of tests 6a, 6b, 6c must pass (the third can be derived using equations P1–P3).
When test 1 fails, or 6c fails because the retrieval has no a priori covariance, the product can still enter the CDF if 6a and 6b pass with the covariance that matches the chosen formulation; the auto-consistency test is then reported as "not applicable", not as failed, and the checks of the Non-OE and incomplete products page apply.