Skip to content

Diagnostics and logs

In addition to radiances and transmittances, JURASSIC can produce a variety of diagnostic output products that are primarily relevant for sensitivity analysis, retrieval development, and scientific interpretation. These diagnostics are most commonly generated by the kernel and retrieval applications.

This page provides an overview of diagnostic outputs, matrix products, and runtime log messages.

Overview of diagnostic products

Depending on the application and configuration, JURASSIC can output:

  • Jacobians / kernels (sensitivities of radiances to state variables)
  • Averaging kernels (retrieval sensitivity and resolution)
  • Error covariance matrices
  • Cost-function diagnostics written to standard output
  • AVK-derived contribution and resolution diagnostics

Most of these products are written to dedicated output files or to matrix-formatted diagnostic files controlled by runtime options.

Jacobians (kernels)

Definition

Jacobians (also called weighting functions or kernels) describe the sensitivity of the simulated radiances to changes in the atmospheric state vector.

Mathematically, the Jacobian matrix \(\mathbf{K}\) is defined as:

\[ K_{ij} = \frac{\partial y_i}{\partial x_j} \]

where:

  • \(y_i\) is the radiance in detector channel i,
  • \(x_j\) is the j-th element of the state vector.

Physical interpretation

Jacobians quantify:

  • which altitude regions contribute most strongly to a given channel,
  • how sensitive a channel is to temperature or trace gas variations,
  • potential parameter correlations in retrievals.

Large Jacobian magnitudes indicate strong sensitivity.

Output format

The kernel application writes the Jacobian as a matrix file, using the output filename passed on the command line, for example kernel.tab. During retrieval diagnostics, the same type of matrix is commonly written as matrix_kernel.*.

Matrix files are controlled by WRITE_MATRIX and MATRIXFMT; see Matrix output control.

Averaging kernels

Definition

The averaging kernel matrix \(\mathbf{A}\) describes the sensitivity of the retrieved state to the true atmospheric state:

\[ \mathbf{A} = \frac{\partial \hat{\mathbf{x}}}{\partial \mathbf{x}} \]

Averaging kernels are a central diagnostic in optimal estimation retrieval theory.

Interpretation

Averaging kernels provide information on:

  • vertical resolution of the retrieval,
  • sensitivity to true atmospheric variations,
  • influence of the a priori constraints.

Rows of \(\mathbf{A}\) close to unity indicate good sensitivity, while broader rows indicate vertical smoothing.

Output

Averaging kernels are written by retrieval applications during error analysis. In practice this means retrieval diagnostics require both RET_ERROR = 1 and WRITE_MATRIX = 1. They are typically stored alongside Jacobians and error covariance matrices.

Retrieval error covariance

Definition

The retrieval error covariance matrix \(\mathbf{S}_x\) quantifies the expected uncertainty of the retrieved atmospheric state:

\[ \mathbf{S}_x = (\mathbf{K}^T \mathbf{S}_\epsilon^{-1} \mathbf{K} + \mathbf{S}_a^{-1})^{-1} \]

Interpretation

Diagonal elements represent variances (squared uncertainties) of the retrieved parameters, while off-diagonal elements describe error correlations.

Error covariance diagnostics are essential for:

  • uncertainty quantification,
  • data assimilation,
  • scientific interpretation of retrieval results.

Cost function and convergence diagnostics

During iterative retrievals, JURASSIC evaluates the optimal-estimation cost function at each iteration.

Typical diagnostic quantities include:

  • iteration number,
  • normalized cost-function value, printed as chi^2/m.

These diagnostics are written to standard output and can be used to assess retrieval stability and convergence behavior. Users can redirect standard output and standard error to a file from the shell if they want to keep a run log.

AVK-derived contribution and resolution diagnostics

Retrieval diagnostics also include AVK-derived contribution and resolution profile outputs (for example atm_cont.* and atm_res.*), which summarize how strongly the retrieval responds to the true state and how vertically resolved the retrieved quantities are.

These products are derived from averaging-kernel analysis rather than from a separate forward-radiative contribution-function calculation.

They are useful for:

  • qualitative interpretation of retrieval sensitivity,
  • diagnosing vertical smoothing and resolution,
  • comparing retrieval setups across test cases.

Matrix output control

Many matrix-valued diagnostic products are written only when matrix output is explicitly enabled via the control parameter:

  • WRITE_MATRIX
  • MATRIXFMT

When enabled:

  • Jacobians, averaging kernels, and error covariance matrices are written in a structured, machine-readable format,
  • output files can be large and are intended for post-processing with external tools (e.g. Python, MATLAB).

MATRIXFMT selects the matrix file format:

Value Format
1 ASCII
2 binary
3 netCDF

The netCDF matrix format can store multiple matrix records in one file, which is useful for shared-file retrieval workflows and large diagnostic sets.

For retrieval workflows, matrix output alone is not sufficient: retrieval-side diagnostics are generated only when RET_ERROR = 1.

Common retrieval diagnostic matrix files include:

File Content
matrix_cov_apr.* a priori covariance
matrix_kernel.* Jacobian / kernel matrix
matrix_cov_ret.* retrieval covariance
matrix_corr.* retrieval correlation matrix
matrix_gain.* gain matrix
matrix_avk.* averaging kernel matrix

Runtime logging

JURASSIC writes runtime log messages to standard output. These messages include:

  • input and output files being read or written,
  • basic data ranges for atmospheric, observation, radiance, and lookup table inputs,
  • iteration and chi^2/m values during retrievals,
  • timer summaries for selected workflow stages,
  • warnings and fatal errors.

Warnings and errors include source-file, function, and line-number context. Fatal errors terminate the program after printing the message.

Tip

For long runs, redirect standard output and standard error to a log file, for example ./retrieval run.ctl dirlist.txt > run.log 2>&1.

Practical considerations

  • Diagnostic output can be large; enable it selectively.
  • Always verify that Jacobians are consistent with forward-model behavior when developing new configurations.
  • Compare averaging kernels and error estimates across test cases to ensure retrieval robustness.

Summary

Diagnostic output products in JURASSIC provide essential insight into radiative transfer sensitivities, retrieval performance, and uncertainty characteristics.

These diagnostics are indispensable for advanced users developing new retrieval setups, validating configurations, or interpreting scientific results.