JEDI-SKYLAB

JEDI-SKYLAB 8.0

JCSDA SkyLab 8.0 is the eighth  roll-up release that provides integrated Earth System Data Assimilation capability. All JEDI code is open source and publicly available at https://github.com/JCSDA. Capabilities are demonstrated via the SkyLab testbed experiments conducted internally at JCSDA for the following components: atmosphere, land/snow, ocean, sea-ice, aerosols, and atmospheric composition. SkyLab testbed output plots are openly available through the SkyLab Data Viewer at https://skylab.jcsda.org.    

The main upgrades for this release include:

  • skylab-weather-land-8_0 experiment:

We added two exciting new features: visible reflectance simulation, and surface soil moisture. The visible reflectance is from the Advanced Baseline Imager (ABI) on GOES 16-18, and is the first demonstration of visible reflectance assimilation using the CRTM. The use of visible reflectance observations is a significant advancement enabled by CRTM 3.1.0. The soil moisture product is produced from the NASA sensor, SMAP. 

  • skylab-weather-variable-grid-8_0 experiment:

The addition of multi-radar multi-sensor (MRMS) ground-based radar incorporates samples from the very dense data stream of CONUS radar reflectivity information. It is demonstrated with the JEDI-MPAS interface with radar data exclusively over CONUS.

  • skylab-marine-8_0 experiment:

We have been cleaning out old, no-longer used code and increasing the efficiency and ease of maintenance of the main SOCA code base. We've removed almost 4,000 lines of code this quarter without changing the marine SkyLab experiment, deleted the non-coupled ocean model and its interface since it’s of limited scientific value, and decreased the number of code lines back to 2022 levels without changing any of SkyLab’s functionality.

  • skylab-aerosol-weather-8_0 experiment:

The SkyLab 4DEnVar aero-weather v8 now uses CRTM AOD using NASA LUTs for aerosol providing more consistent optical properties with the GEOS model for aerosol assimilation. This change has been possible with the AOD (Aerosol Optical Depth) CRTM UFO interface which is now much more generic. It allows users to bring in their aerosol look-up tables for experiments. 

  • skylab-trace-gas-7_0 experiment:

In this release, we added 4DEnVar capability for the trace gas experiment and improved the NICAS/BUMP horizontal localization for atmospheric composition, which will provide better increments at different atmospheric levels. 

  • Software infrastructure 

  • Tested for the following system requirements:

    • Amazon Web Services (AWS) Single Node AMI (RedHat 8; gnu)

    • NASA Discover (gnu & intel)

    • NOAA Mississippi State University Orion (gnu & intel)

    • NOAA Mississippi State University Hercules (gnu & intel)

    • NOAA University of Wisconsin S4 (intel)

    • Limited support for NCAR-Wyoming Derecho (gnu & intel)

    • Limited support for NOAA ParallelWorks AWS, Azure, Gcloud (intel)

Version Date
8.0 2024-04-17
Release Notes

Download
Code

JEDI-SKYLAB 7.0

JCSDA SkyLab 7.0 is the seventh roll-up release that provides integrated Earth System Data Assimilation capability. All JEDI code is open source and publicly available at https://github.com/JCSDA. Capabilities are demonstrated via the SkyLab testbed experiments conducted internally at JCSDA for the following components: atmosphere, land/snow, ocean, sea-ice, aerosols, and atmospheric composition. SkyLab testbed output plots are openly available through the SkyLab Data Viewer at https://skylab.jcsda.org.    

The main upgrades for this release include:

  • skylab-weather-land-7_0 experiment:

    • Full cycling capability for the deterministic and ensemble DA with UFS atmospheric-land global model:

      • Deterministic dual resolution 4DEnVar with hybrid background error covariance matrix. 

        • High-resolution departure computation at c768 resolution

        • Medium-resolution minimization at c384 resolution

      • Ensemble of Data Assimilations: 80 members with 3DVar at c384 resolution.

    • CRTM v3.1.0 implemented  

    • Separation of GNSS-RO quality checks to make independent of selected operator

  • skylab-weather-variable-grid-7_0 experiment:

    • Full cycling capability for the deterministic and ensemble DA with MPAS global atmospheric model:

      • Deterministic dual resolution 4DEnVar with hybrid background error covariance matrix. 

        • High-resolution departure computation at 15km-3km variable resolution centered on CONUS.

        • Medium-resolution minimization at 30km

      • Ensemble of Data Assimilations: 30 members with 3DVar at 30km resolution.

  • skylab-marine-7_0 experiment:

    • new SABER central block (EXPLICIT_DIFFUSION) used as an alternative to BUMP for small scale background error correlations

  • skylab-aerosol-weather-7_0 experiment:

    • 4DEnVar with GEOS at C90 resolution with 32 ensemble members 

      • Weather obs: radiosondes, windborne, satwinds_ssec_amv, AMSU_A (NOAA 15,18,19 and MetOP B,C), MHS (NOAA 19 and MetOP B, C)

      • Aerosol obs: VIIRS (NPP and NOAA 20), MODIS (Aqua and Terra)

  • skylab-trace-gas-7_0 experiment:

    • Minor updates to plots

  • Software infrastructure 

  • Tested for the following system requirements:

    • Amazon Web Services (AWS) Single Node AMI (RedHat 8; gnu)

    • Amazon Web Services (AWS) Parallel Cluster (Ubuntu 20.04; intel)

    • NASA Discover (gnu & intel)

    • NOAA Mississippi State University Orion (gnu & intel)

    • NOAA Mississippi State University Hercules (gnu & intel)

    • NOAA University of Wisconsin S4 (intel)

    • Limited support for NCAR-Wyoming Derecho (gnu & intel)

    • Limited support for NOAA ParallelWorks AWS, Azure, Gcloud (intel)

Version Date
7.0 2024-01-18
Release Notes

Download
Code

JEDI-SKYLAB 6.0

JCSDA SkyLab 6.0 is the sixth roll-up release that provides integrated Earth System Data Assimilation capability. All JEDI code is open source, distributed under the Apache 2 licensing agreement and publicly available at https://github.com/JCSDA. Capabilities are demonstrated via the SkyLab testbed experiments conducted internally at JCSDA for the following components: atmosphere, land/snow, ocean, sea-ice, aerosols, and atmospheric composition. SkyLab testbed output plots are openly available through the SkyLab Data Viewer at https://skylab.jcsda.org.    

The main SkyLab upgrades for release 6.0 include:

  • Atmosphere-land experiment:

    • CRTM v3.0 bug fixes and minor updates compared to the skylab-v5 release.  

      • Updated the coefficients for IR Water emissivity (consistent with v2.4.1)

      • Bugfixes resolving NaNs and underflows 

    • Addition of GMI, AMSR/2, SnowDepth (GHCN), and active all-sky assimilation for GOES water vapor channels.

  • Marine experiment:

    • Quality control filters for in-situ temperature and salinity observations are added (checking: regional value range, profile spikes and gradients, and profile density inversions), these filters should be suitable for ingesting realtime in-situ observations that did not have existing quality control flags.

  • Trace gas experiment:

    • 3D-FGAT assimilation for CO and NO2 with FV3 stretch cubed sphere grid, providing down to 7 km resolution over CONUS. Observations include the new TEMPO proxy NO2 tropospheric columns

  • Software infrastructure 

  • Tested for the following system requirements:

    • Amazon Web Services (AWS) Single Node AMI (RedHat 8; gnu)

    • Amazon Web Services (AWS) Parallel Cluster (Ubuntu 20.04; intel)

    • NASA Discover (gnu & intel)

    • NOAA Mississippi State University Orion (gnu & intel)

    • NOAA University of Wisconsin S4 (intel)

    • NCAR Cheyenne (gnu)

    • Limited support for NOAA ParallelWorks AWS, Azure, Gcloud (intel)

Version Date
6.0 2023-10-06
Release Notes

Download
Code

JEDI-SKYLAB 5.0

JCSDA SkyLab 5.0 is the fifth roll-up release that provides integrated Earth System Data Assimilation capability via a unified end-to-end ecosystem including a single code build, workflow, data store, and diagnostics dashboard. Initial capabilities are demonstrated for the following components: atmosphere, ocean, sea-ice, soil moisture, snow, aerosols, and composition. The diagnostics dashboard is available at https://skylab.jcsda.org   

The main SkyLab upgrades for release 5.0 include:

  • Atmosphere-land experiment:

    • CRTM v3.0 use as observation operator for satellite radiances

    • Addition of monitoring for SSMIS, COWVR and TEMPEST

  • Trace gas experiment:

    • CO and NO2 3DVar assimilation from TropOMI NO2 tropospheric columns, TropOMI CO total columns and MOPITT CO total columns observations. 

  • Software infrastructure 

  • Support is extended to the following system requirements:

    • Amazon Web Services (AWS) Single Node AMI (RedHat 8)

    • Amazon Web Services (AWS) Parallel Cluster (Ubuntu 20.04)

    • NASA Discover (gnu & intel)

    • NOAA Mississippi State University Orion (gnu & intel)

    • NOAA University of Wisconsin S4 (gnu & intel)

    • NCAR Cheyenne (gnu & intel)

Date
2023-07-14

Version
5.0
Release Notes

Download
Code

JEDI-SKYLAB 4.0

JCSDA SkyLab 4.0 is the fourth roll-up release that provides integrated Earth System Data Assimilation capability via a unified end-to-end ecosystem including a single code build, workflow, data store, and diagnostics dashboard. Initial capabilities are demonstrated for the following components: atmosphere, ocean, sea-ice, soil moisture, snow, aerosols, and composition. The diagnostics dashboard is available at https://skylab.jcsda.org    

The main SkyLab upgrades for release 4.0 include:

  • Atmosphere-land experiment:

    • Updated observation converters for AMV satellite wind, ATMS, and TROPICS

    • Correlated observation errors for IASI and CRIS

  • Trace gas experiment:

    • CO and NO2 3DVar assimilation from TropOMI NO2 tropospheric columns, TropOMI CO total columns and MOPITT CO total columns observations. 

  • Software infrastructure 

    • Environment to build: all libraries on SPACK-STACK 1.3.0

    • Observation API: significant gains in I/O efficiency with IODA v.2.5

  • Support is extended to the following system requirements:

    • Amazon Web Services (AWS) Single Node AMI (RedHat 8)

    • Amazon Web Services (AWS) Single Node AMI (Ubuntu 20.04)

    • Amazon Web Services (AWS) Parallel Cluster (Ubuntu 20.04)

    • NASA Discover (gnu & intel)

    • NOAA Mississippi State University Orion (gnu & intel)

    • NOAA University of Wisconsin S4 (gnu & intel)

    • NCAR Cheyenne (gnu & intel)

Date
2023-04-17

Version
4.0
Release Notes

Download
Code

JEDI-SKYLAB 3.0

JCSDA SkyLab 3.0 is the third roll-up release that provides integrated Earth System Data Assimilation capability via a unified end-to-end ecosystem including a single code build, workflow, data store, and diagnostics dashboard. Initial capabilities are demonstrated for the following components: atmosphere, ocean, sea-ice, soil moisture, snow, aerosols, and composition. A diagnostics dashboard is available at https://skylab.jcsda.org    

The main SkyLab upgrades for release 3.0 include:

Atmosphere-land experiment:

  • Non-cycling 25 member 3D-Var EDA with static B at C192 resolution, and deterministic 3D-Var with hybrid B at C384.

  • More observation types: 

    • Ship, buoy

    • Monitoring:  GOES ABI, TROPICS, COWVR

    • Windborne balloon,

    • Commercial GNSS-RO (Spire, GeoOptics, and PlanetiQ)

Composition experiment:

  • Aerosol data assimilation (AOD)  3DVar EDA with 21 members at C96 resolution. VIIRS assimilated, MODIS monitored.

  • New TROPOMI CO converter

Turnkey ecosystem for the community to easily reproduce SkyLab on multiple platforms, 

  • Environment to build: all libraries on SPACK-STACK 1.2.0

  • Workflow: Generic experiments and suites based on EWOK 0.3.1

  • Data store: >100Tb of observation and model data for 15 Feb - 15 Mar 2022 and Aug 2021 on R2D2 data store 1.2.0

Support is extended to the following system requirements:

  • Amazon Web Services (AWS) AMI (Ubuntu & RedHat)

  • NASA Discover (gnu & intel)

  • NOAA Mississippi State University Orion (gnu & intel)

  • NOAA University of Wisconsin S4 (gnu & intel)

  • NCAR Cheyenne (gnu & intel)

Date
2023-01-09

Version
3.0

Release Notes

Download
Code

JEDI-SKYLAB 2.0

JCSDA SkyLab 2.0 is the second roll-up release that provides integrated Earth System Data Assimilation capability via a unified end-to-end ecosystem including a single code build, workflow, data store, and diagnostics dashboard. Initial capabilities are demonstrated for the following components: atmosphere, ocean, sea-ice, soil moisture, snow, aerosols, and composition. A diagnostics dashboard is available at https://skylab.jcsda.org    

The main SkyLab upgrades for release 2.0 include:

Atmosphere-land experiment:

  • Non-cycling 25 member EDA at C192 resolution. Each member is an FGAT-3DEnVar.

  • More observation types: 

    • Synop, METAR, 

    • ASCAT, 

    • CrIS, IASI, 

    • Windborne balloon, 

    • Commercial GNSS-RO (including Spire, GeoOptics)

  • Satellite bias correction

Marine experiment:

  • non-cycling 3DVar

  • New 1/4 degree sea-ice/ocean background provided by NOAA/EMC

  • New instruments: 

    • sea ice concentration retrievals from SSMIS F17/F18

    • insitu temperature and salinity profiles from NAVY/FNMOC

Composition experiment:

  • H(x) capability from GEOS-CF C90 resolution background 

  • New observations 

    • TROPOMI NO2 (tropospheric and total columns) and MOPITT CO (total columns)

    • Aerosol Optical Depth (AOD) from VIIRS NOAA20 and MODIS Terra and Aqua 

Turnkey ecosystem for the community to easily reproduce SkyLab on multiple platforms, 

  • Environment to build: all libraries on SPACK-STACK 1.1.0

  • Workflow: Generic experiments and suites based on EWOK 0.2.0

  • Data store: >50Tb of observation and model data for Aug 2021 on R2D2 data store 1.1.0

Support is extended to the following system requirements:

  • NOAA Orion (gnu & intel)

  • NASA Discover (gnu & intel)

  • University of Wisconsin S4 (intel)

  • AWS AMI (Ubuntu & RedHat)

Date
2022-10-11

Version
2.0

Release Notes

Download
Code

JEDI-SKYLAB 1.0

JCSDA SkyLab 1.0 is a roll-up release that provides the first integrated Earth System Data Assimilation capability via a unified end-to-end ecosystem including a single code build, workflow, data store, and diagnostics dashboard. 

Initial capabilities are demonstrated for the following components: atmosphere, ocean, sea-ice, soil moisture, snow, and aerosols. 

This release represents an important step to bridge research and operations since it introduces: 

a) Skylab flagship application to focus and demonstrate joint development toward future operational deployments. A diagnostics dashboard is available at https://skylab.jcsda.org    

b) Turnkey ecosystem for the community to easily reproduce SkyLab on multiple platforms, 

  • Environment to build: all libraries on SPACK-STACK 1.0.0

  • Workflow: Generic experiments and suites based on EWOK

  • Data store: 10Tb of observation and model data for Aug 2021 on R2D2 data store

c) Continuous Integration: automated testing of Skylab new prototype functions for flagged Github PRs, and hierarchical system development for rapid prototyping.

Support will be limited to capabilities demonstrated in SkyLab flagship application for the following system requirements:

  • NOAA Orion (gnu & intel)

  • NASA Discover (gnu & intel)

  • AWS AMI (Ubuntu & RedHat)

Version
1.0

Release Notes

Download
Code

Date
2022-07-18