CSPP Geo GRB config change to mitigate data loss after Mar 1

Post Reply
graemem
Posts: 14
Joined: Tue Jan 11, 2011 5:44 pm

CSPP Geo GRB config change to mitigate data loss after Mar 1

Post by graemem »

A change in the GOES-R ground system is planned for March 1, 2023, which is expected to result in occasional data loss in products generated by CSPP Geo GRB software. Users are advised to review the information below and consider making a software configuration change to mitigate potential data loss.

Background: In the GRB data stream, the arrival of product metadata normally signals that a complete dataset has arrived. However, in some cases data packets continue to be received after the product metadata has arrived. Currently the CSPP Geo GRB software includes a built-in delay to wait for late data packets, which is applied for each image or dataset. The delay is configurable, with a default value of 0.5 seconds. While in the past this has been sufficient to avoid data loss in almost all cases, recently a user reported missing blocks of data in ABI band 2 images (see the example CONUS image in this thread). The cause was determined to be an increase in late data packets exceeding 0.5 seconds, associated with a ground system update in December 2022. That update has since been reverted, but will go live again on March 1. The issue can be mitigated by increasing the delay via a CSPP Geo software configuration change, at the cost of increased overall latency for all products.

Possible configuration changes are described below, as well as the expected results based on the observed behavior after the ground system update in December. These options assume sites are currently configured with the PUG V4 recommended default delay of 0.5 seconds.

Option 1: No configuration change. Users can expect occasional missing data in ABI Band 2 CONUS, Full Disk and Meso images. Product latency is unchanged from current. This may be the best option if low product latency is a priority and occasional data loss is acceptable.

Option 2: Change delay to 2 seconds. Users can expect missing data only very rarely in ABI Band 2 CONUS, Full Disk and Meso images. Overall latency of all products is increased by 1.5 seconds. This is the delay that was recommended by the ground system team in recent discussions.

Option 3: Change delay to 10 seconds. Users can expect missing data only very rarely in ABI Band 2 CONUS, Full Disk and Meso images, plus there is an additional buffer in case the behavior in the GRB stream deteriorates in the future. Overall latency of all products is increased by 9.5 seconds. This was the default delay in CSPP Geo GRB software prior to version 1.0.24, and may be the best option if the added latency is acceptable.

To change the delay in CSPP Geo software, refer to Section 8.8 in the CSPP Geo GRB Software Users’ Guide, v1.0.27, available from the CSPP Geo website (free registration required).

Please contact csppgeo.issues@ssec.wisc.edu with any questions.
Post Reply