Problem with processing l1b Metop data

Advanced Technology Microwave Sounder Sensor Data Record

Problem with processing l1b Metop data

Postby zion0210 » Tue Jun 22, 2021 1:57 am

Hi colleagues.

I have a problem with processing AMSU/MHS l1b data.
I used MIRS_v2.4 and l1b NOAA data (https://www.avl.class.noaa.gov/).

For example, I run the following command:
Code: Select all
CSPP_MIRS_2_4//scripts/run_mirs.bash -x --noSFR -d /home/terra/NPP_SOFT//MIRS_v2.4/CSPP_MIRS_2_4//data/dynamic_ancillary/ -w /home/terra/NPP_SOFT//WORK/ -s metopB -p 8 -i /home/terra/ATMS_NPP/metop_test/ -vvv


The inputs files are NSS.AMAX.M1.D21166.S0316.E0405.B4535555.MM and NSS.MHSX.M1.D21166.S0316.E0405.B4535555.MM .

And receive the following log:
Code: Select all
DEBUG %options
==============
'dynanc' => '/NFS/snpp2-4/ATMS_GFS/dynamic_ancillary',
'sat' => 'metopB',
'block' => 2,
'input_files' => '/home/terra/ATMS_NPP/metop_test/',
'xfiles' => 1,
'clean' => 1,
'verbose' => 3,
'processors' => 8,
'nothing' => 0,
'SFR' => 0,
'quarantine' => 0,
'logs' => 1,
'yikes' => 0,
'aggregate' => 1,
'resolution' => 'HI',
'work' => '/HDDS/HDD1/public/NPP_SOFT/WORK'
DEBUG %options _END_
---------------
DEBUG %filemask
===============
'metopA' => '*.{AMAX,MHSX}.M2*',
'metopB' => '*.{AMAX,MHSX}.M1*',
'metopC' => '*.{AMAX,MHSX}.M3*',
'n19' => '*.{AMAX,MHSX}.NP*',
'n21' => '{GATMO,SATMS,TATMS}_j02*.h5',
'npp' => '{GATMO,SATMS,TATMS}_npp*.h5',
'n18' => '*.{AMAX,MHSX}.NN*',
'n20' => '{GATMO,SATMS,TATMS}_j01*.h5'
DEBUG %filemask _END_
-----------------
DEBUG %dynanc_loc
=================
'GFSPROTO' => 'https://',
'GFSLOCAL' => '/NFS/snpp2-4/ATMS_GFS/dynamic_ancillary/%Y_%m_%d_%j',
'GFSSITE' => 'jpssdb.ssec.wisc.edu',
'GFSREMOTE' => 'cspp_v_2_0/ancillary/%Y_%m_%d_%j',
'GFSTEMPLATE' => 'gfs.t%Hz.pgrb2.0p50.f0%f.%Y%m%d'
DEBUG %dynanc_loc _END_
chdir to working directory /HDDS/HDD1/public/NPP_SOFT/WORK/TMP_Bzlt_K
PCF file is:  /HDDS/HDD1/public/NPP_SOFT/WORK/TMP_Bzlt_K/cspp_metopB_pcf.bash
SCS file is:  /HDDS/HDD1/public/NPP_SOFT/WORK/TMP_Bzlt_K/cspp_metopB_scs.bash
run_mirs.pl::make_L1B_bigEndian: /home/terra/ATMS_NPP/metop_test/NSS.AMAX.M1.D21166.S0316.E0405.B4535555.MM endian indeterminate
Abort at /home/terra/NPP_SOFT//MIRS_v2.4/CSPP_MIRS_2_4//scripts/run_mirs.pl line 1014.


I also receive identical error for old version MIRS.

Regards,
zion0210
zion0210
 
Posts: 25
Joined: Thu Mar 27, 2014 4:28 am

Re: Problem with processing l1b Metop data

Postby jimd » Tue Jun 22, 2021 11:21 am

The run_mirs.pl script tries to determine whether the input file is big- or little-endian by decoding the header record both ways and checking if the decoded year makes sense. I am wondering if your file has no header record - which would be a problem, not just for endian-ness but, as the code is presently written, also for a filename extracted from the header record.
If you could please send me the two files (AMAX and MHSX) I can take a look. Best way might be to upload to ftp://ftp.ssec.wisc.edu/pub/incoming/ and then drop me a line.
Was there an option on CLASS to get files with/without header? I would have tested CSPP_MIRS with CLASS data, but perhaps something has changed since I did so.
Cheers,
-jim.
jimd
 
Posts: 12
Joined: Thu Sep 05, 2013 3:20 pm

Re: Problem with processing l1b Metop data

Postby zion0210 » Tue Jun 22, 2021 11:58 pm

Hi Jim.

Files NSS.AMAX.M1.D21166.S0316.E0405.B4535555.MM and NSS.MHSX.M1.D21166.S0316.E0405.B4535555.MM located at ftp://ftp.ssec.wisc.edu/pub/incoming/

I used the following form getting l1b metop data
https://www.avl.class.noaa.gov/saa/prod ... submit.y=6

Regards,
zion0210
zion0210
 
Posts: 25
Joined: Thu Mar 27, 2014 4:28 am

Re: Problem with processing l1b Metop data

Postby kathys » Wed Jun 23, 2021 1:08 pm

Hello zion0210,

We have investigated your data and we believe we have a solution.

There is header information that needs to be included in the data that you order from CLASS. To include the header information, you need to follow these instructions:

1). On the CLASS website, along the Left side options, under "User Account", choose "Preferences". This brings you to a "User Preferences" page.

2). Select "Yes" for the Delivery Preference: "Prepend TBM/ARS headers to my data sets"

3). Then scroll to the bottom of the page and select the button "Update".

Once you follow these steps, the data that you order from CLASS will include the header information that you need to execute the MiRS software.

I downloaded the same files that you are working on using this preference, and we were able to process the data using the CSPP MiRS software. I put those new CLASS input files here: https://bin.ssec.wisc.edu/pub/CSPP/test/

Good luck,

Kathy
kathys
 
Posts: 345
Joined: Tue Jun 22, 2010 4:51 pm


Return to ATMS SDR

Who is online

Users browsing this forum: No registered users and 1 guest