Page 1 of 1

exiting CSPP-IAPP message

Posted: Fri Oct 06, 2017 4:08 am
by lapan28
Hello good afternoon,

I process hirs level1 data from NOAA19 data with CSPP-IAPP to get level2 data but currently there is a message problem "Exiting CSPP IAPP".
This is the log processing file :

2017-10-06 08:24:35.495 (INFO) : Starting CSPP IAPP ...
2017-10-06 08:24:35.496 (INFO) : Pass start time: 2017-10-04 08:56:10.636000
2017-10-06 08:24:35.496 (INFO) : Pass mid time: 2017-10-04 09:02:47.136000
2017-10-06 08:24:35.496 (INFO) : Pass end time: 2017-10-04 09:09:24.236000
2017-10-06 08:24:35.496 (INFO) : Pass length: 793 seconds
2017-10-06 08:24:35.497 (INFO) : Retrieving and granulating ancillary data for /home/sbpekayon/CSPP_IAPP_1_0/hirsl1d_noaa19_20171004_0856_44606.l1d...
2017-10-06 08:24:35.497 (INFO) : Retrieving NCEP files for 2017-10-04 09:02:47.136000 ...
2017-10-06 08:43:20.865 (INFO) : Retrieved GRIB file: /home/sbpekayon/CSPP_IAPP_1_0/anc/cache/2017_10_04_277/gdas1.PGrbF00.171004.12z
2017-10-06 08:43:20.865 (INFO) : Retrieved GFS files: ['/home/sbpekayon/CSPP_IAPP_1_0/anc/cache/2017_10_04_277/gdas1.PGrbF00.171004.12z']
2017-10-06 08:43:20.866 (INFO) : Transcoding NCEP file /home/sbpekayon/CSPP_IAPP_1_0/anc/cache/2017_10_04_277/gdas1.PGrbF00.171004.12z to NetCDF...
2017-10-06 08:43:21.360 (INFO) : New NetCDF file successfully created: /home/sbpekayon/CSPP_IAPP_1_0/iapp_ancillary_2017100412-anl.nc
2017-10-06 08:43:21.360 (INFO) : Transcoded GDAS/GFS NetCDF file: /home/sbpekayon/CSPP_IAPP_1_0/anc/cache/2017_10_04_277/iapp_ancillary_2017100412-anl.nc
2017-10-06 08:43:21.361 (INFO) : Creating NetCDF template file /home/sbpekayon/CSPP_IAPP_1_0/uwretrievals.nc ...
2017-10-06 08:43:21.370 (INFO) : New NetCDF retrieval template file successfully created: /home/sbpekayon/CSPP_IAPP_1_0/uwretrievals.nc
2017-10-06 08:43:21.370 (INFO) : Populating NetCDF template file /home/sbpekayon/CSPP_IAPP_1_0/uwretrievals.nc ...
2017-10-06 08:43:21.461 (INFO) : iapp_main ran in 0.0910999774933 seconds.
2017-10-06 08:43:21.461 (INFO) : IAPP completed successfully, and created: /home/sbpekayon/CSPP_IAPP_1_0/uwretrievals.nc
2017-10-06 08:43:21.461 (INFO) : Moving /home/sbpekayon/CSPP_IAPP_1_0/uwretrievals.nc to /home/sbpekayon/CSPP_IAPP_1_0/noaa19_L2_d20171004_t0856106_e0909242_c20171006084321461947_iapp.nc...
2017-10-06 08:43:21.462 (WARNING) : /home/sbpekayon/CSPP_IAPP_1_0/noaa19_L2_d20171004_t0856106_e0909242_c20171006084321461947_iapp.nc has same size (16376 bytes) as uwretrievals.nc, possible IAPP failure.
2017-10-06 08:43:21.462 (INFO) : Performing routine cleanup of working directory...
2017-10-06 08:43:21.462 (INFO) : Exiting CSPP IAPP ...

After this process in CSPP-IAPP the result is noaa19_L2_d20171004_t0856106_e0909242_c20171006084321461947_iapp.nc
with the capacity of data is 16.8KB but with this level2 data I can't open it cause no data inside this file.
Is somebody ever get this problem? I hope someone in this forum can help me.
Thank you.

Re: exiting CSPP-IAPP message

Posted: Mon Oct 09, 2017 2:08 pm
by kathys
Hello,

We advise you to upgrade to the most recent version of CSPP IAPP (version 1.1), which can be found here:

http://cimss.ssec.wisc.edu/cspp/iapp_v1.1.shtml

This version corrected some problems and also includes better logging. If you upgrade, and are still having problems, please let us know.

Kathy

Re: exiting CSPP-IAPP message

Posted: Tue Oct 10, 2017 8:44 pm
by lapan28
Good morning,
I have tried to use the latest version of CSPP_IAPP_v1.1 but still encounter problems like the following :


2017-10-11 01:30:09.984 (INFO) : Starting CSPP IAPP ...
2017-10-11 01:30:09.985 (INFO) :
>>> Processing hirs file /home/sbpekayon/CSPP_IAPP_1_1/hirsl1d_noaa19_20171009_0758_44676.l1d
2017-10-11 01:30:09.987 (INFO) : Pass start time: 2017-10-09 07:59:00.249000
2017-10-11 01:30:09.987 (INFO) : Pass mid time: 2017-10-09 08:05:36.749000
2017-10-11 01:30:09.987 (INFO) : Pass end time: 2017-10-09 08:12:13.849000
2017-10-11 01:30:09.987 (INFO) : Pass length: 793 seconds
2017-10-11 01:30:09.987 (INFO) : Retrieving and granulating ancillary data for /home/sbpekayon/CSPP_IAPP_1_1/hirsl1d_noaa19_20171009_0758_44676.l1d...
2017-10-11 01:30:09.987 (INFO) : Retrieving NCEP files for 2017-10-09 08:05:36.749000 ...
2017-10-11 01:30:24.057 (INFO) : Retrieved GRIB file: /home/sbpekayon/CSPP_IAPP_1_1/anc/cache/2017_10_09_282/gdas1.PGrbF00.171009.06z
2017-10-11 01:30:24.058 (INFO) : Transcoding NCEP file /home/sbpekayon/CSPP_IAPP_1_1/anc/cache/2017_10_09_282/gdas1.PGrbF00.171009.06z to NetCDF...
2017-10-11 01:30:25.067 (INFO) : Transcoded GDAS/GFS NetCDF file: /home/sbpekayon/CSPP_IAPP_1_1/anc/cache/2017_10_09_282/iapp_ancillary_2017100906-anl.nc
2017-10-11 01:30:25.068 (INFO) : Creating NetCDF template file /home/sbpekayon/CSPP_IAPP_1_1/iapp_l2_hirsl1d_noaa19_20171009_0758_44676.l1d_run_0/uwretrievals.nc ...
2017-10-11 01:30:25.078 (INFO) : New NetCDF retrieval template file successfully created: /home/sbpekayon/CSPP_IAPP_1_1/iapp_l2_hirsl1d_noaa19_20171009_0758_44676.l1d_run_0/uwretrievals.nc
2017-10-11 01:30:25.078 (INFO) : Running iapp_main, output to NetCDF template file /home/sbpekayon/CSPP_IAPP_1_1/iapp_l2_hirsl1d_noaa19_20171009_0758_44676.l1d_run_0/uwretrievals.nc ...
2017-10-11 01:30:25.148 (WARNING) : WARNING: AMSUA_Data_Flag fails data quality check in subroutine fill_ret_from_hirs1d
2017-10-11 01:30:25.149 (WARNING) : WARNING: AMSUA_Data_Flag fails data quality check in subroutine fill_ret_from_hirs1d
2017-10-11 01:30:25.149 (WARNING) : WARNING: HIRS_Data_Flag fails data quality check in subroutine fill_ret_from_hirs1d
2017-10-11 01:30:25.149 (WARNING) : WARNING: AMSUA_Data_Flag fails data quality check in subroutine fill_ret_from_hirs1d
2017-10-11 01:30:25.149 (WARNING) : WARNING: HIRS_Data_Flag fails data quality check in subroutine fill_ret_from_hirs1d
2017-10-11 01:30:25.149 (WARNING) : WARNING: AMSUA_Data_Flag fails data quality check in subroutine fill_ret_from_hirs1d
2017-10-11 01:30:25.149 (WARNING) : WARNING: HIRS_Data_Flag fails data quality check in subroutine fill_ret_from_hirs1d
2017-10-11 01:30:25.149 (WARNING) : WARNING: AMSUA_Data_Flag fails data quality check in subroutine fill_ret_from_hirs1d
2017-10-11 01:30:25.149 (WARNING) : Maximum number of "Bad_AMSUA_Data" messages reached, further instances will be counted only
2017-10-11 01:30:25.149 (WARNING) : WARNING: HIRS_Data_Flag fails data quality check in subroutine fill_ret_from_hirs1d
2017-10-11 01:30:25.150 (WARNING) : WARNING: HIRS_Data_Flag fails data quality check in subroutine fill_ret_from_hirs1d
2017-10-11 01:30:25.150 (WARNING) : Maximum number of "Bad_HIRS_Data" messages reached, further instances will be counted only
2017-10-11 01:30:35.206 (WARNING) : Maximum number of attempts (20) of obtaining return code for /home/sbpekayon/CSPP_IAPP_1_1/common/IAPP_VENDOR/iapp/bin/iapp_main reached, setting to zero.
2017-10-11 01:30:35.206 (WARNING) : 75 HIRS fields of regard failed data quality check.
2017-10-11 01:30:35.206 (WARNING) : 406 AMSU-A fields of regard failed data quality check.
2017-10-11 01:30:35.207 (INFO) : iapp_main ran in 10.1288788319 seconds.
2017-10-11 01:30:35.207 (WARNING) : /home/sbpekayon/CSPP_IAPP_1_1/noaa19_L2_d20171009_t0759002_e0812138_c20171011013035207461_iapp.nc has same size (16376 bytes) as uwretrievals.nc, possible IAPP failure.
2017-10-11 01:30:35.207 (WARNING) : No valid retrievals in /home/sbpekayon/CSPP_IAPP_1_1/noaa19_L2_d20171009_t0759002_e0812138_c20171011013035207461_iapp.nc, possible bad l1d file.
2017-10-11 01:30:35.208 (WARNING) : There was a problem running IAPP, moving run files to /home/sbpekayon/CSPP_IAPP_1_1/iapp_l2_hirsl1d_noaa19_20171009_0758_44676.l1d_run_0_crashed
2017-10-11 01:30:35.208 (INFO) : attempted_runs ['hirsl1d_noaa19_20171009_0758_44676.l1d']
2017-10-11 01:30:35.208 (INFO) : successful_runs []
2017-10-11 01:30:35.208 (INFO) : crashed_runs []
2017-10-11 01:30:35.208 (INFO) : problem_runs ['hirsl1d_noaa19_20171009_0758_44676.l1d']
2017-10-11 01:30:35.208 (INFO) : Exiting CSPP IAPP ...

This is the log of data processing, whether the level 1 data I have has a problem? Cause I read in log there isi message "possible bad l1d file" or I mis-installed the software?
Thank you.

Re: exiting CSPP-IAPP message

Posted: Wed Oct 11, 2017 10:35 am
by geoffc
Hi,
the log file you posted seems to indicate that the l1d file might be bad ("75 HIRS fields of regard failed data quality check...", etc...). Do you get an output file at the end (noaa19_L2_d20171009_t0759002_e0812138_c20171011013035207461_iapp.nc)? If you could post the l1d file to ftp://ftp.ssec.wisc.edu/pub/incoming/, I could take a look for you. But otherwise it looks like your installation is ok.



Geoff

Re: exiting CSPP-IAPP message

Posted: Thu Oct 12, 2017 8:32 pm
by lapan28
Good morning,
this is the processing result of NOAA level2 data that I can't open.
I use winrar to upload this data.
Could you please check this file?
Thank you.

Re: exiting CSPP-IAPP message

Posted: Mon Oct 16, 2017 10:51 am
by geoffc
Hi,
can you also post the level-1d file that you used to create the level-2 file?

Thanks,
Geoff

Re: exiting CSPP-IAPP message

Posted: Wed Oct 18, 2017 10:57 pm
by lapan28
Good morning,
this is level1d data of NOAA19.
I use winrar to upload this data.
Could you please check this file?
Thank you.

Re: exiting CSPP-IAPP message

Posted: Thu Oct 19, 2017 11:05 pm
by geoffc
Hi,
in the level-1d file you posted, there is valid HIRS data, but the AMSU-A and AMSU-B BT datasets are all set to the fill value (-10000).

Geoff