exiting CSPP-IAPP message

Post Reply
lapan28
Posts: 4
Joined: Mon Mar 07, 2016 3:58 am

exiting CSPP-IAPP message

Post by lapan28 »

Hello good afternoon,

I process hirs level1 data from NOAA19 data with CSPP-IAPP to get level2 data but currently there is a message problem "Exiting CSPP IAPP".
This is the log processing file :

2017-10-06 08:24:35.495 (INFO) : Starting CSPP IAPP ...
2017-10-06 08:24:35.496 (INFO) : Pass start time: 2017-10-04 08:56:10.636000
2017-10-06 08:24:35.496 (INFO) : Pass mid time: 2017-10-04 09:02:47.136000
2017-10-06 08:24:35.496 (INFO) : Pass end time: 2017-10-04 09:09:24.236000
2017-10-06 08:24:35.496 (INFO) : Pass length: 793 seconds
2017-10-06 08:24:35.497 (INFO) : Retrieving and granulating ancillary data for /home/sbpekayon/CSPP_IAPP_1_0/hirsl1d_noaa19_20171004_0856_44606.l1d...
2017-10-06 08:24:35.497 (INFO) : Retrieving NCEP files for 2017-10-04 09:02:47.136000 ...
2017-10-06 08:43:20.865 (INFO) : Retrieved GRIB file: /home/sbpekayon/CSPP_IAPP_1_0/anc/cache/2017_10_04_277/gdas1.PGrbF00.171004.12z
2017-10-06 08:43:20.865 (INFO) : Retrieved GFS files: ['/home/sbpekayon/CSPP_IAPP_1_0/anc/cache/2017_10_04_277/gdas1.PGrbF00.171004.12z']
2017-10-06 08:43:20.866 (INFO) : Transcoding NCEP file /home/sbpekayon/CSPP_IAPP_1_0/anc/cache/2017_10_04_277/gdas1.PGrbF00.171004.12z to NetCDF...
2017-10-06 08:43:21.360 (INFO) : New NetCDF file successfully created: /home/sbpekayon/CSPP_IAPP_1_0/iapp_ancillary_2017100412-anl.nc
2017-10-06 08:43:21.360 (INFO) : Transcoded GDAS/GFS NetCDF file: /home/sbpekayon/CSPP_IAPP_1_0/anc/cache/2017_10_04_277/iapp_ancillary_2017100412-anl.nc
2017-10-06 08:43:21.361 (INFO) : Creating NetCDF template file /home/sbpekayon/CSPP_IAPP_1_0/uwretrievals.nc ...
2017-10-06 08:43:21.370 (INFO) : New NetCDF retrieval template file successfully created: /home/sbpekayon/CSPP_IAPP_1_0/uwretrievals.nc
2017-10-06 08:43:21.370 (INFO) : Populating NetCDF template file /home/sbpekayon/CSPP_IAPP_1_0/uwretrievals.nc ...
2017-10-06 08:43:21.461 (INFO) : iapp_main ran in 0.0910999774933 seconds.
2017-10-06 08:43:21.461 (INFO) : IAPP completed successfully, and created: /home/sbpekayon/CSPP_IAPP_1_0/uwretrievals.nc
2017-10-06 08:43:21.461 (INFO) : Moving /home/sbpekayon/CSPP_IAPP_1_0/uwretrievals.nc to /home/sbpekayon/CSPP_IAPP_1_0/noaa19_L2_d20171004_t0856106_e0909242_c20171006084321461947_iapp.nc...
2017-10-06 08:43:21.462 (WARNING) : /home/sbpekayon/CSPP_IAPP_1_0/noaa19_L2_d20171004_t0856106_e0909242_c20171006084321461947_iapp.nc has same size (16376 bytes) as uwretrievals.nc, possible IAPP failure.
2017-10-06 08:43:21.462 (INFO) : Performing routine cleanup of working directory...
2017-10-06 08:43:21.462 (INFO) : Exiting CSPP IAPP ...

After this process in CSPP-IAPP the result is noaa19_L2_d20171004_t0856106_e0909242_c20171006084321461947_iapp.nc
with the capacity of data is 16.8KB but with this level2 data I can't open it cause no data inside this file.
Is somebody ever get this problem? I hope someone in this forum can help me.
Thank you.
kathys
Posts: 487
Joined: Tue Jun 22, 2010 4:51 pm

Re: exiting CSPP-IAPP message

Post by kathys »

Hello,

We advise you to upgrade to the most recent version of CSPP IAPP (version 1.1), which can be found here:

http://cimss.ssec.wisc.edu/cspp/iapp_v1.1.shtml

This version corrected some problems and also includes better logging. If you upgrade, and are still having problems, please let us know.

Kathy
lapan28
Posts: 4
Joined: Mon Mar 07, 2016 3:58 am

Re: exiting CSPP-IAPP message

Post by lapan28 »

Good morning,
I have tried to use the latest version of CSPP_IAPP_v1.1 but still encounter problems like the following :


2017-10-11 01:30:09.984 (INFO) : Starting CSPP IAPP ...
2017-10-11 01:30:09.985 (INFO) :
>>> Processing hirs file /home/sbpekayon/CSPP_IAPP_1_1/hirsl1d_noaa19_20171009_0758_44676.l1d
2017-10-11 01:30:09.987 (INFO) : Pass start time: 2017-10-09 07:59:00.249000
2017-10-11 01:30:09.987 (INFO) : Pass mid time: 2017-10-09 08:05:36.749000
2017-10-11 01:30:09.987 (INFO) : Pass end time: 2017-10-09 08:12:13.849000
2017-10-11 01:30:09.987 (INFO) : Pass length: 793 seconds
2017-10-11 01:30:09.987 (INFO) : Retrieving and granulating ancillary data for /home/sbpekayon/CSPP_IAPP_1_1/hirsl1d_noaa19_20171009_0758_44676.l1d...
2017-10-11 01:30:09.987 (INFO) : Retrieving NCEP files for 2017-10-09 08:05:36.749000 ...
2017-10-11 01:30:24.057 (INFO) : Retrieved GRIB file: /home/sbpekayon/CSPP_IAPP_1_1/anc/cache/2017_10_09_282/gdas1.PGrbF00.171009.06z
2017-10-11 01:30:24.058 (INFO) : Transcoding NCEP file /home/sbpekayon/CSPP_IAPP_1_1/anc/cache/2017_10_09_282/gdas1.PGrbF00.171009.06z to NetCDF...
2017-10-11 01:30:25.067 (INFO) : Transcoded GDAS/GFS NetCDF file: /home/sbpekayon/CSPP_IAPP_1_1/anc/cache/2017_10_09_282/iapp_ancillary_2017100906-anl.nc
2017-10-11 01:30:25.068 (INFO) : Creating NetCDF template file /home/sbpekayon/CSPP_IAPP_1_1/iapp_l2_hirsl1d_noaa19_20171009_0758_44676.l1d_run_0/uwretrievals.nc ...
2017-10-11 01:30:25.078 (INFO) : New NetCDF retrieval template file successfully created: /home/sbpekayon/CSPP_IAPP_1_1/iapp_l2_hirsl1d_noaa19_20171009_0758_44676.l1d_run_0/uwretrievals.nc
2017-10-11 01:30:25.078 (INFO) : Running iapp_main, output to NetCDF template file /home/sbpekayon/CSPP_IAPP_1_1/iapp_l2_hirsl1d_noaa19_20171009_0758_44676.l1d_run_0/uwretrievals.nc ...
2017-10-11 01:30:25.148 (WARNING) : WARNING: AMSUA_Data_Flag fails data quality check in subroutine fill_ret_from_hirs1d
2017-10-11 01:30:25.149 (WARNING) : WARNING: AMSUA_Data_Flag fails data quality check in subroutine fill_ret_from_hirs1d
2017-10-11 01:30:25.149 (WARNING) : WARNING: HIRS_Data_Flag fails data quality check in subroutine fill_ret_from_hirs1d
2017-10-11 01:30:25.149 (WARNING) : WARNING: AMSUA_Data_Flag fails data quality check in subroutine fill_ret_from_hirs1d
2017-10-11 01:30:25.149 (WARNING) : WARNING: HIRS_Data_Flag fails data quality check in subroutine fill_ret_from_hirs1d
2017-10-11 01:30:25.149 (WARNING) : WARNING: AMSUA_Data_Flag fails data quality check in subroutine fill_ret_from_hirs1d
2017-10-11 01:30:25.149 (WARNING) : WARNING: HIRS_Data_Flag fails data quality check in subroutine fill_ret_from_hirs1d
2017-10-11 01:30:25.149 (WARNING) : WARNING: AMSUA_Data_Flag fails data quality check in subroutine fill_ret_from_hirs1d
2017-10-11 01:30:25.149 (WARNING) : Maximum number of "Bad_AMSUA_Data" messages reached, further instances will be counted only
2017-10-11 01:30:25.149 (WARNING) : WARNING: HIRS_Data_Flag fails data quality check in subroutine fill_ret_from_hirs1d
2017-10-11 01:30:25.150 (WARNING) : WARNING: HIRS_Data_Flag fails data quality check in subroutine fill_ret_from_hirs1d
2017-10-11 01:30:25.150 (WARNING) : Maximum number of "Bad_HIRS_Data" messages reached, further instances will be counted only
2017-10-11 01:30:35.206 (WARNING) : Maximum number of attempts (20) of obtaining return code for /home/sbpekayon/CSPP_IAPP_1_1/common/IAPP_VENDOR/iapp/bin/iapp_main reached, setting to zero.
2017-10-11 01:30:35.206 (WARNING) : 75 HIRS fields of regard failed data quality check.
2017-10-11 01:30:35.206 (WARNING) : 406 AMSU-A fields of regard failed data quality check.
2017-10-11 01:30:35.207 (INFO) : iapp_main ran in 10.1288788319 seconds.
2017-10-11 01:30:35.207 (WARNING) : /home/sbpekayon/CSPP_IAPP_1_1/noaa19_L2_d20171009_t0759002_e0812138_c20171011013035207461_iapp.nc has same size (16376 bytes) as uwretrievals.nc, possible IAPP failure.
2017-10-11 01:30:35.207 (WARNING) : No valid retrievals in /home/sbpekayon/CSPP_IAPP_1_1/noaa19_L2_d20171009_t0759002_e0812138_c20171011013035207461_iapp.nc, possible bad l1d file.
2017-10-11 01:30:35.208 (WARNING) : There was a problem running IAPP, moving run files to /home/sbpekayon/CSPP_IAPP_1_1/iapp_l2_hirsl1d_noaa19_20171009_0758_44676.l1d_run_0_crashed
2017-10-11 01:30:35.208 (INFO) : attempted_runs ['hirsl1d_noaa19_20171009_0758_44676.l1d']
2017-10-11 01:30:35.208 (INFO) : successful_runs []
2017-10-11 01:30:35.208 (INFO) : crashed_runs []
2017-10-11 01:30:35.208 (INFO) : problem_runs ['hirsl1d_noaa19_20171009_0758_44676.l1d']
2017-10-11 01:30:35.208 (INFO) : Exiting CSPP IAPP ...

This is the log of data processing, whether the level 1 data I have has a problem? Cause I read in log there isi message "possible bad l1d file" or I mis-installed the software?
Thank you.
geoffc
Posts: 34
Joined: Mon Feb 14, 2011 3:02 pm
Location: Madison, WI
Contact:

Re: exiting CSPP-IAPP message

Post by geoffc »

Hi,
the log file you posted seems to indicate that the l1d file might be bad ("75 HIRS fields of regard failed data quality check...", etc...). Do you get an output file at the end (noaa19_L2_d20171009_t0759002_e0812138_c20171011013035207461_iapp.nc)? If you could post the l1d file to ftp://ftp.ssec.wisc.edu/pub/incoming/, I could take a look for you. But otherwise it looks like your installation is ok.



Geoff
Geoff P. Cureton, PhD
Cooperative Institute for Meteorological Satellite Studies
University of Wisconsin-Madison
1225 W. Dayton St.
Madison WI 53706, USA
Phone: +1 608 890 0706
lapan28
Posts: 4
Joined: Mon Mar 07, 2016 3:58 am

Re: exiting CSPP-IAPP message

Post by lapan28 »

Good morning,
this is the processing result of NOAA level2 data that I can't open.
I use winrar to upload this data.
Could you please check this file?
Thank you.
Attachments
noaa19_L2_d20171009_t2046410_e2059290_c20171009210936019724_iapp.rar
level2 data result
(2.29 KiB) Downloaded 564 times
geoffc
Posts: 34
Joined: Mon Feb 14, 2011 3:02 pm
Location: Madison, WI
Contact:

Re: exiting CSPP-IAPP message

Post by geoffc »

Hi,
can you also post the level-1d file that you used to create the level-2 file?

Thanks,
Geoff
Geoff P. Cureton, PhD
Cooperative Institute for Meteorological Satellite Studies
University of Wisconsin-Madison
1225 W. Dayton St.
Madison WI 53706, USA
Phone: +1 608 890 0706
lapan28
Posts: 4
Joined: Mon Mar 07, 2016 3:58 am

Re: exiting CSPP-IAPP message

Post by lapan28 »

Good morning,
this is level1d data of NOAA19.
I use winrar to upload this data.
Could you please check this file?
Thank you.
Attachments
l1d_noaa19_20171009_2046.rar
l1d data include amsua, hirs, mhs
(968.25 KiB) Downloaded 558 times
geoffc
Posts: 34
Joined: Mon Feb 14, 2011 3:02 pm
Location: Madison, WI
Contact:

Re: exiting CSPP-IAPP message

Post by geoffc »

Hi,
in the level-1d file you posted, there is valid HIRS data, but the AMSU-A and AMSU-B BT datasets are all set to the fill value (-10000).

Geoff
Geoff P. Cureton, PhD
Cooperative Institute for Meteorological Satellite Studies
University of Wisconsin-Madison
1225 W. Dayton St.
Madison WI 53706, USA
Phone: +1 608 890 0706
Post Reply