CrIS SDR under ADL 4.0

[rnc]

Good morning
I am having trouble computing a CrIS SDR under the 4.0 environment. The ProSdrCrisController.exe program returns no error message and deposits no SCRIS files in the output directory.

With no error messages to work from, I am not sure how to debug further.

Thanks in advance for your help, even if all you can do is help me frame the question more clearly!

Rich Czerwinski
MIT Lincoln Laboratory

[kbisanz]

There should be a log file in $ADL_HOME/log/ . It'll be named "ProSdrCrisController.exe_*.log".

There will be a directory containing some log files too, but those just contain a small subset of all the info that's contained in the main log file (the ProSdrCrisController.exe_*.log mentioned previously).

You'll want to search the log file for "error" or "fail" case-insensitive. I generally start at the end of the file and search backwards. The reason of this is that sometimes there are certain errors (which output messages with "error" or "fail" wording) which are later recovered from. These are basically false positives if you're looking for the reason the software failed to produce outputs. If you search backwards, you'll find the last error, which is what caused the failure.

The log files can be daunting to look at if you're new to ADL, so feel free to post a section and hopefully someone can help you out.

Could you describe your ADL environment a bit? Did you install ADL from source code or are you using the University of Wisconsin virtual appliance? Are you trying to process real NPP data or are you trying to use the sample data provided with ADL?

[rnc]

Thank you Kevin for your reply.

I have ADL compiled from source code, and I'm processing real data. This is part of a larger effort on my part to produce a CrIMSS EDR from ATMS and CrIS RDRs. I've gotten the ATMS part done, and even made an EDR by leaving the CrIS data out of the pipeline. I'm hoping if I can learn how to make a CrIS SDR the entire process will work for me.

From the log file, I surmise the error is "failed to get cross granule inputs." I have unpacked the four preceeding and succeeding granules adjacent to the one I am attempting to process, and the files are in the same input directory. Is there documentation about what else might be needed so that this would run to completion?

Thanks again

Rich Czerwinski
MIT Lincoln Laboratory

[kbisanz]

Because you can run ATMS and CrIMSS I assume the environment variable $NPP_GRANULE_ID_BASETIME is set correctly. It should be set to 1698019234000000 for real data.

In the log file there should be a line with the text "URs in inventory". After that it should list all the URs in inventory. Basically there is 1 UR per input or output file. For the URIDs of the previous/next granules please verify that they appear in that list. If they're present it's pretty likely the files we read from disk properly

In the file ProSdrCrisControllerLwFile.xml make sure the <debugLevel> is set to DBG_LOW. Then rerun CrIS and look at the debug lines surrounding the error of "failed to get cross granule inputs". Someplace it should mention a collection short name of what was missing. Section "5.8.7.1.2 Input Query Errors" of user manual part 2 ($ADL_HOME/doc/UG60917-IDP-034A_IDPS_ADL_SW_Users_Manual_Part_2.pdf) may help you out.

What is the N_Granule_Version metadata of all the inputs? You can determine that by grepping the *.asc files for N_Granule_Version. Some inputs don't have this metadata. Normally it is A1, but is sometimes A2. A version of A2 often causes issues because it doesn't mix as well with A1 versions. If you have any A2 data, please post back and I can provide more info.

[rnc]

Hi again Kevin

I've verified the environment and Granule versions you indicate,and checked that all the files I know to look for are among the URs included.

By looking at the debug log I discovered I needed an update to the CrIS-SDR-FILL-PACKET-LUT which I have applied.

My next snag is that the code is stopping (inside ProCmnAlgorithm ?), with the log messages appended below. I appreciate all your assistance.

Rich

2012/10/04 16:03:52.632.047 (17641.47360914493792): DBG_HIGH ProCmnMethodAudit.cpp|207|ProCmnAlgorithm[ProSdrCris]::doPstage(NPP000241057398) [0x2b1312d20010] PRO_FAIL doProcessing() call from file ProCmnAlgorithm.cpp, line 3445
2012/10/04 16:03:52.632.064 (17641.47360914493792): DBG_HIGH ProCmnMethodAudit.cpp|207|ProCmnAlgorithm[ProSdrCris]::doIpoModel(NPP000241057398) [0x2b1312d20010] PRO_FAIL doPstage() call from file ProCmnAlgorithm.cpp, line 4156
2012/10/04 16:03:52.632.079 (17641.47360914493792): DBG_HIGH ProCmnMethodAudit.cpp|207|ProCmnAlgorithm[ProSdrCris]::runAlgorithmImpl() [0x2b1312d20010] PRO_FAIL doIpoModel() call from file ProCmnAlgorithm.cpp, line 4259
2012/10/04 16:03:52.632.149 (17641.47360914493792): DBG_LOW ProCmnTimer.cpp|173|ProSdrCris::runAlgorithm() NPP000241057398 PRO_FAIL TIMER-ELAPSED 0.906581 seconds
2012/10/04 16:03:52.632.174 (17641.47360914493792): DBG_HIGH ProCmnMethodAudit.cpp|207|ProCmnAlgorithmBase[ProSdrCris]::runAlgorithm() [0x2b1312d20010] PRO_FAIL runAlgorithmImpl() call from file ProCmnAlgorithmBase.cpp, line 501
2012/10/04 16:03:52.632.192 (17641.47360914493792): DBG_HIGH ProCmnMethodAudit.cpp|207|ProCmnControllerAlgorithm[ProSdrCrisController]::runAlgorithmChain(NPP000241057398) [0x1c17bf0] PRO_FAIL runAlgorithm() call from file ProCmnControllerAlgorithm.cpp, line 476
2012/10/04 16:03:52.632.214 (17641.47360914493792): DBG_HIGH ProCmnMethodAudit.cpp|207|ProCmnControllerBase[ProSdrCrisController]::runAlgorithmImpl(NPP000241057398) [0x1c17bf0] PRO_FAIL runAlgorithmChain() call from file ProCmnControllerBase.cpp, line 242
2012/10/04 16:03:52.632.232 (17641.47360914493792): DBG_LOW ProCmnTimer.cpp|173|ProSdrCrisController::runAlgorithm() NPP000241057398 PRO_FAIL TIMER-ELAPSED 0.907107 seconds
2012/10/04 16:03:52.632.248 (17641.47360914493792): DBG_HIGH ProCmnMethodAudit.cpp|207|ProCmnAlgorithmBase[ProSdrCrisController]::runAlgorithm(NPP000241057398) [0x1c17bf0] PRO_FAIL runAlgorithmImpl() call from file ProCmnAlgorithmBase.cpp, line 501
2012/10/04 16:03:52.632.280 (17641.47360914493792): DBG_HIGH ProCmnMessage.cpp|213|Status: INF_STATUSTYPE_TASK_FAIL | Description: ProSdrCrisController processing did not complete successfully
2012/10/04 16:03:52.632.300 (17641.47360914493792): DBG_HIGH ProCmnMethodAudit.cpp|207|ProCmnAppl[ProCmnCrimssAppl]::run() [0x7fff3be77e60] PRO_FAIL runAlgorithm() call from file ProCmnAppl.cpp, line 782

[kbisanz]

The lines that you posted appear to be the tail end of the error. Unfortunately, there's not enough there to tell what's going on?

Could you post some lines previous to what you've posted? Or could you compress and attach the entire log file if it's not too big. When you reply to a post, there's a tab below the box you type in to "Upload attachment".

[rnc]

Thanks Kevin

Those lines have the time stamp of the final error. I couldn't see anything there either, but I assumed it was just me.

The gzipped logfile is 365K which I assume is small enough to post here... Giving that a try now. As always, thanks for your help.

Rich Czerwinski

[tjensen]

Hi Rich.

Took a look at your log file. Seems like the problem is that your TLE file is too old compared to the granule you're trying to run:

Code: Select all

2012/10/04 16:03:52.471.825 (17641.47360914493792): DBG_HIGH ProSdrCmnGeo.cpp|6731|ProSdrCmnGeo::sgp4_predEA: Cmn Geo Anomaly ERROR:  The difference between the input time and the TLE epoch time is greater than 30 days.
2012/10/04 16:03:52.471.847 (17641.47360914493792): DBG_HIGH ProCmnMessage.cpp|165|Status: INF_STATUSTYPE_PRO_FREE_FORM | Routine: ProSdrCrisController | Description: ProSdrCmnGeo::sgp4_predEA: Cmn Geo Anomaly ERROR:  The difference between the input time and the TLE epoch time is greater than 30 days.
2012/10/04 16:03:52.471.868 (17641.47360914493792): DBG_HIGH ProSdrCmnGeo.cpp|6738|ProSdrCmnGeo::sgp4_predEA: Cmn Geo Anomaly ERROR:  Verify that the correct TLE data file has been retrieved.
2012/10/04 16:03:52.471.882 (17641.47360914493792): DBG_HIGH ProCmnMessage.cpp|165|Status: INF_STATUSTYPE_PRO_FREE_FORM | Routine: ProSdrCrisController | Description: ProSdrCmnGeo::sgp4_predEA: Cmn Geo Anomaly ERROR:  Verify that the correct TLE data file has been retrieved.
2012/10/04 16:03:52.471.903 (17641.47360914493792): DBG_HIGH ProCmnMethodAudit.cpp|207|ProSdrCmnGeo[ProSdrCmnGeo]::calcNpCrossingWorker(ProSdrCris) [0x87082e0] PRO_FAIL sgp4_predEAWorker() call from file ProSdrCmnGeo.cpp, line 1530
2012/10/04 16:03:52.471.927 (17641.47360914493792): DBG_HIGH ProCmnMethodAudit.cpp|207|ProSdrCris[ProSdrCris]::calcNpCrossingTimes(NPP000241057398) [0x2b1312d20010] PRO_FAIL ProSdrCmnGeo::calcNpCrossing() call from file ProSdrCris.cpp, line 1861
2012/10/04 16:03:52.471.944 (17641.47360914493792): DBG_HIGH ProCmnMethodAudit.cpp|207|ProSdrCris[ProSdrCris]::doProcessing(NPP000241057398) [0x2b1312d20010] PRO_FAIL calcNpCrossingTimes() call from file ProSdrCris.cpp, line 927

Try using a TLE file from within 30 days of your CrIS granule. Let me know if you still run into problems.

[rnc]

Thank you Tim,
I'm making progress, I think, with all the help.

I asked a colleague here to obtain an updated TLE-AUX file from CLASS but the h5 files he got for me do not unpack to the format ADL is expecting. I get an ascii readable .msd file and .msd.CMP_FILENAME file -- not the binary and matching .asc I was expecting.

Is there a script somewhere that can convert these or perhaps a different source for them?

Thanks again, and have a great weekend

Rich

[kbisanz]

Yes, you need to "ingest" the data using runMsd.pl.

See the following posts for more info:
viewtopic.php?f=32&t=266&p=925&hilit=runmsd#p925
viewtopic.php?f=27&t=252&p=926&hilit=runmsd#p926

Note that if you're using the virtual appliance there is apparently an error in envSetup.ksh involving $ADL_MSD_LZ_PATH not being set. The second post discusses that.

Please post back if you run in to issues.