Specfying input RDR files

[houchin]

Has anyone gotten ADL up and running and actually processed some RDR files through it. It is not clear through the documentation how the input RDR files are specified. The best I can tell, unless I want to modify several configuration files, my input data must have a very specific file name, and also all my outputs will also have specific file names.

Regards,
Scott Houchin

[bhenders]

Scott,

In the delivered ADL2.0 DVDs there is a data DVD that contains inputs including RDR files to run the ATMS SDR and VIIRS SDR algorithms. The data should be located under $ADL_HOME/data/input/atmsVerifiedRdrInputs and $ADL_HOME/data/input/viirsVerifiedRdrInputs if properly installed.

The Verified ATMS and VIIRS SDR algorithms can then be executed by doing the following after a successful build (compile) of ADL:

cd $ADL_HOME/bin
./ProSdrAtmsVerifiedRDR.exe ../cfg/atmsLwFile.xml #run ATMS Verified RDR
./ProSdrViirsVerifiedRdr.exe ../cfg/viirsVerifiedRdrLwFile.xml #run VIIRS Verified RDR

If you want to stage new input RDR data you are correct that the staging requires very specific file names, which basically match the names listed in the individual algorithm configuration guides. You can follow the same naming conventions as the existing input data shows. You are also correct that the output will be written out into $ADL_HOME/data/output (output directory is specified in the atmsLwFile.xml for example) with the file names that match the outputnames in the algorithm configuration guide.

To run all of the algorithms that are currently compatible with ADL there is a script that can be executed:

$ADL_HOME/runDemoAlgorithms.ksh

Post another message if this does not answer your questions.

Thanks,

Bryan Henderson

[houchin]

Thanks for the input. I am hoping to find a way to make it much easier to either:

• Run a new RDR file through an existing configuration
• Run an existing RDR file through a tweaked configuration (say only one LUT was changed)

Can file paths be specified through the configuration guides, or is there a way to override a specific config guide parameter through the LwFile.xml file.

It would be great if a map/illustration showing the inputs and outputs of each algorithm, what those files are and where they're located (by default) could be produced.

[bhenders]

Scott,

If you want to run with different RDR data or even a new LUT, I would modify the run-time config file to specify a new input path for where the input data is being staged, I've pasted the ATMS Verified RDR run-time config file below bolding the input path. It uses this path along with the actual algorithm config file input file name to retrieve an input file. If I was going to run different data than what was dropped I'd create a new input directory somewhere else, it could be /tmp/atms/input - copy in the current data files that are in the existing input directory and update any files for which I want to replace with new LUTs or RDRs. You can even create soft links back to the original ADL 2.0 dropped files for files you want to re-use. Once the data has been staged in a new input directory just change the line below that is bolded in the run-time config file to be <inputPath>/tmp/atms/input</inputPath> for example. You could potentially have a "empty input directory" and then modify each individual algorithm config file to have a fully qualified path name, but I think that is a little error prone.

bhenders-> cat atmsLwFile.xml
<!--
Description: Config data used by algorithms to set startup parameters

Tag name Description
_________________________________________________________________________

debugLevel DBG_HIGH, DBG_MED, DBG_LOW, DBG_OFF
debugDestination STDOUT, LOGFILE
debugPath The directory where log files should be written
inputPath The directory where input files are stored
outputPath The directory where output files are written
configGuideName The name of the Algorithm level config guide.
sensorName The name of the sensor for this algorithm.
spacecraftName The name of the spacecraft for which this algorithm
is processing data.
dataStartIET The IET time at which the data starts.
dataEndIET The IET time at which the data ends.
actualScans The actual number of scans for this data.
-->

<!-- NOTE: Commenting out lines does NOT work!!!! -->

<AdlStartupConfig>
<debugLevel>DBG_LOW</debugLevel>
<debugDestination>LOGFILE</debugDestination>
<debugPath>$ADL_HOME/log</debugPath>
<inputPath>$ADL_HOME/data/input/atmsInputs</inputPath>
<outputPath>$ADL_HOME/data/output</outputPath>
<guideListName>$ADL_HOME/cfg/ProSdrAtms_GuideList.cfg</guideListName>
<sensorName>ATMS</sensorName>
<spacecraftName>NPP</spacecraftName>
<dataStartIET>1422171153000000</dataStartIET>
<dataEndIET>1422171213000000</dataEndIET>
<actualScans>12</actualScans>
<previousActualScans>12</previousActualScans>
<nextActualScans>12</nextActualScans>
</AdlStartupConfig>

Regards,

Bryan Henderson

[houchin]

After putting this on the back burner for a while, I've finally gotten around to writing a script to automate the process of creating a staging directory for ADL with links to the inputs and then creating a new XML configuration file pointing to the staged directory.

Do I also need to update the dataStartIET and dataEndIET fields, or are those ignored by ADL 2? Our RDR parsing tool can give me the actual number of scans in the input file (although I could just assume for now that for VIIRS its 48), but our tool cannot yet give me the start of scan times for those scans.

Also, if I have 3 input granules (-1, 0 and +1), with which scans do those times correspond?

Regards,
Scott Houchin