Turning HDF5 Output On and Off

[pe_meade_dpe]

ADL 3.0 is shipped with HDF5 output generation turned on by default, that is, all algorithms which operationally produce their outputs in HDF5 format will also do so in ADL 3.0. The ADL Users Manual Part 2 (in sections 5.5 and 5.7.5) indicates that this is configurable - that these algorithms can produce their output in binary format - but doesn't actually indicate how to do that. I asked the Raytheon ADL team about this, and they replied with the following information.

ADL 3.0 supports turning off HDF via the PRO_DPE_CFG.xml and the PRO_CFG.xml file. There is an entry in those files called PRODUCE_HDF_OUTPUTS:

<config>
<name>PRODUCE_HDF_OUTPUTS</name>
<configValue>true</configValue>
</config>

This will need to be set to false. This turns off the production of HDF outputs for the algorithm. However, it's an all or nothing setting. If you only want to have VIIRS SDR Controller not produce HDF outputs you'll have to reset this flag to be true when running another algorithm such as VIIRS CM IP where you do want to produce the HDF outputs during the algorithm run.

Change PRODUCE_HDF_OUTPUTS in both the PRO_CFG.xml and PRO_DPE_CFG.xml files. Technically which one is used comes down to which one is included in the algorithm guide list file. An example guide list file is PRO_SdrViirs_GuideList.cfg, which is specified in the VIIRS SDR execution configuration file. This guide list file, lists all of the other algorithm configuration files needed by the executable, it happens to list only the PRO_CFG.xml file. However other guide list files are including the PRO_DPE_CFG.xml file. These should be consolidated for ADL and we'll add this to our list of items for ADL 3.1 to work on. It looks like about half include one and the other include the other file, some may include both.

Setting them in both files insures that HDF will definitely be turned off, however if you want to configure just one algorithm you could copy PRO_CFG.xml or PRO_DPE_CFG.xml to a new name and update the appropriate guide list file (PRO_SdrViirs_GuideList.cfg as an example) to include the new named file. You would only affect the one algorithm.

[pe_meade_dpe]

I should also point out that, if you DO turn off HDF5 output, you should be able to pack the binary output and the associated ASCII metadata files into HDF5 files after-the-fact using the HDF Packer utility included with the ADL 3.0 release. See the ADL Users Manual Part 2 for details.

On the input side, ADL 3.0 uses ONLY binary input files, NOT HDF5 files. However, the HDF Unpacker utility (also included with ADL 3.0) can be used on the front end of your process to unpack HDF5 inputs to binary files and their associated ASCII metadata files. Again, see the ADL Users Manual Part 2 for guidance on the Unpacker utility.