Page 1 of 1
Add new data field in the Vegetation Index outputs
Posted: Sun Jul 27, 2014 10:50 am
by JeffVI
Dear All,
I truly need some timely help on this.
We need to add a new data field in the VI output, and I did the following:
1. Change xml/VIIRS/VIIRS_VI_EDR.xml to add new data field
2. Change xml/VIIRS/VIIRS_VI_FEDR.xml to add new data field
3. Change EDR/VIIRS/land/VegIndex/src/Calculate_VegIndex.cpp to add the calculation of new data field
4. Rebuild ADL by:
1). Execute “make src” in $ADL_HOME/xml/VIIRS
2). Execute “make clean library” in $ADL_HOME/CMN/Utilities/Dictionary/Entries/VIIRS/src.
3). Execute “make clean library program” in $ADL_HOME/EDR/VIIRS/land/VegIndex/src/
And all seems to be rebuilt successfully.
Before I made any changes, the ADL ran the granule successfully. After I make all these changes and rebuilt the ADL, the error is as follows:
Code: Select all
ProEdrViirsVI processing did not complete successfully
ProCmnAlgorithm[ProEdrViirsVI]::scaleDataItems(NPP000868183614) [0x186dfc0] ROOT PRO_FAIL scaling error from file ProCmnAlgorithm.cpp, line 6641
Are we suppose to make changes in the ProCmnAlgorithm.cpp directly? Or did I miss any files I should have made changes but I didn't? Any timely help and hints are sincerely appreciated!
Look forward to any replies soon!
Thank you very much!
Re: Add new data field in the Vegetation Index outputs
Posted: Sun Jul 27, 2014 11:10 am
by JeffVI
If by any chance helpful, the attached are the two XML files I made changes. Thanks a lot!
Re: Add new data field in the Vegetation Index outputs
Posted: Mon Jul 28, 2014 10:20 am
by JeffVI
I further updated ProCmnProductDictionary_CFG.xml in the folder of:
/data/data132/ADL4.2/ADL/cfg/
And rebuild ADL successfully.
But I got segmentation fault when I ran the VI algorithm through ADL.
The log file is too large to attach to this message. I am putting some information, with 'Error' or 'FAIL' from the log file in case anyone can help me with any hints. Thanks a lot for your timely help!
Code: Select all
2014/07/28 15:03:13.747.192 (6778.47271470909280): DBG_LOW /data/data132/ADL4.2/ADL/include/ProCmnDataItem.h|2007|:getMetadataValue() Error: No metadata for 19ProCmnDQNOutputItem
...
2014/07/28 15:03:14.324.698 (6778.47271470909280): DBG_LOW /data/data132/ADL4.2/ADL/include/ProCmnDataItem.h|2013|:getMetadataValue() Error: No metadata item found for key N_Graceful_Degradation
...
2014/07/28 15:03:14.337.961 (6778.47271470909280): DBG_LOW /data/data132/ADL4.2/ADL/include/ProCmnDataItem.h|2013|:getMetadataValue() Error: No metadata item found for key N_Graceful_Degradation
...
2014/07/28 15:03:10.226.235 (6778.47271470909280): DBG_LOW /data/data132/ADL4.2/ADL/include/ProCmnDataItem.h|2007|:getMetadataValue() Error: No metadata for 13ProCmnSDRItem
...
2014/07/28 15:03:13.743.670 (6778.47271470909280): DBG_HIGH ProCmnMethodAudit.cpp|207|ProCmnAlgorithm[ProEdrViirsVI]::getDataToConvert(NPP000868183614) [0x1beefc0] PRO_FAIL no short name found for input: groupname VI_DQTT at index 1 from file ProCmnAlgorithm.cpp, line 2195
...
2014/07/28 15:03:13.744.694 (6778.47271470909280): DBG_HIGH ProCmnMethodAudit.cpp|207|ProCmnAlgorithm[ProEdrViirsVI]::handleInputQuery(NPP000868183614) [0x1beefc0] PRO_FAIL getDataToConvert() call from file ProCmnAlgorithm.cpp, line 7904
Re: Add new data field in the Vegetation Index outputs
Posted: Mon Jul 28, 2014 2:01 pm
by JeffVI
More interestingly, it seems the new ADL run can generate some outputs files except the h5 file:
Code: Select all
smcd_viirs_vi 5730 Jul 28 11:03 53d66632-4f9c1-020a7c07-e4e6b28a.asc
smcd_viirs_vi 88473624 Jul 28 11:03 53d66632-4f9c1-020a7c07-e4e6b28a.VIIRS-VI-EDR
smcd_viirs_vi 0 Jul 28 11:03 VIVIO_npp_d20140723_t2012406_e2014048_b14180_c20140728150325047132_devl_dev.h5
The h5 file was generated but with 0 size of file. Can anyone provide any hints why this happened?
And when I tried to pack the output to h5 manually:
Code: Select all
/data/data132/ADL4.2/ADL/tools/bin/ADL_Packer.exe 53d66632-4f9c1-020a7c07-e4e6b28a ../adl_out/
Segmentation fault
I got the Segmentation Fault.
I believe I might miss some other files that I should make changes. Any kind and timely help is sincerely appreciated!
Re: Add new data field in the Vegetation Index outputs
Posted: Tue Jul 29, 2014 2:10 pm
by stewart999
Hi,
Could you post the .h, .cpp, and any other updated configuration files to the forum? I would like to try and reproduce your problem (and hopefully offer a solution).
Thanks,
Sara
Re: Add new data field in the Vegetation Index outputs
Posted: Tue Jul 29, 2014 5:44 pm
by tsimpson
We're working on this issue but I had some trouble compiling ADL (not related to your code) and will have to start again tomorrow.
There might be an issue related to HDF5 output, since the algorithm produced an empty h5 file and segfaulted, and the ADL_Packer also segfaulted. You can turn off HDF5 output in the algorithm by editing the file $ADL_HOME/cfg/ADL_CFG.xml; look for PRODUCE_HDF_OUTPUTS and change the word 'true' on the next line to 'false'. I know that's not an ideal solution, because there would be no HDF5 output, but it would prevent a segmentation fault.
Re: Add new data field in the Vegetation Index outputs
Posted: Fri Aug 01, 2014 1:15 pm
by kbisanz
This issue was discussed in an outside email chain.
For future reference, the solution was to update the appropriate file in $ADL_HOME/cfg/DataProducts/*.xml with the updated field information.
Re: Add new data field in the Vegetation Index outputs
Posted: Wed Aug 27, 2014 12:15 pm
by JeffVI
Thanks a lot for your kind update! I was able to add new data fields TOC NDVI and QF4 successfully in the VI EDR Product.
However, I am having new issues with changing granule level meta information for the new data fields:
viewtopic.php?f=27&t=452
Any additional help is appreciated!!!