I just tried to unpack some NPP data and what appears to be every single file had the following type of warnings associated with it:
Processing /home/ADL_3.1/ADL/data/input/CLASS_20111203/hdf_files/SVM16_npp_d20111203_t0030445_e0032087_b00505_c20111206225119317882_noaa_ops.h5...
WARNING 2011-12-08 17:02:19.288414 tid-47696263952048 pid-20015 (DmCoreItemDateTimeRange.cpp line 473) invalid start date: 1958-1-1 - defaulting to IET epoch (1958-01-01 00:00:00.000000Z)
WARNING 2011-12-08 17:02:19.288438 tid-47696263952048 pid-20015 (DmCoreItemDateTimeRange.cpp line 503) invalid end date: 1958-1-1 - defaulting to IET epoch (1958-01-01 00:00:00.000000Z)
WARNING 2011-12-08 17:02:19.457760 tid-47696263952048 pid-20015 (DmCoreItemDateTimeRange.cpp line 473) invalid start date: 1958-1-1 - defaulting to IET epoch (1958-01-01 00:00:00.000000Z)
WARNING 2011-12-08 17:02:19.457790 tid-47696263952048 pid-20015 (DmCoreItemDateTimeRange.cpp line 503) invalid end date: 1958-1-1 - defaulting to IET epoch (1958-01-01 00:00:00.000000Z)
Blob files + metadata were still created, so it doesn't seem to affect that part. Will this affect processing?
HDF Unpacker
Re: HDF Unpacker
That warning will not affect processing. We've seen the error too.
That warning happens due to this sequence of events:
1. The ADL Unpacker is unpacking an HDF that doesn't have a particular time value in the metadata (probably because the CDFCB says it shouldn't). It appears to be RangeDateTime.
2. The Unpacker thinks/knows that this item should have RangeDateTime metadata, so it adds it, but defaults it to the IET epoch time (Jan 1, 1958) because it doesn't have a better value.
3. Some code in ADL's DMS implementation makes a system call to mktime with the 1958 time.
4. The mktime system call (which normally returns the number of seconds elapsed since Jan 1, 1970, the Unix epoch time) returns a negative number as an error because Jan 1, 1958 is before the Jan 1, 1970 Unix epoch time.
5. The ADL DMS code handles the "error" and defaults the time to Jan 1, 1958.
6. A warning message is printed saying that an error happened.
That warning happens due to this sequence of events:
1. The ADL Unpacker is unpacking an HDF that doesn't have a particular time value in the metadata (probably because the CDFCB says it shouldn't). It appears to be RangeDateTime.
2. The Unpacker thinks/knows that this item should have RangeDateTime metadata, so it adds it, but defaults it to the IET epoch time (Jan 1, 1958) because it doesn't have a better value.
3. Some code in ADL's DMS implementation makes a system call to mktime with the 1958 time.
4. The mktime system call (which normally returns the number of seconds elapsed since Jan 1, 1970, the Unix epoch time) returns a negative number as an error because Jan 1, 1958 is before the Jan 1, 1970 Unix epoch time.
5. The ADL DMS code handles the "error" and defaults the time to Jan 1, 1958.
6. A warning message is printed saying that an error happened.
Kevin Bisanz
Raytheon Company
Raytheon Company
Re: HDF Unpacker
I am using NPP data from GTP. The original file of cloud mask is IICMO_npp_d20111219_t2105433_e2107075_b00745_c20111220032955067527_noaa_ops.h5
Unpack is fine but the LST code gives following error.
2011/12/20 18:00:19.528.300 (940.47301446215264): DBG_MED ProCmnMethodAudit.cpp|225|ProCmnURSorter[ProCmnURSorter]::sortURs() [0x7fffa5b47ca0] PRO_SUCCESS
2011/12/20 18:00:19.528.314 (940.47301446215264): DBG_MED ProCmnMethodAudit.cpp|225|ProCmnInputItem[15ProCmnInputItem]::sortURs(VIIRS-CM-IP) [0x20f77760] PRO_SUCCESS
2011/12/20 18:00:19.528.330 (940.47301446215264): DBG_HIGH ProCmnMethodAudit.cpp|206|ProCmnInputItem[15ProCmnInputItem]::acquireURs(VIIRS-CM-IP) [0x20f77760] UNKNOWN_ERROR checkAcquireResults() call from file ProCmnInputItem.cpp, line 280
2011/12/20 18:00:19.528.350 (940.47301446215264): DBG_HIGH ProCmnMethodAudit.cpp|206|ProCmnDataItem[15ProCmnInputItem]::getData(VIIRS-CM-IP) [0x20f77760] UNKNOWN_ERROR acquireURs() call from file ProCmnDataItem.cpp, line 941
2011/12/20 18:00:19.528.363 (940.47301446215264): DBG_HIGH ProCmnMethodAudit.cpp|206|ProCmnInputItem[15ProCmnInputItem]::getData(VIIRS-CM-IP) [0x20f77760] UNKNOWN_ERROR ProCmnDataItem::getData() failed from file ProCmnInputItem.cpp, line 183
2011/12/20 18:00:19.528.375 (940.47301446215264): DBG_HIGH ProCmnMethodAudit.cpp|206|ProCmnDataItem[15ProCmnInputItem]::get(VIIRS-CM-IP) [0x20f77760] UNKNOWN_ERROR getData() call from file ProCmnDataItem.cpp, line 552
2011/12/20 18:00:19.528.397 (940.47301446215264): DBG_LOW ProCmnDataItemModel.cpp|574|WARNING: ::getMetadata(0) returning null metadata pointer to ProCmnDataItem.cpp on line 1976
2011/12/20 18:00:19.528.409 (940.47301446215264): DBG_LOW /home/ADL_3.1/ADL/include/ProCmnDataItem.h|1993|:getMetadataValue() Error: No metadata for 15ProCmnInputItem
2011/12/20 18:00:19.528.420 (940.47301446215264): DBG_MED ProCmnAlgorithm.cpp|7634|Error retrieving PRIMARY data for VIIRS-CM-IP
2011/12/20 18:00:19.528.429 (940.47301446215264): DBG_MED ProCmnAlgorithm.cpp|7639|Checking for CONVERSION data (if any) for VIIRS-CM-IP
2011/12/20 18:00:19.528.444 (940.47301446215264): DBG_MED ProCmnMethodAudit.cpp|123|ProCmnAlgorithm[ProEdrViirsLst]::getDataToConvert(NPP000050007418) [0x20f6abc0] entered
2011/12/20 18:00:19.528.582 (940.47301446215264): DBG_LOW ProCmnShortNameGenerator.cpp|207|getInputShortName(): CASE ConversionDataShortName
2011/12/20 18:00:19.528.809 (940.47301446215264): DBG_LOW ProCmnConfigGuideReader.cpp|137|Unable to read value for ProEdrViirsLst.Inputs.CloudMask_IP.OfficialShortNameConversionProduct_1: findname Unable to find Name: ProEdrViirsLst.Inputs.CloudMask_IP.OfficialShortNameConversionProduct_1
2011/12/20 18:00:19.528.835 (940.47301446215264): DBG_HIGH ProCmnMethodAudit.cpp|206|ProCmnAlgorithm[ProEdrViirsLst]::getDataToConvert(NPP000050007418) [0x20f6abc0] PRO_FAIL no short name found for input: groupname CloudMask_IP at index 1 from file ProCmnAlgorithm.cpp, line 2175
2011/12/20 18:00:19.528.861 (940.47301446215264): DBG_HIGH ProCmnAlgorithm.cpp|7659|Error retrieving data for VIIRS-CM-IP
2011/12/20 18:00:19.528.872 (940.47301446215264): DBG_MED ProCmnAlgorithm.cpp|7664|Retrieving alternate data for VIIRS-CM-IP
2011/12/20 18:00:19.528.884 (940.47301446215264): DBG_MED ProCmnMethodAudit.cpp|123|ProCmnAlgorithm[ProEdrViirsLst]::getAlternateData(NPP000050007418) [0x20f6abc0] entered
2011/12/20 18:00:19.529.236 (940.47301446215264): DBG_LOW ProCmnConfigGuideReader.cpp|137|Unable to read value for ProEdrViirsLst.Inputs.CloudMask_IP.OfficialShortName_2: findname Unable to find Name: ProEdrViirsLst.Inputs.CloudMask_IP.OfficialShortName_2
2011/12/20 18:00:19.529.259 (940.47301446215264): DBG_MED ProCmnMethodAudit.cpp|173|ProCmnProductDictionary[ProCmnProductDictionary]::getProductDataType() [0x1fceba30] entered
2011/12/20 18:00:19.529.276 (940.47301446215264): DBG_MED ProCmnMethodAudit.cpp|225|ProCmnProductDictionary[ProCmnProductDictionary]::getProductDataType() [0x1fceba30] PRO_SUCCESS
2011/12/20 18:00:19.529.288 (940.47301446215264): DBG_MED ProCmnMethodAudit.cpp|173|ProCmnProductDictionary[ProCmnProductDictionary]::getProductDataType() [0x1fceba30] entered
2011/12/20 18:00:19.529.302 (940.47301446215264): DBG_MED ProCmnMethodAudit.cpp|225|ProCmnProductDictionary[ProCmnProductDictionary]::getProductDataType() [0x1fceba30] PRO_SUCCESS
2011/12/20 18:00:19.529.524 (940.47301446215264): DBG_LOW ProCmnConfigGuideReader.cpp|137|Unable to read value for ProEdrViirsLst.Inputs.CloudMask_IP.Mode: findname Unable to find Name: ProEdrViirsLst.Inputs.CloudMask_IP.Mode
2011/12/20 18:00:19.529.547 (940.47301446215264): DBG_MED ProCmnMethodAudit.cpp|225|ProCmnAlgorithm[ProEdrViirsLst]::getAlternateData(NPP000050007418) [0x20f6abc0] PRO_SUCCESS
2011/12/20 18:00:19.529.574 (940.47301446215264): DBG_LOW ProCmnAlgorithmBase.cpp|362|INFSTATUS infStatusData: ProEdrViirsLst INF_STATUSTYPE_EDR_PREREQ_ABSENT VIIRS-CM-IP ( Required input not available for Shortname: VIIRS-CM-IP , Granule ID: NPP000050007418 Failure(s): (VIIRS-CM-IP, NPP000050007418) ) severity level: INF_SEVERITYLEVEL_NORMAL
2011/12/20 18:00:19.529.587 (940.47301446215264): DBG_HIGH ProCmnMessage.cpp|191|Status: INF_STATUSTYPE_EDR_PREREQ_ABSENT | ShortName: VIIRS-CM-IP | Description: Required input not available for Shortname: VIIRS-CM-IP , Granule ID: NPP000050007418 Failure(s): (VIIRS-CM-IP, NPP000050007418)
2011/12/20 18:00:19.529.600 (940.47301446215264): DBG_HIGH ProCmnMethodAudit.cpp|206|ProCmnAlgorithm[ProEdrViirsLst]::handleInputQuery(NPP000050007418) [0x20f6abc0] PRO_FAIL Required input not available for Shortname: VIIRS-CM-IP , Granule ID: NPP000050007418 Failure(s): (VIIRS-CM-IP, NPP000050007418) from file ProCmnAlgorithm.cpp, line 7699
2011/12/20 18:00:19.529.624 (940.47301446215264): DBG_HIGH ProCmnMethodAudit.cpp|206|ProCmnAlgorithm[ProEdrViirsLst]::getDataItems(NPP000050007418) [0x20f6abc0] PRO_FAIL handleInputQuery() call from file ProCmnAlgorithm.cpp, line 971
So I checked the settings for cloud mask.
This is the asc for cloud mask input.
("N_Collection_Short_Name" STRING EQ "VIIRS-CM-IP")
and this is how the xml set the cloud mask input.
<group name="CloudMask_IP">
<config>
<name>OfficialShortName_1</name>
<configValue>VIIRS-CM-IP</configValue>
The short name matches each other. I do not know why adl can not read this file. Would you please help? Thanks.
Unpack is fine but the LST code gives following error.
2011/12/20 18:00:19.528.300 (940.47301446215264): DBG_MED ProCmnMethodAudit.cpp|225|ProCmnURSorter[ProCmnURSorter]::sortURs() [0x7fffa5b47ca0] PRO_SUCCESS
2011/12/20 18:00:19.528.314 (940.47301446215264): DBG_MED ProCmnMethodAudit.cpp|225|ProCmnInputItem[15ProCmnInputItem]::sortURs(VIIRS-CM-IP) [0x20f77760] PRO_SUCCESS
2011/12/20 18:00:19.528.330 (940.47301446215264): DBG_HIGH ProCmnMethodAudit.cpp|206|ProCmnInputItem[15ProCmnInputItem]::acquireURs(VIIRS-CM-IP) [0x20f77760] UNKNOWN_ERROR checkAcquireResults() call from file ProCmnInputItem.cpp, line 280
2011/12/20 18:00:19.528.350 (940.47301446215264): DBG_HIGH ProCmnMethodAudit.cpp|206|ProCmnDataItem[15ProCmnInputItem]::getData(VIIRS-CM-IP) [0x20f77760] UNKNOWN_ERROR acquireURs() call from file ProCmnDataItem.cpp, line 941
2011/12/20 18:00:19.528.363 (940.47301446215264): DBG_HIGH ProCmnMethodAudit.cpp|206|ProCmnInputItem[15ProCmnInputItem]::getData(VIIRS-CM-IP) [0x20f77760] UNKNOWN_ERROR ProCmnDataItem::getData() failed from file ProCmnInputItem.cpp, line 183
2011/12/20 18:00:19.528.375 (940.47301446215264): DBG_HIGH ProCmnMethodAudit.cpp|206|ProCmnDataItem[15ProCmnInputItem]::get(VIIRS-CM-IP) [0x20f77760] UNKNOWN_ERROR getData() call from file ProCmnDataItem.cpp, line 552
2011/12/20 18:00:19.528.397 (940.47301446215264): DBG_LOW ProCmnDataItemModel.cpp|574|WARNING: ::getMetadata(0) returning null metadata pointer to ProCmnDataItem.cpp on line 1976
2011/12/20 18:00:19.528.409 (940.47301446215264): DBG_LOW /home/ADL_3.1/ADL/include/ProCmnDataItem.h|1993|:getMetadataValue() Error: No metadata for 15ProCmnInputItem
2011/12/20 18:00:19.528.420 (940.47301446215264): DBG_MED ProCmnAlgorithm.cpp|7634|Error retrieving PRIMARY data for VIIRS-CM-IP
2011/12/20 18:00:19.528.429 (940.47301446215264): DBG_MED ProCmnAlgorithm.cpp|7639|Checking for CONVERSION data (if any) for VIIRS-CM-IP
2011/12/20 18:00:19.528.444 (940.47301446215264): DBG_MED ProCmnMethodAudit.cpp|123|ProCmnAlgorithm[ProEdrViirsLst]::getDataToConvert(NPP000050007418) [0x20f6abc0] entered
2011/12/20 18:00:19.528.582 (940.47301446215264): DBG_LOW ProCmnShortNameGenerator.cpp|207|getInputShortName(): CASE ConversionDataShortName
2011/12/20 18:00:19.528.809 (940.47301446215264): DBG_LOW ProCmnConfigGuideReader.cpp|137|Unable to read value for ProEdrViirsLst.Inputs.CloudMask_IP.OfficialShortNameConversionProduct_1: findname Unable to find Name: ProEdrViirsLst.Inputs.CloudMask_IP.OfficialShortNameConversionProduct_1
2011/12/20 18:00:19.528.835 (940.47301446215264): DBG_HIGH ProCmnMethodAudit.cpp|206|ProCmnAlgorithm[ProEdrViirsLst]::getDataToConvert(NPP000050007418) [0x20f6abc0] PRO_FAIL no short name found for input: groupname CloudMask_IP at index 1 from file ProCmnAlgorithm.cpp, line 2175
2011/12/20 18:00:19.528.861 (940.47301446215264): DBG_HIGH ProCmnAlgorithm.cpp|7659|Error retrieving data for VIIRS-CM-IP
2011/12/20 18:00:19.528.872 (940.47301446215264): DBG_MED ProCmnAlgorithm.cpp|7664|Retrieving alternate data for VIIRS-CM-IP
2011/12/20 18:00:19.528.884 (940.47301446215264): DBG_MED ProCmnMethodAudit.cpp|123|ProCmnAlgorithm[ProEdrViirsLst]::getAlternateData(NPP000050007418) [0x20f6abc0] entered
2011/12/20 18:00:19.529.236 (940.47301446215264): DBG_LOW ProCmnConfigGuideReader.cpp|137|Unable to read value for ProEdrViirsLst.Inputs.CloudMask_IP.OfficialShortName_2: findname Unable to find Name: ProEdrViirsLst.Inputs.CloudMask_IP.OfficialShortName_2
2011/12/20 18:00:19.529.259 (940.47301446215264): DBG_MED ProCmnMethodAudit.cpp|173|ProCmnProductDictionary[ProCmnProductDictionary]::getProductDataType() [0x1fceba30] entered
2011/12/20 18:00:19.529.276 (940.47301446215264): DBG_MED ProCmnMethodAudit.cpp|225|ProCmnProductDictionary[ProCmnProductDictionary]::getProductDataType() [0x1fceba30] PRO_SUCCESS
2011/12/20 18:00:19.529.288 (940.47301446215264): DBG_MED ProCmnMethodAudit.cpp|173|ProCmnProductDictionary[ProCmnProductDictionary]::getProductDataType() [0x1fceba30] entered
2011/12/20 18:00:19.529.302 (940.47301446215264): DBG_MED ProCmnMethodAudit.cpp|225|ProCmnProductDictionary[ProCmnProductDictionary]::getProductDataType() [0x1fceba30] PRO_SUCCESS
2011/12/20 18:00:19.529.524 (940.47301446215264): DBG_LOW ProCmnConfigGuideReader.cpp|137|Unable to read value for ProEdrViirsLst.Inputs.CloudMask_IP.Mode: findname Unable to find Name: ProEdrViirsLst.Inputs.CloudMask_IP.Mode
2011/12/20 18:00:19.529.547 (940.47301446215264): DBG_MED ProCmnMethodAudit.cpp|225|ProCmnAlgorithm[ProEdrViirsLst]::getAlternateData(NPP000050007418) [0x20f6abc0] PRO_SUCCESS
2011/12/20 18:00:19.529.574 (940.47301446215264): DBG_LOW ProCmnAlgorithmBase.cpp|362|INFSTATUS infStatusData: ProEdrViirsLst INF_STATUSTYPE_EDR_PREREQ_ABSENT VIIRS-CM-IP ( Required input not available for Shortname: VIIRS-CM-IP , Granule ID: NPP000050007418 Failure(s): (VIIRS-CM-IP, NPP000050007418) ) severity level: INF_SEVERITYLEVEL_NORMAL
2011/12/20 18:00:19.529.587 (940.47301446215264): DBG_HIGH ProCmnMessage.cpp|191|Status: INF_STATUSTYPE_EDR_PREREQ_ABSENT | ShortName: VIIRS-CM-IP | Description: Required input not available for Shortname: VIIRS-CM-IP , Granule ID: NPP000050007418 Failure(s): (VIIRS-CM-IP, NPP000050007418)
2011/12/20 18:00:19.529.600 (940.47301446215264): DBG_HIGH ProCmnMethodAudit.cpp|206|ProCmnAlgorithm[ProEdrViirsLst]::handleInputQuery(NPP000050007418) [0x20f6abc0] PRO_FAIL Required input not available for Shortname: VIIRS-CM-IP , Granule ID: NPP000050007418 Failure(s): (VIIRS-CM-IP, NPP000050007418) from file ProCmnAlgorithm.cpp, line 7699
2011/12/20 18:00:19.529.624 (940.47301446215264): DBG_HIGH ProCmnMethodAudit.cpp|206|ProCmnAlgorithm[ProEdrViirsLst]::getDataItems(NPP000050007418) [0x20f6abc0] PRO_FAIL handleInputQuery() call from file ProCmnAlgorithm.cpp, line 971
So I checked the settings for cloud mask.
This is the asc for cloud mask input.
("N_Collection_Short_Name" STRING EQ "VIIRS-CM-IP")
and this is how the xml set the cloud mask input.
<group name="CloudMask_IP">
<config>
<name>OfficialShortName_1</name>
<configValue>VIIRS-CM-IP</configValue>
The short name matches each other. I do not know why adl can not read this file. Would you please help? Thanks.
Re: HDF Unpacker
The short name looks good. What about the granule ID? I would recommend looking at section 5.8.7.1.2 of User Manual Part 2 in $ADL_HOME/doc/ for some information on how to debug the problem of a missing input.
You will probably find a query that says "DMS query finish: query returned 0 item(s) containing these items" and assuming that's the one for the cloud mask you want to find, that'd be the query you want to see why it was excluding the URID you think it should be finding.
Since it sounds like you're using real NPP data, you may also need to update the $NPP_GRANULE_ID_BASETIME in $ADL_HOME/build/envSetup.*sh. See viewtopic.php?f=31&t=159 for more information.
Please post back if you continue to have issues.
You will probably find a query that says "DMS query finish: query returned 0 item(s) containing these items" and assuming that's the one for the cloud mask you want to find, that'd be the query you want to see why it was excluding the URID you think it should be finding.
Since it sounds like you're using real NPP data, you may also need to update the $NPP_GRANULE_ID_BASETIME in $ADL_HOME/build/envSetup.*sh. See viewtopic.php?f=31&t=159 for more information.
Please post back if you continue to have issues.
Kevin Bisanz
Raytheon Company
Raytheon Company
Re: HDF Unpacker
Right, I am using real NPP data and I changed the value for NPP_GRANULE_ID_BASETIME.
I followed your suggestion on debugging problematic inputs and found that all UIDS are there in the inventory.
Here is the message.
DMS initialization complete for directory: /home/ADL_3.1/ADL/data/input/withMetadata/ProEdrViirsLstControllerInputs/20120109/NPP000068154535
URs in inventory:
URID = 4f0c9659-7d382-007f0100-9356b7ab Collection Short Name = VIIRS-M12-SDR
URID = 4f0c965a-201c9-007f0100-36bc78f3 Collection Short Name = VIIRS-M13-SDR
URID = 4f0c965b-e9927-007f0100-4f984cf2 Collection Short Name = VIIRS-M16-SDR
URID = 4f0c965a-7ca5d-007f0100-6bc4e8a7 Collection Short Name = VIIRS-M15-SDR
URID = 4d911427-2af91-0a4f180b-57e295e3 Collection Short Name = VIIRS-LST-Coef-LUT
URID = 4dc4242d-1f6f2-9b9dea23-e963b8e2 Collection Short Name = VIIRS-LST-EDR-AC-Int
URID = 4f0c965c-9ba11-007f0100-07e5f76d Collection Short Name = VIIRS-ST-EDR
URID = 4f0c9658-374d9-007f0100-d5a86f21 Collection Short Name = VIIRS-MOD-GEO-TC
URID = 4f0c9659-249e9-007f0100-ef409b42 Collection Short Name = VIIRS-CM-IP
2012/01/10 19:51:39.169.401 (8183.47500526003808): DBG_LOW DmCoreURMetadataFilter.cpp|144| tid-47500526003808 DMS query start: Searching for items matching the following metadata:
Metadata Collection METADATALIST (size 4)
item 0, 0x16a34e20 id=0 urId=0 parent=0 parentType=0 ItemIndex_=0 type=STRING name=N_Collection_Short_Name, comparison=EQ, value=VIIRS-CM-IP
item 1, 0x16a358f8 id=0 urId=0 parent=0 parentType=0 ItemIndex_=0 type=STRING name=N_Granule_ID, comparison=EQ, value=NPP000068154535
item 2, 0x16a357b0 id=0 urId=0 parent=0 parentType=0 ItemIndex_=0 type=STRING name=Platform_Short_Name, comparison=EQ, value=NPP
item 3, 0x16a33d20 id=0 urId=0 parent=0 parentType=0 ItemIndex_=0 type=INTEGER name=DatasetLock, comparison=EQ,value=0
2012/01/10 19:51:39.169.436 (8183.47500526003808): DBG_LOW DmCoreURMetadataFilter.cpp|216| tid-47500526003808 Excluding URID 4f0c9659-7d382-007f0100-9356b7ab because the following metadata does not match: 0x15e21b10 id=0 urId=0 parent=0 parentType=0 ItemIndex_=0 type=STRING name=N_Collection_Short_Name, comparison=EQ, value=VIIRS-M12-SDR
2012/01/10 19:51:39.169.522 (8183.47500526003808): DBG_LOW DmCoreURMetadataFilter.cpp|216| tid-47500526003808 Excluding URID 4f0c965a-201c9-007f0100-36bc78f3 because the following metadata does not match: 0x169dadd0 id=0 urId=0 parent=0 parentType=0 ItemIndex_=0 type=STRING name=N_Collection_Short_Name, comparison=EQ, value=VIIRS-M13-SDR
2012/01/10 19:51:39.169.584 (8183.47500526003808): DBG_LOW DmCoreURMetadataFilter.cpp|216| tid-47500526003808 Excluding URID 4f0c965b-e9927-007f0100-4f984cf2 because the following metadata does not match: 0x169ec578 id=0 urId=0 parent=0 parentType=0 ItemIndex_=0 type=STRING name=N_Collection_Short_Name, comparison=EQ, value=VIIRS-M16-SDR
2012/01/10 19:51:39.169.646 (8183.47500526003808): DBG_LOW DmCoreURMetadataFilter.cpp|216| tid-47500526003808 Excluding URID 4f0c965a-7ca5d-007f0100-6bc4e8a7 because the following metadata does not match: 0x169fc7f0 id=0 urId=0 parent=0 parentType=0 ItemIndex_=0 type=STRING name=N_Collection_Short_Name, comparison=EQ, value=VIIRS-M15-SDR
2012/01/10 19:51:39.169.714 (8183.47500526003808): DBG_LOW DmCoreURMetadataFilter.cpp|216| tid-47500526003808 Excluding URID 4d911427-2af91-0a4f180b-57e295e3 because the following metadata does not match: 0x16a0cde8 id=0 urId=0 parent=0 parentType=0 ItemIndex_=0 type=STRING name=N_Collection_Short_Name, comparison=EQ, value=VIIRS-LST-Coef-LUT
2012/01/10 19:51:39.169.746 (8183.47500526003808): DBG_LOW DmCoreURMetadataFilter.cpp|216| tid-47500526003808 Excluding URID 4dc4242d-1f6f2-9b9dea23-e963b8e2 because the following metadata does not match: 0x16a0eb60 id=0 urId=0 parent=0 parentType=0 ItemIndex_=0 type=STRING name=N_Collection_Short_Name, comparison=EQ, value=VIIRS-LST-EDR-AC-Int
2012/01/10 19:51:39.169.772 (8183.47500526003808): DBG_LOW DmCoreURMetadataFilter.cpp|216| tid-47500526003808 Excluding URID 4f0c965c-9ba11-007f0100-07e5f76d because the following metadata does not match: 0x16a12e90 id=0 urId=0 parent=0 parentType=0 ItemIndex_=0 type=STRING name=N_Collection_Short_Name, comparison=EQ, value=VIIRS-ST-EDR
2012/01/10 19:51:39.169.827 (8183.47500526003808): DBG_LOW DmCoreURMetadataFilter.cpp|216| tid-47500526003808 Excluding URID 4f0c9658-374d9-007f0100-d5a86f21 because the following metadata does not match: 0x16a1c9c8 id=0 urId=0 parent=0 parentType=0 ItemIndex_=0 type=STRING name=N_Collection_Short_Name, comparison=EQ, value=VIIRS-MOD-GEO-TC
2012/01/10 19:51:39.169.881 (8183.47500526003808): DBG_MED DmCoreURMetadataFilter.cpp|275| tid-47500526003808 DMS query finish: query returned 1 item(s) containing these items:
URID = 4f0c9659-249e9-007f0100-ef409b42 Collection Short Name = VIIRS-CM-IP
-------------------------------------------------------------------------------------------------------
However, with above message I can not locate the problem with cloud mask file because the four query items: GID, short-name, datasetLock and Platform_Short_Name are the same in asc file.
I attach the beginning part of asc file of cloud mask for your reference.
4f0c964e-2727d-007f0100-eb9cdcfb.asc
("4f0c964e-2727d-007f0100-eb9cdcfb" UR "2012-01-10 19:49:34.000000"
("FILELIST" COLLECTION
("/home/ADL_3.1/ADL/data/input/withMetadata/ProEdrViirsLstControllerInputs/20120109/NPP000068152828/4f0c964e-2727d-007f0100-eb9cdcfb.VIIRS-CM-IP" FILE
("METADATALIST" COLLECTION)
)
)
("METADATALIST" COLLECTION
("Mission_Name" STRING EQ "NPP")
("N_Dataset_Source" STRING EQ "noaa")
("Platform_Short_Name" STRING EQ "NPP")
("Instrument_Short_Name" STRING EQ "VIIRS")
("N_Anc_Type_Tasked" STRING EQ "Official")
("N_Collection_Short_Name" STRING EQ "VIIRS-CM-IP")
("N_Dataset_Type_Tag" STRING EQ "IP")
("InstrumentFlightSW" GROUP EQ
("METADATALIST" COLLECTION
("N_Instrument_Flight_SW_Version" INTEGER EQ 20)
.....
This problem is not only for one file but for 6 among all 27 granules in 20120109.
Thanks.
I followed your suggestion on debugging problematic inputs and found that all UIDS are there in the inventory.
Here is the message.
DMS initialization complete for directory: /home/ADL_3.1/ADL/data/input/withMetadata/ProEdrViirsLstControllerInputs/20120109/NPP000068154535
URs in inventory:
URID = 4f0c9659-7d382-007f0100-9356b7ab Collection Short Name = VIIRS-M12-SDR
URID = 4f0c965a-201c9-007f0100-36bc78f3 Collection Short Name = VIIRS-M13-SDR
URID = 4f0c965b-e9927-007f0100-4f984cf2 Collection Short Name = VIIRS-M16-SDR
URID = 4f0c965a-7ca5d-007f0100-6bc4e8a7 Collection Short Name = VIIRS-M15-SDR
URID = 4d911427-2af91-0a4f180b-57e295e3 Collection Short Name = VIIRS-LST-Coef-LUT
URID = 4dc4242d-1f6f2-9b9dea23-e963b8e2 Collection Short Name = VIIRS-LST-EDR-AC-Int
URID = 4f0c965c-9ba11-007f0100-07e5f76d Collection Short Name = VIIRS-ST-EDR
URID = 4f0c9658-374d9-007f0100-d5a86f21 Collection Short Name = VIIRS-MOD-GEO-TC
URID = 4f0c9659-249e9-007f0100-ef409b42 Collection Short Name = VIIRS-CM-IP
2012/01/10 19:51:39.169.401 (8183.47500526003808): DBG_LOW DmCoreURMetadataFilter.cpp|144| tid-47500526003808 DMS query start: Searching for items matching the following metadata:
Metadata Collection METADATALIST (size 4)
item 0, 0x16a34e20 id=0 urId=0 parent=0 parentType=0 ItemIndex_=0 type=STRING name=N_Collection_Short_Name, comparison=EQ, value=VIIRS-CM-IP
item 1, 0x16a358f8 id=0 urId=0 parent=0 parentType=0 ItemIndex_=0 type=STRING name=N_Granule_ID, comparison=EQ, value=NPP000068154535
item 2, 0x16a357b0 id=0 urId=0 parent=0 parentType=0 ItemIndex_=0 type=STRING name=Platform_Short_Name, comparison=EQ, value=NPP
item 3, 0x16a33d20 id=0 urId=0 parent=0 parentType=0 ItemIndex_=0 type=INTEGER name=DatasetLock, comparison=EQ,value=0
2012/01/10 19:51:39.169.436 (8183.47500526003808): DBG_LOW DmCoreURMetadataFilter.cpp|216| tid-47500526003808 Excluding URID 4f0c9659-7d382-007f0100-9356b7ab because the following metadata does not match: 0x15e21b10 id=0 urId=0 parent=0 parentType=0 ItemIndex_=0 type=STRING name=N_Collection_Short_Name, comparison=EQ, value=VIIRS-M12-SDR
2012/01/10 19:51:39.169.522 (8183.47500526003808): DBG_LOW DmCoreURMetadataFilter.cpp|216| tid-47500526003808 Excluding URID 4f0c965a-201c9-007f0100-36bc78f3 because the following metadata does not match: 0x169dadd0 id=0 urId=0 parent=0 parentType=0 ItemIndex_=0 type=STRING name=N_Collection_Short_Name, comparison=EQ, value=VIIRS-M13-SDR
2012/01/10 19:51:39.169.584 (8183.47500526003808): DBG_LOW DmCoreURMetadataFilter.cpp|216| tid-47500526003808 Excluding URID 4f0c965b-e9927-007f0100-4f984cf2 because the following metadata does not match: 0x169ec578 id=0 urId=0 parent=0 parentType=0 ItemIndex_=0 type=STRING name=N_Collection_Short_Name, comparison=EQ, value=VIIRS-M16-SDR
2012/01/10 19:51:39.169.646 (8183.47500526003808): DBG_LOW DmCoreURMetadataFilter.cpp|216| tid-47500526003808 Excluding URID 4f0c965a-7ca5d-007f0100-6bc4e8a7 because the following metadata does not match: 0x169fc7f0 id=0 urId=0 parent=0 parentType=0 ItemIndex_=0 type=STRING name=N_Collection_Short_Name, comparison=EQ, value=VIIRS-M15-SDR
2012/01/10 19:51:39.169.714 (8183.47500526003808): DBG_LOW DmCoreURMetadataFilter.cpp|216| tid-47500526003808 Excluding URID 4d911427-2af91-0a4f180b-57e295e3 because the following metadata does not match: 0x16a0cde8 id=0 urId=0 parent=0 parentType=0 ItemIndex_=0 type=STRING name=N_Collection_Short_Name, comparison=EQ, value=VIIRS-LST-Coef-LUT
2012/01/10 19:51:39.169.746 (8183.47500526003808): DBG_LOW DmCoreURMetadataFilter.cpp|216| tid-47500526003808 Excluding URID 4dc4242d-1f6f2-9b9dea23-e963b8e2 because the following metadata does not match: 0x16a0eb60 id=0 urId=0 parent=0 parentType=0 ItemIndex_=0 type=STRING name=N_Collection_Short_Name, comparison=EQ, value=VIIRS-LST-EDR-AC-Int
2012/01/10 19:51:39.169.772 (8183.47500526003808): DBG_LOW DmCoreURMetadataFilter.cpp|216| tid-47500526003808 Excluding URID 4f0c965c-9ba11-007f0100-07e5f76d because the following metadata does not match: 0x16a12e90 id=0 urId=0 parent=0 parentType=0 ItemIndex_=0 type=STRING name=N_Collection_Short_Name, comparison=EQ, value=VIIRS-ST-EDR
2012/01/10 19:51:39.169.827 (8183.47500526003808): DBG_LOW DmCoreURMetadataFilter.cpp|216| tid-47500526003808 Excluding URID 4f0c9658-374d9-007f0100-d5a86f21 because the following metadata does not match: 0x16a1c9c8 id=0 urId=0 parent=0 parentType=0 ItemIndex_=0 type=STRING name=N_Collection_Short_Name, comparison=EQ, value=VIIRS-MOD-GEO-TC
2012/01/10 19:51:39.169.881 (8183.47500526003808): DBG_MED DmCoreURMetadataFilter.cpp|275| tid-47500526003808 DMS query finish: query returned 1 item(s) containing these items:
URID = 4f0c9659-249e9-007f0100-ef409b42 Collection Short Name = VIIRS-CM-IP
-------------------------------------------------------------------------------------------------------
However, with above message I can not locate the problem with cloud mask file because the four query items: GID, short-name, datasetLock and Platform_Short_Name are the same in asc file.
I attach the beginning part of asc file of cloud mask for your reference.
4f0c964e-2727d-007f0100-eb9cdcfb.asc
("4f0c964e-2727d-007f0100-eb9cdcfb" UR "2012-01-10 19:49:34.000000"
("FILELIST" COLLECTION
("/home/ADL_3.1/ADL/data/input/withMetadata/ProEdrViirsLstControllerInputs/20120109/NPP000068152828/4f0c964e-2727d-007f0100-eb9cdcfb.VIIRS-CM-IP" FILE
("METADATALIST" COLLECTION)
)
)
("METADATALIST" COLLECTION
("Mission_Name" STRING EQ "NPP")
("N_Dataset_Source" STRING EQ "noaa")
("Platform_Short_Name" STRING EQ "NPP")
("Instrument_Short_Name" STRING EQ "VIIRS")
("N_Anc_Type_Tasked" STRING EQ "Official")
("N_Collection_Short_Name" STRING EQ "VIIRS-CM-IP")
("N_Dataset_Type_Tag" STRING EQ "IP")
("InstrumentFlightSW" GROUP EQ
("METADATALIST" COLLECTION
("N_Instrument_Flight_SW_Version" INTEGER EQ 20)
.....
This problem is not only for one file but for 6 among all 27 granules in 20120109.
Thanks.
Re: HDF Unpacker
It appears like you're using multiple input directories, one for each granule ID. So you should see multiple "DMS initialization complete" messages. One for each directory. The list of inputs will build on each other. So each "DMS initialization complete" will get longer than the next.
I would guess you're looking at the wrong set of "DMS query start" messages? Looks like the one you pasted me is for NPP000068154535, but the granule ID for .asc file you pasted is in the NPP000068152828 directory.
Please post back if you still have issues.
I would guess you're looking at the wrong set of "DMS query start" messages? Looks like the one you pasted me is for NPP000068154535, but the granule ID for .asc file you pasted is in the NPP000068152828 directory.
Please post back if you still have issues.
Kevin Bisanz
Raytheon Company
Raytheon Company
Re: HDF Unpacker
you are right. I am putting input files together in a folder with name as granule id. I do see multiple "DMS initialization complete" in log file but they are located in different log files. It is quite difficult to find the right one containing a certain granule. Is this the logic for log creation in ADL? Create each log for a input directory? That is my question.
Sorry, the asc file I put is for another file. Here it is the complete asc file.
("4f0c9659-249e9-007f0100-ef409b42" UR "2012-01-10 19:49:45.000000"
("FILELIST" COLLECTION
("/home/ADL_3.1/ADL/data/input/withMetadata/ProEdrViirsLstControllerInputs/20120109/NPP000068154535/4f0c9659-249e9-007f0100-ef409b42.VIIRS-CM-IP" FILE
("METADATALIST" COLLECTION)
)
)
("METADATALIST" COLLECTION
("Mission_Name" STRING EQ "NPP")
("N_Dataset_Source" STRING EQ "noaa")
("Platform_Short_Name" STRING EQ "NPP")
("Instrument_Short_Name" STRING EQ "VIIRS")
("N_Anc_Type_Tasked" STRING EQ "Official")
("N_Collection_Short_Name" STRING EQ "VIIRS-CM-IP")
("N_Dataset_Type_Tag" STRING EQ "IP")
("InstrumentFlightSW" GROUP EQ
("METADATALIST" COLLECTION
("N_Instrument_Flight_SW_Version" INTEGER EQ 20)
)
)
("N_Processing_Domain" STRING EQ "ops")
("Operational_Mode" STRING EQ "NPP Unknown, VIIRS Operational")
("Ascending/Descending_Indicator" UINTEGER EQ 0)
("East_Bounding_Coordinate" FLOAT EQ 3268660298)
("GRingLatitude" GROUP EQ
("METADATALIST" COLLECTION
("Value" FLOAT EQ 1110795302)
("Value" FLOAT EQ 1110118944)
("Value" FLOAT EQ 1109474909)
("Value" FLOAT EQ 1109146940)
("Value" FLOAT EQ 1108110397)
("Value" FLOAT EQ 1108707770)
("Value" FLOAT EQ 1109329183)
("Value" FLOAT EQ 1110445785)
)
)
("GRingLongitude" GROUP EQ
("METADATALIST" COLLECTION
("Value" FLOAT EQ 3268681138)
("Value" FLOAT EQ 3268672225)
("Value" FLOAT EQ 3268661740)
("Value" FLOAT EQ 3270986695)
("Value" FLOAT EQ 3272360395)
("Value" FLOAT EQ 3272446968)
("Value" FLOAT EQ 3272544092)
("Value" FLOAT EQ 3271210828)
)
)
("AlgorithmVersion" STRING EQ "1.O.000.001")
("AncFilename" GROUP EQ
("METADATALIST" COLLECTION
("N_Anc_Filename" STRING EQ "Terrain-Eco-ANC-Tile_20030125000000Z_ee00000000000000Z_NA_NA_N0585_1.O.0.0")
("N_Anc_Filename" STRING EQ "Terrain-Eco-ANC-Tile_20030125000000Z_ee00000000000000Z_NA_NA_N0586_1.O.0.0")
("N_Anc_Filename" STRING EQ "Terrain-Eco-ANC-Tile_20030125000000Z_ee00000000000000Z_NA_NA_N0616_1.O.0.0")
("N_Anc_Filename" STRING EQ "Terrain-Eco-ANC-Tile_20030125000000Z_ee00000000000000Z_NA_NA_N0617_1.O.0.0")
("N_Anc_Filename" STRING EQ "Terrain-Eco-ANC-Tile_20030125000000Z_ee00000000000000Z_NA_NA_N0618_1.O.0.0")
("N_Anc_Filename" STRING EQ "Terrain-Eco-ANC-Tile_20030125000000Z_ee00000000000000Z_NA_NA_N0619_1.O.0.0")
("N_Anc_Filename" STRING EQ "Terrain-Eco-ANC-Tile_20030125000000Z_ee00000000000000Z_NA_NA_N0648_1.O.0.0")
("N_Anc_Filename" STRING EQ "Terrain-Eco-ANC-Tile_20030125000000Z_ee00000000000000Z_NA_NA_N0649_1.O.0.0")
("N_Anc_Filename" STRING EQ "Terrain-Eco-ANC-Tile_20030125000000Z_ee00000000000000Z_NA_NA_N0650_1.O.0.0")
("N_Anc_Filename" STRING EQ "Terrain-Eco-ANC-Tile_20030125000000Z_ee00000000000000Z_NA_NA_N0651_1.O.0.0")
("N_Anc_Filename" STRING EQ "Terrain-Eco-ANC-Tile_20030125000000Z_ee00000000000000Z_NA_NA_N0652_1.O.0.0")
("N_Anc_Filename" STRING EQ "Terrain-Eco-ANC-Tile_20030125000000Z_ee00000000000000Z_NA_NA_N0653_1.O.0.0")
("N_Anc_Filename" STRING EQ "Terrain-Eco-ANC-Tile_20030125000000Z_ee00000000000000Z_NA_NA_N0680_1.O.0.0")
("N_Anc_Filename" STRING EQ "Terrain-Eco-ANC-Tile_20030125000000Z_ee00000000000000Z_NA_NA_N0681_1.O.0.0")
("N_Anc_Filename" STRING EQ "Terrain-Eco-ANC-Tile_20030125000000Z_ee00000000000000Z_NA_NA_N0682_1.O.0.0")
("N_Anc_Filename" STRING EQ "Terrain-Eco-ANC-Tile_20030125000000Z_ee00000000000000Z_NA_NA_N0683_1.O.0.0")
("N_Anc_Filename" STRING EQ "Terrain-Eco-ANC-Tile_20030125000000Z_ee00000000000000Z_NA_NA_N0684_1.O.0.0")
("N_Anc_Filename" STRING EQ "Terrain-Eco-ANC-Tile_20030125000000Z_ee00000000000000Z_NA_NA_N0685_1.O.0.0")
("N_Anc_Filename" STRING EQ "Terrain-Eco-ANC-Tile_20030125000000Z_ee00000000000000Z_NA_NA_N0714_1.O.0.0")
("N_Anc_Filename" STRING EQ "Terrain-Eco-ANC-Tile_20030125000000Z_ee00000000000000Z_NA_NA_N0715_1.O.0.0")
("N_Anc_Filename" STRING EQ "Terrain-Eco-ANC-Tile_20030125000000Z_ee00000000000000Z_NA_NA_N0716_1.O.0.0")
("N_Anc_Filename" STRING EQ "Terrain-Eco-ANC-Tile_20030125000000Z_ee00000000000000Z_NA_NA_N0717_1.O.0.0")
("N_Anc_Filename" STRING EQ "Terrain-Eco-ANC-Tile_20030125000000Z_ee00000000000000Z_NA_NA_N0747_1.O.0.0")
("N_Anc_Filename" STRING EQ "Terrain-Eco-ANC-Tile_20030125000000Z_ee00000000000000Z_NA_NA_N0748_1.O.0.0")
("N_Anc_Filename" STRING EQ "off_NCEP-GFS-09HR-ANC_GFS_NCEP_009f_20120109_201201091200Z_20120109154123Z_ee20120110000000Z_np")
("N_Anc_Filename" STRING EQ "off_Planet-Eph-ANC_Static_JPL_000f_20000101_200001010000Z_20000101000000Z_ee00000000000000Z_np")
("N_Anc_Filename" STRING EQ "off_USNO-PolarWander-UT1-ANC_Ser7_USNO_000f_20120105_201201051200Z_20120105120515Z_ee20120112120000Z_np")
)
)
("AuxFilename" GROUP EQ
("METADATALIST" COLLECTION
("N_Aux_Filename" STRING EQ "VIIRS-AF-EDR-AC_npp_20020101010000Z_20020101010000Z_ee00000000000000Z_1_devl_dev_noaa_ops")
("N_Aux_Filename" STRING EQ "VIIRS-AF-EDR-DQTT_npp_20020101010000Z_20020101010000Z_ee00000000000000Z_BASELINE-MON-1_devl_dev_noaa_ops")
("N_Aux_Filename" STRING EQ "VIIRS-CM-IP-AC_npp_20020101010000Z_20020101010000Z_ee00000000000000Z_1_devl_dev_noaa_ops")
)
)
("BeginningOrbitNumber" UINTEGER EQ 1043)
("N_Graceful_Degradation" STRING EQ "No")
("N_Granule_ID" STRING EQ "NPP000068154535")
("N_Granule_Version" STRING EQ "A2")
("InputProd" GROUP EQ
("METADATALIST" COLLECTION
("N_Input_Prod" STRING EQ "4f0b6480-6bf60-0a180213-5929c1b3")
("N_Input_Prod" STRING EQ "4f0b6480-8cac8-0a180213-592bcd1b")
("N_Input_Prod" STRING EQ "4f0b6480-b5848-0a180213-592e5a9b")
("N_Input_Prod" STRING EQ "4f0b6595-8e58a-0a180213-592be8f2")
("N_Input_Prod" STRING EQ "4f0b6595-911b8-0a180213-592c1520")
("N_Input_Prod" STRING EQ "4f0b6595-9618e-0a180213-592c64f6")
("N_Input_Prod" STRING EQ "4f0b6595-a07ae-0a180213-592d0b16")
("N_Input_Prod" STRING EQ "4f0b6595-a3a89-0a180213-592d3df1")
("N_Input_Prod" STRING EQ "4f0b6595-ea744-0a180213-5931aaac")
("N_Input_Prod" STRING EQ "4f0b6595-ecb88-0a180213-5931cef0")
("N_Input_Prod" STRING EQ "4f0b6595-f1d1f-0a180213-59322087")
("N_Input_Prod" STRING EQ "4f0b6596-035e6-0a180213-5923394f")
("N_Input_Prod" STRING EQ "4f0b6596-05b8f-0a180213-59235ef8")
("N_Input_Prod" STRING EQ "4f0b6596-08198-0a180213-59238501")
("N_Input_Prod" STRING EQ "4f0b6596-0a2c0-0a180213-5923a629")
("N_Input_Prod" STRING EQ "4f0b6596-0c13f-0a180213-5923c4a8")
("N_Input_Prod" STRING EQ "4f0b6596-0d195-0a180213-5923d4fe")
("N_Input_Prod" STRING EQ "4f0b6596-1c822-0a180213-5924cb8b")
("N_Input_Prod" STRING EQ "4f0b6596-365ba-0a180213-59266923")
("N_Input_Prod" STRING EQ "4f0b660d-81907-0a180213-592b1007")
("N_Input_Prod" STRING EQ "4f0b66a7-a871c-0a180213-592d8b96")
("N_Input_Prod" STRING EQ "4f0b66a7-ab851-0a180213-592dbccb")
("N_Input_Prod" STRING EQ "4f0b66a7-b2e09-0a180213-592e3283")
("N_Input_Prod" STRING EQ "4f0b676a-f3d32-0a180212-5932426e")
("N_Input_Prod" STRING EQ "4f0b676b-02e79-0a180212-592333b6")
("N_Input_Prod" STRING EQ "4f0b676b-0758f-0a180212-59237acc")
("N_Input_Prod" STRING EQ "4f0b67cb-b7d78-0a180213-592e8316")
("N_Input_Prod" STRING EQ "4f0b67cb-ba2fa-0a180213-592ea898")
("N_Input_Prod" STRING EQ "4f0b67cb-be991-0a180213-592eef2f")
("N_Input_Prod" STRING EQ "4f0b67cb-c0b0d-0a180213-592f10ab")
("N_Input_Prod" STRING EQ "4f0b67cb-c2dc9-0a180213-592f3367")
("N_Input_Prod" STRING EQ "4f0b67cb-d8a7a-0a180213-59309018")
("N_Input_Prod" STRING EQ "4f0b67cb-dacd3-0a180213-5930b271")
("N_Input_Prod" STRING EQ "4f0b67cb-df0ab-0a180213-5930f649")
("N_Input_Prod" STRING EQ "4f0b67cb-e31bf-0a180213-5931375d")
("N_Input_Prod" STRING EQ "4f0b67cb-e522c-0a180213-593157ca")
("N_Input_Prod" STRING EQ "4f0b67cb-e72d3-0a180213-59317871")
("N_Input_Prod" STRING EQ "4f0b67cb-e94ca-0a180213-59319a68")
("N_Input_Prod" STRING EQ "4f0b67cb-eb46f-0a180213-5931ba0d")
("N_Input_Prod" STRING EQ "4f0b67cb-ec557-0a180213-5931caf5")
("N_Input_Prod" STRING EQ "4f0b67cb-ee7da-0a180213-5931ed78")
("N_Input_Prod" STRING EQ "4f0b67cb-f09b9-0a180213-59320f57")
("N_Input_Prod" STRING EQ "4f0b683e-bbda0-0a180213-592eb6d1")
("N_Input_Prod" STRING EQ "4f0b688e-91b2e-0a180212-592c218e")
("N_Input_Prod" STRING EQ "4f0b688e-93bcf-0a180212-592c422f")
("N_Input_Prod" STRING EQ "4f0b688e-97a46-0a180212-592c80a6")
("N_Input_Prod" STRING EQ "4f0b688e-a26f2-0a180212-592d2d52")
("N_Input_Prod" STRING EQ "4f0b688e-a4720-0a180212-592d4d80")
("N_Input_Prod" STRING EQ "4f0b688e-ac443-0a180212-592dcaa3")
("N_Input_Prod" STRING EQ "4f0b688e-ae182-0a180212-592de7e2")
("N_Input_Prod" STRING EQ "4f0b688e-b22a3-0a180212-592e2903")
("N_Input_Prod" STRING EQ "4f0b688e-b6446-0a180212-592e6aa6")
("N_Input_Prod" STRING EQ "4f0b688e-b869a-0a180212-592e8cfa")
("N_Input_Prod" STRING EQ "4f0b688e-ba9f7-0a180212-592eb057")
("N_Input_Prod" STRING EQ "4f0b688e-bc7f4-0a180212-592ece54")
("N_Input_Prod" STRING EQ "4f0b688e-bedda-0a180212-592ef43a")
("N_Input_Prod" STRING EQ "4f0b688e-bffb6-0a180212-592f0616")
("N_Input_Prod" STRING EQ "4f0b688e-c23bc-0a180212-592f2a1c")
("N_Input_Prod" STRING EQ "4f0b688e-c45f9-0a180212-592f4c59")
("N_Input_Prod" STRING EQ "4f0b6900-4b56d-0a180212-5927af5f")
)
)
("N_LEOA_Flag" STRING EQ "On")
("N_Nadir_Latitude_Max" FLOAT EQ 1110433438)
("N_Nadir_Latitude_Min" FLOAT EQ 1109161175)
("N_Nadir_Longitude_Max" FLOAT EQ 3270987612)
("N_Nadir_Longitude_Min" FLOAT EQ 3271208075)
("N_Number_of_Scans" INTEGER EQ 48)
("QualityNames" GROUP EQ
("METADATALIST" COLLECTION
("N_Quality_Names" STRING EQ "N/A")
)
)
("QualityValues" GROUP EQ
("METADATALIST" COLLECTION
("N_Quality_Values" INTEGER EQ -993)
)
)
("N_Satellite/Local_Azimuth_Angle_Max" FLOAT EQ 1127477871)
("N_Satellite/Local_Azimuth_Angle_Min" FLOAT EQ 3274964533)
("N_Satellite/Local_Zenith_Angle_Max" FLOAT EQ 1116488650)
("N_Satellite/Local_Zenith_Angle_Min" FLOAT EQ 1008869124)
("N_Solar_Azimuth_Angle_Max" FLOAT EQ 1127481344)
("N_Solar_Azimuth_Angle_Min" FLOAT EQ 3274964992)
("N_Solar_Zenith_Angle_Max" FLOAT EQ 1116845742)
("N_Solar_Zenith_Angle_Min" FLOAT EQ 1113939509)
("N_Spacecraft_Maneuver" STRING EQ "Unknown")
("North_Bounding_Coordinate" FLOAT EQ 1110798421)
("South_Bounding_Coordinate" FLOAT EQ 1108110397)
("West_Bounding_Coordinate" FLOAT EQ 3272544092)
("N_Day_Night_Flag" STRING EQ "Day")
("RangeDateTime" DATETIMERANGE EQ "1958-01-01 00:00:00.000000" "1958-01-01 00:00:00.000000")
("ObservedDateTime" DATETIMERANGE EQ "2012-01-09 21:10:53.710583" "2012-01-09 21:12:17.886236")
("Shell_Status" STRING EQ "No")
("ByteOrder" STRING EQ "LE")
("DatasetLock" INTEGER EQ 0)
("CreationDateTime" DATETIME EQ "2012-01-10 19:49:45.150533")
("N_Software_Version" STRING EQ "Unknown")
("UpdateDateTime" DATETIME EQ "2012-01-10 19:49:45.160700")
)
)
I compared the four items in the query list and found that they are same(in bold).
2012/01/10 19:51:39.169.401 (8183.47500526003808): DBG_LOW DmCoreURMetadataFilter.cpp|144| tid-47500526003808 DMS query start: Searching for items matching the following metadata:
Metadata Collection METADATALIST (size 4)
item 0, 0x16a34e20 id=0 urId=0 parent=0 parentType=0 ItemIndex_=0 type=STRING name=N_Collection_Short_Name, comparison=EQ, value=VIIRS-CM-IP
item 1, 0x16a358f8 id=0 urId=0 parent=0 parentType=0 ItemIndex_=0 type=STRING name=N_Granule_ID, comparison=EQ, value=NPP000068154535
item 2, 0x16a357b0 id=0 urId=0 parent=0 parentType=0 ItemIndex_=0 type=STRING name=Platform_Short_Name, comparison=EQ, value=NPP
item 3, 0x16a33d20 id=0 urId=0 parent=0 parentType=0 ItemIndex_=0 type=INTEGER name=DatasetLock, comparison=EQ,value=0
This is error information from log file.
2012/01/10 19:51:39.170.515 (8183.47500526003808): DBG_MED ProCmnMethodAudit.cpp|225|ProCmnURSorter[ProCmnURSorter]::sortURs() [0x7fff766ae890] PRO_SUCCESS
2012/01/10 19:51:39.170.528 (8183.47500526003808): DBG_MED ProCmnMethodAudit.cpp|225|ProCmnInputItem[15ProCmnInputItem]::sortURs(VIIRS-CM-IP) [0x169c85c0] PRO_SUCCESS
2012/01/10 19:51:39.170.545 (8183.47500526003808): DBG_HIGH ProCmnMethodAudit.cpp|206|ProCmnInputItem[15ProCmnInputItem]::acquireURs(VIIRS-CM-IP) [0x169c85c0] UNKNOWN_ERROR checkAcquireResults() call from file ProCmnInputItem.cpp, line 280
2012/01/10 19:51:39.170.564 (8183.47500526003808): DBG_HIGH ProCmnMethodAudit.cpp|206|ProCmnDataItem[15ProCmnInputItem]::getData(VIIRS-CM-IP) [0x169c85c0] UNKNOWN_ERROR acquireURs() call from file ProCmnDataItem.cpp, line 941
2012/01/10 19:51:39.170.577 (8183.47500526003808): DBG_HIGH ProCmnMethodAudit.cpp|206|ProCmnInputItem[15ProCmnInputItem]::getData(VIIRS-CM-IP) [0x169c85c0] UNKNOWN_ERROR ProCmnDataItem::getData() failed from file ProCmnInputItem.cpp, line 183
2012/01/10 19:51:39.170.589 (8183.47500526003808): DBG_HIGH ProCmnMethodAudit.cpp|206|ProCmnDataItem[15ProCmnInputItem]::get(VIIRS-CM-IP) [0x169c85c0] UNKNOWN_ERROR getData() call from file ProCmnDataItem.cpp, line 552
2012/01/10 19:51:39.170.612 (8183.47500526003808): DBG_LOW ProCmnDataItemModel.cpp|574|WARNING: ::getMetadata(0) returning null metadata pointer to ProCmnDataItem.cpp on line 1976
2012/01/10 19:51:39.170.625 (8183.47500526003808): DBG_LOW /home/ADL_3.1/ADL/include/ProCmnDataItem.h|1993|:getMetadataValue() Error: No metadata for 15ProCmnInputItem
2012/01/10 19:51:39.170.635 (8183.47500526003808): DBG_MED ProCmnAlgorithm.cpp|7634|Error retrieving PRIMARY data for VIIRS-CM-IP
2012/01/10 19:51:39.170.645 (8183.47500526003808): DBG_MED ProCmnAlgorithm.cpp|7639|Checking for CONVERSION data (if any) for VIIRS-CM-IP
2012/01/10 19:51:39.170.660 (8183.47500526003808): DBG_MED ProCmnMethodAudit.cpp|123|ProCmnAlgorithm[ProEdrViirsLst]::getDataToConvert(NPP000068154535) [0x169bba20] entered
2012/01/10 19:51:39.170.812 (8183.47500526003808): DBG_LOW ProCmnShortNameGenerator.cpp|207|getInputShortName(): CASE ConversionDataShortName
2012/01/10 19:51:39.171.042 (8183.47500526003808): DBG_LOW ProCmnConfigGuideReader.cpp|137|Unable to read value for ProEdrViirsLst.Inputs.CloudMask_IP.OfficialShortNameConversionProduct_1: findname Unable to find Name: ProEdrViirsLst.Inputs.CloudMask_IP.OfficialShortNameConversionProduct_1
2012/01/10 19:51:39.171.074 (8183.47500526003808): DBG_HIGH ProCmnMethodAudit.cpp|206|ProCmnAlgorithm[ProEdrViirsLst]::getDataToConvert(NPP000068154535) [0x169bba20] PRO_FAIL no short name found for input: groupname CloudMask_IP at index 1 from file ProCmnAlgorithm.cpp, line 2175
2012/01/10 19:51:39.171.087 (8183.47500526003808): DBG_HIGH ProCmnAlgorithm.cpp|7659|Error retrieving data for VIIRS-CM-IP
2012/01/10 19:51:39.171.097 (8183.47500526003808): DBG_MED ProCmnAlgorithm.cpp|7664|Retrieving alternate data for VIIRS-CM-IP
2012/01/10 19:51:39.171.108 (8183.47500526003808): DBG_MED ProCmnMethodAudit.cpp|123|ProCmnAlgorithm[ProEdrViirsLst]::getAlternateData(NPP000068154535) [0x169bba20] entered
2012/01/10 19:51:39.171.446 (8183.47500526003808): DBG_LOW ProCmnConfigGuideReader.cpp|137|Unable to read value for ProEdrViirsLst.Inputs.CloudMask_IP.OfficialShortName_2: findname Unable to find Name: ProEdrViirsLst.Inputs.CloudMask_IP.OfficialShortName_2
2012/01/10 19:51:39.171.468 (8183.47500526003808): DBG_MED ProCmnMethodAudit.cpp|173|ProCmnProductDictionary[ProCmnProductDictionary]::getProductDataType() [0x1586e5e0] entered
2012/01/10 19:51:39.171.486 (8183.47500526003808): DBG_MED ProCmnMethodAudit.cpp|225|ProCmnProductDictionary[ProCmnProductDictionary]::getProductDataType() [0x1586e5e0] PRO_SUCCESS
2012/01/10 19:51:39.171.499 (8183.47500526003808): DBG_MED ProCmnMethodAudit.cpp|173|ProCmnProductDictionary[ProCmnProductDictionary]::getProductDataType() [0x1586e5e0] entered
2012/01/10 19:51:39.171.513 (8183.47500526003808): DBG_MED ProCmnMethodAudit.cpp|225|ProCmnProductDictionary[ProCmnProductDictionary]::getProductDataType() [0x1586e5e0] PRO_SUCCESS
2012/01/10 19:51:39.171.744 (8183.47500526003808): DBG_LOW ProCmnConfigGuideReader.cpp|137|Unable to read value for ProEdrViirsLst.Inputs.CloudMask_IP.Mode: findname Unable to find Name: ProEdrViirsLst.Inputs.CloudMask_IP.Mode
2012/01/10 19:51:39.171.768 (8183.47500526003808): DBG_MED ProCmnMethodAudit.cpp|225|ProCmnAlgorithm[ProEdrViirsLst]::getAlternateData(NPP000068154535) [0x169bba20] PRO_SUCCESS
2012/01/10 19:51:39.171.796 (8183.47500526003808): DBG_LOW ProCmnAlgorithmBase.cpp|362|INFSTATUS infStatusData: ProEdrViirsLst INF_STATUSTYPE_EDR_PREREQ_ABSENT VIIRS-CM-IP ( Required input not available for Shortname: VIIRS-CM-IP , Granule ID: NPP000068154535 Failure(s): (VIIRS-CM-IP, NPP000068154535) ) severity level: INF_SEVERITYLEVEL_NORMAL
2012/01/10 19:51:39.171.810 (8183.47500526003808): DBG_HIGH ProCmnMessage.cpp|191|Status: INF_STATUSTYPE_EDR_PREREQ_ABSENT | ShortName: VIIRS-CM-IP | Description: Required input not available for Shortname: VIIRS-CM-IP , Granule ID: NPP000068154535 Failure(s): (VIIRS-CM-IP, NPP000068154535)
2012/01/10 19:51:39.171.823 (8183.47500526003808): DBG_HIGH ProCmnMethodAudit.cpp|206|ProCmnAlgorithm[ProEdrViirsLst]::handleInputQuery(NPP000068154535) [0x169bba20] PRO_FAIL Required input not available for Shortname: VIIRS-CM-IP , Granule ID: NPP000068154535 Failure(s): (VIIRS-CM-IP, NPP000068154535) from file ProCmnAlgorithm.cpp, line 7699
2012/01/10 19:51:39.171.851 (8183.47500526003808): DBG_HIGH ProCmnMethodAudit.cpp|206|ProCmnAlgorithm[ProEdrViirsLst]::getDataItems(NPP000068154535) [0x169bba20] PRO_FAIL handleInputQuery() call from file ProCmnAlgorithm.cpp, line 971
I found these error happen with those granules with multiple geolocation file from GTP. These files have different creation time but the data looks the same. So I use one of them as input. From the log, no complain is found for geolocation file.
This problem happens quite often, for 20120110, there are 5 granules with this problem. Donot know if there is any issue with data itself.
Thanks,
Sorry, the asc file I put is for another file. Here it is the complete asc file.
("4f0c9659-249e9-007f0100-ef409b42" UR "2012-01-10 19:49:45.000000"
("FILELIST" COLLECTION
("/home/ADL_3.1/ADL/data/input/withMetadata/ProEdrViirsLstControllerInputs/20120109/NPP000068154535/4f0c9659-249e9-007f0100-ef409b42.VIIRS-CM-IP" FILE
("METADATALIST" COLLECTION)
)
)
("METADATALIST" COLLECTION
("Mission_Name" STRING EQ "NPP")
("N_Dataset_Source" STRING EQ "noaa")
("Platform_Short_Name" STRING EQ "NPP")
("Instrument_Short_Name" STRING EQ "VIIRS")
("N_Anc_Type_Tasked" STRING EQ "Official")
("N_Collection_Short_Name" STRING EQ "VIIRS-CM-IP")
("N_Dataset_Type_Tag" STRING EQ "IP")
("InstrumentFlightSW" GROUP EQ
("METADATALIST" COLLECTION
("N_Instrument_Flight_SW_Version" INTEGER EQ 20)
)
)
("N_Processing_Domain" STRING EQ "ops")
("Operational_Mode" STRING EQ "NPP Unknown, VIIRS Operational")
("Ascending/Descending_Indicator" UINTEGER EQ 0)
("East_Bounding_Coordinate" FLOAT EQ 3268660298)
("GRingLatitude" GROUP EQ
("METADATALIST" COLLECTION
("Value" FLOAT EQ 1110795302)
("Value" FLOAT EQ 1110118944)
("Value" FLOAT EQ 1109474909)
("Value" FLOAT EQ 1109146940)
("Value" FLOAT EQ 1108110397)
("Value" FLOAT EQ 1108707770)
("Value" FLOAT EQ 1109329183)
("Value" FLOAT EQ 1110445785)
)
)
("GRingLongitude" GROUP EQ
("METADATALIST" COLLECTION
("Value" FLOAT EQ 3268681138)
("Value" FLOAT EQ 3268672225)
("Value" FLOAT EQ 3268661740)
("Value" FLOAT EQ 3270986695)
("Value" FLOAT EQ 3272360395)
("Value" FLOAT EQ 3272446968)
("Value" FLOAT EQ 3272544092)
("Value" FLOAT EQ 3271210828)
)
)
("AlgorithmVersion" STRING EQ "1.O.000.001")
("AncFilename" GROUP EQ
("METADATALIST" COLLECTION
("N_Anc_Filename" STRING EQ "Terrain-Eco-ANC-Tile_20030125000000Z_ee00000000000000Z_NA_NA_N0585_1.O.0.0")
("N_Anc_Filename" STRING EQ "Terrain-Eco-ANC-Tile_20030125000000Z_ee00000000000000Z_NA_NA_N0586_1.O.0.0")
("N_Anc_Filename" STRING EQ "Terrain-Eco-ANC-Tile_20030125000000Z_ee00000000000000Z_NA_NA_N0616_1.O.0.0")
("N_Anc_Filename" STRING EQ "Terrain-Eco-ANC-Tile_20030125000000Z_ee00000000000000Z_NA_NA_N0617_1.O.0.0")
("N_Anc_Filename" STRING EQ "Terrain-Eco-ANC-Tile_20030125000000Z_ee00000000000000Z_NA_NA_N0618_1.O.0.0")
("N_Anc_Filename" STRING EQ "Terrain-Eco-ANC-Tile_20030125000000Z_ee00000000000000Z_NA_NA_N0619_1.O.0.0")
("N_Anc_Filename" STRING EQ "Terrain-Eco-ANC-Tile_20030125000000Z_ee00000000000000Z_NA_NA_N0648_1.O.0.0")
("N_Anc_Filename" STRING EQ "Terrain-Eco-ANC-Tile_20030125000000Z_ee00000000000000Z_NA_NA_N0649_1.O.0.0")
("N_Anc_Filename" STRING EQ "Terrain-Eco-ANC-Tile_20030125000000Z_ee00000000000000Z_NA_NA_N0650_1.O.0.0")
("N_Anc_Filename" STRING EQ "Terrain-Eco-ANC-Tile_20030125000000Z_ee00000000000000Z_NA_NA_N0651_1.O.0.0")
("N_Anc_Filename" STRING EQ "Terrain-Eco-ANC-Tile_20030125000000Z_ee00000000000000Z_NA_NA_N0652_1.O.0.0")
("N_Anc_Filename" STRING EQ "Terrain-Eco-ANC-Tile_20030125000000Z_ee00000000000000Z_NA_NA_N0653_1.O.0.0")
("N_Anc_Filename" STRING EQ "Terrain-Eco-ANC-Tile_20030125000000Z_ee00000000000000Z_NA_NA_N0680_1.O.0.0")
("N_Anc_Filename" STRING EQ "Terrain-Eco-ANC-Tile_20030125000000Z_ee00000000000000Z_NA_NA_N0681_1.O.0.0")
("N_Anc_Filename" STRING EQ "Terrain-Eco-ANC-Tile_20030125000000Z_ee00000000000000Z_NA_NA_N0682_1.O.0.0")
("N_Anc_Filename" STRING EQ "Terrain-Eco-ANC-Tile_20030125000000Z_ee00000000000000Z_NA_NA_N0683_1.O.0.0")
("N_Anc_Filename" STRING EQ "Terrain-Eco-ANC-Tile_20030125000000Z_ee00000000000000Z_NA_NA_N0684_1.O.0.0")
("N_Anc_Filename" STRING EQ "Terrain-Eco-ANC-Tile_20030125000000Z_ee00000000000000Z_NA_NA_N0685_1.O.0.0")
("N_Anc_Filename" STRING EQ "Terrain-Eco-ANC-Tile_20030125000000Z_ee00000000000000Z_NA_NA_N0714_1.O.0.0")
("N_Anc_Filename" STRING EQ "Terrain-Eco-ANC-Tile_20030125000000Z_ee00000000000000Z_NA_NA_N0715_1.O.0.0")
("N_Anc_Filename" STRING EQ "Terrain-Eco-ANC-Tile_20030125000000Z_ee00000000000000Z_NA_NA_N0716_1.O.0.0")
("N_Anc_Filename" STRING EQ "Terrain-Eco-ANC-Tile_20030125000000Z_ee00000000000000Z_NA_NA_N0717_1.O.0.0")
("N_Anc_Filename" STRING EQ "Terrain-Eco-ANC-Tile_20030125000000Z_ee00000000000000Z_NA_NA_N0747_1.O.0.0")
("N_Anc_Filename" STRING EQ "Terrain-Eco-ANC-Tile_20030125000000Z_ee00000000000000Z_NA_NA_N0748_1.O.0.0")
("N_Anc_Filename" STRING EQ "off_NCEP-GFS-09HR-ANC_GFS_NCEP_009f_20120109_201201091200Z_20120109154123Z_ee20120110000000Z_np")
("N_Anc_Filename" STRING EQ "off_Planet-Eph-ANC_Static_JPL_000f_20000101_200001010000Z_20000101000000Z_ee00000000000000Z_np")
("N_Anc_Filename" STRING EQ "off_USNO-PolarWander-UT1-ANC_Ser7_USNO_000f_20120105_201201051200Z_20120105120515Z_ee20120112120000Z_np")
)
)
("AuxFilename" GROUP EQ
("METADATALIST" COLLECTION
("N_Aux_Filename" STRING EQ "VIIRS-AF-EDR-AC_npp_20020101010000Z_20020101010000Z_ee00000000000000Z_1_devl_dev_noaa_ops")
("N_Aux_Filename" STRING EQ "VIIRS-AF-EDR-DQTT_npp_20020101010000Z_20020101010000Z_ee00000000000000Z_BASELINE-MON-1_devl_dev_noaa_ops")
("N_Aux_Filename" STRING EQ "VIIRS-CM-IP-AC_npp_20020101010000Z_20020101010000Z_ee00000000000000Z_1_devl_dev_noaa_ops")
)
)
("BeginningOrbitNumber" UINTEGER EQ 1043)
("N_Graceful_Degradation" STRING EQ "No")
("N_Granule_ID" STRING EQ "NPP000068154535")
("N_Granule_Version" STRING EQ "A2")
("InputProd" GROUP EQ
("METADATALIST" COLLECTION
("N_Input_Prod" STRING EQ "4f0b6480-6bf60-0a180213-5929c1b3")
("N_Input_Prod" STRING EQ "4f0b6480-8cac8-0a180213-592bcd1b")
("N_Input_Prod" STRING EQ "4f0b6480-b5848-0a180213-592e5a9b")
("N_Input_Prod" STRING EQ "4f0b6595-8e58a-0a180213-592be8f2")
("N_Input_Prod" STRING EQ "4f0b6595-911b8-0a180213-592c1520")
("N_Input_Prod" STRING EQ "4f0b6595-9618e-0a180213-592c64f6")
("N_Input_Prod" STRING EQ "4f0b6595-a07ae-0a180213-592d0b16")
("N_Input_Prod" STRING EQ "4f0b6595-a3a89-0a180213-592d3df1")
("N_Input_Prod" STRING EQ "4f0b6595-ea744-0a180213-5931aaac")
("N_Input_Prod" STRING EQ "4f0b6595-ecb88-0a180213-5931cef0")
("N_Input_Prod" STRING EQ "4f0b6595-f1d1f-0a180213-59322087")
("N_Input_Prod" STRING EQ "4f0b6596-035e6-0a180213-5923394f")
("N_Input_Prod" STRING EQ "4f0b6596-05b8f-0a180213-59235ef8")
("N_Input_Prod" STRING EQ "4f0b6596-08198-0a180213-59238501")
("N_Input_Prod" STRING EQ "4f0b6596-0a2c0-0a180213-5923a629")
("N_Input_Prod" STRING EQ "4f0b6596-0c13f-0a180213-5923c4a8")
("N_Input_Prod" STRING EQ "4f0b6596-0d195-0a180213-5923d4fe")
("N_Input_Prod" STRING EQ "4f0b6596-1c822-0a180213-5924cb8b")
("N_Input_Prod" STRING EQ "4f0b6596-365ba-0a180213-59266923")
("N_Input_Prod" STRING EQ "4f0b660d-81907-0a180213-592b1007")
("N_Input_Prod" STRING EQ "4f0b66a7-a871c-0a180213-592d8b96")
("N_Input_Prod" STRING EQ "4f0b66a7-ab851-0a180213-592dbccb")
("N_Input_Prod" STRING EQ "4f0b66a7-b2e09-0a180213-592e3283")
("N_Input_Prod" STRING EQ "4f0b676a-f3d32-0a180212-5932426e")
("N_Input_Prod" STRING EQ "4f0b676b-02e79-0a180212-592333b6")
("N_Input_Prod" STRING EQ "4f0b676b-0758f-0a180212-59237acc")
("N_Input_Prod" STRING EQ "4f0b67cb-b7d78-0a180213-592e8316")
("N_Input_Prod" STRING EQ "4f0b67cb-ba2fa-0a180213-592ea898")
("N_Input_Prod" STRING EQ "4f0b67cb-be991-0a180213-592eef2f")
("N_Input_Prod" STRING EQ "4f0b67cb-c0b0d-0a180213-592f10ab")
("N_Input_Prod" STRING EQ "4f0b67cb-c2dc9-0a180213-592f3367")
("N_Input_Prod" STRING EQ "4f0b67cb-d8a7a-0a180213-59309018")
("N_Input_Prod" STRING EQ "4f0b67cb-dacd3-0a180213-5930b271")
("N_Input_Prod" STRING EQ "4f0b67cb-df0ab-0a180213-5930f649")
("N_Input_Prod" STRING EQ "4f0b67cb-e31bf-0a180213-5931375d")
("N_Input_Prod" STRING EQ "4f0b67cb-e522c-0a180213-593157ca")
("N_Input_Prod" STRING EQ "4f0b67cb-e72d3-0a180213-59317871")
("N_Input_Prod" STRING EQ "4f0b67cb-e94ca-0a180213-59319a68")
("N_Input_Prod" STRING EQ "4f0b67cb-eb46f-0a180213-5931ba0d")
("N_Input_Prod" STRING EQ "4f0b67cb-ec557-0a180213-5931caf5")
("N_Input_Prod" STRING EQ "4f0b67cb-ee7da-0a180213-5931ed78")
("N_Input_Prod" STRING EQ "4f0b67cb-f09b9-0a180213-59320f57")
("N_Input_Prod" STRING EQ "4f0b683e-bbda0-0a180213-592eb6d1")
("N_Input_Prod" STRING EQ "4f0b688e-91b2e-0a180212-592c218e")
("N_Input_Prod" STRING EQ "4f0b688e-93bcf-0a180212-592c422f")
("N_Input_Prod" STRING EQ "4f0b688e-97a46-0a180212-592c80a6")
("N_Input_Prod" STRING EQ "4f0b688e-a26f2-0a180212-592d2d52")
("N_Input_Prod" STRING EQ "4f0b688e-a4720-0a180212-592d4d80")
("N_Input_Prod" STRING EQ "4f0b688e-ac443-0a180212-592dcaa3")
("N_Input_Prod" STRING EQ "4f0b688e-ae182-0a180212-592de7e2")
("N_Input_Prod" STRING EQ "4f0b688e-b22a3-0a180212-592e2903")
("N_Input_Prod" STRING EQ "4f0b688e-b6446-0a180212-592e6aa6")
("N_Input_Prod" STRING EQ "4f0b688e-b869a-0a180212-592e8cfa")
("N_Input_Prod" STRING EQ "4f0b688e-ba9f7-0a180212-592eb057")
("N_Input_Prod" STRING EQ "4f0b688e-bc7f4-0a180212-592ece54")
("N_Input_Prod" STRING EQ "4f0b688e-bedda-0a180212-592ef43a")
("N_Input_Prod" STRING EQ "4f0b688e-bffb6-0a180212-592f0616")
("N_Input_Prod" STRING EQ "4f0b688e-c23bc-0a180212-592f2a1c")
("N_Input_Prod" STRING EQ "4f0b688e-c45f9-0a180212-592f4c59")
("N_Input_Prod" STRING EQ "4f0b6900-4b56d-0a180212-5927af5f")
)
)
("N_LEOA_Flag" STRING EQ "On")
("N_Nadir_Latitude_Max" FLOAT EQ 1110433438)
("N_Nadir_Latitude_Min" FLOAT EQ 1109161175)
("N_Nadir_Longitude_Max" FLOAT EQ 3270987612)
("N_Nadir_Longitude_Min" FLOAT EQ 3271208075)
("N_Number_of_Scans" INTEGER EQ 48)
("QualityNames" GROUP EQ
("METADATALIST" COLLECTION
("N_Quality_Names" STRING EQ "N/A")
)
)
("QualityValues" GROUP EQ
("METADATALIST" COLLECTION
("N_Quality_Values" INTEGER EQ -993)
)
)
("N_Satellite/Local_Azimuth_Angle_Max" FLOAT EQ 1127477871)
("N_Satellite/Local_Azimuth_Angle_Min" FLOAT EQ 3274964533)
("N_Satellite/Local_Zenith_Angle_Max" FLOAT EQ 1116488650)
("N_Satellite/Local_Zenith_Angle_Min" FLOAT EQ 1008869124)
("N_Solar_Azimuth_Angle_Max" FLOAT EQ 1127481344)
("N_Solar_Azimuth_Angle_Min" FLOAT EQ 3274964992)
("N_Solar_Zenith_Angle_Max" FLOAT EQ 1116845742)
("N_Solar_Zenith_Angle_Min" FLOAT EQ 1113939509)
("N_Spacecraft_Maneuver" STRING EQ "Unknown")
("North_Bounding_Coordinate" FLOAT EQ 1110798421)
("South_Bounding_Coordinate" FLOAT EQ 1108110397)
("West_Bounding_Coordinate" FLOAT EQ 3272544092)
("N_Day_Night_Flag" STRING EQ "Day")
("RangeDateTime" DATETIMERANGE EQ "1958-01-01 00:00:00.000000" "1958-01-01 00:00:00.000000")
("ObservedDateTime" DATETIMERANGE EQ "2012-01-09 21:10:53.710583" "2012-01-09 21:12:17.886236")
("Shell_Status" STRING EQ "No")
("ByteOrder" STRING EQ "LE")
("DatasetLock" INTEGER EQ 0)
("CreationDateTime" DATETIME EQ "2012-01-10 19:49:45.150533")
("N_Software_Version" STRING EQ "Unknown")
("UpdateDateTime" DATETIME EQ "2012-01-10 19:49:45.160700")
)
)
I compared the four items in the query list and found that they are same(in bold).
2012/01/10 19:51:39.169.401 (8183.47500526003808): DBG_LOW DmCoreURMetadataFilter.cpp|144| tid-47500526003808 DMS query start: Searching for items matching the following metadata:
Metadata Collection METADATALIST (size 4)
item 0, 0x16a34e20 id=0 urId=0 parent=0 parentType=0 ItemIndex_=0 type=STRING name=N_Collection_Short_Name, comparison=EQ, value=VIIRS-CM-IP
item 1, 0x16a358f8 id=0 urId=0 parent=0 parentType=0 ItemIndex_=0 type=STRING name=N_Granule_ID, comparison=EQ, value=NPP000068154535
item 2, 0x16a357b0 id=0 urId=0 parent=0 parentType=0 ItemIndex_=0 type=STRING name=Platform_Short_Name, comparison=EQ, value=NPP
item 3, 0x16a33d20 id=0 urId=0 parent=0 parentType=0 ItemIndex_=0 type=INTEGER name=DatasetLock, comparison=EQ,value=0
This is error information from log file.
2012/01/10 19:51:39.170.515 (8183.47500526003808): DBG_MED ProCmnMethodAudit.cpp|225|ProCmnURSorter[ProCmnURSorter]::sortURs() [0x7fff766ae890] PRO_SUCCESS
2012/01/10 19:51:39.170.528 (8183.47500526003808): DBG_MED ProCmnMethodAudit.cpp|225|ProCmnInputItem[15ProCmnInputItem]::sortURs(VIIRS-CM-IP) [0x169c85c0] PRO_SUCCESS
2012/01/10 19:51:39.170.545 (8183.47500526003808): DBG_HIGH ProCmnMethodAudit.cpp|206|ProCmnInputItem[15ProCmnInputItem]::acquireURs(VIIRS-CM-IP) [0x169c85c0] UNKNOWN_ERROR checkAcquireResults() call from file ProCmnInputItem.cpp, line 280
2012/01/10 19:51:39.170.564 (8183.47500526003808): DBG_HIGH ProCmnMethodAudit.cpp|206|ProCmnDataItem[15ProCmnInputItem]::getData(VIIRS-CM-IP) [0x169c85c0] UNKNOWN_ERROR acquireURs() call from file ProCmnDataItem.cpp, line 941
2012/01/10 19:51:39.170.577 (8183.47500526003808): DBG_HIGH ProCmnMethodAudit.cpp|206|ProCmnInputItem[15ProCmnInputItem]::getData(VIIRS-CM-IP) [0x169c85c0] UNKNOWN_ERROR ProCmnDataItem::getData() failed from file ProCmnInputItem.cpp, line 183
2012/01/10 19:51:39.170.589 (8183.47500526003808): DBG_HIGH ProCmnMethodAudit.cpp|206|ProCmnDataItem[15ProCmnInputItem]::get(VIIRS-CM-IP) [0x169c85c0] UNKNOWN_ERROR getData() call from file ProCmnDataItem.cpp, line 552
2012/01/10 19:51:39.170.612 (8183.47500526003808): DBG_LOW ProCmnDataItemModel.cpp|574|WARNING: ::getMetadata(0) returning null metadata pointer to ProCmnDataItem.cpp on line 1976
2012/01/10 19:51:39.170.625 (8183.47500526003808): DBG_LOW /home/ADL_3.1/ADL/include/ProCmnDataItem.h|1993|:getMetadataValue() Error: No metadata for 15ProCmnInputItem
2012/01/10 19:51:39.170.635 (8183.47500526003808): DBG_MED ProCmnAlgorithm.cpp|7634|Error retrieving PRIMARY data for VIIRS-CM-IP
2012/01/10 19:51:39.170.645 (8183.47500526003808): DBG_MED ProCmnAlgorithm.cpp|7639|Checking for CONVERSION data (if any) for VIIRS-CM-IP
2012/01/10 19:51:39.170.660 (8183.47500526003808): DBG_MED ProCmnMethodAudit.cpp|123|ProCmnAlgorithm[ProEdrViirsLst]::getDataToConvert(NPP000068154535) [0x169bba20] entered
2012/01/10 19:51:39.170.812 (8183.47500526003808): DBG_LOW ProCmnShortNameGenerator.cpp|207|getInputShortName(): CASE ConversionDataShortName
2012/01/10 19:51:39.171.042 (8183.47500526003808): DBG_LOW ProCmnConfigGuideReader.cpp|137|Unable to read value for ProEdrViirsLst.Inputs.CloudMask_IP.OfficialShortNameConversionProduct_1: findname Unable to find Name: ProEdrViirsLst.Inputs.CloudMask_IP.OfficialShortNameConversionProduct_1
2012/01/10 19:51:39.171.074 (8183.47500526003808): DBG_HIGH ProCmnMethodAudit.cpp|206|ProCmnAlgorithm[ProEdrViirsLst]::getDataToConvert(NPP000068154535) [0x169bba20] PRO_FAIL no short name found for input: groupname CloudMask_IP at index 1 from file ProCmnAlgorithm.cpp, line 2175
2012/01/10 19:51:39.171.087 (8183.47500526003808): DBG_HIGH ProCmnAlgorithm.cpp|7659|Error retrieving data for VIIRS-CM-IP
2012/01/10 19:51:39.171.097 (8183.47500526003808): DBG_MED ProCmnAlgorithm.cpp|7664|Retrieving alternate data for VIIRS-CM-IP
2012/01/10 19:51:39.171.108 (8183.47500526003808): DBG_MED ProCmnMethodAudit.cpp|123|ProCmnAlgorithm[ProEdrViirsLst]::getAlternateData(NPP000068154535) [0x169bba20] entered
2012/01/10 19:51:39.171.446 (8183.47500526003808): DBG_LOW ProCmnConfigGuideReader.cpp|137|Unable to read value for ProEdrViirsLst.Inputs.CloudMask_IP.OfficialShortName_2: findname Unable to find Name: ProEdrViirsLst.Inputs.CloudMask_IP.OfficialShortName_2
2012/01/10 19:51:39.171.468 (8183.47500526003808): DBG_MED ProCmnMethodAudit.cpp|173|ProCmnProductDictionary[ProCmnProductDictionary]::getProductDataType() [0x1586e5e0] entered
2012/01/10 19:51:39.171.486 (8183.47500526003808): DBG_MED ProCmnMethodAudit.cpp|225|ProCmnProductDictionary[ProCmnProductDictionary]::getProductDataType() [0x1586e5e0] PRO_SUCCESS
2012/01/10 19:51:39.171.499 (8183.47500526003808): DBG_MED ProCmnMethodAudit.cpp|173|ProCmnProductDictionary[ProCmnProductDictionary]::getProductDataType() [0x1586e5e0] entered
2012/01/10 19:51:39.171.513 (8183.47500526003808): DBG_MED ProCmnMethodAudit.cpp|225|ProCmnProductDictionary[ProCmnProductDictionary]::getProductDataType() [0x1586e5e0] PRO_SUCCESS
2012/01/10 19:51:39.171.744 (8183.47500526003808): DBG_LOW ProCmnConfigGuideReader.cpp|137|Unable to read value for ProEdrViirsLst.Inputs.CloudMask_IP.Mode: findname Unable to find Name: ProEdrViirsLst.Inputs.CloudMask_IP.Mode
2012/01/10 19:51:39.171.768 (8183.47500526003808): DBG_MED ProCmnMethodAudit.cpp|225|ProCmnAlgorithm[ProEdrViirsLst]::getAlternateData(NPP000068154535) [0x169bba20] PRO_SUCCESS
2012/01/10 19:51:39.171.796 (8183.47500526003808): DBG_LOW ProCmnAlgorithmBase.cpp|362|INFSTATUS infStatusData: ProEdrViirsLst INF_STATUSTYPE_EDR_PREREQ_ABSENT VIIRS-CM-IP ( Required input not available for Shortname: VIIRS-CM-IP , Granule ID: NPP000068154535 Failure(s): (VIIRS-CM-IP, NPP000068154535) ) severity level: INF_SEVERITYLEVEL_NORMAL
2012/01/10 19:51:39.171.810 (8183.47500526003808): DBG_HIGH ProCmnMessage.cpp|191|Status: INF_STATUSTYPE_EDR_PREREQ_ABSENT | ShortName: VIIRS-CM-IP | Description: Required input not available for Shortname: VIIRS-CM-IP , Granule ID: NPP000068154535 Failure(s): (VIIRS-CM-IP, NPP000068154535)
2012/01/10 19:51:39.171.823 (8183.47500526003808): DBG_HIGH ProCmnMethodAudit.cpp|206|ProCmnAlgorithm[ProEdrViirsLst]::handleInputQuery(NPP000068154535) [0x169bba20] PRO_FAIL Required input not available for Shortname: VIIRS-CM-IP , Granule ID: NPP000068154535 Failure(s): (VIIRS-CM-IP, NPP000068154535) from file ProCmnAlgorithm.cpp, line 7699
2012/01/10 19:51:39.171.851 (8183.47500526003808): DBG_HIGH ProCmnMethodAudit.cpp|206|ProCmnAlgorithm[ProEdrViirsLst]::getDataItems(NPP000068154535) [0x169bba20] PRO_FAIL handleInputQuery() call from file ProCmnAlgorithm.cpp, line 971
I found these error happen with those granules with multiple geolocation file from GTP. These files have different creation time but the data looks the same. So I use one of them as input. From the log, no complain is found for geolocation file.
This problem happens quite often, for 20120110, there are 5 granules with this problem. Donot know if there is any issue with data itself.
Thanks,
Re: HDF Unpacker
All log information should go to the same log file. However, once a log file gets above a certain size (I think it's about 100MB), it get put into a second (or third or forth, etc) log file. Most algorithms do not output enough log messages to get above that threshold for 1 file. Some things like VIIRS SDR do. The logging for inputs is fairly verbose, so if you have a large number of inputs, you may get enough log messages to require multiple files. If you want to reduce the number of log files, I'd recommend removing inputs you know you do not need. You could also change <debugLevel> in your LW file to be "<debugLevel>DBG_MED</debugLevel>". This will output DBG_HIGH and DBG_MED messages, but not DBG_LOW.
4f0c9659-249e9-007f0100-ef409b42 looks to be the URID of the file you're having trouble with. So, someplace is should show up in the inventory audit at the beginning. Try
grep "URID = 4f0c9659-249e9-007f0100-ef409b42" *.log
You may want to remove previous log files.
You should also see messages about "Excluding URID 4f0c9659-249e9-007f0100-ef409b42". You will see them for each query where that item doesn't match the query. For example, if you're querying for a geolocation, you'd see that message. However, someplace there should be "Excluding URID 4f0c9659-249e9-007f0100-ef409b42" when you're querying for VIIRS-CM-IP for NPP000068154535. That's the time it should be finding that URID and not excluding it.
The metadata looks correct (that you colored blue). So, I suspect it's not getting read in correctly for some reason (and therefore not available for consideration when querying).
What is the total size of your log(s) for 1 attempted execution. (I'm wondering if it's feasible to email the logs or post them someplace so we can view the entire log.)
4f0c9659-249e9-007f0100-ef409b42 looks to be the URID of the file you're having trouble with. So, someplace is should show up in the inventory audit at the beginning. Try
grep "URID = 4f0c9659-249e9-007f0100-ef409b42" *.log
You may want to remove previous log files.
You should also see messages about "Excluding URID 4f0c9659-249e9-007f0100-ef409b42". You will see them for each query where that item doesn't match the query. For example, if you're querying for a geolocation, you'd see that message. However, someplace there should be "Excluding URID 4f0c9659-249e9-007f0100-ef409b42" when you're querying for VIIRS-CM-IP for NPP000068154535. That's the time it should be finding that URID and not excluding it.
The metadata looks correct (that you colored blue). So, I suspect it's not getting read in correctly for some reason (and therefore not available for consideration when querying).
What is the total size of your log(s) for 1 attempted execution. (I'm wondering if it's feasible to email the logs or post them someplace so we can view the entire log.)
Kevin Bisanz
Raytheon Company
Raytheon Company
Re: HDF Unpacker
The log file is about 60k but it is too big to be posted here. Could you give me your email address so that I can send it to you?
I have a question about N_Granule_Version, what is the difference between A1 and A2? I found this value is equal to A2 in those problematic cloud mask files. Thanks.
I have a question about N_Granule_Version, what is the difference between A1 and A2? I found this value is equal to A2 in those problematic cloud mask files. Thanks.
Re: HDF Unpacker
N_Granule_Version is used to track different versions of an RDR that is released in the system. Ingest (ING) is the software item in IDPS that creates the RDR items. The ING software will release an "A1" version of an RDR when a time threshold is exceeded and a sufficient number of packets have been received. However, additional packets may be received later for this same RDR granule (time period), possibly on the next contact and another RDR will be released but the version is incremented to be "A2". This version of the granule would then also be tasked all the way through the VIIRS chain.
I suspect the issue you are running into now is that you have a mixture of inputs for LST, some of these inputs are A1 versions and others like the VIIRS-CM-IP are now A2 versions. In order to successfully task LST for a given granule the versions have to match up to be the same. For instance, all of the granule inputs need to be "A1" or either "A2".
The actual version that is executed in ADL is controlled by the run-time configuration file, specifically the taskDetails2 item in the tasking detail block:
<task>
<taskType>EDR</taskType>
<taskDetails1>NPP001212026331</taskDetails1>
<taskDetails2>A1</taskDetails2>
<taskDetails3>NPP</taskDetails3>
<taskDetails4>VIIRS</taskDetails4>
</task>
I suspect you have a disjoint group of inputs in your current input directory, or your modified ProEdrViirsLstControllerLwFile.xml does not specify "A2".
Bryan Henderson
Raytheon Company
I suspect the issue you are running into now is that you have a mixture of inputs for LST, some of these inputs are A1 versions and others like the VIIRS-CM-IP are now A2 versions. In order to successfully task LST for a given granule the versions have to match up to be the same. For instance, all of the granule inputs need to be "A1" or either "A2".
The actual version that is executed in ADL is controlled by the run-time configuration file, specifically the taskDetails2 item in the tasking detail block:
<task>
<taskType>EDR</taskType>
<taskDetails1>NPP001212026331</taskDetails1>
<taskDetails2>A1</taskDetails2>
<taskDetails3>NPP</taskDetails3>
<taskDetails4>VIIRS</taskDetails4>
</task>
I suspect you have a disjoint group of inputs in your current input directory, or your modified ProEdrViirsLstControllerLwFile.xml does not specify "A2".
Bryan Henderson
Raytheon Company