If you look at the log file(s) produced for a process which is a controller (like the VIIRS SDR Controller you're running), you'll see messages talking about either "will create" or "will not create" an algorithm. For example:
2011/10/03 14:12:24.627.071 (25931.47472956151424): DBG_LOW ProCmnControllerAlgorithm.cpp|606| will create ProSdrViirsVerifiedRdr
or
2011/10/03 14:12:24.789.394 (25931.47472956151424): DBG_LOW ProCmnControllerAlgorithm.cpp|618| will not create ProAncViirsGranulateNhfSurfTemp
Messages of "...will create..." indicate an algorithm was found in an Algorithm_N list (where N is an integer) and the <EnvironmentRunNameOfficial> found in the algorithm's *_CFG.xml file matches the currently executing process. Later, the algorithm will be executed.
Messages of "...will not create ..." indicate an algorithm will not be created and will not be executed.
I said "log file(s)" because some large processes produce multiple log files on a single execution. A second file will be created if the size exceeds about 100MB. For example, when running the VIIRS SDR controller on DBG_LOW I get 3 log files (for process ID 25931):
ProSdrViirsController.exe_auislnx01x_25931.log
ProSdrViirsController.exe_auislnx01x_25931.log.1
ProSdrViirsController.exe_auislnx01x_25931.log.2
Ancillary Data Inputs -- which to pick? How to granulate?
Re: Ancillary Data Inputs -- which to pick? How to granulize
Kevin Bisanz
Raytheon Company
Raytheon Company
Re: Ancillary Data Inputs -- which to pick? How to granulize
I saw the following 'will not create' in my Anc Granulation process in the ProSdrViirs run. It has the same "Error getting algorithm configuration information" type of error as what I get in my Aerosol EDR runs. Any further clues please? Does this mean some Anc inputs were not properly generated from the Anc granulation processes and cause failures in the EDR runs?
Code: Select all
2012/02/02 16:46:42.549.519 (8394.47677930533952): DBG_HIGH ProCmnMessage.cpp|165|Status: INF_STATUSTYPE_PRO_ERR | Routine: ProAncViirsController | Description: Error getting algorithm configuration informationfindname Unable to find Name: ProAncViirsGranulateNhfSurfTemp.ConfigValues.EnvironmentRunNameOfficial
2012/02/02 16:46:42.549.542 (8394.47677930533952): DBG_LOW ProCmnControllerAlgorithm.cpp|618| will not create ProAncViirsGranulateNhfSurfTemp
2012/02/02 16:46:42.549.564 (8394.47677930533952): DBG_HIGH ProCmnControllerAlgorithm.cpp|253|WARNING: ProAncViirsGranulateNhfSurfTemp will not be created
2012/02/02 16:46:42.550.254 (8394.47677930533952): DBG_HIGH ProCmnMessage.cpp|165|Status: INF_STATUSTYPE_PRO_ERR | Routine: ProAncViirsController | Description: Error getting algorithm configuration informationfindname Unable to find Name: ProAncViirsGranulateNhfSurfPres.ConfigValues.EnvironmentRunNameOfficial
2012/02/02 16:46:42.550.283 (8394.47677930533952): DBG_LOW ProCmnControllerAlgorithm.cpp|618| will not create ProAncViirsGranulateNhfSurfPres
2012/02/02 16:46:42.550.297 (8394.47677930533952): DBG_HIGH ProCmnControllerAlgorithm.cpp|253|WARNING: ProAncViirsGranulateNhfSurfPres will not be created
2012/02/02 16:46:42.550.992 (8394.47677930533952): DBG_HIGH ProCmnMessage.cpp|165|Status: INF_STATUSTYPE_PRO_ERR | Routine: ProAncViirsController | Description: Error getting algorithm configuration informationfindname Unable to find Name: ProAncViirsGranulateNhfSpecSurfHumidity.ConfigValues.EnvironmentRunNameOfficial
2012/02/02 16:46:42.551.017 (8394.47677930533952): DBG_LOW ProCmnControllerAlgorithm.cpp|618| will not create ProAncViirsGranulateNhfSpecSurfHumidity
2012/02/02 16:46:42.551.031 (8394.47677930533952): DBG_HIGH ProCmnControllerAlgorithm.cpp|253|WARNING: ProAncViirsGranulateNhfSpecSurfHumidity will not be created
2012/02/02 16:46:42.551.759 (8394.47677930533952): DBG_HIGH ProCmnMessage.cpp|165|Status: INF_STATUSTYPE_PRO_ERR | Routine: ProAncViirsController | Description: Error getting algorithm configuration informationfindname Unable to find Name: ProAncViirsGranulateNhfPresLevelTemp.ConfigValues.EnvironmentRunNameOfficial
2012/02/02 16:46:42.551.782 (8394.47677930533952): DBG_LOW ProCmnControllerAlgorithm.cpp|618| will not create ProAncViirsGranulateNhfPresLevelTemp
2012/02/02 16:46:42.551.796 (8394.47677930533952): DBG_HIGH ProCmnControllerAlgorithm.cpp|253|WARNING: ProAncViirsGranulateNhfPresLevelTemp will not be created
2012/02/02 16:46:42.552.611 (8394.47677930533952): DBG_HIGH ProCmnMessage.cpp|165|Status: INF_STATUSTYPE_PRO_ERR | Routine: ProAncViirsController | Description: Error getting algorithm configuration informationfindname Unable to find Name: ProAncViirsGranulateNhfWaterVaporMixRatio.ConfigValues.EnvironmentRunNameOfficial
2012/02/02 16:46:42.552.633 (8394.47677930533952): DBG_LOW ProCmnControllerAlgorithm.cpp|618| will not create ProAncViirsGranulateNhfWaterVaporMixRatio
2012/02/02 16:46:42.552.647 (8394.47677930533952): DBG_HIGH ProCmnControllerAlgorithm.cpp|253|WARNING: ProAncViirsGranulateNhfWaterVaporMixRatio will not be created
2012/02/02 16:46:42.553.453 (8394.47677930533952): DBG_HIGH ProCmnMessage.cpp|165|Status: INF_STATUSTYPE_PRO_ERR | Routine: ProAncViirsController | Description: Error getting algorithm configuration informationfindname Unable to find Name: ProAncViirsGranulateNhfOzone.ConfigValues.EnvironmentRunNameOfficial
2012/02/02 16:46:42.553.474 (8394.47677930533952): DBG_LOW ProCmnControllerAlgorithm.cpp|618| will not create ProAncViirsGranulateNhfOzone
2012/02/02 16:46:42.553.488 (8394.47677930533952): DBG_HIGH ProCmnControllerAlgorithm.cpp|253|WARNING: ProAncViirsGranulateNhfOzone will not be created
2012/02/02 16:46:42.553.946 (8394.47677930533952): DBG_MED ProCmnMethodAudit.cpp|225|ProCmnControllerAlgorithm[ProAncViirsController]::populateAlgorithmMap() [0x2e7aed0] PRO_SUCCESS
kbisanz wrote:If you look at the log file(s) produced for a process which is a controller (like the VIIRS SDR Controller you're running), you'll see messages talking about either "will create" or "will not create" an algorithm.
Messages of "...will not create ..." indicate an algorithm will not be created and will not be executed.
Re: Ancillary Data Inputs -- which to pick? How to granulize
Those all appear to be specific for the VIIRS Net Heat Flux algorithm. (That's what the "Nhf" in the algorithm is for.)
It's not finding "ProAncViirsGranulateNhfSurfTemp.ConfigValues.EnvironmentRunNameOfficial" because the code only looks in *_CFG.xml files which are present in the algorithm's guide list, as specified by the <configGuideName> tag in the LW file. You're probably using ProSdrViirs_GuideList.cfg. Because ProSdrViirs_GuideList.cfg doesn't list ProAncViirsGranulateNhf*_CFG.xml it can't look in those files to find "ProAncViirsGranulateNhfSurfTemp.ConfigValues.EnvironmentRunNameOfficial". If you add ProAncViirsGranulateNhf*_CFG.xml to ProSdrViirs_GuideList.cfg it should get rid of your error, assuming you changed the EnvironmentRunNameOfficial in those files. If you don't plan to run Net Heat Flux, there is no need to do this.
Actually, looking at ProAncViirsGranulateNhf*_CFG.xml, it shows that all the outputs are HEAP, meaning they won't get written to disk (because only NHF needs them). So there isn't a point to trying to run the ProAncViirsGranulateNhf* algorithms. If you plan to run Net Heat Flux, I'd recommend leaving the EnvironmentRunNameOfficial for the NHF gran anc to be the original value.
It's not finding "ProAncViirsGranulateNhfSurfTemp.ConfigValues.EnvironmentRunNameOfficial" because the code only looks in *_CFG.xml files which are present in the algorithm's guide list, as specified by the <configGuideName> tag in the LW file. You're probably using ProSdrViirs_GuideList.cfg. Because ProSdrViirs_GuideList.cfg doesn't list ProAncViirsGranulateNhf*_CFG.xml it can't look in those files to find "ProAncViirsGranulateNhfSurfTemp.ConfigValues.EnvironmentRunNameOfficial". If you add ProAncViirsGranulateNhf*_CFG.xml to ProSdrViirs_GuideList.cfg it should get rid of your error, assuming you changed the EnvironmentRunNameOfficial in those files. If you don't plan to run Net Heat Flux, there is no need to do this.
Actually, looking at ProAncViirsGranulateNhf*_CFG.xml, it shows that all the outputs are HEAP, meaning they won't get written to disk (because only NHF needs them). So there isn't a point to trying to run the ProAncViirsGranulateNhf* algorithms. If you plan to run Net Heat Flux, I'd recommend leaving the EnvironmentRunNameOfficial for the NHF gran anc to be the original value.
Kevin Bisanz
Raytheon Company
Raytheon Company
Re: Ancillary Data Inputs -- which to pick? How to granulize
Thanks, Kevin,
It is not the NHF bothers me, it is the ANC-Optical-Depth that I just cannot granulate it from NAAPS-ANC-Int. Tried your recommendation in the other thread. Still not working. Very frustrated. Appreciate your help!
Cheers,
Jingfeng
It is not the NHF bothers me, it is the ANC-Optical-Depth that I just cannot granulate it from NAAPS-ANC-Int. Tried your recommendation in the other thread. Still not working. Very frustrated. Appreciate your help!
Cheers,
Jingfeng
Re: Ancillary Data Inputs -- which to pick? How to granulize
Moreover, when I run the Aerosol EDR algorithm:
1) If I use the staged example run, it does not contain any ANC-Optical-Depth inputs, and it worked out fine;
2) If I use my new granule, somehow it started complaining about ANC-Optical-Depth inputs, how come?
Are they not based on the same configuration files?
The only difference I made is the BASETIME. Does the aerosol algorithm actually know that I am running proxy data or NPP data based on the BASETIME, then it decides it has to use ANC-Optical-Depth inputs for NPP, but not for Proxy runs? Totally clueless.
If I change such inputs to optional, it will give me 'Segmentation Fault (Core Dumped)', like more headache to worry about.
1) If I use the staged example run, it does not contain any ANC-Optical-Depth inputs, and it worked out fine;
2) If I use my new granule, somehow it started complaining about ANC-Optical-Depth inputs, how come?
Are they not based on the same configuration files?
The only difference I made is the BASETIME. Does the aerosol algorithm actually know that I am running proxy data or NPP data based on the BASETIME, then it decides it has to use ANC-Optical-Depth inputs for NPP, but not for Proxy runs? Totally clueless.
If I change such inputs to optional, it will give me 'Segmentation Fault (Core Dumped)', like more headache to worry about.
Re: Ancillary Data Inputs -- which to pick? How to granulize
$NPP_GRANULE_ID_BASETIME is only used to compute granule IDs. The aerosols code doesn't look at. It should use ANC-Optical-Depth always.
You do not want to change it to optional. I replied to the other post. It has more details.
You do not want to change it to optional. I replied to the other post. It has more details.
Kevin Bisanz
Raytheon Company
Raytheon Company
Re: Ancillary Data Inputs -- which to pick? How to granulize
Thanks, Kevin,
Yes, I now know better about this 'Optional' mode.
I think, as what I was told by Sid, is that if the aerosol code cannot find the ANC-Optical-Depth files, it will use static AOT climatology instead.
It seems was the case for the staged example run. The example run worked smoothly without ANC-Optical-Depth in the input folder.
But for my NPP new run, it did not work out as smoothly as the example run. Maybe missing ANC-Optical-Depth files is not the key that the code did not generate any outputs. Still head scratching...
Yes, I now know better about this 'Optional' mode.
I think, as what I was told by Sid, is that if the aerosol code cannot find the ANC-Optical-Depth files, it will use static AOT climatology instead.
It seems was the case for the staged example run. The example run worked smoothly without ANC-Optical-Depth in the input folder.
But for my NPP new run, it did not work out as smoothly as the example run. Maybe missing ANC-Optical-Depth files is not the key that the code did not generate any outputs. Still head scratching...
kbisanz wrote:$NPP_GRANULE_ID_BASETIME is only used to compute granule IDs. The aerosols code doesn't look at. It should use ANC-Optical-Depth always.
You do not want to change it to optional. I replied to the other post. It has more details.
Re: Ancillary Data Inputs -- which to pick? How to granulize
I finally got the VIIRS Aerosol EDR new NPP granule run worked out.
Thanks a lot for all the kind and helpful feedbacks I got from this forum!
Thanks a lot for all the kind and helpful feedbacks I got from this forum!