"double free or corruption" type of error

[jhuangadl]

Hi All,
We are trying to add new QF data field to the existing QF structure in aerosol outputs. We are running into this interesting error that the aerosol output files including h5 files were generated successfully, but the screen gave out the following error after it said "ProEdrViirsAerosolController processing completed successfully". An examination of the output files looks fine with output datafields.

Code: Select all

*** glibc detected *** /data/data003/ADL4.2/ADL/bin/ProEdrViirsAerosolController.exe: double free or corruption (!prev): 0x0000000002ee9e60 ***
======= Backtrace: =========
/lib64/libc.so.6[0x3b7ae75e76]
/lib64/libc.so.6[0x3b7ae789b3]
/data/data003/ADL4.2/ADL/lib/libProEdrViirsAerosol.so(_ZN18ProEdrViirsAerosolD0Ev+0x24)[0x2b206d57e67c]
/data/data003/ADL4.2/ADL/lib/libProCmnIPO.so(_ZN28ProCmnSubAlgorithmOperationsD2Ev+0x18e)[0x2b2064a1ad46]
/data/data003/ADL4.2/ADL/lib/libProCmnIPO.so(_ZN28ProCmnSubAlgorithmOperationsD0Ev+0x18)[0x2b2064a1aef6]
/data/data003/ADL4.2/ADL/lib/libProCmnIPO.so(_ZN25ProCmnControllerAlgorithm25cleanTaskIndependentStateEv+0x49)[0x2b2064a0f3df]
/data/data003/ADL4.2/ADL/lib/libProCmnIPO.so(_ZN25ProCmnControllerAlgorithmD1Ev+0x52)[0x2b2064a0f244]
/data/data003/ADL4.2/ADL/bin/ProEdrViirsAerosolController.exe(_ZN28ProEdrViirsAerosolControllerD1Ev+0x43)[0x427e6b]
/data/data003/ADL4.2/ADL/bin/ProEdrViirsAerosolController.exe(_ZN28ProEdrViirsAerosolControllerD0Ev+0x18)[0x427eac]
/data/data003/ADL4.2/ADL/lib/libAdlScienceAppl.so(_ZN10ProCmnApplD1Ev+0x7d)[0x2b20614ee657]
/data/data003/ADL4.2/ADL/lib/libProCmnViirsAppl.so(_ZN15ProCmnViirsApplD1Ev+0x5e)[0x2b2062593cdc]
/data/data003/ADL4.2/ADL/bin/ProEdrViirsAerosolController.exe(main+0xb3)[0x41911d]
/lib64/libc.so.6(__libc_start_main+0xfd)[0x3b7ae1ed5d]
/data/data003/ADL4.2/ADL/bin/ProEdrViirsAerosolController.exe[0x4181f9]
======= Memory map: ========
00400000-00827000 r-xp 00000000 00:1e 111180                             /data/data003/ADL4.2/ADL/bin/ProEdrViirsAerosolController.exe
00a26000-00a33000 rw-p 00426000 00:1e 111180                             /data/data003/ADL4.2/ADL/bin/ProEdrViirsAerosolController.exe
00a33000-01048000 rw-p 00000000 00:00 0
01793000-04ed4000 rw-p 00000000 00:00 0                                  [heap]
3b7aa00000-3b7aa20000 r-xp 00000000 08:05 1700747                        /lib64/ld-2.12.so
3b7ac1f000-3b7ac20000 r--p 0001f000 08:05 1700747                        /lib64/ld-2.12.so
3b7ac20000-3b7ac21000 rw-p 00020000 08:05 1700747                        /lib64/ld-2.12.so
3b7ac21000-3b7ac22000 rw-p 00000000 00:00 0
3b7ae00000-3b7af8a000 r-xp 00000000 08:05 1700768                        /lib64/libc-2.12.so
3b7af8a000-3b7b18a000 ---p 0018a000 08:05 1700768                        /lib64/libc-2.12.so
3b7b18a000-3b7b18e000 r--p 0018a000 08:05 1700768                        /lib64/libc-2.12.so
3b7b18e000-3b7b18f000 rw-p 0018e000 08:05 1700768                        /lib64/libc-2.12.so
3b7b18f000-3b7b194000 rw-p 00000000 00:00 0
3b7b200000-3b7b283000 r-xp 00000000 08:05 1701591                        /lib64/libm-2.12.so
3b7b283000-3b7b482000 ---p 00083000 08:05 1701591                        /lib64/libm-2.12.so
3b7b482000-3b7b483000 r--p 00082000 08:05 1701591                        /lib64/libm-2.12.so
3b7b483000-3b7b484000 rw-p 00083000 08:05 1701591                        /lib64/libm-2.12.so
3b7b600000-3b7b617000 r-xp 00000000 08:05 1701488                        /lib64/libpthread-2.12.so
3b7b617000-3b7b817000 ---p 00017000 08:05 1701488                        /lib64/libpthread-2.12.so
3b7b817000-3b7b818000 r--p 00017000 08:05 1701488                        /lib64/libpthread-2.12.so
3b7b818000-3b7b819000 rw-p 00018000 08:05 1701488                        /lib64/libpthread-2.12.so
3b7b819000-3b7b81d000 rw-p 00000000 00:00 0
3b7c200000-3b7c207000 r-xp 00000000 08:05 1701541                        /lib64/librt-2.12.so
3b7c207000-3b7c406000 ---p 00007000 08:05 1701541                        /lib64/librt-2.12.so
3b7c406000-3b7c407000 r--p 00006000 08:05 1701541                        /lib64/librt-2.12.so
3b7c407000-3b7c408000 rw-p 00007000 08:05 1701541                        /lib64/librt-2.12.so
3b7ca00000-3b7ca0d000 r-xp 00000000 08:05 679099                         /usr/lib64/libgomp.so.1.0.0
3b7ca0d000-3b7cc0c000 ---p 0000d000 08:05 679099                         /usr/lib64/libgomp.so.1.0.0
3b7cc0c000-3b7cc0d000 rw-p 0000c000 08:05 679099                         /usr/lib64/libgomp.so.1.0.0
3b7ce00000-3b7ce16000 r-xp 00000000 08:05 1701653                        /lib64/libresolv-2.12.so
3b7ce16000-3b7d016000 ---p 00016000 08:05 1701653                        /lib64/libresolv-2.12.so
3b7d016000-3b7d017000 r--p 00016000 08:05 1701653                        /lib64/libresolv-2.12.so
3b7d017000-3b7d018000 rw-p 00017000 08:05 1701653                        /lib64/libresolv-2.12.so
3b7d018000-3b7d01a000 rw-p 00000000 00:00 0
3b7d400000-3b7d4f0000 r-xp 00000000 08:05 654992                         /usr/lib64/libgfortran.so.3.0.0
3b7d4f0000-3b7d6ef000 ---p 000f0000 08:05 654992                         /usr/lib64/libgfortran.so.3.0.0
3b7d6ef000-3b7d6f1000 rw-p 000ef000 08:05 654992                         /usr/lib64/libgfortran.so.3.0.0
3b7d6f1000-3b7d6f2000 rw-p 00000000 00:00 0
3b86600000-3b86616000 r-xp 00000000 08:05 1701609                        /lib64/libgcc_s-4.4.7-20120601.so.1
3b86616000-3b86815000 ---p 00016000 08:05 1701609                        /lib64/libgcc_s-4.4.7-20120601.so.1
3b86815000-3b86816000 rw-p 00015000 08:05 1701609                        /lib64/libgcc_s-4.4.7-20120601.so.1
3b87600000-3b876e8000 r-xp 00000000 08:05 656803                         /usr/lib64/libstdc++.so.6.0.13
3b876e8000-3b878e8000 ---p 000e8000 08:05 656803                         /usr/lib64/libstdc++.so.6.0.13
3b878e8000-3b878ef000 r--p 000e8000 08:05 656803                         /usr/lib64/libstdc++.so.6.0.13
3b878ef000-3b878f1000 rw-p 000ef000 08:05 656803                         /usr/lib64/libstdc++.so.6.0.13
3b878f1000-3b87906000 rw-p 00000000 00:00 0
2b206105c000-2b206105d000 rw-p 00000000 00:00 0
2b206105d000-2b2061073000 r-xp 00000000 00:1d 13278216                   /data/data020/JPSS_AIT/ADL_Versions/Production_ADL/CSPP/common/local/lib/libz.so.1.2.7
2b2061073000-2b2061272000 ---p 00016000 00:1d 13278216                   /data/data020/JPSS_AIT/ADL_Versions/Production_ADL/CSPP/common/local/lib/libz.so.1.2.7
2b2061272000-2b2061273000 rw-p 00015000 00:1d 13278216                   /data/data020/JPSS_AIT/ADL_Versions/Production_ADL/CSPP/common/local/lib/libz.so.1.2.7
2b2061288000-2b206128a000 rw-p 00000000 00:00 0
2b206128a000-2b20612d9000 r-xp 00000000 00:1d 13276936                   /data/data020/JPSS_AIT/ADL_Versions/Production_ADL/CSPP/common/local/lib/liblog4cplus.so.2.0.0
2b20612d9000-2b20614d9000 ---p 0004f000 00:1d 13276936                   /data/data020/JPSS_AIT/ADL_Versions/Production_ADL/CSPP/common/local/lib/liblog4cplus.so.2.0.0
2b20614d9000-2b20614dd000 rw-p 0004f000 00:1d 13276936                   /data/data020/JPSS_AIT/ADL_Versions/Production_ADL/CSPP/common/local/lib/liblog4cplus.so.2.0.0

And I rebuilt ADL successfully after all the code changes.

It appears to be our changes in the VIIRS_AOT_IP_Heap.xml file caused the problem.

Any hints on where we might set things wrong or there are any other files we should make changes to? I used to add a new QF bit successfully in the past without seeing such kind of errors. But has no luck this time, although the outputs seem fine.

Thanks a lot! Any kind feedback or guidance will be sincerely appreciated!

Jingfeng

[jhuangadl]

The following is the displayed messages when I compile the code changes in algorithm source code location:

Code: Select all

[jhuang@rhw9103 src]$ make clean library program
/bin/rm -f /data/data003/ADL4.2/ADL/EDR/VIIRS/aerosol/include/AutoGeneratedProEdrViirsAerosol.h /data/data003/ADL4.2/ADL/EDR/VIIRS/aerosol/src/AutoGeneratedProEdrViirsAerosol.cpp ProEdrViirsAerosol.o ProEdrViirsAerosolDelegate.o AutoGeneratedProEdrViirsAerosol.o ProEdrViirsAerosolMain.o /data/data003/ADL4.2/ADL/lib/libProEdrViirsAerosol.a /data/data003/ADL4.2/ADL/lib/libProEdrViirsAerosol.so ;
/bin/rm -f -rf ProCmnGenerateDerivedAlgorithm.exe_*;
/bin/rm -f -f *_AdlCmnGenerateDerivedAlgorithm.exe.log
/bin/rm -f /data/data003/ADL4.2/ADL/EDR/VIIRS/aerosol/include/AutoGeneratedProEdrViirsAerosol.h /data/data003/ADL4.2/ADL/EDR/VIIRS/aerosol/src/AutoGeneratedProEdrViirsAerosol.cpp ProEdrViirsAerosolMain.cpp
/data/data003/ADL4.2/ADL/tools/bin/ProCmnGenerateDerivedAlgorithm.exe -lw /data/data003/ADL4.2/ADL/cfg/srcgen.xml ProEdrViirsAerosol 48 VIIRS EDR
mv AutoGenerated*.h ../include
chmod 775 ProCmnGenerateDerivedAlgorithm.exe_*
/usr/bin/g++ -DBYTE_ORDER_LE -DADL_ENV -D_USE_FLAT_FILE_ -D_THREAD_SAFE -DGCC -m64 -fPIC -Wall -Wno-unknown-pragmas -ggdb3     -DUSE_UNDERSCORE   -I../include            -I/data/data003/ADL4.2/ADL/CMN/Utilities/INF/include            -I/data/data003/ADL4.2/ADL/CMN/Utilities/INF/common/exceptions/include            -I/data/data003/ADL4.2/ADL/CMN/Utilities/INF/util/time/include            -I/data/data003/ADL4.2/ADL/Geolocation/CmnGeo/include            -I/data/data003/ADL4.2/ADL/Geolocation/novas-c201/include                   -I/data/data003/ADL4.2/ADL/include               -I/data/data003/ADL4.2/ADL/CMN/Utilities/DMS/include               -I/data/data003/ADL4.2/ADL/CMN/Utilities/INF/include               -I/data/data003/ADL4.2/ADL/CMN/Utilities/INF/common/exceptions/include               -I/data/data003/ADL4.2/ADL/CMN/Utilities/INF/common/util/include               -I/data/data003/ADL4.2/ADL/CMN/Utilities/INF/util/cfg/include               -I/data/data003/ADL4.2/ADL/CMN/Utilities/INF/util/time/include            -I/data/data003/ADL4.2/ADL/CMN/Utilities/INF/tk/util/include               -I/data/data003/ADL4.2/ADL/CMN/Utilities/ING/include               -I/data/data020/JPSS_AIT/ADL_Versions/CSPP/common/local/include               -I/data/data020/JPSS_AIT/ADL_Versions/CSPP/common/local/include               -I/data/data020/JPSS_AIT/ADL_Versions/CSPP/common/local/include -c ProEdrViirsAerosol.cpp -o ProEdrViirsAerosol.o
In file included from ../include/ProEdrViirsAerosol.h:102,
                 from ProEdrViirsAerosol.cpp:191:
../include/ProEdrViirsAerosolStruct.h:481: warning: ‘typedef’ was ignored in this declaration
../include/ProEdrViirsAerosolStruct.h:498: warning: ‘typedef’ was ignored in this declaration
../include/ProEdrViirsAerosolStruct.h:886: warning: ‘typedef’ was ignored in this declaration
ProEdrViirsAerosol.cpp: In member function ‘void ProEdrViirsAerosol::fillAotIpHeapValues()’:
ProEdrViirsAerosol.cpp:1191: warning: comparison between signed and unsigned integer expressions
ProEdrViirsAerosol.cpp: In member function ‘void ProEdrViirsAerosol::OzoneTrans(MDL_STRUCT*, Int32, Int32, Float32*, Float32*) const’:
ProEdrViirsAerosol.cpp:2762: warning: unused variable ‘transo2’
ProEdrViirsAerosol.cpp:2763: warning: unused variable ‘transco2’
ProEdrViirsAerosol.cpp:2764: warning: unused variable ‘transn2o’
ProEdrViirsAerosol.cpp:2765: warning: unused variable ‘transch4’
ProEdrViirsAerosol.cpp: In member function ‘void ProEdrViirsAerosol::DetermineHC(EDR_STRUCT_AEROSOL*, SM_EDR_STRUCT*, UInt32) const’:
ProEdrViirsAerosol.cpp:3174: warning: unused variable ‘qual’
ProEdrViirsAerosol.cpp:3181: warning: unused variable ‘twoScanOffset’
ProEdrViirsAerosol.cpp: In member function ‘void ProEdrViirsAerosol::AggregateHCS(EDR_STRUCT_AEROSOL*, SM_EDR_STRUCT*, Int32, Int32, Int32) const’:
ProEdrViirsAerosol.cpp:3628: warning: comparison between signed and unsigned integer expressions
ProEdrViirsAerosol.cpp:3660: warning: comparison between signed and unsigned integer expressions
ProEdrViirsAerosol.cpp:3661: warning: comparison between signed and unsigned integer expressions
ProEdrViirsAerosol.cpp:3870: warning: comparison between signed and unsigned integer expressions
ProEdrViirsAerosol.cpp:3880: warning: comparison between signed and unsigned integer expressions
ProEdrViirsAerosol.cpp:3916: warning: comparison between signed and unsigned integer expressions
ProEdrViirsAerosol.cpp:3926: warning: comparison between signed and unsigned integer expressions
ProEdrViirsAerosol.cpp:3963: warning: comparison between signed and unsigned integer expressions
ProEdrViirsAerosol.cpp:3967: warning: comparison between signed and unsigned integer expressions
ProEdrViirsAerosol.cpp:3971: warning: comparison between signed and unsigned integer expressions
ProEdrViirsAerosol.cpp: In member function ‘Int32 ProEdrViirsAerosol::AOT_main()’:
ProEdrViirsAerosol.cpp:4365: warning: comparison between signed and unsigned integer expressions
ProEdrViirsAerosol.cpp: In member function ‘Int32 ProEdrViirsAerosol::ChsMdlLand(SDR_STRUCT_AEROSOL*, MDL_STRUCT*, AOT_STRUCT*, STRUCT_LUT*, ViirsAotLutType*, int)’:
ProEdrViirsAerosol.cpp:5454: warning: comparison between signed and unsigned integer expressions
ProEdrViirsAerosol.cpp:5630: warning: comparison between signed and unsigned integer expressions
ProEdrViirsAerosol.cpp:5645: warning: comparison between signed and unsigned integer expressions
ProEdrViirsAerosol.cpp:5409: warning: unused variable ‘toarefl’
ProEdrViirsAerosol.cpp:5412: warning: unused variable ‘interpReflM5’
ProEdrViirsAerosol.cpp: In member function ‘int ProEdrViirsAerosol::ChsMdlWater(SDR_STRUCT_AEROSOL*, MDL_STRUCT*, AOT_STRUCT*, STRUCT_LUT*, ViirsAOTSunGlintInfoType*, ViirsAotLutType*, int)’:
ProEdrViirsAerosol.cpp:5889: warning: ‘typedef’ was ignored in this declaration
ProEdrViirsAerosol.cpp: In member function ‘Int32 ProEdrViirsAerosol::Watertests(SDR_STRUCT_AEROSOL*, MDL_STRUCT*, FLAG_STRUCT*, int)’:
ProEdrViirsAerosol.cpp:7377: warning: unused variable ‘standardDeviation’
ProEdrViirsAerosol.cpp: In member function ‘Int32 ProEdrViirsAerosol::AOT_interp(Float32*, Float32 (*)[2], Float32 (*)[2], Float32 (*)[12][5], Int32)’:
ProEdrViirsAerosol.cpp:7804: warning: large integer implicitly truncated to unsigned type
/data/data003/ADL4.2/ADL/CMN/Utilities/INF/include/InfTk_Defs.h: At global scope:
/data/data003/ADL4.2/ADL/CMN/Utilities/INF/include/InfTk_Defs.h:120: warning: ‘INFTK_TASK_SHUTDOWNSTR’ defined but not used
/data/data003/ADL4.2/ADL/CMN/Utilities/INF/include/InfTk_StatusTypeMap.h:83: warning: ‘InfTk_StatusTypeMap’ defined but not used
/data/data003/ADL4.2/ADL/CMN/Utilities/INF/include/InfTk_DetailTypeMap.h:83: warning: ‘InfTk_DetailTypeMap’ defined but not used
/usr/bin/g++ -DBYTE_ORDER_LE -DADL_ENV -D_USE_FLAT_FILE_ -D_THREAD_SAFE -DGCC -m64 -fPIC -Wall -Wno-unknown-pragmas -ggdb3     -DUSE_UNDERSCORE   -I../include            -I/data/data003/ADL4.2/ADL/CMN/Utilities/INF/include            -I/data/data003/ADL4.2/ADL/CMN/Utilities/INF/common/exceptions/include            -I/data/data003/ADL4.2/ADL/CMN/Utilities/INF/util/time/include            -I/data/data003/ADL4.2/ADL/Geolocation/CmnGeo/include            -I/data/data003/ADL4.2/ADL/Geolocation/novas-c201/include                   -I/data/data003/ADL4.2/ADL/include               -I/data/data003/ADL4.2/ADL/CMN/Utilities/DMS/include               -I/data/data003/ADL4.2/ADL/CMN/Utilities/INF/include               -I/data/data003/ADL4.2/ADL/CMN/Utilities/INF/common/exceptions/include               -I/data/data003/ADL4.2/ADL/CMN/Utilities/INF/common/util/include               -I/data/data003/ADL4.2/ADL/CMN/Utilities/INF/util/cfg/include               -I/data/data003/ADL4.2/ADL/CMN/Utilities/INF/util/time/include            -I/data/data003/ADL4.2/ADL/CMN/Utilities/INF/tk/util/include               -I/data/data003/ADL4.2/ADL/CMN/Utilities/ING/include               -I/data/data020/JPSS_AIT/ADL_Versions/CSPP/common/local/include               -I/data/data020/JPSS_AIT/ADL_Versions/CSPP/common/local/include               -I/data/data020/JPSS_AIT/ADL_Versions/CSPP/common/local/include -c ProEdrViirsAerosolDelegate.cpp -o ProEdrViirsAerosolDelegate.o
In file included from ../include/ProEdrViirsAerosol.h:102,
                 from ../include/ProEdrViirsAerosolDelegate.h:69,
                 from ProEdrViirsAerosolDelegate.cpp:73:
../include/ProEdrViirsAerosolStruct.h:481: warning: ‘typedef’ was ignored in this declaration
../include/ProEdrViirsAerosolStruct.h:498: warning: ‘typedef’ was ignored in this declaration
../include/ProEdrViirsAerosolStruct.h:886: warning: ‘typedef’ was ignored in this declaration
../include/ProEdrViirsAerosolDelegate.h: In constructor ‘ProEdrViirsAerosolDelegate::ProEdrViirsAerosolDelegate(ProCmnAlgorithm*)’:
../include/ProEdrViirsAerosolDelegate.h:235: warning: ‘ProEdrViirsAerosolDelegate::identity_’ will be initialized after
../include/ProEdrViirsAerosolDelegate.h:229: warning:   ‘ProCmnCallerIDImpl ProEdrViirsAerosolDelegate::myCallerID_’
ProEdrViirsAerosolDelegate.cpp:96: warning:   when initialized here
../include/ProEdrViirsAerosolDelegate.h:229: warning: ‘ProEdrViirsAerosolDelegate::myCallerID_’ will be initialized after
../include/ProEdrViirsAerosolDelegate.h:218: warning:   ‘UInt8* ProEdrViirsAerosolDelegate::susmatDetectionExclusionSummaryDataPtr_’
ProEdrViirsAerosolDelegate.cpp:96: warning:   when initialized here
ProEdrViirsAerosolDelegate.cpp: In member function ‘virtual void ProEdrViirsAerosolDelegate::createQualityTests()’:
ProEdrViirsAerosolDelegate.cpp:237: warning: comparison between signed and unsigned integer expressions
In file included from ProEdrViirsAerosolDelegate.cpp:82:
/data/data003/ADL4.2/ADL/include/ProCmnPercentQualityFlag.h: In constructor ‘ProCmnPercentQualityFlag<Type, Type2>::ProCmnPercentQualityFlag(Int32, bool, ProCmnQualityRunPhase, Type*, ProCmnDqnPixelType, Type, Type, std::string, Int32, bool, Type2*, bool, bool, bool, Type2*, Type*, Type, Type) [with Type = unsigned char, Type2 = float]’:
ProEdrViirsAerosolDelegate.cpp:263:   instantiated from here
/data/data003/ADL4.2/ADL/include/ProCmnPercentQualityFlag.h:307: warning: ‘ProCmnPercentQualityFlag<unsigned char, float>::flagValue_’ will be initialized after
/data/data003/ADL4.2/ADL/include/ProCmnPercentQualityFlag.h:274: warning:   ‘Int32 ProCmnPercentQualityFlag<unsigned char, float>::pixelCount_’
/data/data003/ADL4.2/ADL/include/ProCmnPercentQualityFlag.h:420: warning:   when initialized here
/data/data003/ADL4.2/ADL/include/ProCmnPercentQualityFlag.h:285: warning: ‘ProCmnPercentQualityFlag<unsigned char, float>::nonExcludedCount_’ will be initialized after
/data/data003/ADL4.2/ADL/include/ProCmnPercentQualityFlag.h:269: warning:   ‘std::string ProCmnPercentQualityFlag<unsigned char, float>::metaFieldName_’
/data/data003/ADL4.2/ADL/include/ProCmnPercentQualityFlag.h:420: warning:   when initialized here
/data/data003/ADL4.2/ADL/include/ProCmnPercentQualityFlag.h:340: warning: ‘ProCmnPercentQualityFlag<unsigned char, float>::copyTest_’ will be initialized after
/data/data003/ADL4.2/ADL/include/ProCmnPercentQualityFlag.h:312: warning:   ‘Int32 ProCmnPercentQualityFlag<unsigned char, float>::numElements_’
/data/data003/ADL4.2/ADL/include/ProCmnPercentQualityFlag.h:420: warning:   when initialized here
/data/data003/ADL4.2/ADL/include/ProCmnPercentQualityFlag.h:345: warning: ‘ProCmnPercentQualityFlag<unsigned char, float>::percentPtr_’ will be initialized after
/data/data003/ADL4.2/ADL/include/ProCmnPercentQualityFlag.h:335: warning:   ‘bool ProCmnPercentQualityFlag<unsigned char, float>::allBitsSetOn_’
/data/data003/ADL4.2/ADL/include/ProCmnPercentQualityFlag.h:420: warning:   when initialized here
/data/data003/ADL4.2/ADL/include/ProCmnPercentQualityFlag.h:366: warning: ‘ProCmnPercentQualityFlag<unsigned char, float>::qualityFillPtr_’ will be initialized after
/data/data003/ADL4.2/ADL/include/ProCmnPercentQualityFlag.h:350: warning:   ‘unsigned char* ProCmnPercentQualityFlag<unsigned char, float>::qualityPtr_’
/data/data003/ADL4.2/ADL/include/ProCmnPercentQualityFlag.h:420: warning:   when initialized here
/data/data003/ADL4.2/ADL/include/ProCmnPercentQualityFlag.h: In constructor ‘ProCmnPercentQualityFlag<Type, Type2>::ProCmnPercentQualityFlag(Int32, bool, ProCmnQualityRunPhase, Type2*) [with Type = unsigned char, Type2 = float]’:
ProEdrViirsAerosolDelegate.cpp:475:   instantiated from here
/data/data003/ADL4.2/ADL/include/ProCmnPercentQualityFlag.h:307: warning: ‘ProCmnPercentQualityFlag<unsigned char, float>::flagValue_’ will be initialized after
/data/data003/ADL4.2/ADL/include/ProCmnPercentQualityFlag.h:274: warning:   ‘Int32 ProCmnPercentQualityFlag<unsigned char, float>::pixelCount_’
/data/data003/ADL4.2/ADL/include/ProCmnPercentQualityFlag.h:468: warning:   when initialized here
/data/data003/ADL4.2/ADL/include/ProCmnPercentQualityFlag.h:285: warning: ‘ProCmnPercentQualityFlag<unsigned char, float>::nonExcludedCount_’ will be initialized after
/data/data003/ADL4.2/ADL/include/ProCmnPercentQualityFlag.h:269: warning:   ‘std::string ProCmnPercentQualityFlag<unsigned char, float>::metaFieldName_’
/data/data003/ADL4.2/ADL/include/ProCmnPercentQualityFlag.h:468: warning:   when initialized here
/data/data003/ADL4.2/ADL/include/ProCmnPercentQualityFlag.h:340: warning: ‘ProCmnPercentQualityFlag<unsigned char, float>::copyTest_’ will be initialized after
/data/data003/ADL4.2/ADL/include/ProCmnPercentQualityFlag.h:312: warning:   ‘Int32 ProCmnPercentQualityFlag<unsigned char, float>::numElements_’
/data/data003/ADL4.2/ADL/include/ProCmnPercentQualityFlag.h:468: warning:   when initialized here
/data/data003/ADL4.2/ADL/include/ProCmnPercentQualityFlag.h:345: warning: ‘ProCmnPercentQualityFlag<unsigned char, float>::percentPtr_’ will be initialized after
/data/data003/ADL4.2/ADL/include/ProCmnPercentQualityFlag.h:335: warning:   ‘bool ProCmnPercentQualityFlag<unsigned char, float>::allBitsSetOn_’
/data/data003/ADL4.2/ADL/include/ProCmnPercentQualityFlag.h:468: warning:   when initialized here
/data/data003/ADL4.2/ADL/include/ProCmnPercentQualityFlag.h:366: warning: ‘ProCmnPercentQualityFlag<unsigned char, float>::qualityFillPtr_’ will be initialized after
/data/data003/ADL4.2/ADL/include/ProCmnPercentQualityFlag.h:350: warning:   ‘unsigned char* ProCmnPercentQualityFlag<unsigned char, float>::qualityPtr_’
/data/data003/ADL4.2/ADL/include/ProCmnPercentQualityFlag.h:468: warning:   when initialized here
/data/data003/ADL4.2/ADL/include/ProCmnPercentQualityFlag.h: In member function ‘Int32 ProCmnPercentQualityFlag<Type, Type2>::produceQuality() [with Type = unsigned char, Type2 = float]’:
ProEdrViirsAerosolDelegate.cpp:912:   instantiated from here
/data/data003/ADL4.2/ADL/include/ProCmnPercentQualityFlag.h:513: warning: unused variable ‘colMin’
/data/data003/ADL4.2/ADL/include/ProCmnPercentQualityFlag.h:514: warning: unused variable ‘colMax’
/data/data003/ADL4.2/ADL/include/ProCmnPercentQualityFlag.h:517: warning: unused variable ‘bitTestSubject’
/data/data003/ADL4.2/ADL/include/ProCmnPercentQualityFlag.h:518: warning: unused variable ‘bitsSetOn’
/data/data003/ADL4.2/ADL/include/ProCmnPercentQualityFlag.h: In member function ‘void ProCmnPercentQualityFlag<Type, Type2>::executeBitCount() [with Type = unsigned char, Type2 = float]’:
/data/data003/ADL4.2/ADL/include/ProCmnPercentQualityFlag.h:548:   instantiated from ‘Int32 ProCmnPercentQualityFlag<Type, Type2>::produceQuality() [with Type = unsigned char, Type2 = float]’
ProEdrViirsAerosolDelegate.cpp:912:   instantiated from here
/data/data003/ADL4.2/ADL/include/ProCmnPercentQualityFlag.h:821: warning: comparison between signed and unsigned integer expressions
/data/data003/ADL4.2/ADL/CMN/Utilities/INF/include/InfTk_Defs.h: At global scope:
/data/data003/ADL4.2/ADL/CMN/Utilities/INF/include/InfTk_Defs.h:120: warning: ‘INFTK_TASK_SHUTDOWNSTR’ defined but not used
/data/data003/ADL4.2/ADL/CMN/Utilities/INF/include/InfTk_StatusTypeMap.h:83: warning: ‘InfTk_StatusTypeMap’ defined but not used
/data/data003/ADL4.2/ADL/CMN/Utilities/INF/include/InfTk_DetailTypeMap.h:83: warning: ‘InfTk_DetailTypeMap’ defined but not used
/usr/bin/g++ -DBYTE_ORDER_LE -DADL_ENV -D_USE_FLAT_FILE_ -D_THREAD_SAFE -DGCC -m64 -fPIC -Wall -Wno-unknown-pragmas -ggdb3     -DUSE_UNDERSCORE   -I../include            -I/data/data003/ADL4.2/ADL/CMN/Utilities/INF/include            -I/data/data003/ADL4.2/ADL/CMN/Utilities/INF/common/exceptions/include            -I/data/data003/ADL4.2/ADL/CMN/Utilities/INF/util/time/include            -I/data/data003/ADL4.2/ADL/Geolocation/CmnGeo/include            -I/data/data003/ADL4.2/ADL/Geolocation/novas-c201/include                   -I/data/data003/ADL4.2/ADL/include               -I/data/data003/ADL4.2/ADL/CMN/Utilities/DMS/include               -I/data/data003/ADL4.2/ADL/CMN/Utilities/INF/include               -I/data/data003/ADL4.2/ADL/CMN/Utilities/INF/common/exceptions/include               -I/data/data003/ADL4.2/ADL/CMN/Utilities/INF/common/util/include               -I/data/data003/ADL4.2/ADL/CMN/Utilities/INF/util/cfg/include               -I/data/data003/ADL4.2/ADL/CMN/Utilities/INF/util/time/include            -I/data/data003/ADL4.2/ADL/CMN/Utilities/INF/tk/util/include               -I/data/data003/ADL4.2/ADL/CMN/Utilities/ING/include               -I/data/data020/JPSS_AIT/ADL_Versions/CSPP/common/local/include               -I/data/data020/JPSS_AIT/ADL_Versions/CSPP/common/local/include               -I/data/data020/JPSS_AIT/ADL_Versions/CSPP/common/local/include -c AutoGeneratedProEdrViirsAerosol.cpp -o AutoGeneratedProEdrViirsAerosol.o
In file included from ../include/ProEdrViirsAerosol.h:102,
                 from ../include/ProEdrViirsAerosolDelegate.h:69,
                 from AutoGeneratedProEdrViirsAerosol.cpp:64:
../include/ProEdrViirsAerosolStruct.h:481: warning: ‘typedef’ was ignored in this declaration
../include/ProEdrViirsAerosolStruct.h:498: warning: ‘typedef’ was ignored in this declaration
../include/ProEdrViirsAerosolStruct.h:886: warning: ‘typedef’ was ignored in this declaration
/data/data003/ADL4.2/ADL/CMN/Utilities/INF/include/InfTk_Defs.h:120: warning: ‘INFTK_TASK_SHUTDOWNSTR’ defined but not used
/data/data003/ADL4.2/ADL/CMN/Utilities/INF/include/InfTk_StatusTypeMap.h:83: warning: ‘InfTk_StatusTypeMap’ defined but not used
/data/data003/ADL4.2/ADL/CMN/Utilities/INF/include/InfTk_DetailTypeMap.h:83: warning: ‘InfTk_DetailTypeMap’ defined but not used
Creating Static Library ...
/bin/rm -f /data/data003/ADL4.2/ADL/lib/libProEdrViirsAerosol.a
/usr/bin/ar -ru /data/data003/ADL4.2/ADL/lib/libProEdrViirsAerosol.a ProEdrViirsAerosol.o ProEdrViirsAerosolDelegate.o AutoGeneratedProEdrViirsAerosol.o
/usr/bin/ar: creating /data/data003/ADL4.2/ADL/lib/libProEdrViirsAerosol.a
Creating Shared Library ...
/bin/rm -f /data/data003/ADL4.2/ADL/lib/libProEdrViirsAerosol.so
/usr/bin/g++ -mfull-toc -m64 -Xlinker -zmuldefs -lc -shared  -o /data/data003/ADL4.2/ADL/lib/libProEdrViirsAerosol.so ProEdrViirsAerosol.o ProEdrViirsAerosolDelegate.o AutoGeneratedProEdrViirsAerosol.o
make: Nothing to be done for `program'.

[jhuangadl]

We pinpoint to the VIIRS_AOT_IP_Heap.xml that caused the problem. If we changed the other but not change this one, everything seems running fine. If we made changes (adding snowadj QF to the end, the last bit of QF5) to this xml file, there will be this error:

Code: Select all

*** glibc detected *** /data/data003/ADL4.2/ADL/bin/ProEdrViirsAerosolController.exe: free(): invalid next size (normal): 0x00000000033bb120 ***

I am attaching the changed XML file for your reference.

The problem is I did not see anything wrong with it. Any chance you can kindly help examine whether it is padding bit problem again?

[jhuangadl]

I conducted more experiments on this based on a clean version of the ADL4.2 with Mx8.4 operational code.
My experiments indicated that, adding a QF to the last bit of the last byte (QF5) in the VIIRS_AOT_IP_Heap.xml will cause such memory issue, no matter what name I gave to this QF. We couldn't figure out where in the ADL that the memory will be dellocated at the end of the algorithm running.
If you want to reproduce the issue, simply use a clean version of ADL4.2 Mx8.4 code, and replace the VIIRS_AOT_IP_Heap.xml with the one I provided previously, then run any granule through aerosol code. You will see the issue at the end. And actually no matter the algorithm is completed successfully or not, this memory dellocation issue is happening.
Any hints or guidance is greatly appreciated!

[pe_meade_dpe]

Jingfeng -

It's been quite a while since I've tried to do anything like what you are trying here, but I THINK that maybe ADL's Data Dictionary isn't getting updated when you rebuild the algorithm. I think that could result in two different parts of the framework having different definitions of the data structure. Do I interpret correctly that you are doing the "make clean library program" from within the algorithm's source code directory? If so, you could try doing a full rebuild from your $ADL_HOME directory, e.g.:

cd $ADL_HOME
./buildAdl.csh

It takes a while to rebuild, but I recall that at one point in ADL's history, at least, it was necessary to rebuild not only the algorithm but the framework when data product XML was changed.

If this doesn't work, let me know. I don't think that I have time today to try to recreate your issue, but I possibly could give it a try next week.

Also: If I DO get around to trying to recreate, is this a purely-internal data product, or an output, or do I need a modified input file to go along with the modified XML?

- Paul Meade

[tsimpson]

I think Paul Meade has identified the cause and the solution to this problem. I found that if I change VIIRS_AOT_IP_Heap.xml and do a full rebuild, then the algorithm completes successfully and I don't see the memory deallocation issue.

There are three files that need to be updated if VIIRS_AOT_IP_Heap.xml changes, but are not updated when you run "make clean library program" in the $ADL_HOME/EDR/VIIRS/aerosol/src directory. These files are:

$ADL_HOME/include/AutoGeneratedViirsProductsGbl.h
$ADL_HOME/include/ProCmnViirsXmlDictionaryEntries.h
$ADL_HOME/CMN/Utilities/Dictionary/Entries/VIIRS/src/ProCmnViirsXmlDictionaryEntries.cpp

The preferred way to rebuild all the software is with the $ADL_HOME/script/build_adl.pl script, as in the command:
$ADL_HOME/script/build_adl.pl -clean -src -library -program

The -src option causes the files mentioned above to be updated.

[jhuangadl]

Thank you very much, Todd and Paul,
I am rebuilding ADL from scratch, and will update you once I test running the updated system.
Thanks a lot for helping us out!

[jhuangadl]

Dear Paul, Todd and All,
I am glad reporting back to you that a testing ADl run is successful without the memory dellocation error.
Thank you very much for your kind guidance and sorry for taking your time on this. Appreciate your timely help very much!
Wish you a very nice day and a splendid week!
Sincerely,
Jingfeng