ADL_Unpacker causes Segmentation Fault errors

Issues related to installation of ADL
jhuangadl
Posts: 142
Joined: Fri Apr 29, 2011 7:57 am

ADL_Unpacker causes Segmentation Fault errors

Post by jhuangadl »

Code: Select all

[scientist@localhost S]$ /home/scientist/CSPP/ADL/tools/bin/ADL_Unpacker.exe VAOOO_npp_d20120618_t1909025_e1910267_b03326_c20120619014327746381_noaa_ops.h5 
Segmentation fault (core dumped)
I met this problem when I tried the first attempt of VA ADL4.0.
Then it seems the problem was gone after I reinstalled the ADL4.0 VA for the second time.
But it came back again this week, despite that it was working properly last week.
Very frustrating with such Segmentation Fault issues.
If I only run /home/scientist/CSPP/ADL/tools/bin/ADL_Unpacker.exe, it seems give me the help options listed successfully.
What else could be wrong? I am not sure it is entirely the ADL problem, might be my PC problem. I am using VMware Player under 64-bit Windows7 to run the VA ADL4.0. So far it did not work out as smoothly as we did for ADL3.1.
Any hints are greatly appreciated!
kbisanz
Posts: 280
Joined: Wed Jan 05, 2011 7:02 pm
Location: Omaha NE

Re: ADL_Unpacker causes Segmentation Fault errors

Post by kbisanz »

The ADL HDF packer and unpacker tend to be picky about the paths provided them. In the case of the packer, the only way to provide the output path is to modify the value of "DMS_PATH" in $ADL_HOME/cfg/DDSADL_CFG.xml. Environment variables are OK, but make sure the path is an absolute path and it's writable.

Assuming that's not your problem (I doubt it is), I think you're going to have to try to run the unpacker in a debugger. I'm not sure what debugger you use, but gdb is popular on Linux. You can use it like this:

Code: Select all

~/tmp > cat error.cpp
#include <iostream>
using namespace std;
int main(int argc, char* argv[])
{
   for(int i = 0; i < argc; ++i)
   {
      cout << "argv[i] is " << argv[i] << endl;
   }
   cout << "starting..." << endl;
   int array[5];
   int x = -100000;
   int y = array[x];
   cout << "finished..." << endl;
   return 0;
}
~/tmp > g++44 -g error.cpp
~/tmp > ./a.out
argv[i] is ./a.out
starting...
Segmentation fault
~/tmp > gdb --args a.out file1
GNU gdb (GDB) Red Hat Enterprise Linux (7.0.1-32.el5_6.2)
Copyright (C) 2009 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /home/adluser/tmp/a.out...done.
(gdb) run
Starting program: /home/adluser/tmp/a.out file1
argv[i] is /home/adluser/tmp/a.out
argv[i] is file1
starting...

Program received signal SIGSEGV, Segmentation fault.
0x000000000040087d in main (argc=2, argv=0x7fffffffe5a8) at error.cpp:12
12         int y = array[x];
(gdb) where
#0  0x000000000040087d in main (argc=2, argv=0x7fffffffe5a8) at error.cpp:12
(gdb) print x
$1 = -100000
(gdb)
I created a sample C++ program with a segmentation fault as an example. You must compile with the "-g" flag to g++. The -g turns on debugging symbols when the code is compiled. The -g is already the default for ADL, so you shouldn't have to do anything special.

Then run the program with "gdb --args /home/scientist/CSPP/ADL/tools/bin/ADL_Unpacker.exe VAOOO_npp_d20120618_t1909025_e1910267_b03326_c20120619014327746381_noaa_ops.h5"

Once gdb starts, type "run" to execute the program. gdb should stop at the point it detects a seg fault. At that point, you can type "where" to get a stack trace and type "print aVarNameHere" to see the value of variables. If you get a stack trace, please post it to this thread.

Do other h5 files cause a segmentation fault? Or is it just that one? How large of a file is it? Is it able to be emailed? If it's easily emailed, I can try to run unpack it and see if I see the segmentation fault. If I could reproduce it, you wouldn't need to use gdb.

As always, please post back if something doesn't make sense.
Kevin Bisanz
Raytheon Company
jhuangadl
Posts: 142
Joined: Fri Apr 29, 2011 7:57 am

Re: ADL_Unpacker causes Segmentation Fault errors

Post by jhuangadl »

I do have a good news!
I don't know how, but this fixed the problem:
source /home/scientist/CSPP/ADL/build/envSetup.ksh
I think Kevin is right that it is the path info that the unpacker feels picky about that caused the segmentation fault.
Many thanks for all the hints!
Cheers,
jingfeng
ljiang
Posts: 58
Joined: Mon Jul 11, 2011 10:57 pm

Re: ADL_Unpacker causes Segmentation Fault errors

Post by ljiang »

Hi, I also have some issues with ADL_Unpacker in the ADL4.1 package. I got this:

Processing /data/data041/VIIRS_DATA/VIIRS-M11-SDR/20111121/SVM11_npp_d20111121_t1942260_e1943502_b00347_c20120830214928682851_noaa_ops.h5...

Finished
HDF5 error:
HDF5-DIAG: Error detected in HDF5 (1.8.7) thread 0:
#000: H5E.c line 1592 in H5Eget_auto2(): wrong API function, H5Eset_auto1 has been called
major: Error API
minor: Can't get value


Aborted (core dumped)



seems like hdf5 library compatible problem but ADL4.1 is using the intended library:

ldd /data/data041/ADL4.1/ADL/tools/bin/ADL_Unpacker.exe | grep hdf
libhdf5.so.7 => /data/data041/ADL4.1/common/local/lib/libhdf5.so.7 (0x00002b84c1cf7000)

so any ideas?
Lide Jiang
CIRA @ NOAA/NESDIS/STAR
houchin
Posts: 128
Joined: Mon Jan 10, 2011 6:20 am

Re: ADL_Unpacker causes Segmentation Fault errors

Post by houchin »

ljiang wrote:Hi, I also have some issues with ADL_Unpacker in the ADL4.1 package. I got this:
#000: H5E.c line 1592 in H5Eget_auto2(): wrong API function, H5Eset_auto1 has been called
Aborted (core dumped)
I have had this issue ever since the Mx6.2 tarball release. It is not a path issue, as if I run against the original ADL 4.0 version with the exact same setup, it doesn't crash. Kevin and Bryan looked into it, but couldn't find anything. Maybe now that I'm not the only one with the problem...

I did use gdb to track this down a bit, and the good news is that the crash is happening in the cleanup code inserted by the compiler that gets run when the application itself causes exit. So the unpacker has actually finished up, written all files, and thinks it ran successfully.

So in absence of a fix, I modified the AeroADL setup file to not dump core and added some code to my unpacker wrapper script that hides that error message from the user.

The problem is likely a memory leak or bad pointer somewhere in the code.
Scott Houchin, Senior Engineering Specialist, The Aerospace Corporation
15049 Conference Center Dr CH3/310, Chantilly, VA 20151; 571-307-3914; scott.houchin@aero.org
scottm

Re: ADL_Unpacker causes Segmentation Fault errors

Post by scottm »

This is a problem with the H5 interface. To solve this problem. Save you envSetup.ksh file. Reinstall ADL 4.1 and appliy the SSWEC patch. (This patch was added after the initial posting of software) Replace the setupEnv with the one you saved and recompile.
jhuangadl
Posts: 142
Joined: Fri Apr 29, 2011 7:57 am

Re: ADL_Unpacker causes Segmentation Fault errors

Post by jhuangadl »

Hi Scott,
I know I sound silly here, do you mind giving more details how to apply the patch SSEC_ADL4_1.patch?
Something like:
patch -p1 < SSEC_ADL4_1.patch
???
scottm

Re: ADL_Unpacker causes Segmentation Fault errors

Post by scottm »

The patch should be applied over the ADL tree

ls
ADL
SSEC_ADL4_1.patch

patch -p1 -i SSEC_ADL4_1.patch
jhuangadl
Posts: 142
Joined: Fri Apr 29, 2011 7:57 am

Re: ADL_Unpacker causes Segmentation Fault errors

Post by jhuangadl »

It seems it works for NCEP and NAAPS ancillary data unpacking, but not for USNO Polar Wonder data unpacking.
The unpacked results were still generated though.

Code: Select all

Processing off_NAAPS-12HR-ANC_NAAPS_FNMOC_012f_20121016_201210161200Z_20121016151501Z_ee20121017030000Z_np.h5... 

--------------------------------------------------------------------------------------

Processing off_NCEP-GFS-09HR-ANC_GFS_NCEP_009f_20121016_201210161200Z_20121016153025Z_ee20121017000000Z_np.h5... 

--------------------------------------------------------------------------------------

Processing off_NCEP-GFS-12HR-ANC_GFS_NCEP_012f_20121016_201210161200Z_20121016153039Z_ee20121017030000Z_np.h5... 

--------------------------------------------------------------------------------------

Processing off_USNO-PolarWander-UT1-ANC_Ser7_USNO_000f_20121012_201210120000Z_20121012000002Z_ee20121019120000Z_np.h5... 

Finished
HDF5 error: 
HDF5-DIAG: Error detected in HDF5 (1.8.7) thread 0:
  #000: H5E.c line 1592 in H5Eget_auto2(): wrong API function, H5Eset_auto1 has been called
    major: Error API
    minor: Can't get value


Aborted (core dumped)
scottm

Re: ADL_Unpacker causes Segmentation Fault errors

Post by scottm »

Could you please you send me the command you ran that caused this error.
Post Reply