Fail running command on stderr.log

VIIRS Environmental Data Records
Post Reply
megasaputra2
Posts: 7
Joined: Fri Nov 20, 2020 2:45 am

Fail running command on stderr.log

Post by megasaputra2 »

Dear all,

I try to run acspo.bash. But, there is an error like this :

[mega@localhost CSPP_ACSPO_TESTDATA_1_2]$ run_acspo.bash -s npp -i /home/mega/Documents/CSPP_ACSPO_TESTDATA_1_2/input/jpss -w work/NPP -b bias -d dynamic_ancillary -p 4 -vv -q = 1
--------------
DEBUG %options
==============
'dynanc' => '/home/mega/Documents/CSPP_ACSPO_TESTDATA_1_2/dynamic_ancillary',
'clean' => 1,
'verbose' => 2,
'bias' => '/home/mega/Documents/CSPP_ACSPO_TESTDATA_1_2/bias',
'ghrsst' => 'by_date',
'aggregate' => 1,
'nothing' => 0,
'granule' => 0,
'xfiles' => 1,
'sat' => 'npp',
'input_files' => '/home/mega/Documents/CSPP_ACSPO_TESTDATA_1_2/input/jpss',
'processors' => 4,
'work' => '/home/mega/Documents/CSPP_ACSPO_TESTDATA_1_2/work/NPP',
'quarantine' => 1
DEBUG %options _END_
chdir to working directory /home/mega/Documents/CSPP_ACSPO_TESTDATA_1_2/work/NPP/TMP_2soPmc
--------------
DEBUG %sources
==============
'sim' => 'simDataConfigFile.txt',
'sst' => 'SST_Regression_Config.txt',
'pcf' => 'Output_Data_Structure.txt',
'sen' => 'sensorConfigFile.txt',
'cld' => 'Cloud_Mask_Config.txt',
'aer' => 'aerConfigFile.txt'
DEBUG %sources _END_
--------------
DEBUG %targets
==============
'aer' => 'aerConfigFile.txt',
'cld' => 'Cloud_Mask_Config.txt',
'pcf' => 'Output_Data_Structure.txt',
'sen' => 'sensorConfigFile.txt',
'sim' => 'simDataConfigFile.txt',
'sst' => 'SST_Regression_Config.txt'
DEBUG %targets _END_
run_acspo.pl::acspo_dynanc_management: INFO Fetching CMC files
run_acspo.pl::acspo_dynanc_management: INFO Fetching GFS files
Executing (time /home/mega/Documents/CSPP_ACSPO_1_2/DAP/ACSPO-VIIRS_V2-61_20190503/bin/ACSPO-V_V2.61) 2> stderr.log
Chunk size 1024 will be used
Distance value 5.000000 will be used
Overlap size 20 will be used
run_acspo.pl:: Could not run command (time /home/mega/Documents/CSPP_ACSPO_1_2/DAP/ACSPO-VIIRS_V2-61_20190503/bin/ACSPO-V_V2.61) 2> stderr.log
Abort at /home/mega/Documents/CSPP_ACSPO_1_2/scripts/run_acspo.pl line 593.

Any solution?

Regard,
Mega
kathys
Posts: 487
Joined: Tue Jun 22, 2010 4:51 pm

Re: Fail running command on stderr.log

Post by kathys »

Hello,

One thing we noticed in your command was the use of -q=1. There are no options for -q, which tells the ACSPO execution script not to download the required ancillary file from the CSPP external ancillary archive.

Can you please provide us with more information?

What is the name of your input file(s)?

What operating system and version are you using?

How much memory and CPUs does your computer have?

Sorry that you are having problems.

Kathy
megasaputra2
Posts: 7
Joined: Fri Nov 20, 2020 2:45 am

Re: Fail running command on stderr.log

Post by megasaputra2 »

The name of input files (I put in input/jpss) :
GMTCO_npp_d20201114_t1824272_e1832585_b46895_c20201114185703578941_cspp_dev.h5
GMODO_npp_d20201114_t1824272_e1832585_b46895_c20201114185829777322_cspp_dev.h5
SVM05_npp_d20201114_t1824272_e1832585_b46895_c20201114185852818359_cspp_dev.h5
SVM07_npp_d20201114_t1824272_e1832585_b46895_c20201114185902732773_cspp_dev.h5
SVM10_npp_d20201114_t1824272_e1832585_b46895_c20201114185916076152_cspp_dev.h5
SVM12_npp_d20201114_t1824272_e1832585_b46895_c20201114185921392078_cspp_dev.h5
SVM14_npp_d20201114_t1824272_e1832585_b46895_c20201114185921253331_cspp_dev.h5
SVM15_npp_d20201114_t1824272_e1832585_b46895_c20201114185922104699_cspp_dev.h5
SVM16_npp_d20201114_t1824272_e1832585_b46895_c20201114185923116817_cspp_dev.h5

Because I am offline, so I choose -q=1. Then I download GHRSST and other and I put in dynamic_ancillary/2020_11_14_319 :
20201114120000-CMC-L4_GHRSST-SSTfnd-CMC0.1deg-GLOB-v02.0-fv03.0.nc
gfs.t00.201114.pgrbf12
gfs.t06.201114.pgrbf12
gfs.t12.201114.pgrbf12

I install centos 8 in virtualbox. RAM 6 GB.

Regard,
Mega
kathys
Posts: 487
Joined: Tue Jun 22, 2010 4:51 pm

Re: Fail running command on stderr.log

Post by kathys »

Hello Mega,

We think the problem is that you have not allocated enough memory in your Virtual Machine. The requirement for ACSPO is 8GB of RAM. We suggest you allocate 10 GB to start with. You can allows change to 8GB once you know it works.

Also, you should also use -q . You do not need to use -q=1. Using the -q flag means you do not want to download the external ancillary files. If you do not use -q, the software will try to download external files.

Good luck,

Kathy
megasaputra2
Posts: 7
Joined: Fri Nov 20, 2020 2:45 am

Re: Fail running command on stderr.log

Post by megasaputra2 »

Hello Kathy,

Now, I run in MX Linux 64 bit.
If I don't use -q, the result is like this :

mega@mega:~/Documents/CSPP_ACSPO_TESTDATA_1_2
$ run_acspo.bash -s npp -i input/jpss -w work/NPP -b bias -d dynamic_ancillary -p 4 -vv
--------------
DEBUG %options
==============
'clean' => 1,
'work' => '/home/mega/Documents/CSPP_ACSPO_TESTDATA_1_2/work/NPP',
'input_files' => 'input/jpss',
'quarantine' => 0,
'verbose' => 2,
'dynanc' => '/home/mega/Documents/CSPP_ACSPO_TESTDATA_1_2/dynamic_ancillary',
'nothing' => 0,
'granule' => 0,
'processors' => 4,
'ghrsst' => 'by_date',
'bias' => '/home/mega/Documents/CSPP_ACSPO_TESTDATA_1_2/bias',
'aggregate' => 1,
'xfiles' => 1,
'sat' => 'npp'
DEBUG %options _END_
run_acspo.pl:: Required utility lftp not found
Abort at /home/mega/Documents/CSPP_ACSPO_1_2/scripts/run_acspo.pl line 331.
mega@mega:~/Documents/CSPP_ACSPO_TESTDATA_1_2

While I have install lftp. You can see this :

mega@mega:~
$ lftp --version
LFTP | Version 4.8.4 | Copyright (c) 1996-2017 Alexander V. Lukyanov

LFTP is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with LFTP. If not, see <http://www.gnu.org/licenses/>.

Send bug reports and questions to the mailing list <lftp@uniyar.ac.ru>.

Libraries used: GnuTLS 3.6.7, idn2 2.0.5, Readline 7.0, zlib 1.2.11

So, I use -q. The result is like this :

mega@mega:~/Documents/CSPP_ACSPO_TESTDATA_1_2
$ run_acspo.bash -s npp -i input/jpss -w work/NPP -b bias -d dynamic_ancillary -p 4 -vv -q 1
--------------
DEBUG %options
==============
'granule' => 0,
'quarantine' => 1,
'xfiles' => 1,
'bias' => '/home/mega/Documents/CSPP_ACSPO_TESTDATA_1_2/bias',
'dynanc' => '/home/mega/Documents/CSPP_ACSPO_TESTDATA_1_2/dynamic_ancillary',
'processors' => 4,
'ghrsst' => 'by_date',
'verbose' => 2,
'nothing' => 0,
'input_files' => 'input/jpss',
'clean' => 1,
'work' => '/home/mega/Documents/CSPP_ACSPO_TESTDATA_1_2/work/NPP',
'aggregate' => 1,
'sat' => 'npp'
DEBUG %options _END_
chdir to working directory /home/mega/Documents/CSPP_ACSPO_TESTDATA_1_2/work/NPP/TMP_kt9ozV
run_acspo.pl::cspp_preprocess: Could not run command /home/mega/Documents/CSPP_ACSPO_1_2/bin/nagg -S -g GMODO -t SVM05,SVM07,SVM10,SVM12,SVM14,SVM15,SVM16 -n 1 -d disagg -O cspp -D dev links/* > stdout.nagg 2> stderr.nagg
Abort at /home/mega/Documents/CSPP_ACSPO_1_2/scripts/run_acspo.pl line 992.

About RAM, you can see this :

mega@mega:~
$ free -h
total used free shared buff/cache available
Mem: 31Gi 2.3Gi 335Mi 173Mi 28Gi 28Gi
Swap: 0B 0B 0B

So, what is the problem? I think, the requirement for RAM is enough.

Regard,
Mega
kathys
Posts: 487
Joined: Tue Jun 22, 2010 4:51 pm

Re: Fail running command on stderr.log

Post by kathys »

We are not sure why your execution of CSPP ACSPO is failing.

Have you ever been able to successfully execute the software?
Did you verify your installation by processing the verification data as described in Section 3.2 of the CSPP ACSPO Installation Guide?

The system requirements for CSPP ACSPO version 1.2 include:
  • CentOS-7 64-bit Linux
We try to test on other Operating Systems, but CentOS7 is the target machine. If possible, can you try to execute on a computer or Virtual Machine that is CentOS7 with at least 8 GB of RAM and lftp, perl and perl-core installed?

If you are still not successful, I suggest that you upload your input files to our anonymous ftp site: ftp://ftp.ssec.wisc.edu/pub/incoming

and we will try to process them here.

Good Luck,

Kathy
megasaputra2
Posts: 7
Joined: Fri Nov 20, 2020 2:45 am

Re: Fail running command on stderr.log

Post by megasaputra2 »

Dear all,

I try run in centos 8 on virtualbox with RAM 10 GB. It is successfull.
But, when I run in MX Linux 64 bit, it fail.
In other side, I try polar2grid om MX Linux. It is successful.
So, what is problem in ACSPO bash?

Mega
kathys
Posts: 487
Joined: Tue Jun 22, 2010 4:51 pm

Re: Fail running command on stderr.log

Post by kathys »

I am glad that you were able to get the ACSPO software to work on your CentOS8 computer.

We are not sure why it would not work on MX Linux, but I am afraid we are probably not going to be able to help you further without spending alot more time and effort to investigate now that you have a working version on CentOS8.

Kathy
Post Reply