SSEC forums

discussion forums for the Space Science & Engineering Center at UW-Madison
https://forums.ssec.wisc.edu/

6.2 CSPP MiRS Microwave Retrieval TEST File

https://forums.ssec.wisc.edu/viewtopic.php?t=59316

Page 1 of 1

6.2 CSPP MiRS Microwave Retrieval TEST File

Posted: Thu Nov 18, 2021 8:15 am
by rdirienzo
Hello everyone,
I have a problem with the tests defined in the manual:
https://bin.ssec.wisc.edu/pub/CSPP/hidd ... e_v2.4.pdf --> 3.3 Running CSPP_MIRS Test Cases

Code: Select all

[user@machine work]# ../scripts/verify_mirs.bash

Run this in CSPP_MIRS_TESTDATA_2_4/work
---------------------------------------
Typical total runtime for 6 missions is 1 to 2 hours on single CPU
For 8 CPUs, runtime for 6 missions might be about 10 to 20 minutes
Running: run_mirs.bash -v -q -d ../dynanc -s metopA -i ../input -p 8 > metopA.log
Running: run_mirs.bash -v -q -d ../dynanc -s metopB -i ../input -p 8 > metopB.log
Running: run_mirs.bash -v -q -d ../dynanc -s metopC -i ../input -p 8 > metopC.log
Running: run_mirs.bash -v -q -d ../dynanc -s n19 -i ../input -p 8 > n19.log
Running: run_mirs.bash -v -q -d ../dynanc -s npp --noagg -i ../input -p 8 > npp.log
Running: run_mirs.bash -v -q -d ../dynanc -s n20 -i ../input -p 8 > n20.log

Begin verification
------------------
Verifying: metopA
Verifying: metopB
Verifying: metopC
Verifying: n19
compare_mirs.pl:: Number of user-generated and test files do not match
FAIL at ../scripts/compare_mirs.pl line 120.
Verifying: npp
Verifying: n20
envs:

Code: Select all

XDG_SESSION_ID=365
HOSTNAME=xxxxxxxx
SELINUX_ROLE_REQUESTED=
TERM=screen
SHELL=/bin/bash
HISTSIZE=1000
SSH_CLIENT=93.41.252.46 62270 22
SELINUX_USE_CURRENT_RANGE=
OLDPWD=/opt/CSPP_MIRS_TESTDATA_2_4
SSH_TTY=/dev/pts/0
CSPP_MIRS_REVDATE=20201005
USER=root
LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=01;05;37;41:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.axv=01;35:*.anx=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=01;36:*.au=01;36:*.flac=01;36:*.mid=01;36:*.midi=01;36:*.mka=01;36:*.mp3=01;36:*.mpc=01;36:*.ogg=01;36:*.ra=01;36:*.wav=01;36:*.axa=01;36:*.oga=01;36:*.spx=01;36:*.xspf=01;36:
LD_LIBRARY_PATH=:/common/local/lib/:/common/local/lib/
JPSS_REMOTE_ANC_DIR=https://jpssdb.ssec.wisc.edu/cspp_v_2_0/ancillary
CSPP_MIRS_REVNAME=DAP/mirs_v11r6_r110620219
TMUX=/tmp/tmux-0/default,16496,0
PATH=/opt/CSPP/CSPP_MIRS_2_4/scripts:/usr/local/go/bin:/usr/local/go/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/go/bin:/root/bin:/root/go/bin:/root/bin
MAIL=/var/spool/mail/root
HDF5_DISABLE_VERSION_CHECK=2
PWD=/opt/CSPP_MIRS_TESTDATA_2_4/work
LANG=en_US.UTF-8
TMUX_PANE=%0
CSPP_MIRS_HOME=/opt/CSPP/CSPP_MIRS_2_4
SELINUX_LEVEL_REQUESTED=
CSPP_DYNAMIC_ANCIL_DIR=/opt/CSPP/CSPP_MIRS_2_4/data/dynamic_ancillary
HISTCONTROL=ignoredups
SHLVL=2
LESSOPEN=||/usr/bin/lesspipe.sh %s
GOPATH=/root/go
XDG_RUNTIME_DIR=/run/user/0
_=/usr/bin/env

Code: Select all

top - 13:10:43 up 12 days, 19:16,  1 user,  load average: 0.00, 0.01, 0.05
Tasks: 123 total,   2 running, 121 sleeping,   0 stopped,   0 zombie
%Cpu(s):  0.1 us,  0.0 sy,  0.0 ni, 99.9 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem : 32005448 total, 26188708 free,   420324 used,  5396416 buff/cache
KiB Swap:        0 total,        0 free,        0 used. 30865248 avail Mem
Thanks for your help and time,
Roberto

Re: 6.2 CSPP MiRS Microwave Retrieval TEST File

Posted: Thu Nov 18, 2021 12:11 pm
by jimd
Hi Roberto,
I have been unable to reproduce the error.
Please can you do the following for me?
1) Advise me of the OS that you are using.
2) Upload the n19.log file that was generated by the verification script.
Cheers,
-jim.

Re: 6.2 CSPP MiRS Microwave Retrieval TEST File

Posted: Thu Nov 18, 2021 2:09 pm
by jimd
One last thing that would be helpful, Roberto.
3) The listing from command: $ ls -l output/ work/
-jim

Re: 6.2 CSPP MiRS Microwave Retrieval TEST File

Posted: Mon Nov 22, 2021 5:40 am
by rdirienzo
Ciao Jim,
thanks for your help:
I'm using Centos7

Code: Select all

[root@xxxxCSPP_MIRS_TESTDATA_2_4]# uname -a
Linux xxxx3.10.0-1160.45.1.el7.x86_64 #1 SMP Wed Oct 13 17:20:51 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Code: Select all

[root@xxxxx CSPP_MIRS_TESTDATA_2_4]#  ls -l output/ work/
output/:
total 283276
-rw-r--r--. 1 53485 700    82302 Sep 25  2020 metopA.log
-rw-r--r--. 1 53485 700     3943 Sep 25  2020 metopA_logFile_M2.D20238.S1344.E1349.B7187171.WE.dat
-rw-r--r--. 1 53485 700    82285 Sep 25  2020 metopB.log
-rw-r--r--. 1 53485 700     3943 Sep 25  2020 metopB_logFile_M1.D20238.S1453.E1504.B4118585.WE.dat
-rw-r--r--. 1 53485 700    79945 Sep 25  2020 metopC.log
-rw-r--r--. 1 53485 700       52 Sep 25  2020 metopC_logFile_M3.D20238.S1408.E1415.B0934242.WE.dat
-rw-r--r--. 1 53485 700    85473 Sep 25  2020 n19.log
-rw-r--r--. 1 53485 700       52 Sep 25  2020 n19_logFile_NP.D20238.S1441.E1447.B5951616.WE.dat
-rw-r--r--. 1 53485 700    77007 Sep 25  2020 n20.log
-rw-r--r--. 1 53485 700       52 Sep 25  2020 n20_logFile_d20200825_t1715226_e1723223_b14349.dat
-rw-r--r--. 1 53485 700    80459 Sep 25  2020 npp.log
-rw-r--r--. 1 53485 700       52 Sep 25  2020 npp_logFile_d20200825_t1803546_e1814343_b45746.dat
-rw-r--r--. 1 53485 700  5591500 Sep 25  2020 NPR-MIRS-IMG_v11r6_ma1_s202008251453446_e202008251504300_c202009252230150.nc
-rw-r--r--. 1 53485 700  2185900 Sep 25  2020 NPR-MIRS-IMG_v11r6_ma2_s202008251344555_e202008251349009_c202009252227310.nc
-rw-r--r--. 1 53485 700  3139468 Sep 25  2020 NPR-MIRS-IMG_v11r6_ma3_s202008251408469_e202008251414441_c202009252232450.nc
-rw-r--r--. 1 53485 700  2730796 Sep 25  2020 NPR-MIRS-IMG_v11r6_n19_s202008251441526_e202008251447020_c202009252234280.nc
-rw-r--r--. 1 53485 700  4689502 Sep 25  2020 NPR-MIRS-IMG_v11r6_n20_s202008251715226_e202008251723223_c202009252240470.nc
-rw-r--r--. 1 53485 700  6234622 Sep 25  2020 NPR-MIRS-IMG_v11r6_npp_s202008251803546_e202008251814343_c202009252237480.nc
-rw-r--r--. 1 53485 700 62131392 Sep 25  2020 NPR-MIRS-SND_v11r6_ma1_s202008251453446_e202008251504300_c202009252230150.nc
-rw-r--r--. 1 53485 700 23814792 Sep 25  2020 NPR-MIRS-SND_v11r6_ma2_s202008251344555_e202008251349009_c202009252227310.nc
-rw-r--r--. 1 53485 700 34543440 Sep 25  2020 NPR-MIRS-SND_v11r6_ma3_s202008251408469_e202008251414441_c202009252232450.nc
-rw-r--r--. 1 53485 700 29945448 Sep 25  2020 NPR-MIRS-SND_v11r6_n19_s202008251441526_e202008251447020_c202009252234280.nc
-rw-r--r--. 1 53485 700 49084126 Sep 25  2020 NPR-MIRS-SND_v11r6_n20_s202008251715226_e202008251723223_c202009252240470.nc
-rw-r--r--. 1 53485 700 65432446 Sep 25  2020 NPR-MIRS-SND_v11r6_npp_s202008251803546_e202008251814343_c202009252237480.nc
-rw-r--r--. 1 53485 700      486 Sep 25  2020 timing.txt

work/:
total 315176
-rw-r--r--. 1 root root    79500 Nov 18 11:40 metopA.log
-rw-r--r--. 1 root root     3943 Nov 18 11:40 metopA_logFile_M2.D20238.S1344.E1349.B7187171.WE.dat
-rw-r--r--. 1 root root    79535 Nov 18 11:44 metopB.log
-rw-r--r--. 1 root root     3943 Nov 18 11:44 metopB_logFile_M1.D20238.S1453.E1504.B4118585.WE.dat
-rw-r--r--. 1 root root    77208 Nov 18 11:47 metopC.log
-rw-r--r--. 1 root root       52 Nov 18 11:47 metopC_logFile_M3.D20238.S1408.E1415.B0934242.WE.dat
-rw-r--r--. 1 root root    82515 Nov 18 11:49 n19.log
-rw-r--r--. 1 root root       52 Nov 18 11:49 n19_logFile_NP.D20238.S1441.E1447.B5951616.WE.dat
-rw-r--r--. 1 root root    74491 Nov 18 11:58 n20.log
-rw-r--r--. 1 root root       52 Nov 18 11:58 n20_logFile_d20200825_t1715226_e1723223_b14349.dat
-rw-r--r--. 1 root root    77930 Nov 18 11:54 npp.log
-rw-r--r--. 1 root root       52 Nov 18 11:54 npp_logFile_d20200825_t1803546_e1814343_b45746.dat
-rw-r--r--. 1 root root  5591500 Nov 18 11:44 NPR-MIRS-IMG_v11r6_ma1_s202008251453446_e202008251504300_c202111181144190.nc
-rw-r--r--. 1 root root  2185900 Nov 18 11:40 NPR-MIRS-IMG_v11r6_ma2_s202008251344555_e202008251349009_c202111181140490.nc
-rw-r--r--. 1 root root  3139468 Nov 18 11:47 NPR-MIRS-IMG_v11r6_ma3_s202008251408469_e202008251414441_c202111181147300.nc
-rw-r--r--. 1 root root  2730796 Nov 18 11:36 NPR-MIRS-IMG_v11r6_n19_s202008251441526_e202008251447020_c202111181136380.nc
-rw-r--r--. 1 root root  2730796 Nov 18 11:49 NPR-MIRS-IMG_v11r6_n19_s202008251441526_e202008251447020_c202111181149390.nc
-rw-r--r--. 1 root root  4689502 Nov 18 11:58 NPR-MIRS-IMG_v11r6_n20_s202008251715226_e202008251723223_c202111181158270.nc
-rw-r--r--. 1 root root  6234622 Nov 18 11:54 NPR-MIRS-IMG_v11r6_npp_s202008251803546_e202008251814343_c202111181154190.nc
-rw-r--r--. 1 root root 62131392 Nov 18 11:44 NPR-MIRS-SND_v11r6_ma1_s202008251453446_e202008251504300_c202111181144200.nc
-rw-r--r--. 1 root root 23814792 Nov 18 11:40 NPR-MIRS-SND_v11r6_ma2_s202008251344555_e202008251349009_c202111181140500.nc
-rw-r--r--. 1 root root 34543440 Nov 18 11:47 NPR-MIRS-SND_v11r6_ma3_s202008251408469_e202008251414441_c202111181147300.nc
-rw-r--r--. 1 root root 29945448 Nov 18 11:36 NPR-MIRS-SND_v11r6_n19_s202008251441526_e202008251447020_c202111181136380.nc
-rw-r--r--. 1 root root 29945448 Nov 18 11:49 NPR-MIRS-SND_v11r6_n19_s202008251441526_e202008251447020_c202111181149390.nc
-rw-r--r--. 1 root root 49084126 Nov 18 11:58 NPR-MIRS-SND_v11r6_n20_s202008251715226_e202008251723223_c202111181158270.nc
-rw-r--r--. 1 root root 65432446 Nov 18 11:54 NPR-MIRS-SND_v11r6_npp_s202008251803546_e202008251814343_c202111181154190.nc
-rw-r--r--. 1 root root      486 Nov 18 11:58 timing.txt
Again,
Thanks for your help!!
Roberto.

Re: 6.2 CSPP MiRS Microwave Retrieval TEST File

Posted: Mon Nov 22, 2021 11:56 am
by jimd
Hi Roberto,

In the work directory I see files from TWO runs of CSPP_MIRS for n19:

-rw-r--r--. 1 root root 2730796 Nov 18 11:36 NPR-MIRS-IMG_v11r6_n19_s202008251441526_e202008251447020_c202111181136380.nc
-rw-r--r--. 1 root root 2730796 Nov 18 11:49 NPR-MIRS-IMG_v11r6_n19_s202008251441526_e202008251447020_c202111181149390.nc

One run at 11:36, the other at 11:49. The verification script is not very smart - it expects there to be one file in work/ (that you generate when you run the verification script) for each matching file in output/ (that I supplied).

So the question is: Why is there output for TWO runs for n19 in the work/ directory? Is it possible you did a test run for just n19, with output directed to work/, before you ran the verification script? If so, you can just delete the work/ directory and start again.

If not, I am puzzled still.

Cheers,

-jim.
All times are UTC-05:00
Page 1 of 1

Powered by phpBB® Forum Software © phpBB Limited