VIIRS Flood Detection Processing Failed

VIIRS Environmental Data Records

VIIRS Flood Detection Processing Failed

Postby sverrem » Wed May 20, 2020 2:30 am

I have created a docker image with the CSPP EDR FLOOD Software.
Running the docker image works fine, but when I am trying to run it in Kubernetes it fails.

If anyone can look at the failed execution output and tell me what/why it is failing?

Since the ancillary data is around 12GB, I have moved this out of the image, and provided it as externally mounted resource
Code: Select all
docker run -e "SESSION=j01_d20200322_t1738241_e1751123_b00001" \
  --volume /drive/cspp/CSPP_FLOOD_1_1/ancillary:/cspp/CSPP_FLOOD_1_1/ancillary:rw
  --volume /drive/cspp/data:/cspp/data:rw
  cspp-flood:latest


Code: Select all
$CSPP_FLOOD_HOME/scripts/cspp_flood.sh --verbose --debug --output ${OUTPUT}/${SESSION} ${INPUT}/${SESSION}


Failed execution output:
Code: Select all
Produced   9 granules in SVI05_j01_d20200322_t1738255_e1751125_b12136_c20200520072003503483_cspp_dev.h5
DEBUG:__main__:is_day sz check: nday == 430, if > 0 we continue
INFO:__main__:input: /cspp-floods-phixbxvm/ GITCO_j01_d20200322_t1738255_e1751125_b12136_c20200520071856189095_cspp_dev.h5
DEBUG:__main__:VIIRS_Swath_Projection -i /cspp-floods-phixbxvm/ -g GITCO_j01_d20200322_t1738255_e1751125_b12136_c20200520071856189095_cspp_dev.h5 -o /cspp-floods-phixbxvm/ -l /cspp-floods-phixbxvm/ -n 0
DEBUG:__main__:/cspp-floods-phixbxvm/GITCO_Prj_SVI_j01_d20200322_t1738255_e1751125_b12136_cspp_dev.h5
DEBUG:__main__:VIIRS_Flood_Detection -h /cspp-floods-phixbxvm/ -v GITCO_Prj_SVI_j01_d20200322_t1738255_e1751125_b12136_cspp_dev.h5 -a /kspt/cspp/CSPP_FLOOD_1_1/ancillary -o . -l /cspp-floods-phixbxvm/
ERROR:__main__:Found 0 flood detection output files.
ERROR:__main__:While looking for: WATER_VIIRS_Prj_SVI_j01_d20200322_t1738255_e1751125_b12136_cspp_dev*.hdf
Traceback (most recent call last):
  File "/kspt/cspp/CSPP_FLOOD_1_1/bin/cspp_flood", line 295, in <module>
    main()
  File "/kspt/cspp/CSPP_FLOOD_1_1/bin/cspp_flood", line 274, in main
    water_file = water_files[0]
IndexError: list index out of range


Successful execution output:
Code: Select all
Produced   9 granules in SVI05_j01_d20200322_t1738255_e1751125_b12136_c20200520064806409835_cspp_dev.h5
DEBUG:__main__:is_day sz check: nday == 430, if > 0 we continue
INFO:__main__:input: /cspp-floods-25advr7k/ GITCO_j01_d20200322_t1738255_e1751125_b12136_c20200520064709298686_cspp_dev.h5
DEBUG:__main__:VIIRS_Swath_Projection -i /cspp-floods-25advr7k/ -g GITCO_j01_d20200322_t1738255_e1751125_b12136_c20200520064709298686_cspp_dev.h5 -o /cspp-floods-25advr7k/ -l /cspp-floods-25advr7k/ -n 0
DEBUG:__main__:/cspp-floods-25advr7k/GITCO_Prj_SVI_j01_d20200322_t1738255_e1751125_b12136_cspp_dev.h5
DEBUG:__main__:VIIRS_Flood_Detection -h /cspp-floods-25advr7k/ -v GITCO_Prj_SVI_j01_d20200322_t1738255_e1751125_b12136_cspp_dev.h5 -a /kspt/cspp/CSPP_FLOOD_1_1/ancillary -o . -l /cspp-floods-25advr7k/
DEBUG:__main__:Moving WATER_VIIRS_Prj_SVI_j01_d20200322_t1738255_e1751125_b12136_cspp_dev_19351_14530_01.hdf -> /kspt/cspp/data/flood/output/j01_d20200322_t1738241_e1751123_b00001
sverrem
 
Posts: 7
Joined: Tue May 19, 2020 4:38 pm

Re: VIIRS Flood Detection Processing Failed

Postby kathys » Thu May 21, 2020 11:28 am

Hello. We have looked at your deployment, and believe this is a bit outside the scope of CSPP. It seems that you were able to successfully install the software in your container. The problems is with the Kubernetes deployment.

We suggest that you examine the log files:

VIIRS_Flood_Detection_Log.txt
flood_detection.log

to see if there is anything that can explain why your execution was not successful.

Kathy
kathys
 
Posts: 291
Joined: Tue Jun 22, 2010 4:51 pm

Re: VIIRS Flood Detection Processing Failed

Postby sverrem » Fri May 22, 2020 3:41 am

Running in docker environment works just fine.
It is probably the Kubernetes deployment that is the problem.

However, I cannot find any problem with my kubernetes manifest for CSPP FLOOD.
I have the same deployment for CSPP SDR witch works in Kubernetes.

I was hoping something from the FLOOD v1.1 SW output could explain why it does not work in Kubernetes.
sverrem
 
Posts: 7
Joined: Tue May 19, 2020 4:38 pm


Return to VIIRS EDRs

Who is online

Users browsing this forum: No registered users and 1 guest

cron