Page 1 of 1

VIIRS Flood Detection Processing Failed

PostPosted: Wed May 20, 2020 2:30 am
by sverrem
I have created a docker image with the CSPP EDR FLOOD Software.
Running the docker image works fine, but when I am trying to run it in Kubernetes it fails.

If anyone can look at the failed execution output and tell me what/why it is failing?

Since the ancillary data is around 12GB, I have moved this out of the image, and provided it as externally mounted resource
Code: Select all
docker run -e "SESSION=j01_d20200322_t1738241_e1751123_b00001" \
  --volume /drive/cspp/CSPP_FLOOD_1_1/ancillary:/cspp/CSPP_FLOOD_1_1/ancillary:rw
  --volume /drive/cspp/data:/cspp/data:rw
  cspp-flood:latest


Code: Select all
$CSPP_FLOOD_HOME/scripts/cspp_flood.sh --verbose --debug --output ${OUTPUT}/${SESSION} ${INPUT}/${SESSION}


Failed execution output:
Code: Select all
Produced   9 granules in SVI05_j01_d20200322_t1738255_e1751125_b12136_c20200520072003503483_cspp_dev.h5
DEBUG:__main__:is_day sz check: nday == 430, if > 0 we continue
INFO:__main__:input: /cspp-floods-phixbxvm/ GITCO_j01_d20200322_t1738255_e1751125_b12136_c20200520071856189095_cspp_dev.h5
DEBUG:__main__:VIIRS_Swath_Projection -i /cspp-floods-phixbxvm/ -g GITCO_j01_d20200322_t1738255_e1751125_b12136_c20200520071856189095_cspp_dev.h5 -o /cspp-floods-phixbxvm/ -l /cspp-floods-phixbxvm/ -n 0
DEBUG:__main__:/cspp-floods-phixbxvm/GITCO_Prj_SVI_j01_d20200322_t1738255_e1751125_b12136_cspp_dev.h5
DEBUG:__main__:VIIRS_Flood_Detection -h /cspp-floods-phixbxvm/ -v GITCO_Prj_SVI_j01_d20200322_t1738255_e1751125_b12136_cspp_dev.h5 -a /kspt/cspp/CSPP_FLOOD_1_1/ancillary -o . -l /cspp-floods-phixbxvm/
ERROR:__main__:Found 0 flood detection output files.
ERROR:__main__:While looking for: WATER_VIIRS_Prj_SVI_j01_d20200322_t1738255_e1751125_b12136_cspp_dev*.hdf
Traceback (most recent call last):
  File "/kspt/cspp/CSPP_FLOOD_1_1/bin/cspp_flood", line 295, in <module>
    main()
  File "/kspt/cspp/CSPP_FLOOD_1_1/bin/cspp_flood", line 274, in main
    water_file = water_files[0]
IndexError: list index out of range


Successful execution output:
Code: Select all
Produced   9 granules in SVI05_j01_d20200322_t1738255_e1751125_b12136_c20200520064806409835_cspp_dev.h5
DEBUG:__main__:is_day sz check: nday == 430, if > 0 we continue
INFO:__main__:input: /cspp-floods-25advr7k/ GITCO_j01_d20200322_t1738255_e1751125_b12136_c20200520064709298686_cspp_dev.h5
DEBUG:__main__:VIIRS_Swath_Projection -i /cspp-floods-25advr7k/ -g GITCO_j01_d20200322_t1738255_e1751125_b12136_c20200520064709298686_cspp_dev.h5 -o /cspp-floods-25advr7k/ -l /cspp-floods-25advr7k/ -n 0
DEBUG:__main__:/cspp-floods-25advr7k/GITCO_Prj_SVI_j01_d20200322_t1738255_e1751125_b12136_cspp_dev.h5
DEBUG:__main__:VIIRS_Flood_Detection -h /cspp-floods-25advr7k/ -v GITCO_Prj_SVI_j01_d20200322_t1738255_e1751125_b12136_cspp_dev.h5 -a /kspt/cspp/CSPP_FLOOD_1_1/ancillary -o . -l /cspp-floods-25advr7k/
DEBUG:__main__:Moving WATER_VIIRS_Prj_SVI_j01_d20200322_t1738255_e1751125_b12136_cspp_dev_19351_14530_01.hdf -> /kspt/cspp/data/flood/output/j01_d20200322_t1738241_e1751123_b00001

Re: VIIRS Flood Detection Processing Failed

PostPosted: Thu May 21, 2020 11:28 am
by kathys
Hello. We have looked at your deployment, and believe this is a bit outside the scope of CSPP. It seems that you were able to successfully install the software in your container. The problems is with the Kubernetes deployment.

We suggest that you examine the log files:

VIIRS_Flood_Detection_Log.txt
flood_detection.log

to see if there is anything that can explain why your execution was not successful.

Kathy

Re: VIIRS Flood Detection Processing Failed

PostPosted: Fri May 22, 2020 3:41 am
by sverrem
Running in docker environment works just fine.
It is probably the Kubernetes deployment that is the problem.

However, I cannot find any problem with my kubernetes manifest for CSPP FLOOD.
I have the same deployment for CSPP SDR witch works in Kubernetes.

I was hoping something from the FLOOD v1.1 SW output could explain why it does not work in Kubernetes.