VIIRS Flood Detection Processing Failed

VIIRS Environmental Data Records
Post Reply
sverrem
Posts: 10
Joined: Tue May 19, 2020 4:38 pm

VIIRS Flood Detection Processing Failed

Post by sverrem »

I have created a docker image with the CSPP EDR FLOOD Software.
Running the docker image works fine, but when I am trying to run it in Kubernetes it fails.

If anyone can look at the failed execution output and tell me what/why it is failing?

Since the ancillary data is around 12GB, I have moved this out of the image, and provided it as externally mounted resource

Code: Select all

docker run -e "SESSION=j01_d20200322_t1738241_e1751123_b00001" \
  --volume /drive/cspp/CSPP_FLOOD_1_1/ancillary:/cspp/CSPP_FLOOD_1_1/ancillary:rw
  --volume /drive/cspp/data:/cspp/data:rw
  cspp-flood:latest

Code: Select all

$CSPP_FLOOD_HOME/scripts/cspp_flood.sh --verbose --debug --output ${OUTPUT}/${SESSION} ${INPUT}/${SESSION}
Failed execution output:

Code: Select all

Produced   9 granules in SVI05_j01_d20200322_t1738255_e1751125_b12136_c20200520072003503483_cspp_dev.h5
DEBUG:__main__:is_day sz check: nday == 430, if > 0 we continue
INFO:__main__:input: /cspp-floods-phixbxvm/ GITCO_j01_d20200322_t1738255_e1751125_b12136_c20200520071856189095_cspp_dev.h5
DEBUG:__main__:VIIRS_Swath_Projection -i /cspp-floods-phixbxvm/ -g GITCO_j01_d20200322_t1738255_e1751125_b12136_c20200520071856189095_cspp_dev.h5 -o /cspp-floods-phixbxvm/ -l /cspp-floods-phixbxvm/ -n 0
DEBUG:__main__:/cspp-floods-phixbxvm/GITCO_Prj_SVI_j01_d20200322_t1738255_e1751125_b12136_cspp_dev.h5
DEBUG:__main__:VIIRS_Flood_Detection -h /cspp-floods-phixbxvm/ -v GITCO_Prj_SVI_j01_d20200322_t1738255_e1751125_b12136_cspp_dev.h5 -a /kspt/cspp/CSPP_FLOOD_1_1/ancillary -o . -l /cspp-floods-phixbxvm/
ERROR:__main__:Found 0 flood detection output files.
ERROR:__main__:While looking for: WATER_VIIRS_Prj_SVI_j01_d20200322_t1738255_e1751125_b12136_cspp_dev*.hdf
Traceback (most recent call last):
  File "/kspt/cspp/CSPP_FLOOD_1_1/bin/cspp_flood", line 295, in <module>
    main()
  File "/kspt/cspp/CSPP_FLOOD_1_1/bin/cspp_flood", line 274, in main
    water_file = water_files[0]
IndexError: list index out of range
Successful execution output:

Code: Select all

Produced   9 granules in SVI05_j01_d20200322_t1738255_e1751125_b12136_c20200520064806409835_cspp_dev.h5
DEBUG:__main__:is_day sz check: nday == 430, if > 0 we continue
INFO:__main__:input: /cspp-floods-25advr7k/ GITCO_j01_d20200322_t1738255_e1751125_b12136_c20200520064709298686_cspp_dev.h5
DEBUG:__main__:VIIRS_Swath_Projection -i /cspp-floods-25advr7k/ -g GITCO_j01_d20200322_t1738255_e1751125_b12136_c20200520064709298686_cspp_dev.h5 -o /cspp-floods-25advr7k/ -l /cspp-floods-25advr7k/ -n 0
DEBUG:__main__:/cspp-floods-25advr7k/GITCO_Prj_SVI_j01_d20200322_t1738255_e1751125_b12136_cspp_dev.h5
DEBUG:__main__:VIIRS_Flood_Detection -h /cspp-floods-25advr7k/ -v GITCO_Prj_SVI_j01_d20200322_t1738255_e1751125_b12136_cspp_dev.h5 -a /kspt/cspp/CSPP_FLOOD_1_1/ancillary -o . -l /cspp-floods-25advr7k/
DEBUG:__main__:Moving WATER_VIIRS_Prj_SVI_j01_d20200322_t1738255_e1751125_b12136_cspp_dev_19351_14530_01.hdf -> /kspt/cspp/data/flood/output/j01_d20200322_t1738241_e1751123_b00001
kathys
Posts: 487
Joined: Tue Jun 22, 2010 4:51 pm

Re: VIIRS Flood Detection Processing Failed

Post by kathys »

Hello. We have looked at your deployment, and believe this is a bit outside the scope of CSPP. It seems that you were able to successfully install the software in your container. The problems is with the Kubernetes deployment.

We suggest that you examine the log files:

VIIRS_Flood_Detection_Log.txt
flood_detection.log

to see if there is anything that can explain why your execution was not successful.

Kathy
sverrem
Posts: 10
Joined: Tue May 19, 2020 4:38 pm

Re: VIIRS Flood Detection Processing Failed

Post by sverrem »

Running in docker environment works just fine.
It is probably the Kubernetes deployment that is the problem.

However, I cannot find any problem with my kubernetes manifest for CSPP FLOOD.
I have the same deployment for CSPP SDR witch works in Kubernetes.

I was hoping something from the FLOOD v1.1 SW output could explain why it does not work in Kubernetes.
Post Reply