Page 1 of 1

Drop-outs in RGB

Posted: Wed Oct 21, 2015 7:45 am
by arb
polar2grid has been mostly working fine but occasionally the RGB will exhibit a drop-out that's not present in the individually-extracted bands.
I've attached an example of the dropout (flipped vertically to match the direction of scans as seen by hdfview).
Also attached is the data as seen by hdfview from the CREFL file (CREFLM_npp_d20151019_t1335523_e1337165.hdf) where you can see (as indicated by red arrows) that the pixels do exist (I've checked both CREFLI and CREFLM).
Is there some way we can debug this?
Thanks for your help

Re: Drop-outs in RGB

Posted: Mon Oct 26, 2015 11:32 am
by kathys
Hi Andrew, we are investigating. Kathy

Re: Drop-outs in RGB

Posted: Mon Oct 26, 2015 11:41 am
by kathys
Hi Andrew, can you please stage your SDR data and your CREFL files for us (we only really need the CREFL and geolocation files for now). Feel free to tar them up and post on our anonymous ftp site if that is easiest for you:

Could you also please post or include the log file created during execution of P2G?



Re: Drop-outs in RGB

Posted: Fri Oct 30, 2015 8:54 am
by arb
(sorry for the delay, I didn't see the notification of your reply because it was in my spam folder!)

I'm now uploading the logs, the crefl files and the geo files to your ftp site.

Re: Drop-outs in RGB

Posted: Mon Nov 23, 2015 10:44 am
by davidh
Hi Andrew,

I finally had a chance to look at this and as far as I can tell this missing data is actually in the input files. I didn't have access to the original SDRs so I'm not sure if this is in the SDRs or is created by something in the crefl software. I was able to reproduce your results and I've made some PNGs of the CREFL files. I only see these drop outs in the M band and only in some bands. First is the CorrRefl_02 band (VIIRS SDR M07) that was what you looked at in your original post, you can see that it looks fine:
CorrRefl_02_20151019_133552.png (114.46 KiB) Viewed 2889 times
Second, is the CorrRefl_03 band (VIIRS SDR M03) which is actually used when making the true color:
CorrRefl_03_20151019_133552.png (107.52 KiB) Viewed 2889 times
As you can see near the top of the image there is some red striping where there are missing data values. These are images of just a single granule. If you'd like me to look in to this further I'll need the original SDRs that were used to generate the crefl hdf files. Otherwise, my answer right now is that this is missing data in the input files. The values in the crefl files for this bad data is 32767 which is the designated fill value. If the SDR band data looks fine my next guess would be that the crefl software is misinterpreting an SDR fill value or that the reflectance couldn't be corrected for some reason (bad SZA data, etc), but either way the data would still probably need to be marked as missing to produce an image.