Page 1 of 2

Obtain and convert daily VIIRS DNB hdf SDR files to geotiffs

PostPosted: Fri Aug 31, 2018 12:36 pm
by leonidas
Is there any step-by-step turorial to generate daily VIIRS DNB geotiffs from hdf5 SDR files. I need them for time series analysis.
There is so much information about VIIRS data. Could anyone point me to the right direction?
Where can I download appropriate daily VIIRS DNB SDR data ?
Which dataset/files and how to convert hdf5 SDR hdf5 files to geotiffs?

Re: Obtain and convert daily VIIRS DNB hdf SDR files to geot

PostPosted: Thu Sep 06, 2018 2:40 pm
by kathys
The CSPP Polar2Grid package can create GeoTIFF outputs from input VIIRS SDRs.

http://cimss.ssec.wisc.edu/cspp/npp_pol ... .2.1.shtml

It has a few different options for enhancing the output DNB images, however, the output is stored in 8 bit 0-255 "brightness" values. The GeoTIFF will not contain geophysical science data. I would happy to post instructions on how you would go about creating DNB images if this 8 bit GeoTIFF format is what you are looking for. Just let me know.

Kathy

Re: Obtain and convert daily VIIRS DNB hdf SDR files to geot

PostPosted: Thu Sep 06, 2018 4:51 pm
by kathys
Sorry, I forgot that you can create reprojected HDF5 radiance output files from VIIRS DNB SDRs using Polar2Grid. Again, if this is something you are interested in doing, then let me know.

Kathy

Re: Obtain and convert daily VIIRS DNB hdf SDR files to geot

PostPosted: Fri Sep 07, 2018 12:38 pm
by leonidas
Thank you! I created some 8-bit geotiffs using Polar2Grid.
I also tried satpy to generate floating point geotiffs. But the generated tiffs have no CRS assigned.
Code available at:https://gist.github.com/kokkytos/2bacd2b04181f5df3b170eb7fea6c023
Should I use pyresample to resample satpy swath data to a grid? Or any other step?
Thank you again!

Re: Obtain and convert daily VIIRS DNB hdf SDR files to geot

PostPosted: Mon Sep 10, 2018 4:21 pm
by davidh
Hi,

I'm the lead developer on polar2grid, satpy, and pyresample so hopefully I can help a little. The reason your geotiff generated with satpy does not include any CRS information is because the data has not been resampled yet. The VIIRS SDR "raw" data is unprojected and ungridded so it can't be represented properly by a geotiff. A geotiff needs pixels to be evenly spaced which VIIRS swath data is not. To resample data you can add something like the following after your call to `.load` and before `.save_datasets`:

new_scn = scene.resample(area_def)
new_scn.save_datasets(...)

Where 'area_def' is a pyresample AreaDefinition object. For more information on that I would recommend reading the pyresample documentation (https://pyresample.readthedocs.io/en/latest/) or look at the various satpy examples (https://satpy.readthedocs.io/en/latest/examples.html). Using SatPy instead of polar2grid will give you more control over what and how things are generated, but polar2grid will provide a simpler interface. Since it seems like you are comfortable coding a python script and using github the choice is up to you. You can also join the pytroll slack team (mentioned under "getting in touch": http://pytroll.github.io/) and the rest of the satpy team can help you out. Or you can file a bug on github.

Polar2Grid should allow you to create float geotiffs as well by specifying `--dtype real4` on the command line.

Re: Obtain and convert daily VIIRS DNB hdf SDR files to geot

PostPosted: Wed Sep 12, 2018 12:08 pm
by leonidas
Thank you! I projected my data. I defined ETRS89/LAEA as follows:

Code: Select all

area_id = 'etrs89-etrs-laea'
description = 'ETRS89 / ETRS-LAEA'
proj_id = 'etrs89_3035'
x_size = 3867
y_size = 3918
area_extent = (2426378.0132, 1528101.2618, 6293974.6215, 5446513.5222)
proj_dict = {'units': 'm',
             'proj': 'laea',
             'lat_0': 52,
             'lon_0': 10,
             'x_0' : 4321000,
             'y_0' : 3210000,
             'ellps':'GRS80'
             }
area_def = geometry.AreaDefinition(area_id, description, proj_id, proj_dict, x_size, y_size, area_extent)
newscene = scene.resample(area_def)
newscene.save_dataset("DNB", filename='./dnb.tif')

Is it possible to enhance my DNB image with satpy (I mean Adaptive, Dynamic Simplified HNCC scaling like Polar2Grid)?
Also, is it possible to save my DNB data as radiance geotiff file using satpy?

P.S. Do I need an invitation for https://pytroll.slack.com/ ?

Re: Obtain and convert daily VIIRS DNB hdf SDR files to geot

PostPosted: Wed Sep 12, 2018 12:13 pm
by davidh
The pytroll page had a link to invite yourself to the slack team. The link is: https://pytrollslackin.herokuapp.com/

All of the DNB enhancements available in P2G are available in satpy, usually under the same or similar name. You can use `scn.available_composite_names()` to get the full list of available composites. If I recall correctly you should see "adaptive_dnb", "histogram_dnb", "dynamic_dnb" ("erf_dnb"?), and maybe "hncc_dnb". You can save a float geotiff in satpy, see the documentation for the geotiff writer for more details (sorry for the bad HTML formatting): https://satpy.readthedocs.io/en/latest/ ... TIFFWriter

Re: Obtain and convert daily VIIRS DNB hdf SDR files to geot

PostPosted: Mon Sep 24, 2018 1:05 pm
by leonidas
davidh wrote:The pytroll page had a link to invite yourself to the slack team. The link is: https://pytrollslackin.herokuapp.com/

All of the DNB enhancements available in P2G are available in satpy, usually under the same or similar name. You can use `scn.available_composite_names()` to get the full list of available composites. If I recall correctly you should see "adaptive_dnb", "histogram_dnb", "dynamic_dnb" ("erf_dnb"?), and maybe "hncc_dnb". You can save a float geotiff in satpy, see the documentation for the geotiff writer for more details (sorry for the bad HTML formatting): https://satpy.readthedocs.io/en/latest/ ... TIFFWriter

So, setting dtype=np.float32 to geotiff writer will export radiance values as W cm^-2 sr^-1? Did the writer apply any scaling to output values during export? Because it doesn't return values in the order of magnitude of 10^-9

Re: Obtain and convert daily VIIRS DNB hdf SDR files to geot

PostPosted: Fri Sep 28, 2018 9:57 am
by davidh
You need to also set the
Code: Select all
enhancement_config=False
so the full call would look something like:

Code: Select all
scn.save_datasets(writer=’geotiff’, dtype=np.float32, enhancement_config=False)

Re: Obtain and convert daily VIIRS DNB hdf SDR files to geot

PostPosted: Fri Oct 26, 2018 4:03 am
by PearceRJ
kathys wrote:Sorry, I forgot that you can create reprojected HDF5 radiance output files from VIIRS DNB SDRs using Polar2Grid. Again, if this is something you are interested in doing, then let me know.

Kathy

Hi Kathy, I am trying to do this also but cannot get satpy to work, please could you explain how to do this :)