No filenames found for reader ahi_hsd

torresrodyvito · Post by **torresrodyvito** » Wed Jul 24, 2019 3:10 am

Is this the .log file you're talking about. If so, this is an example of a .log file using the code for "corrected" band you said.

Code: Select all

[2019-07-04 14:21:45,692] : PID     30 : DEBUG    : ahi_hsd_geotiff : main : Starting script with arguments: /mnt/c/Users/BlueCARES/Downloads/geo2grid_v_1_0_0_0/lib/python3.6/site-packages/polar2grid/glue.py -r ahi_hsd -w geotiff -p B01 B02 B03 B04 B05 -f trial2 -vv
[2019-07-04 14:21:45,696] : PID     30 : INFO     : ahi_hsd_geotiff : main : Sorting and reading input files...
[2019-07-04 14:21:45,697] : PID     30 : DEBUG    : satpy.scene : __init__ : Setting 'PPP_CONFIG_DIR' to '/mnt/c/Users/BlueCARES/Downloads/geo2grid_v_1_0_0_0/etc/satpy'
[2019-07-04 14:21:45,698] : PID     30 : DEBUG    : satpy.readers : read_reader_config : Reading ['/mnt/c/Users/BlueCARES/Downloads/geo2grid_v_1_0_0_0/lib/python3.6/site-packages/satpy/etc/readers/ahi_hsd.yaml']
[2019-07-04 14:21:45,785] : PID     30 : DEBUG    : satpy.readers.yaml_reader : create_filehandlers : Assigning to ahi_hsd: ['trial2/HS_H08_20181015_0200_B01_FLDK_R10_S0410.DAT', 'trial2/HS_H08_20181015_0200_B02_FLDK_R10_S0410.DAT', 'trial2/HS_H08_20181015_0200_B03_FLDK_R05_S0410.DAT', 'trial2/HS_H08_20181015_0200_B04_FLDK_R10_S0410.DAT']
[2019-07-04 14:21:45,802] : PID     30 : DEBUG    : satpy.composites : load_sensor_composites : Looking for composites config file ahi.yaml
[2019-07-04 14:21:45,820] : PID     30 : DEBUG    : satpy.composites : load_sensor_composites : Looking for composites config file visir.yaml
[2019-07-04 14:21:45,855] : PID     30 : DEBUG    : polar2grid.core.script_utils : rename_log_file : Log renamed from '/home/bluecares/ahi_hsd_geotiff_fail.log' to 'ahi_hsd_geotiff_20181015_020236.log'
[2019-07-04 14:21:45,855] : PID     30 : INFO     : ahi_hsd_geotiff : main : Loading product metadata from files...
[2019-07-04 14:21:45,857] : PID     30 : DEBUG    : ahi_hsd : _read_header : Band number = 3
[2019-07-04 14:21:45,857] : PID     30 : DEBUG    : ahi_hsd : _read_header : Time_interval: 2018-10-15 02:02:37.287367 - 2018-10-15 02:04:03.560212
[2019-07-04 14:21:45,861] : PID     30 : DEBUG    : ahi_hsd : read_band : Reading time 0:00:00.004128
[2019-07-04 14:21:45,912] : PID     30 : DEBUG    : ahi_hsd : calibrate : Calibration time 0:00:00.050759
[2019-07-04 14:21:45,947] : PID     30 : DEBUG    : ahi_hsd : _read_header : Band number = 1
[2019-07-04 14:21:45,947] : PID     30 : DEBUG    : ahi_hsd : _read_header : Time_interval: 2018-10-15 02:02:37.548508 - 2018-10-15 02:04:03.818888
[2019-07-04 14:21:45,949] : PID     30 : DEBUG    : ahi_hsd : read_band : Reading time 0:00:00.002413
[2019-07-04 14:21:45,967] : PID     30 : DEBUG    : ahi_hsd : calibrate : Calibration time 0:00:00.017361
[2019-07-04 14:21:45,988] : PID     30 : DEBUG    : ahi_hsd : _read_header : Band number = 2
[2019-07-04 14:21:45,988] : PID     30 : DEBUG    : ahi_hsd : _read_header : Time_interval: 2018-10-15 02:02:37.797630 - 2018-10-15 02:04:04.068010
[2019-07-04 14:21:45,990] : PID     30 : DEBUG    : ahi_hsd : read_band : Reading time 0:00:00.002497
[2019-07-04 14:21:45,992] : PID     30 : DEBUG    : ahi_hsd : calibrate : Calibration time 0:00:00.001804
[2019-07-04 14:21:46,008] : PID     30 : WARNING  : satpy.readers.yaml_reader : _get_file_handlers : Required file type 'hsd_b05' not found or loaded for 'B05'
[2019-07-04 14:21:46,009] : PID     30 : DEBUG    : ahi_hsd : _read_header : Band number = 4
[2019-07-04 14:21:46,009] : PID     30 : DEBUG    : ahi_hsd : _read_header : Time_interval: 2018-10-15 02:02:36.949136 - 2018-10-15 02:04:03.219514
[2019-07-04 14:21:46,011] : PID     30 : DEBUG    : ahi_hsd : read_band : Reading time 0:00:00.002569
[2019-07-04 14:21:46,013] : PID     30 : DEBUG    : ahi_hsd : calibrate : Calibration time 0:00:00.001845
[2019-07-04 14:21:46,030] : PID     30 : WARNING  : satpy.scene : load : The following datasets were not created and may require resampling to be generated: DatasetID(name='B05', wavelength=(1.5, 1.6, 1.7), resolution=2000, polarization=None, calibration='reflectance', level=None, modifiers=())
[2019-07-04 14:21:46,147] : PID     30 : DEBUG    : polar2grid.grids.manager : __init__ : Using default grid configuration: 'grids.conf' 
[2019-07-04 14:21:46,147] : PID     30 : DEBUG    : polar2grid.grids.manager : __init__ : Loading grid configuration 'grids.conf'
[2019-07-04 14:21:46,154] : PID     30 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid 'dwd_germany' origin from (lon: -2.000000, lat: 56.000000) to (x: -755570.370463, y: -3554679.114710)
[2019-07-04 14:21:46,154] : PID     30 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid 'australia' origin from (lon: 105.000000, lat: 5.000000) to (x: -3896182.177765, y: 557305.257275)
[2019-07-04 14:21:46,154] : PID     30 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid 'australia2' origin from (lon: 105.000000, lat: 5.000000) to (x: -3896182.177765, y: 557305.257275)
[2019-07-04 14:21:46,154] : PID     30 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '211e' origin from (lon: -123.044000, lat: 59.844000) to (x: -1950510.636800, y: 4368587.226913)
[2019-07-04 14:21:46,154] : PID     30 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '211e_10km' origin from (lon: -123.044000, lat: 59.844000) to (x: -1950510.636800, y: 4368587.226913)
[2019-07-04 14:21:46,154] : PID     30 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '211e_hi' origin from (lon: -123.044000, lat: 59.844000) to (x: -1950510.636800, y: 4368587.226913)
[2019-07-04 14:21:46,154] : PID     30 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '211w' origin from (lon: -152.855000, lat: 54.536000) to (x: -4226045.730436, y: 4368634.765713)
[2019-07-04 14:21:46,154] : PID     30 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '211w_10km' origin from (lon: -152.855000, lat: 54.536000) to (x: -4226045.730436, y: 4368634.765713)
[2019-07-04 14:21:46,155] : PID     30 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '203' origin from (lon: 115.601000, lat: 44.646000) to (x: -4952960.856044, y: 381022.610378)
[2019-07-04 14:21:46,155] : PID     30 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '203_10km' origin from (lon: 115.601000, lat: 44.646000) to (x: -4952960.856044, y: 381022.610378)
[2019-07-04 14:21:46,155] : PID     30 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '204' origin from (lon: 110.000000, lat: 60.644000) to (x: -7314463.252793, y: 8020503.279488)
[2019-07-04 14:21:46,155] : PID     30 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '204_10km' origin from (lon: 110.000000, lat: 60.644000) to (x: -7314463.252793, y: 8020503.279488)
[2019-07-04 14:21:46,155] : PID     30 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '205' origin from (lon: -115.304000, lat: 36.257000) to (x: -4952924.911284, y: -3429056.260492)
[2019-07-04 14:21:46,155] : PID     30 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '210' origin from (lon: -77.000000, lat: 26.422000) to (x: 0.000000, y: 2864298.891608)
[2019-07-04 14:21:46,155] : PID     30 : DEBUG    : satpy.writers : read_writer_config : Reading ['/mnt/c/Users/BlueCARES/Downloads/geo2grid_v_1_0_0_0/lib/python3.6/site-packages/satpy/etc/writers/geotiff.yaml']
[2019-07-04 14:21:46,305] : PID     30 : DEBUG    : satpy.writers : apply : Enhancement configuration options: [{'name': 'linear_stretch', 'method': <function stretch at 0x7f9450a15ae8>, 'kwargs': {'stretch': 'crude', 'min_stretch': 0.0, 'max_stretch': 100.0}}, {'name': 'gamma', 'method': <function gamma at 0x7f9450a15c80>, 'kwargs': {'gamma': 2.0}}]
[2019-07-04 14:21:46,305] : PID     30 : DEBUG    : trollimage.xrimage : stretch : Applying stretch crude with parameters {'min_stretch': 0.0, 'max_stretch': 100.0}
[2019-07-04 14:21:46,307] : PID     30 : DEBUG    : trollimage.xrimage : gamma : Applying gamma 2.0
[2019-07-04 14:21:46,322] : PID     30 : DEBUG    : satpy.writers : apply : Enhancement configuration options: [{'name': 'linear_stretch', 'method': <function stretch at 0x7f9450a15ae8>, 'kwargs': {'stretch': 'crude', 'min_stretch': 0.0, 'max_stretch': 100.0}}, {'name': 'gamma', 'method': <function gamma at 0x7f9450a15c80>, 'kwargs': {'gamma': 2.0}}]
[2019-07-04 14:21:46,322] : PID     30 : DEBUG    : trollimage.xrimage : stretch : Applying stretch crude with parameters {'min_stretch': 0.0, 'max_stretch': 100.0}
[2019-07-04 14:21:46,323] : PID     30 : DEBUG    : trollimage.xrimage : gamma : Applying gamma 2.0
[2019-07-04 14:21:46,336] : PID     30 : DEBUG    : satpy.writers : apply : Enhancement configuration options: [{'name': 'linear_stretch', 'method': <function stretch at 0x7f9450a15ae8>, 'kwargs': {'stretch': 'crude', 'min_stretch': 0.0, 'max_stretch': 100.0}}, {'name': 'gamma', 'method': <function gamma at 0x7f9450a15c80>, 'kwargs': {'gamma': 2.0}}]
[2019-07-04 14:21:46,336] : PID     30 : DEBUG    : trollimage.xrimage : stretch : Applying stretch crude with parameters {'min_stretch': 0.0, 'max_stretch': 100.0}
[2019-07-04 14:21:46,337] : PID     30 : DEBUG    : trollimage.xrimage : gamma : Applying gamma 2.0
[2019-07-04 14:21:46,349] : PID     30 : DEBUG    : satpy.writers : apply : Enhancement configuration options: [{'name': 'linear_stretch', 'method': <function stretch at 0x7f9450a15ae8>, 'kwargs': {'stretch': 'crude', 'min_stretch': 0.0, 'max_stretch': 100.0}}, {'name': 'gamma', 'method': <function gamma at 0x7f9450a15c80>, 'kwargs': {'gamma': 2.0}}]
[2019-07-04 14:21:46,350] : PID     30 : DEBUG    : trollimage.xrimage : stretch : Applying stretch crude with parameters {'min_stretch': 0.0, 'max_stretch': 100.0}
[2019-07-04 14:21:46,350] : PID     30 : DEBUG    : trollimage.xrimage : gamma : Applying gamma 2.0
[2019-07-04 14:21:46,361] : PID     30 : DEBUG    : ahi_hsd_geotiff : main : Products to preserve resolution for: {DatasetID(name='B03', wavelength=(0.62, 0.64, 0.66), resolution=500, polarization=None, calibration='reflectance', level=None, modifiers=()), DatasetID(name='B01', wavelength=(0.45, 0.47, 0.49), resolution=1000, polarization=None, calibration='reflectance', level=None, modifiers=()), DatasetID(name='B02', wavelength=(0.49, 0.51, 0.53), resolution=1000, polarization=None, calibration='reflectance', level=None, modifiers=()), DatasetID(name='B04', wavelength=(0.83, 0.85, 0.87), resolution=1000, polarization=None, calibration='reflectance', level=None, modifiers=())}
[2019-07-04 14:21:46,361] : PID     30 : DEBUG    : ahi_hsd_geotiff : main : Products to use new resolution for: set()
[2019-07-04 14:21:46,361] : PID     30 : DEBUG    : ahi_hsd_geotiff : main : Setting default resampling to 'native' for grid 'MAX'
[2019-07-04 14:21:46,362] : PID     30 : INFO     : ahi_hsd_geotiff : main : Resampling data to 'MAX'
[2019-07-04 14:21:46,362] : PID     30 : DEBUG    : satpy.scene : __init__ : Setting 'PPP_CONFIG_DIR' to '/mnt/c/Users/BlueCARES/Downloads/geo2grid_v_1_0_0_0/etc/satpy'
[2019-07-04 14:21:46,362] : PID     30 : DEBUG    : satpy.scene : _resampled_scene : Resampling DatasetID(name='B03', wavelength=(0.62, 0.64, 0.66), resolution=500, polarization=None, calibration='reflectance', level=None, modifiers=())
[2019-07-04 14:21:46,363] : PID     30 : DEBUG    : pyresample.geometry : get_area_slices : Projections for data and slice areas are identical: geos
[2019-07-04 14:21:46,375] : PID     30 : DEBUG    : satpy.scene : _resampled_scene : Resampling DatasetID(name='B01', wavelength=(0.45, 0.47, 0.49), resolution=1000, polarization=None, calibration='reflectance', level=None, modifiers=())
[2019-07-04 14:21:46,375] : PID     30 : DEBUG    : pyresample.geometry : get_area_slices : Projections for data and slice areas are identical: geos
[2019-07-04 14:21:46,399] : PID     30 : DEBUG    : satpy.scene : _resampled_scene : Resampling DatasetID(name='B02', wavelength=(0.49, 0.51, 0.53), resolution=1000, polarization=None, calibration='reflectance', level=None, modifiers=())
[2019-07-04 14:21:46,399] : PID     30 : DEBUG    : pyresample.geometry : get_area_slices : Projections for data and slice areas are identical: geos
[2019-07-04 14:21:46,423] : PID     30 : DEBUG    : satpy.scene : _resampled_scene : Resampling DatasetID(name='B04', wavelength=(0.83, 0.85, 0.87), resolution=1000, polarization=None, calibration='reflectance', level=None, modifiers=())
[2019-07-04 14:21:46,423] : PID     30 : DEBUG    : pyresample.geometry : get_area_slices : Projections for data and slice areas are identical: geos
[2019-07-04 14:21:46,447] : PID     30 : INFO     : ahi_hsd_geotiff : main : Computing products and saving data to writers...
[2019-07-04 14:21:52,599] : PID     30 : INFO     : ahi_hsd_geotiff : main : SUCCESS
[2019-07-11 10:04:31,950] : PID    930 : DEBUG    : ahi_hsd_geotiff : main : Starting script with arguments: /mnt/c/Users/BlueCARES/Downloads/geo2grid_v_1_0_0_0/lib/python3.6/site-packages/polar2grid/glue.py -r ahi_hsd -w geotiff -p B01C B02C B03C B04C B05C -f count2tbb/oct15/20 -vv
[2019-07-11 10:04:31,955] : PID    930 : INFO     : ahi_hsd_geotiff : main : Sorting and reading input files...
[2019-07-11 10:04:31,956] : PID    930 : DEBUG    : satpy.scene : __init__ : Setting 'PPP_CONFIG_DIR' to '/mnt/c/Users/BlueCARES/Downloads/geo2grid_v_1_0_0_0/etc/satpy'
[2019-07-11 10:04:31,958] : PID    930 : DEBUG    : satpy.readers : read_reader_config : Reading ['/mnt/c/Users/BlueCARES/Downloads/geo2grid_v_1_0_0_0/lib/python3.6/site-packages/satpy/etc/readers/ahi_hsd.yaml']
[2019-07-11 10:04:32,044] : PID    930 : DEBUG    : satpy.readers.yaml_reader : create_filehandlers : Assigning to ahi_hsd: ['count2tbb/oct15/20/HS_H08_20181015_0200_B01_FLDK_R10_S0410.DAT', 'count2tbb/oct15/20/HS_H08_20181015_0200_B02_FLDK_R10_S0410.DAT', 'count2tbb/oct15/20/HS_H08_20181015_0200_B03_FLDK_R05_S0410.DAT', 'count2tbb/oct15/20/HS_H08_20181015_0200_B04_FLDK_R10_S0410.DAT', 'count2tbb/oct15/20/HS_H08_20181015_0200_B05_FLDK_R20_S0410.DAT']
[2019-07-11 10:04:32,068] : PID    930 : DEBUG    : satpy.composites : load_sensor_composites : Looking for composites config file ahi.yaml
[2019-07-11 10:04:32,089] : PID    930 : DEBUG    : satpy.composites : load_sensor_composites : Looking for composites config file visir.yaml
[2019-07-11 10:04:32,158] : PID    930 : DEBUG    : polar2grid.core.script_utils : rename_log_file : Log renamed from '/home/bluecares/ahi_hsd_geotiff_fail.log' to 'ahi_hsd_geotiff_20181015_020236.log'
[2019-07-11 10:04:32,158] : PID    930 : INFO     : ahi_hsd_geotiff : main : Loading product metadata from files...
[2019-07-11 10:04:32,172] : PID    930 : DEBUG    : ahi_hsd : _read_header : Band number = 5
[2019-07-11 10:04:32,172] : PID    930 : DEBUG    : ahi_hsd : _read_header : Time_interval: 2018-10-15 02:02:38.070794 - 2018-10-15 02:04:04.341171
[2019-07-11 10:04:32,175] : PID    930 : DEBUG    : ahi_hsd : read_band : Reading time 0:00:00.003746
[2019-07-11 10:04:32,229] : PID    930 : DEBUG    : ahi_hsd : calibrate : Calibration time 0:00:00.053327
[2019-07-11 10:04:32,263] : PID    930 : DEBUG    : ahi_hsd : _read_header : Band number = 2
[2019-07-11 10:04:32,263] : PID    930 : DEBUG    : ahi_hsd : _read_header : Time_interval: 2018-10-15 02:02:37.797630 - 2018-10-15 02:04:04.068010
[2019-07-11 10:04:32,265] : PID    930 : DEBUG    : ahi_hsd : read_band : Reading time 0:00:00.002549
[2019-07-11 10:04:32,267] : PID    930 : DEBUG    : ahi_hsd : calibrate : Calibration time 0:00:00.001803
[2019-07-11 10:04:32,284] : PID    930 : DEBUG    : ahi_hsd : _read_header : Band number = 4
[2019-07-11 10:04:32,285] : PID    930 : DEBUG    : ahi_hsd : _read_header : Time_interval: 2018-10-15 02:02:36.949136 - 2018-10-15 02:04:03.219514
[2019-07-11 10:04:32,287] : PID    930 : DEBUG    : ahi_hsd : read_band : Reading time 0:00:00.002478
[2019-07-11 10:04:32,289] : PID    930 : DEBUG    : ahi_hsd : calibrate : Calibration time 0:00:00.001828
[2019-07-11 10:04:32,306] : PID    930 : DEBUG    : ahi_hsd : _read_header : Band number = 1
[2019-07-11 10:04:32,306] : PID    930 : DEBUG    : ahi_hsd : _read_header : Time_interval: 2018-10-15 02:02:37.548508 - 2018-10-15 02:04:03.818888
[2019-07-11 10:04:32,308] : PID    930 : DEBUG    : ahi_hsd : read_band : Reading time 0:00:00.002515
[2019-07-11 10:04:32,310] : PID    930 : DEBUG    : ahi_hsd : calibrate : Calibration time 0:00:00.001866
[2019-07-11 10:04:32,327] : PID    930 : DEBUG    : ahi_hsd : _read_header : Band number = 3
[2019-07-11 10:04:32,327] : PID    930 : DEBUG    : ahi_hsd : _read_header : Time_interval: 2018-10-15 02:02:37.287367 - 2018-10-15 02:04:03.560212
[2019-07-11 10:04:32,330] : PID    930 : DEBUG    : ahi_hsd : read_band : Reading time 0:00:00.003261
[2019-07-11 10:04:32,332] : PID    930 : DEBUG    : ahi_hsd : calibrate : Calibration time 0:00:00.001855
[2019-07-11 10:04:32,358] : PID    930 : DEBUG    : satpy.scene : unload : Unloading dataset: DatasetID(name='B05', wavelength=(1.5, 1.6, 1.7), resolution=2000, polarization=None, calibration='reflectance', level=None, modifiers=())
[2019-07-11 10:04:32,358] : PID    930 : DEBUG    : satpy.scene : unload : Unloading dataset: DatasetID(name='B02', wavelength=(0.49, 0.51, 0.53), resolution=1000, polarization=None, calibration='reflectance', level=None, modifiers=())
[2019-07-11 10:04:32,358] : PID    930 : DEBUG    : satpy.scene : unload : Unloading dataset: DatasetID(name='B04', wavelength=(0.83, 0.85, 0.87), resolution=1000, polarization=None, calibration='reflectance', level=None, modifiers=())
[2019-07-11 10:04:32,358] : PID    930 : DEBUG    : satpy.scene : unload : Unloading dataset: DatasetID(name='B01', wavelength=(0.45, 0.47, 0.49), resolution=1000, polarization=None, calibration='reflectance', level=None, modifiers=())
[2019-07-11 10:04:32,358] : PID    930 : DEBUG    : satpy.scene : unload : Unloading dataset: DatasetID(name='B03', wavelength=(0.62, 0.64, 0.66), resolution=500, polarization=None, calibration='reflectance', level=None, modifiers=())
[2019-07-11 10:04:32,477] : PID    930 : DEBUG    : polar2grid.grids.manager : __init__ : Using default grid configuration: 'grids.conf' 
[2019-07-11 10:04:32,477] : PID    930 : DEBUG    : polar2grid.grids.manager : __init__ : Loading grid configuration 'grids.conf'
[2019-07-11 10:04:32,484] : PID    930 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid 'dwd_germany' origin from (lon: -2.000000, lat: 56.000000) to (x: -755570.370463, y: -3554679.114710)
[2019-07-11 10:04:32,484] : PID    930 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid 'australia' origin from (lon: 105.000000, lat: 5.000000) to (x: -3896182.177765, y: 557305.257275)
[2019-07-11 10:04:32,484] : PID    930 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid 'australia2' origin from (lon: 105.000000, lat: 5.000000) to (x: -3896182.177765, y: 557305.257275)
[2019-07-11 10:04:32,484] : PID    930 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '211e' origin from (lon: -123.044000, lat: 59.844000) to (x: -1950510.636800, y: 4368587.226913)
[2019-07-11 10:04:32,484] : PID    930 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '211e_10km' origin from (lon: -123.044000, lat: 59.844000) to (x: -1950510.636800, y: 4368587.226913)
[2019-07-11 10:04:32,484] : PID    930 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '211e_hi' origin from (lon: -123.044000, lat: 59.844000) to (x: -1950510.636800, y: 4368587.226913)
[2019-07-11 10:04:32,484] : PID    930 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '211w' origin from (lon: -152.855000, lat: 54.536000) to (x: -4226045.730436, y: 4368634.765713)
[2019-07-11 10:04:32,484] : PID    930 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '211w_10km' origin from (lon: -152.855000, lat: 54.536000) to (x: -4226045.730436, y: 4368634.765713)
[2019-07-11 10:04:32,484] : PID    930 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '203' origin from (lon: 115.601000, lat: 44.646000) to (x: -4952960.856044, y: 381022.610378)
[2019-07-11 10:04:32,485] : PID    930 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '203_10km' origin from (lon: 115.601000, lat: 44.646000) to (x: -4952960.856044, y: 381022.610378)
[2019-07-11 10:04:32,485] : PID    930 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '204' origin from (lon: 110.000000, lat: 60.644000) to (x: -7314463.252793, y: 8020503.279488)
[2019-07-11 10:04:32,485] : PID    930 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '204_10km' origin from (lon: 110.000000, lat: 60.644000) to (x: -7314463.252793, y: 8020503.279488)
[2019-07-11 10:04:32,485] : PID    930 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '205' origin from (lon: -115.304000, lat: 36.257000) to (x: -4952924.911284, y: -3429056.260492)
[2019-07-11 10:04:32,485] : PID    930 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '210' origin from (lon: -77.000000, lat: 26.422000) to (x: 0.000000, y: 2864298.891608)
[2019-07-11 10:04:32,485] : PID    930 : DEBUG    : satpy.writers : read_writer_config : Reading ['/mnt/c/Users/BlueCARES/Downloads/geo2grid_v_1_0_0_0/lib/python3.6/site-packages/satpy/etc/writers/geotiff.yaml']
[2019-07-11 10:04:32,639] : PID    930 : DEBUG    : satpy.writers : apply : Enhancement configuration options: [{'name': 'stretch', 'method': <function stretch at 0x7f72e4f50268>, 'kwargs': {'stretch': 'linear'}}]
[2019-07-11 10:04:32,639] : PID    930 : DEBUG    : trollimage.xrimage : stretch : Applying stretch linear with parameters {}
[2019-07-11 10:04:32,639] : PID    930 : DEBUG    : trollimage.xrimage : stretch_linear : Perform a linear contrast stretch.
[2019-07-11 10:04:32,639] : PID    930 : DEBUG    : trollimage.xrimage : stretch_linear : Calculate the histogram quantiles: 
[2019-07-11 10:04:32,639] : PID    930 : DEBUG    : trollimage.xrimage : stretch_linear : Left and right quantiles: 0.005 0.005
[2019-07-11 10:04:32,668] : PID    930 : DEBUG    : satpy.writers : apply : Enhancement configuration options: [{'name': 'stretch', 'method': <function stretch at 0x7f72e4f50268>, 'kwargs': {'stretch': 'linear'}}]
[2019-07-11 10:04:32,668] : PID    930 : DEBUG    : trollimage.xrimage : stretch : Applying stretch linear with parameters {}
[2019-07-11 10:04:32,668] : PID    930 : DEBUG    : trollimage.xrimage : stretch_linear : Perform a linear contrast stretch.
[2019-07-11 10:04:32,668] : PID    930 : DEBUG    : trollimage.xrimage : stretch_linear : Calculate the histogram quantiles: 
[2019-07-11 10:04:32,668] : PID    930 : DEBUG    : trollimage.xrimage : stretch_linear : Left and right quantiles: 0.005 0.005
[2019-07-11 10:04:32,688] : PID    930 : DEBUG    : satpy.writers : apply : Enhancement configuration options: [{'name': 'stretch', 'method': <function stretch at 0x7f72e4f50268>, 'kwargs': {'stretch': 'linear'}}]
[2019-07-11 10:04:32,688] : PID    930 : DEBUG    : trollimage.xrimage : stretch : Applying stretch linear with parameters {}
[2019-07-11 10:04:32,689] : PID    930 : DEBUG    : trollimage.xrimage : stretch_linear : Perform a linear contrast stretch.
[2019-07-11 10:04:32,689] : PID    930 : DEBUG    : trollimage.xrimage : stretch_linear : Calculate the histogram quantiles: 
[2019-07-11 10:04:32,689] : PID    930 : DEBUG    : trollimage.xrimage : stretch_linear : Left and right quantiles: 0.005 0.005
[2019-07-11 10:04:32,709] : PID    930 : DEBUG    : satpy.writers : apply : Enhancement configuration options: [{'name': 'stretch', 'method': <function stretch at 0x7f72e4f50268>, 'kwargs': {'stretch': 'linear'}}]
[2019-07-11 10:04:32,709] : PID    930 : DEBUG    : trollimage.xrimage : stretch : Applying stretch linear with parameters {}
[2019-07-11 10:04:32,710] : PID    930 : DEBUG    : trollimage.xrimage : stretch_linear : Perform a linear contrast stretch.
[2019-07-11 10:04:32,710] : PID    930 : DEBUG    : trollimage.xrimage : stretch_linear : Calculate the histogram quantiles: 
[2019-07-11 10:04:32,710] : PID    930 : DEBUG    : trollimage.xrimage : stretch_linear : Left and right quantiles: 0.005 0.005
[2019-07-11 10:04:32,730] : PID    930 : DEBUG    : satpy.writers : apply : Enhancement configuration options: [{'name': 'stretch', 'method': <function stretch at 0x7f72e4f50268>, 'kwargs': {'stretch': 'linear'}}]
[2019-07-11 10:04:32,730] : PID    930 : DEBUG    : trollimage.xrimage : stretch : Applying stretch linear with parameters {}
[2019-07-11 10:04:32,730] : PID    930 : DEBUG    : trollimage.xrimage : stretch_linear : Perform a linear contrast stretch.
[2019-07-11 10:04:32,730] : PID    930 : DEBUG    : trollimage.xrimage : stretch_linear : Calculate the histogram quantiles: 
[2019-07-11 10:04:32,731] : PID    930 : DEBUG    : trollimage.xrimage : stretch_linear : Left and right quantiles: 0.005 0.005
[2019-07-11 10:04:32,748] : PID    930 : DEBUG    : ahi_hsd_geotiff : main : Products to preserve resolution for: {DatasetID(name='B03C', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=('sunz_corrected', 'rayleigh_corrected')), DatasetID(name='B02C', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=('sunz_corrected', 'rayleigh_corrected')), DatasetID(name='B01C', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=('sunz_corrected', 'rayleigh_corrected')), DatasetID(name='B04C', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=('sunz_corrected', 'rayleigh_corrected')), DatasetID(name='B05C', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=('sunz_corrected', 'rayleigh_corrected'))}
[2019-07-11 10:04:32,748] : PID    930 : DEBUG    : ahi_hsd_geotiff : main : Products to use new resolution for: set()
[2019-07-11 10:04:32,748] : PID    930 : DEBUG    : ahi_hsd_geotiff : main : Setting default resampling to 'native' for grid 'MAX'
[2019-07-11 10:04:32,748] : PID    930 : INFO     : ahi_hsd_geotiff : main : Resampling data to 'MAX'
[2019-07-11 10:04:32,748] : PID    930 : DEBUG    : satpy.scene : __init__ : Setting 'PPP_CONFIG_DIR' to '/mnt/c/Users/BlueCARES/Downloads/geo2grid_v_1_0_0_0/etc/satpy'
[2019-07-11 10:04:32,749] : PID    930 : DEBUG    : satpy.scene : _resampled_scene : Resampling DatasetID(name='B03C', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=('sunz_corrected', 'rayleigh_corrected'))
[2019-07-11 10:04:32,749] : PID    930 : DEBUG    : pyresample.geometry : get_area_slices : Projections for data and slice areas are identical: geos
[2019-07-11 10:04:32,763] : PID    930 : DEBUG    : satpy.scene : _resampled_scene : Resampling DatasetID(name='B02C', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=('sunz_corrected', 'rayleigh_corrected'))
[2019-07-11 10:04:32,763] : PID    930 : DEBUG    : pyresample.geometry : get_area_slices : Projections for data and slice areas are identical: geos
[2019-07-11 10:04:32,788] : PID    930 : DEBUG    : satpy.scene : _resampled_scene : Resampling DatasetID(name='B01C', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=('sunz_corrected', 'rayleigh_corrected'))
[2019-07-11 10:04:32,788] : PID    930 : DEBUG    : pyresample.geometry : get_area_slices : Projections for data and slice areas are identical: geos
[2019-07-11 10:04:32,813] : PID    930 : DEBUG    : satpy.scene : _resampled_scene : Resampling DatasetID(name='B04C', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=('sunz_corrected', 'rayleigh_corrected'))
[2019-07-11 10:04:32,813] : PID    930 : DEBUG    : pyresample.geometry : get_area_slices : Projections for data and slice areas are identical: geos
[2019-07-11 10:04:32,837] : PID    930 : DEBUG    : satpy.scene : _resampled_scene : Resampling DatasetID(name='B05C', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=('sunz_corrected', 'rayleigh_corrected'))
[2019-07-11 10:04:32,837] : PID    930 : DEBUG    : pyresample.geometry : get_area_slices : Projections for data and slice areas are identical: geos
[2019-07-11 10:04:32,901] : PID    930 : INFO     : ahi_hsd_geotiff : main : Computing products and saving data to writers...
[2019-07-11 10:04:36,107] : PID    930 : DEBUG    : trollimage.xrimage : _compute_quantile : Interval: left=<xarray.DataArray 'where-158a8ec6a9eea4bd11f7188c41c2afd8' (bands: 1)>
array([10.529412])
Coordinates:
  * bands     (bands) <U1 'L'
    quantile  float64 0.005, right=<xarray.DataArray 'where-158a8ec6a9eea4bd11f7188c41c2afd8' (bands: 1)>
array([86.999996])
Coordinates:
  * bands     (bands) <U1 'L'
    quantile  float64 0.995
[2019-07-11 10:04:36,553] : PID    930 : DEBUG    : trollimage.xrimage : _compute_quantile : Interval: left=<xarray.DataArray 'where-75cba78c48b6bb9330a2c1aefcb03ef5' (bands: 1)>
array([2.705882])
Coordinates:
  * bands     (bands) <U1 'L'
    quantile  float64 0.005, right=<xarray.DataArray 'where-75cba78c48b6bb9330a2c1aefcb03ef5' (bands: 1)>
array([85.411761])
Coordinates:
  * bands     (bands) <U1 'L'
    quantile  float64 0.995
[2019-07-11 10:04:36,557] : PID    930 : DEBUG    : trollimage.xrimage : _compute_quantile : Interval: left=<xarray.DataArray 'where-1f7ae94d51c31e7fd543a059cde07220' (bands: 1)>
array([0.235294])
Coordinates:
  * bands     (bands) <U1 'L'
    quantile  float64 0.005, right=<xarray.DataArray 'where-1f7ae94d51c31e7fd543a059cde07220' (bands: 1)>
array([39.411764])
Coordinates:
  * bands     (bands) <U1 'L'
    quantile  float64 0.995
[2019-07-11 10:04:36,845] : PID    930 : DEBUG    : trollimage.xrimage : _compute_quantile : Interval: left=<xarray.DataArray 'where-d31c8f9ccdc510c258dc874871fede59' (bands: 1)>
array([0.941176])
Coordinates:
  * bands     (bands) <U1 'L'
    quantile  float64 0.005, right=<xarray.DataArray 'where-d31c8f9ccdc510c258dc874871fede59' (bands: 1)>
array([87.76471])
Coordinates:
  * bands     (bands) <U1 'L'
    quantile  float64 0.995
[2019-07-11 10:04:36,868] : PID    930 : DEBUG    : trollimage.xrimage : _compute_quantile : Interval: left=<xarray.DataArray 'where-28602118327c2a3fddf9e7f74a64abfc' (bands: 1)>
array([7.])
Coordinates:
  * bands     (bands) <U1 'L'
    quantile  float64 0.005, right=<xarray.DataArray 'where-28602118327c2a3fddf9e7f74a64abfc' (bands: 1)>
array([85.000001])
Coordinates:
  * bands     (bands) <U1 'L'
    quantile  float64 0.995
[2019-07-11 10:04:39,251] : PID    930 : INFO     : ahi_hsd_geotiff : main : SUCCESS
[2019-07-11 10:16:29,047] : PID   1826 : DEBUG    : ahi_hsd_geotiff : main : Starting script with arguments: /mnt/c/Users/BlueCARES/Downloads/geo2grid_v_1_0_0_0/lib/python3.6/site-packages/polar2grid/glue.py -r ahi_hsd -w geotiff -p B01C B02C B03C B04C B05C -f count2tbb/oct15/20 -vv
[2019-07-11 10:16:29,052] : PID   1826 : INFO     : ahi_hsd_geotiff : main : Sorting and reading input files...
[2019-07-11 10:16:29,052] : PID   1826 : DEBUG    : satpy.scene : __init__ : Setting 'PPP_CONFIG_DIR' to '/mnt/c/Users/BlueCARES/Downloads/geo2grid_v_1_0_0_0/etc/satpy'
[2019-07-11 10:16:29,053] : PID   1826 : DEBUG    : satpy.readers : read_reader_config : Reading ['/mnt/c/Users/BlueCARES/Downloads/geo2grid_v_1_0_0_0/lib/python3.6/site-packages/satpy/etc/readers/ahi_hsd.yaml']
[2019-07-11 10:16:29,143] : PID   1826 : DEBUG    : satpy.readers.yaml_reader : create_filehandlers : Assigning to ahi_hsd: ['count2tbb/oct15/20/HS_H08_20181015_0200_B01_FLDK_R10_S0410.DAT', 'count2tbb/oct15/20/HS_H08_20181015_0200_B02_FLDK_R10_S0410.DAT', 'count2tbb/oct15/20/HS_H08_20181015_0200_B03_FLDK_R05_S0410.DAT', 'count2tbb/oct15/20/HS_H08_20181015_0200_B04_FLDK_R10_S0410.DAT', 'count2tbb/oct15/20/HS_H08_20181015_0200_B05_FLDK_R20_S0410.DAT']
[2019-07-11 10:16:29,149] : PID   1826 : DEBUG    : satpy.composites : load_sensor_composites : Looking for composites config file ahi.yaml
[2019-07-11 10:16:29,171] : PID   1826 : DEBUG    : satpy.composites : load_sensor_composites : Looking for composites config file visir.yaml
[2019-07-11 10:16:29,211] : PID   1826 : DEBUG    : polar2grid.core.script_utils : rename_log_file : Log renamed from '/home/bluecares/ahi_hsd_geotiff_fail.log' to 'ahi_hsd_geotiff_20181015_020236.log'
[2019-07-11 10:16:29,211] : PID   1826 : INFO     : ahi_hsd_geotiff : main : Loading product metadata from files...
[2019-07-11 10:16:29,219] : PID   1826 : DEBUG    : ahi_hsd : _read_header : Band number = 2
[2019-07-11 10:16:29,219] : PID   1826 : DEBUG    : ahi_hsd : _read_header : Time_interval: 2018-10-15 02:02:37.797630 - 2018-10-15 02:04:04.068010
[2019-07-11 10:16:29,222] : PID   1826 : DEBUG    : ahi_hsd : read_band : Reading time 0:00:00.003861
[2019-07-11 10:16:29,276] : PID   1826 : DEBUG    : ahi_hsd : calibrate : Calibration time 0:00:00.053751
[2019-07-11 10:16:29,314] : PID   1826 : DEBUG    : ahi_hsd : _read_header : Band number = 5
[2019-07-11 10:16:29,314] : PID   1826 : DEBUG    : ahi_hsd : _read_header : Time_interval: 2018-10-15 02:02:38.070794 - 2018-10-15 02:04:04.341171
[2019-07-11 10:16:29,316] : PID   1826 : DEBUG    : ahi_hsd : read_band : Reading time 0:00:00.002395
[2019-07-11 10:16:29,318] : PID   1826 : DEBUG    : ahi_hsd : calibrate : Calibration time 0:00:00.001881
[2019-07-11 10:16:29,331] : PID   1826 : DEBUG    : ahi_hsd : _read_header : Band number = 1
[2019-07-11 10:16:29,331] : PID   1826 : DEBUG    : ahi_hsd : _read_header : Time_interval: 2018-10-15 02:02:37.548508 - 2018-10-15 02:04:03.818888
[2019-07-11 10:16:29,333] : PID   1826 : DEBUG    : ahi_hsd : read_band : Reading time 0:00:00.002653
[2019-07-11 10:16:29,335] : PID   1826 : DEBUG    : ahi_hsd : calibrate : Calibration time 0:00:00.001909
[2019-07-11 10:16:29,352] : PID   1826 : DEBUG    : ahi_hsd : _read_header : Band number = 3
[2019-07-11 10:16:29,352] : PID   1826 : DEBUG    : ahi_hsd : _read_header : Time_interval: 2018-10-15 02:02:37.287367 - 2018-10-15 02:04:03.560212
[2019-07-11 10:16:29,355] : PID   1826 : DEBUG    : ahi_hsd : read_band : Reading time 0:00:00.003241
[2019-07-11 10:16:29,358] : PID   1826 : DEBUG    : ahi_hsd : calibrate : Calibration time 0:00:00.002952
[2019-07-11 10:16:29,384] : PID   1826 : DEBUG    : ahi_hsd : _read_header : Band number = 4
[2019-07-11 10:16:29,384] : PID   1826 : DEBUG    : ahi_hsd : _read_header : Time_interval: 2018-10-15 02:02:36.949136 - 2018-10-15 02:04:03.219514
[2019-07-11 10:16:29,386] : PID   1826 : DEBUG    : ahi_hsd : read_band : Reading time 0:00:00.002618
[2019-07-11 10:16:29,394] : PID   1826 : DEBUG    : ahi_hsd : calibrate : Calibration time 0:00:00.001927
[2019-07-11 10:16:29,413] : PID   1826 : DEBUG    : satpy.scene : unload : Unloading dataset: DatasetID(name='B02', wavelength=(0.49, 0.51, 0.53), resolution=1000, polarization=None, calibration='reflectance', level=None, modifiers=())
[2019-07-11 10:16:29,413] : PID   1826 : DEBUG    : satpy.scene : unload : Unloading dataset: DatasetID(name='B05', wavelength=(1.5, 1.6, 1.7), resolution=2000, polarization=None, calibration='reflectance', level=None, modifiers=())
[2019-07-11 10:16:29,414] : PID   1826 : DEBUG    : satpy.scene : unload : Unloading dataset: DatasetID(name='B01', wavelength=(0.45, 0.47, 0.49), resolution=1000, polarization=None, calibration='reflectance', level=None, modifiers=())
[2019-07-11 10:16:29,414] : PID   1826 : DEBUG    : satpy.scene : unload : Unloading dataset: DatasetID(name='B03', wavelength=(0.62, 0.64, 0.66), resolution=500, polarization=None, calibration='reflectance', level=None, modifiers=())
[2019-07-11 10:16:29,414] : PID   1826 : DEBUG    : satpy.scene : unload : Unloading dataset: DatasetID(name='B04', wavelength=(0.83, 0.85, 0.87), resolution=1000, polarization=None, calibration='reflectance', level=None, modifiers=())
[2019-07-11 10:16:29,542] : PID   1826 : DEBUG    : polar2grid.grids.manager : __init__ : Using default grid configuration: 'grids.conf' 
[2019-07-11 10:16:29,542] : PID   1826 : DEBUG    : polar2grid.grids.manager : __init__ : Loading grid configuration 'grids.conf'
[2019-07-11 10:16:29,549] : PID   1826 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid 'dwd_germany' origin from (lon: -2.000000, lat: 56.000000) to (x: -755570.370463, y: -3554679.114710)
[2019-07-11 10:16:29,549] : PID   1826 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid 'australia' origin from (lon: 105.000000, lat: 5.000000) to (x: -3896182.177765, y: 557305.257275)
[2019-07-11 10:16:29,549] : PID   1826 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid 'australia2' origin from (lon: 105.000000, lat: 5.000000) to (x: -3896182.177765, y: 557305.257275)
[2019-07-11 10:16:29,549] : PID   1826 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '211e' origin from (lon: -123.044000, lat: 59.844000) to (x: -1950510.636800, y: 4368587.226913)
[2019-07-11 10:16:29,549] : PID   1826 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '211e_10km' origin from (lon: -123.044000, lat: 59.844000) to (x: -1950510.636800, y: 4368587.226913)
[2019-07-11 10:16:29,549] : PID   1826 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '211e_hi' origin from (lon: -123.044000, lat: 59.844000) to (x: -1950510.636800, y: 4368587.226913)
[2019-07-11 10:16:29,549] : PID   1826 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '211w' origin from (lon: -152.855000, lat: 54.536000) to (x: -4226045.730436, y: 4368634.765713)
[2019-07-11 10:16:29,550] : PID   1826 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '211w_10km' origin from (lon: -152.855000, lat: 54.536000) to (x: -4226045.730436, y: 4368634.765713)
[2019-07-11 10:16:29,550] : PID   1826 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '203' origin from (lon: 115.601000, lat: 44.646000) to (x: -4952960.856044, y: 381022.610378)
[2019-07-11 10:16:29,550] : PID   1826 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '203_10km' origin from (lon: 115.601000, lat: 44.646000) to (x: -4952960.856044, y: 381022.610378)
[2019-07-11 10:16:29,550] : PID   1826 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '204' origin from (lon: 110.000000, lat: 60.644000) to (x: -7314463.252793, y: 8020503.279488)
[2019-07-11 10:16:29,550] : PID   1826 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '204_10km' origin from (lon: 110.000000, lat: 60.644000) to (x: -7314463.252793, y: 8020503.279488)
[2019-07-11 10:16:29,550] : PID   1826 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '205' origin from (lon: -115.304000, lat: 36.257000) to (x: -4952924.911284, y: -3429056.260492)
[2019-07-11 10:16:29,550] : PID   1826 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '210' origin from (lon: -77.000000, lat: 26.422000) to (x: 0.000000, y: 2864298.891608)
[2019-07-11 10:16:29,551] : PID   1826 : DEBUG    : satpy.writers : read_writer_config : Reading ['/mnt/c/Users/BlueCARES/Downloads/geo2grid_v_1_0_0_0/lib/python3.6/site-packages/satpy/etc/writers/geotiff.yaml']
[2019-07-11 10:16:29,706] : PID   1826 : DEBUG    : satpy.writers : apply : Enhancement configuration options: [{'name': 'stretch', 'method': <function stretch at 0x7f1647e25158>, 'kwargs': {'stretch': 'linear'}}]
[2019-07-11 10:16:29,706] : PID   1826 : DEBUG    : trollimage.xrimage : stretch : Applying stretch linear with parameters {}
[2019-07-11 10:16:29,706] : PID   1826 : DEBUG    : trollimage.xrimage : stretch_linear : Perform a linear contrast stretch.
[2019-07-11 10:16:29,707] : PID   1826 : DEBUG    : trollimage.xrimage : stretch_linear : Calculate the histogram quantiles: 
[2019-07-11 10:16:29,707] : PID   1826 : DEBUG    : trollimage.xrimage : stretch_linear : Left and right quantiles: 0.005 0.005
[2019-07-11 10:16:29,845] : PID   1826 : DEBUG    : satpy.writers : apply : Enhancement configuration options: [{'name': 'stretch', 'method': <function stretch at 0x7f1647e25158>, 'kwargs': {'stretch': 'linear'}}]
[2019-07-11 10:16:29,845] : PID   1826 : DEBUG    : trollimage.xrimage : stretch : Applying stretch linear with parameters {}
[2019-07-11 10:16:29,845] : PID   1826 : DEBUG    : trollimage.xrimage : stretch_linear : Perform a linear contrast stretch.
[2019-07-11 10:16:29,845] : PID   1826 : DEBUG    : trollimage.xrimage : stretch_linear : Calculate the histogram quantiles: 
[2019-07-11 10:16:29,845] : PID   1826 : DEBUG    : trollimage.xrimage : stretch_linear : Left and right quantiles: 0.005 0.005
[2019-07-11 10:16:29,873] : PID   1826 : DEBUG    : satpy.writers : apply : Enhancement configuration options: [{'name': 'stretch', 'method': <function stretch at 0x7f1647e25158>, 'kwargs': {'stretch': 'linear'}}]
[2019-07-11 10:16:29,873] : PID   1826 : DEBUG    : trollimage.xrimage : stretch : Applying stretch linear with parameters {}
[2019-07-11 10:16:29,873] : PID   1826 : DEBUG    : trollimage.xrimage : stretch_linear : Perform a linear contrast stretch.
[2019-07-11 10:16:29,873] : PID   1826 : DEBUG    : trollimage.xrimage : stretch_linear : Calculate the histogram quantiles: 
[2019-07-11 10:16:29,873] : PID   1826 : DEBUG    : trollimage.xrimage : stretch_linear : Left and right quantiles: 0.005 0.005
[2019-07-11 10:16:29,909] : PID   1826 : DEBUG    : satpy.writers : apply : Enhancement configuration options: [{'name': 'stretch', 'method': <function stretch at 0x7f1647e25158>, 'kwargs': {'stretch': 'linear'}}]
[2019-07-11 10:16:29,909] : PID   1826 : DEBUG    : trollimage.xrimage : stretch : Applying stretch linear with parameters {}
[2019-07-11 10:16:29,909] : PID   1826 : DEBUG    : trollimage.xrimage : stretch_linear : Perform a linear contrast stretch.
[2019-07-11 10:16:29,909] : PID   1826 : DEBUG    : trollimage.xrimage : stretch_linear : Calculate the histogram quantiles: 
[2019-07-11 10:16:29,909] : PID   1826 : DEBUG    : trollimage.xrimage : stretch_linear : Left and right quantiles: 0.005 0.005
[2019-07-11 10:16:29,937] : PID   1826 : DEBUG    : satpy.writers : apply : Enhancement configuration options: [{'name': 'stretch', 'method': <function stretch at 0x7f1647e25158>, 'kwargs': {'stretch': 'linear'}}]
[2019-07-11 10:16:29,937] : PID   1826 : DEBUG    : trollimage.xrimage : stretch : Applying stretch linear with parameters {}
[2019-07-11 10:16:29,937] : PID   1826 : DEBUG    : trollimage.xrimage : stretch_linear : Perform a linear contrast stretch.
[2019-07-11 10:16:29,937] : PID   1826 : DEBUG    : trollimage.xrimage : stretch_linear : Calculate the histogram quantiles: 
[2019-07-11 10:16:29,937] : PID   1826 : DEBUG    : trollimage.xrimage : stretch_linear : Left and right quantiles: 0.005 0.005
[2019-07-11 10:16:29,961] : PID   1826 : DEBUG    : ahi_hsd_geotiff : main : Products to preserve resolution for: {DatasetID(name='B04C', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=('sunz_corrected', 'rayleigh_corrected')), DatasetID(name='B02C', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=('sunz_corrected', 'rayleigh_corrected')), DatasetID(name='B03C', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=('sunz_corrected', 'rayleigh_corrected')), DatasetID(name='B01C', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=('sunz_corrected', 'rayleigh_corrected')), DatasetID(name='B05C', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=('sunz_corrected', 'rayleigh_corrected'))}
[2019-07-11 10:16:29,961] : PID   1826 : DEBUG    : ahi_hsd_geotiff : main : Products to use new resolution for: set()
[2019-07-11 10:16:29,961] : PID   1826 : DEBUG    : ahi_hsd_geotiff : main : Setting default resampling to 'native' for grid 'MAX'
[2019-07-11 10:16:29,961] : PID   1826 : INFO     : ahi_hsd_geotiff : main : Resampling data to 'MAX'
[2019-07-11 10:16:29,962] : PID   1826 : DEBUG    : satpy.scene : __init__ : Setting 'PPP_CONFIG_DIR' to '/mnt/c/Users/BlueCARES/Downloads/geo2grid_v_1_0_0_0/etc/satpy'
[2019-07-11 10:16:29,962] : PID   1826 : DEBUG    : satpy.scene : _resampled_scene : Resampling DatasetID(name='B04C', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=('sunz_corrected', 'rayleigh_corrected'))
[2019-07-11 10:16:29,963] : PID   1826 : DEBUG    : pyresample.geometry : get_area_slices : Projections for data and slice areas are identical: geos
[2019-07-11 10:16:29,988] : PID   1826 : DEBUG    : satpy.scene : _resampled_scene : Resampling DatasetID(name='B02C', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=('sunz_corrected', 'rayleigh_corrected'))
[2019-07-11 10:16:29,988] : PID   1826 : DEBUG    : pyresample.geometry : get_area_slices : Projections for data and slice areas are identical: geos
[2019-07-11 10:16:30,013] : PID   1826 : DEBUG    : satpy.scene : _resampled_scene : Resampling DatasetID(name='B03C', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=('sunz_corrected', 'rayleigh_corrected'))
[2019-07-11 10:16:30,013] : PID   1826 : DEBUG    : pyresample.geometry : get_area_slices : Projections for data and slice areas are identical: geos
[2019-07-11 10:16:30,023] : PID   1826 : DEBUG    : satpy.scene : _resampled_scene : Resampling DatasetID(name='B01C', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=('sunz_corrected', 'rayleigh_corrected'))
[2019-07-11 10:16:30,023] : PID   1826 : DEBUG    : pyresample.geometry : get_area_slices : Projections for data and slice areas are identical: geos
[2019-07-11 10:16:30,048] : PID   1826 : DEBUG    : satpy.scene : _resampled_scene : Resampling DatasetID(name='B05C', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=('sunz_corrected', 'rayleigh_corrected'))
[2019-07-11 10:16:30,048] : PID   1826 : DEBUG    : pyresample.geometry : get_area_slices : Projections for data and slice areas are identical: geos
[2019-07-11 10:16:30,112] : PID   1826 : INFO     : ahi_hsd_geotiff : main : Computing products and saving data to writers...
[2019-07-11 10:16:33,191] : PID   1826 : DEBUG    : trollimage.xrimage : _compute_quantile : Interval: left=<xarray.DataArray 'where-158a8ec6a9eea4bd11f7188c41c2afd8' (bands: 1)>
array([10.529412])
Coordinates:
  * bands     (bands) <U1 'L'
    quantile  float64 0.005, right=<xarray.DataArray 'where-158a8ec6a9eea4bd11f7188c41c2afd8' (bands: 1)>
array([86.999996])
Coordinates:
  * bands     (bands) <U1 'L'
    quantile  float64 0.995
[2019-07-11 10:16:33,211] : PID   1826 : DEBUG    : trollimage.xrimage : _compute_quantile : Interval: left=<xarray.DataArray 'where-28602118327c2a3fddf9e7f74a64abfc' (bands: 1)>
array([7.])
Coordinates:
  * bands     (bands) <U1 'L'
    quantile  float64 0.005, right=<xarray.DataArray 'where-28602118327c2a3fddf9e7f74a64abfc' (bands: 1)>
array([85.000001])
Coordinates:
  * bands     (bands) <U1 'L'
    quantile  float64 0.995
[2019-07-11 10:16:33,832] : PID   1826 : DEBUG    : trollimage.xrimage : _compute_quantile : Interval: left=<xarray.DataArray 'where-1f7ae94d51c31e7fd543a059cde07220' (bands: 1)>
array([0.235294])
Coordinates:
  * bands     (bands) <U1 'L'
    quantile  float64 0.005, right=<xarray.DataArray 'where-1f7ae94d51c31e7fd543a059cde07220' (bands: 1)>
array([39.411764])
Coordinates:
  * bands     (bands) <U1 'L'
    quantile  float64 0.995
[2019-07-11 10:16:33,957] : PID   1826 : DEBUG    : trollimage.xrimage : _compute_quantile : Interval: left=<xarray.DataArray 'where-75cba78c48b6bb9330a2c1aefcb03ef5' (bands: 1)>
array([2.705882])
Coordinates:
  * bands     (bands) <U1 'L'
    quantile  float64 0.005, right=<xarray.DataArray 'where-75cba78c48b6bb9330a2c1aefcb03ef5' (bands: 1)>
array([85.411761])
Coordinates:
  * bands     (bands) <U1 'L'
    quantile  float64 0.995
[2019-07-11 10:16:34,002] : PID   1826 : DEBUG    : trollimage.xrimage : _compute_quantile : Interval: left=<xarray.DataArray 'where-d31c8f9ccdc510c258dc874871fede59' (bands: 1)>
array([0.941176])
Coordinates:
  * bands     (bands) <U1 'L'
    quantile  float64 0.005, right=<xarray.DataArray 'where-d31c8f9ccdc510c258dc874871fede59' (bands: 1)>
array([87.76471])
Coordinates:
  * bands     (bands) <U1 'L'
    quantile  float64 0.995
[2019-07-11 10:16:36,430] : PID   1826 : INFO     : ahi_hsd_geotiff : main : SUCCESS

Do you need to see all the .log for each band (bands 1 to 5)? Please tell me anything that can help you analyze and verify the process.

torresrodyvito · Post by **torresrodyvito** » Wed Jul 24, 2019 3:28 am

Follow up, this is what's inside the ahi.yml

Code: Select all

sensor_name: visir/ahi

composites:
  true_color_night:
    compositor: !!python/name:satpy.composites.DayNightCompositor
    prerequisites:
      - true_color
      - B14
    standard_name: day_night_mix
  B01C:
    compositor: !!python/name:satpy.composites.GenericCompositor
    prerequisites:
      - name: B01
    modifiers: [sunz_corrected, rayleigh_corrected]
    standard_name: B01C
  B04C:
    compositor: !!python/name:satpy.composites.GenericCompositor
    prerequisites:
      - name: B04
    modifiers: [sunz_corrected, rayleigh_corrected]
    standard_name: B04C
  B02C:
    compositor: !!python/name:satpy.composites.GenericCompositor
    prerequisites:
      - name: B02
    modifiers: [sunz_corrected, rayleigh_corrected]
    standard_name: B02C
  B03C:
    compositor: !!python/name:satpy.composites.GenericCompositor
    prerequisites:
      - name: B03
    modifiers: [sunz_corrected, rayleigh_corrected]
    standard_name: B03C
  B05C:
    compositor: !!python/name:satpy.composites.GenericCompositor
    prerequisites:
      - name: B05
    modifiers: [sunz_corrected, rayleigh_corrected]
    standard_name: B05C

davidh · Post by **davidh** » Wed Jul 24, 2019 10:00 am

That will work. I can tell from the log output that the wrong enhancement is being applied (the step to convert reflectances to image pixels). It is doing a dynamic linear stretch from the minimum value of the data to the maximum value. The default enhancement for reflectances in Geo2Grid is supposed to be a square root enhancement. I think the main issue is your `standard_name: B01C` for each composite. Try either removing those standard_name lines from your composites in the YAML file or change it to "standard_name: toa_bidirectional_reflectance". This will cause Geo2Grid (Satpy) to treat them as reflectances and apply the square root enhancement.

torresrodyvito · Post by **torresrodyvito** » Wed Jul 24, 2019 7:52 pm

Was the "corrected" image properly processed now?

Code: Select all

[2019-07-25 08:39:15,759] : PID     30 : DEBUG    : ahi_hsd_geotiff : main : Starting script with arguments: /mnt/c/Users/BlueCARES/Downloads/geo2grid_v_1_0_0_0/lib/python3.6/site-packages/polar2grid/glue.py -r ahi_hsd -w geotiff -p B01_corrected -f HS_H08_20181008_0200_B01_FLDK_R10_S0410.DAT -vv
[2019-07-25 08:39:15,767] : PID     30 : INFO     : ahi_hsd_geotiff : main : Sorting and reading input files...
[2019-07-25 08:39:15,767] : PID     30 : DEBUG    : satpy.scene : __init__ : Setting 'PPP_CONFIG_DIR' to '/mnt/c/Users/BlueCARES/Downloads/geo2grid_v_1_0_0_0/etc/satpy'
[2019-07-25 08:39:15,768] : PID     30 : DEBUG    : satpy.readers : read_reader_config : Reading ['/mnt/c/Users/BlueCARES/Downloads/geo2grid_v_1_0_0_0/lib/python3.6/site-packages/satpy/etc/readers/ahi_hsd.yaml']
[2019-07-25 08:39:15,862] : PID     30 : DEBUG    : satpy.readers.yaml_reader : create_filehandlers : Assigning to ahi_hsd: ['HS_H08_20181008_0200_B01_FLDK_R10_S0410.DAT']
[2019-07-25 08:39:15,870] : PID     30 : DEBUG    : satpy.composites : load_sensor_composites : Looking for composites config file ahi.yaml
[2019-07-25 08:39:15,894] : PID     30 : DEBUG    : satpy.composites : load_sensor_composites : Looking for composites config file visir.yaml
[2019-07-25 08:39:15,931] : PID     30 : DEBUG    : polar2grid.core.script_utils : rename_log_file : Log renamed from '/home/bluecares/ahi_hsd_geotiff_fail.log' to 'ahi_hsd_geotiff_20181008_020237.log'
[2019-07-25 08:39:15,931] : PID     30 : INFO     : ahi_hsd_geotiff : main : Loading product metadata from files...
[2019-07-25 08:39:15,936] : PID     30 : DEBUG    : ahi_hsd : _read_header : Band number = 1
[2019-07-25 08:39:15,936] : PID     30 : DEBUG    : ahi_hsd : _read_header : Time_interval: 2018-10-08 02:02:37.548255 - 2018-10-08 02:04:03.821116
[2019-07-25 08:39:15,939] : PID     30 : DEBUG    : ahi_hsd : read_band : Reading time 0:00:00.003363
[2019-07-25 08:39:16,063] : PID     30 : DEBUG    : ahi_hsd : calibrate : Calibration time 0:00:00.123694
[2019-07-25 08:39:16,085] : PID     30 : DEBUG    : satpy.scene : unload : Unloading dataset: DatasetID(name='B01', wavelength=(0.45, 0.47, 0.49), resolution=1000, polarization=None, calibration='reflectance', level=None, modifiers=())
[2019-07-25 08:39:16,412] : PID     30 : DEBUG    : polar2grid.grids.manager : __init__ : Using default grid configuration: 'grids.conf' 
[2019-07-25 08:39:16,413] : PID     30 : DEBUG    : polar2grid.grids.manager : __init__ : Loading grid configuration 'grids.conf'
[2019-07-25 08:39:16,422] : PID     30 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid 'dwd_germany' origin from (lon: -2.000000, lat: 56.000000) to (x: -755570.370463, y: -3554679.114710)
[2019-07-25 08:39:16,423] : PID     30 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid 'australia' origin from (lon: 105.000000, lat: 5.000000) to (x: -3896182.177765, y: 557305.257275)
[2019-07-25 08:39:16,423] : PID     30 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid 'australia2' origin from (lon: 105.000000, lat: 5.000000) to (x: -3896182.177765, y: 557305.257275)
[2019-07-25 08:39:16,423] : PID     30 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '211e' origin from (lon: -123.044000, lat: 59.844000) to (x: -1950510.636800, y: 4368587.226913)
[2019-07-25 08:39:16,423] : PID     30 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '211e_10km' origin from (lon: -123.044000, lat: 59.844000) to (x: -1950510.636800, y: 4368587.226913)
[2019-07-25 08:39:16,423] : PID     30 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '211e_hi' origin from (lon: -123.044000, lat: 59.844000) to (x: -1950510.636800, y: 4368587.226913)
[2019-07-25 08:39:16,423] : PID     30 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '211w' origin from (lon: -152.855000, lat: 54.536000) to (x: -4226045.730436, y: 4368634.765713)
[2019-07-25 08:39:16,423] : PID     30 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '211w_10km' origin from (lon: -152.855000, lat: 54.536000) to (x: -4226045.730436, y: 4368634.765713)
[2019-07-25 08:39:16,423] : PID     30 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '203' origin from (lon: 115.601000, lat: 44.646000) to (x: -4952960.856044, y: 381022.610378)
[2019-07-25 08:39:16,423] : PID     30 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '203_10km' origin from (lon: 115.601000, lat: 44.646000) to (x: -4952960.856044, y: 381022.610378)
[2019-07-25 08:39:16,423] : PID     30 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '204' origin from (lon: 110.000000, lat: 60.644000) to (x: -7314463.252793, y: 8020503.279488)
[2019-07-25 08:39:16,424] : PID     30 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '204_10km' origin from (lon: 110.000000, lat: 60.644000) to (x: -7314463.252793, y: 8020503.279488)
[2019-07-25 08:39:16,424] : PID     30 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '205' origin from (lon: -115.304000, lat: 36.257000) to (x: -4952924.911284, y: -3429056.260492)
[2019-07-25 08:39:16,424] : PID     30 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '210' origin from (lon: -77.000000, lat: 26.422000) to (x: 0.000000, y: 2864298.891608)
[2019-07-25 08:39:16,424] : PID     30 : DEBUG    : satpy.writers : read_writer_config : Reading ['/mnt/c/Users/BlueCARES/Downloads/geo2grid_v_1_0_0_0/lib/python3.6/site-packages/satpy/etc/writers/geotiff.yaml']
[2019-07-25 08:39:16,586] : PID     30 : DEBUG    : satpy.writers : apply : Enhancement configuration options: [{'name': 'linear_stretch', 'method': <function stretch at 0x7fc9eb8abe18>, 'kwargs': {'stretch': 'crude', 'min_stretch': 0.0, 'max_stretch': 100.0}}, {'name': 'gamma', 'method': <function gamma at 0x7fc9eb711048>, 'kwargs': {'gamma': 2.0}}]
[2019-07-25 08:39:16,586] : PID     30 : DEBUG    : trollimage.xrimage : stretch : Applying stretch crude with parameters {'min_stretch': 0.0, 'max_stretch': 100.0}
[2019-07-25 08:39:16,588] : PID     30 : DEBUG    : trollimage.xrimage : gamma : Applying gamma 2.0
[2019-07-25 08:39:16,602] : PID     30 : DEBUG    : ahi_hsd_geotiff : main : Products to preserve resolution for: {DatasetID(name='B01_corrected', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=('sunz_corrected', 'rayleigh_corrected'))}
[2019-07-25 08:39:16,602] : PID     30 : DEBUG    : ahi_hsd_geotiff : main : Products to use new resolution for: set()
[2019-07-25 08:39:16,602] : PID     30 : DEBUG    : ahi_hsd_geotiff : main : Setting default resampling to 'native' for grid 'MAX'
[2019-07-25 08:39:16,602] : PID     30 : INFO     : ahi_hsd_geotiff : main : Resampling data to 'MAX'
[2019-07-25 08:39:16,603] : PID     30 : DEBUG    : satpy.scene : __init__ : Setting 'PPP_CONFIG_DIR' to '/mnt/c/Users/BlueCARES/Downloads/geo2grid_v_1_0_0_0/etc/satpy'
[2019-07-25 08:39:16,603] : PID     30 : DEBUG    : satpy.scene : _resampled_scene : Resampling DatasetID(name='B01_corrected', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=('sunz_corrected', 'rayleigh_corrected'))
[2019-07-25 08:39:16,603] : PID     30 : DEBUG    : pyresample.geometry : get_area_slices : Projections for data and slice areas are identical: geos
[2019-07-25 08:39:16,613] : PID     30 : INFO     : ahi_hsd_geotiff : main : Computing products and saving data to writers...
[2019-07-25 08:39:17,831] : PID     30 : INFO     : ahi_hsd_geotiff : main : SUCCESS

This is what's inside the ahi.yml now:

Code: Select all

sensor_name: visir/ahi

composites:
  true_color_night:
    compositor: !!python/name:satpy.composites.DayNightCompositor
    prerequisites:
      - true_color
      - B14
    standard_name: day_night_mix
  B01_corrected:
    compositor: !!python/name:satpy.composites.GenericCompositor
    prerequisites:
      - name: B01
    modifiers: [sunz_corrected, rayleigh_corrected]
  B04_corrected:
    compositor: !!python/name:satpy.composites.GenericCompositor
    prerequisites:
      - name: B04
    modifiers: [sunz_corrected, rayleigh_corrected]
  B02_corrected:
    compositor: !!python/name:satpy.composites.GenericCompositor
    prerequisites:
      - name: B02
    modifiers: [sunz_corrected, rayleigh_corrected]
  B03_corrected:
    compositor: !!python/name:satpy.composites.GenericCompositor
    prerequisites:
      - name: B03
    modifiers: [sunz_corrected, rayleigh_corrected]
  B05_corrected:
    compositor: !!python/name:satpy.composites.GenericCompositor
    prerequisites:
      - name: B05
    modifiers: [sunz_corrected, rayleigh_corrected]

torresrodyvito · Post by **torresrodyvito** » Wed Jul 24, 2019 8:20 pm

Follow up.

However, when we checked the geotiff images produced using "B01_corrected" and "B01",both images have the same DN (0-255) for every pixel when inspected. Does that mean B01 was also already modified by the sunz correction and rayleigh correction by changing the ahi.yml as stated above?

davidh · Post by **davidh** » Thu Jul 25, 2019 10:42 am

I think the correct enhancement was applied based on this output in the log:

Code: Select all

[2019-07-25 08:39:16,586] : PID     30 : DEBUG    : trollimage.xrimage : stretch : Applying stretch crude with parameters {'min_stretch': 0.0, 'max_stretch': 100.0}
[2019-07-25 08:39:16,588] : PID     30 : DEBUG    : trollimage.xrimage : gamma : Applying gamma 2.0

However, I noticed something in your composite config. The `modifiers` parameter should be at the same level as the "n" in "name" like this:

Code: Select all

  B01_corrected:
    compositor: !!python/name:satpy.composites.GenericCompositor
    prerequisites:
      - name: B01
        modifiers: [sunz_corrected, rayleigh_corrected]

torresrodyvito · Post by **torresrodyvito** » Thu Jul 25, 2019 8:30 pm

Hello! It seems that the corrections were applied now. However, when trying to process for the corrected images of B01, B02, B04, and B05, the process asks for B03. And even though B03 is in the same folder/directory, it seems to not work and return an error.

.log for B03_corrected (no error)

Code: Select all

[2019-07-26 09:13:53,179] : PID     55 : DEBUG    : ahi_hsd_geotiff : main : Starting script with arguments: /mnt/c/Users/BlueCARES/Downloads/geo2grid_v_1_0_0_0/lib/python3.6/site-packages/polar2grid/glue.py -r ahi_hsd -w geotiff -p B03_corrected -f HS_H08_20181008_0200_B03_FLDK_R05_S0410.DAT -vv
[2019-07-26 09:13:53,184] : PID     55 : INFO     : ahi_hsd_geotiff : main : Sorting and reading input files...
[2019-07-26 09:13:53,184] : PID     55 : DEBUG    : satpy.scene : __init__ : Setting 'PPP_CONFIG_DIR' to '/mnt/c/Users/BlueCARES/Downloads/geo2grid_v_1_0_0_0/etc/satpy'
[2019-07-26 09:13:53,185] : PID     55 : DEBUG    : satpy.readers : read_reader_config : Reading ['/mnt/c/Users/BlueCARES/Downloads/geo2grid_v_1_0_0_0/lib/python3.6/site-packages/satpy/etc/readers/ahi_hsd.yaml']
[2019-07-26 09:13:53,269] : PID     55 : DEBUG    : satpy.readers.yaml_reader : create_filehandlers : Assigning to ahi_hsd: ['HS_H08_20181008_0200_B03_FLDK_R05_S0410.DAT']
[2019-07-26 09:13:53,276] : PID     55 : DEBUG    : satpy.composites : load_sensor_composites : Looking for composites config file ahi.yaml
[2019-07-26 09:13:53,295] : PID     55 : DEBUG    : satpy.composites : load_sensor_composites : Looking for composites config file visir.yaml
[2019-07-26 09:13:53,334] : PID     55 : DEBUG    : polar2grid.core.script_utils : rename_log_file : Log renamed from '/home/bluecares/ahi_hsd_geotiff_fail.log' to 'ahi_hsd_geotiff_20181008_020237.log'
[2019-07-26 09:13:53,334] : PID     55 : INFO     : ahi_hsd_geotiff : main : Loading product metadata from files...
[2019-07-26 09:13:53,338] : PID     55 : DEBUG    : satpy.node : _get_compositor_prereqs : Skipping optional solar_zenith_angle: Unknown dataset solar_zenith_angle
[2019-07-26 09:13:53,340] : PID     55 : DEBUG    : satpy.node : _get_compositor_prereqs : Skipping optional satellite_azimuth_angle: Unknown dataset satellite_azimuth_angle
[2019-07-26 09:13:53,341] : PID     55 : DEBUG    : satpy.node : _get_compositor_prereqs : Skipping optional satellite_zenith_angle: Unknown dataset satellite_zenith_angle
[2019-07-26 09:13:53,342] : PID     55 : DEBUG    : satpy.node : _get_compositor_prereqs : Skipping optional solar_azimuth_angle: Unknown dataset solar_azimuth_angle
[2019-07-26 09:13:53,343] : PID     55 : DEBUG    : satpy.node : _get_compositor_prereqs : Skipping optional solar_zenith_angle: Unknown dataset solar_zenith_angle
[2019-07-26 09:13:53,345] : PID     55 : DEBUG    : ahi_hsd : _read_header : Band number = 3
[2019-07-26 09:13:53,345] : PID     55 : DEBUG    : ahi_hsd : _read_header : Time_interval: 2018-10-08 02:02:37.287115 - 2018-10-08 02:04:03.559973
[2019-07-26 09:13:53,349] : PID     55 : DEBUG    : ahi_hsd : read_band : Reading time 0:00:00.004339
[2019-07-26 09:13:53,402] : PID     55 : DEBUG    : ahi_hsd : calibrate : Calibration time 0:00:00.052783
[2019-07-26 09:13:53,444] : PID     55 : DEBUG    : satpy.composites : __call__ : Applying sun zen correction
[2019-07-26 09:13:53,445] : PID     55 : DEBUG    : satpy.composites : __call__ : Computing sun zenith angles.
[2019-07-26 09:13:53,454] : PID     55 : DEBUG    : satpy.composites : _apply_correction : Apply the standard sun-zenith correction [1/cos(sunz)]
[2019-07-26 09:13:53,462] : PID     55 : DEBUG    : satpy.composites : __call__ : Sun-zenith correction applied. Computation time:   0.0 (sec)
[2019-07-26 09:13:57,700] : PID     55 : INFO     : satpy.composites : __call__ : Removing Rayleigh scattering with atmosphere 'us-standard' and aerosol type 'marine_clean_aerosol' for 'B03'
[2019-07-26 09:13:57,701] : PID     55 : INFO     : pyspectral.rayleigh : __init__ : Atmosphere chosen: us-standard
[2019-07-26 09:13:57,704] : PID     55 : DEBUG    : pyspectral.rayleigh : __init__ : LUT filename: /mnt/c/Users/BlueCARES/Downloads/geo2grid_v_1_0_0_0/pyspectral_data/marine_clean_aerosol/rayleigh_lut_us-standard.h5
[2019-07-26 09:13:57,707] : PID     55 : DEBUG    : pyspectral.rsr_reader : _get_filename : Filename: /mnt/c/Users/BlueCARES/Downloads/geo2grid_v_1_0_0_0/pyspectral_data/rsr_ahi_Himawari-8.h5
[2019-07-26 09:13:57,707] : PID     55 : DEBUG    : pyspectral.rsr_reader : __init__ : Filename: /mnt/c/Users/BlueCARES/Downloads/geo2grid_v_1_0_0_0/pyspectral_data/rsr_ahi_Himawari-8.h5
[2019-07-26 09:13:57,814] : PID     55 : DEBUG    : pyspectral.rsr_reader : load : No detectors found - assume only one...
[2019-07-26 09:13:57,841] : PID     55 : DEBUG    : pyspectral.rayleigh : get_effective_wavelength : Band name: ch3  Effective wavelength: 0.635378
[2019-07-26 09:13:57,970] : PID     55 : DEBUG    : pyspectral.rayleigh : get_reflectance : Time - Interpolation: 0.086962
[2019-07-26 09:13:57,973] : PID     55 : DEBUG    : satpy.scene : unload : Unloading dataset: DatasetID(name='B03', wavelength=(0.62, 0.64, 0.66), resolution=500, polarization=None, calibration='reflectance', level=None, modifiers=())
[2019-07-26 09:13:57,974] : PID     55 : DEBUG    : satpy.scene : unload : Unloading dataset: DatasetID(name='B03', wavelength=(0.62, 0.64, 0.66), resolution=500, polarization=None, calibration='reflectance', level=None, modifiers=('sunz_corrected',))
[2019-07-26 09:13:57,974] : PID     55 : DEBUG    : satpy.scene : unload : Unloading dataset: DatasetID(name='B03', wavelength=(0.62, 0.64, 0.66), resolution=500, polarization=None, calibration='reflectance', level=None, modifiers=('sunz_corrected', 'rayleigh_corrected'))
[2019-07-26 09:13:57,977] : PID     55 : DEBUG    : polar2grid.grids.manager : __init__ : Using default grid configuration: 'grids.conf' 
[2019-07-26 09:13:57,977] : PID     55 : DEBUG    : polar2grid.grids.manager : __init__ : Loading grid configuration 'grids.conf'
[2019-07-26 09:13:57,983] : PID     55 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid 'dwd_germany' origin from (lon: -2.000000, lat: 56.000000) to (x: -755570.370463, y: -3554679.114710)
[2019-07-26 09:13:57,983] : PID     55 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid 'australia' origin from (lon: 105.000000, lat: 5.000000) to (x: -3896182.177765, y: 557305.257275)
[2019-07-26 09:13:57,983] : PID     55 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid 'australia2' origin from (lon: 105.000000, lat: 5.000000) to (x: -3896182.177765, y: 557305.257275)
[2019-07-26 09:13:57,983] : PID     55 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '211e' origin from (lon: -123.044000, lat: 59.844000) to (x: -1950510.636800, y: 4368587.226913)
[2019-07-26 09:13:57,984] : PID     55 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '211e_10km' origin from (lon: -123.044000, lat: 59.844000) to (x: -1950510.636800, y: 4368587.226913)
[2019-07-26 09:13:57,984] : PID     55 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '211e_hi' origin from (lon: -123.044000, lat: 59.844000) to (x: -1950510.636800, y: 4368587.226913)
[2019-07-26 09:13:57,984] : PID     55 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '211w' origin from (lon: -152.855000, lat: 54.536000) to (x: -4226045.730436, y: 4368634.765713)
[2019-07-26 09:13:57,984] : PID     55 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '211w_10km' origin from (lon: -152.855000, lat: 54.536000) to (x: -4226045.730436, y: 4368634.765713)
[2019-07-26 09:13:57,984] : PID     55 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '203' origin from (lon: 115.601000, lat: 44.646000) to (x: -4952960.856044, y: 381022.610378)
[2019-07-26 09:13:57,984] : PID     55 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '203_10km' origin from (lon: 115.601000, lat: 44.646000) to (x: -4952960.856044, y: 381022.610378)
[2019-07-26 09:13:57,984] : PID     55 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '204' origin from (lon: 110.000000, lat: 60.644000) to (x: -7314463.252793, y: 8020503.279488)
[2019-07-26 09:13:57,984] : PID     55 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '204_10km' origin from (lon: 110.000000, lat: 60.644000) to (x: -7314463.252793, y: 8020503.279488)
[2019-07-26 09:13:57,984] : PID     55 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '205' origin from (lon: -115.304000, lat: 36.257000) to (x: -4952924.911284, y: -3429056.260492)
[2019-07-26 09:13:57,985] : PID     55 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '210' origin from (lon: -77.000000, lat: 26.422000) to (x: 0.000000, y: 2864298.891608)
[2019-07-26 09:13:57,985] : PID     55 : DEBUG    : satpy.writers : read_writer_config : Reading ['/mnt/c/Users/BlueCARES/Downloads/geo2grid_v_1_0_0_0/lib/python3.6/site-packages/satpy/etc/writers/geotiff.yaml']
[2019-07-26 09:13:58,138] : PID     55 : DEBUG    : satpy.writers : apply : Enhancement configuration options: [{'name': 'linear_stretch', 'method': <function stretch at 0x7f92cd5abd08>, 'kwargs': {'stretch': 'crude', 'min_stretch': 0.0, 'max_stretch': 100.0}}, {'name': 'gamma', 'method': <function gamma at 0x7f92cd5abea0>, 'kwargs': {'gamma': 2.0}}]
[2019-07-26 09:13:58,138] : PID     55 : DEBUG    : trollimage.xrimage : stretch : Applying stretch crude with parameters {'min_stretch': 0.0, 'max_stretch': 100.0}
[2019-07-26 09:13:58,139] : PID     55 : DEBUG    : trollimage.xrimage : gamma : Applying gamma 2.0
[2019-07-26 09:13:58,179] : PID     55 : DEBUG    : ahi_hsd_geotiff : main : Products to preserve resolution for: {DatasetID(name='B03_corrected', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None)}
[2019-07-26 09:13:58,179] : PID     55 : DEBUG    : ahi_hsd_geotiff : main : Products to use new resolution for: set()
[2019-07-26 09:13:58,179] : PID     55 : DEBUG    : ahi_hsd_geotiff : main : Setting default resampling to 'native' for grid 'MAX'
[2019-07-26 09:13:58,179] : PID     55 : INFO     : ahi_hsd_geotiff : main : Resampling data to 'MAX'
[2019-07-26 09:13:58,180] : PID     55 : DEBUG    : satpy.scene : __init__ : Setting 'PPP_CONFIG_DIR' to '/mnt/c/Users/BlueCARES/Downloads/geo2grid_v_1_0_0_0/etc/satpy'
[2019-07-26 09:13:58,180] : PID     55 : DEBUG    : satpy.scene : _resampled_scene : Resampling DatasetID(name='B03_corrected', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None)
[2019-07-26 09:13:58,180] : PID     55 : DEBUG    : pyresample.geometry : get_area_slices : Projections for data and slice areas are identical: geos
[2019-07-26 09:13:58,191] : PID     55 : INFO     : ahi_hsd_geotiff : main : Computing products and saving data to writers...
[2019-07-26 09:14:19,054] : PID     55 : INFO     : ahi_hsd_geotiff : main : SUCCESS

.log for B01_corrected (error: no 'hsd_b03', same errors for B02_corrected,B04_corrected,B05_corrected)

Code: Select all

[2019-07-26 09:19:50,467] : PID    136 : DEBUG    : ahi_hsd_geotiff : main : Starting script with arguments: /mnt/c/Users/BlueCARES/Downloads/geo2grid_v_1_0_0_0/lib/python3.6/site-packages/polar2grid/glue.py -r ahi_hsd -w geotiff -p B01_corrected -f HS_H08_20181008_0200_B01_FLDK_R10_S0410.DAT -vvv -vv
[2019-07-26 09:19:50,473] : PID    136 : INFO     : ahi_hsd_geotiff : main : Sorting and reading input files...
[2019-07-26 09:19:50,473] : PID    136 : DEBUG    : satpy.scene : __init__ : Setting 'PPP_CONFIG_DIR' to '/mnt/c/Users/BlueCARES/Downloads/geo2grid_v_1_0_0_0/etc/satpy'
[2019-07-26 09:19:50,474] : PID    136 : DEBUG    : satpy.readers : read_reader_config : Reading ['/mnt/c/Users/BlueCARES/Downloads/geo2grid_v_1_0_0_0/lib/python3.6/site-packages/satpy/etc/readers/ahi_hsd.yaml']
[2019-07-26 09:19:50,559] : PID    136 : DEBUG    : satpy.readers.yaml_reader : create_filehandlers : Assigning to ahi_hsd: ['HS_H08_20181008_0200_B01_FLDK_R10_S0410.DAT']
[2019-07-26 09:19:50,563] : PID    136 : DEBUG    : satpy.composites : load_sensor_composites : Looking for composites config file ahi.yaml
[2019-07-26 09:19:50,582] : PID    136 : DEBUG    : satpy.composites : load_sensor_composites : Looking for composites config file visir.yaml
[2019-07-26 09:19:50,622] : PID    136 : DEBUG    : polar2grid.core.script_utils : rename_log_file : Log renamed from '/home/bluecares/ahi_hsd_geotiff_fail.log' to 'ahi_hsd_geotiff_20181008_020237.log'
[2019-07-26 09:19:50,622] : PID    136 : INFO     : ahi_hsd_geotiff : main : Loading product metadata from files...
[2019-07-26 09:19:50,626] : PID    136 : DEBUG    : satpy.node : _get_compositor_prereqs : Skipping optional solar_zenith_angle: Unknown dataset solar_zenith_angle
[2019-07-26 09:19:50,630] : PID    136 : DEBUG    : satpy.node : _get_compositor_prereqs : Skipping optional solar_zenith_angle: Unknown dataset solar_zenith_angle
[2019-07-26 09:19:50,631] : PID    136 : DEBUG    : satpy.node : _get_compositor_prereqs : Skipping optional satellite_azimuth_angle: Unknown dataset satellite_azimuth_angle
[2019-07-26 09:19:50,632] : PID    136 : DEBUG    : satpy.node : _get_compositor_prereqs : Skipping optional satellite_zenith_angle: Unknown dataset satellite_zenith_angle
[2019-07-26 09:19:50,633] : PID    136 : DEBUG    : satpy.node : _get_compositor_prereqs : Skipping optional solar_azimuth_angle: Unknown dataset solar_azimuth_angle
[2019-07-26 09:19:50,635] : PID    136 : DEBUG    : satpy.node : _get_compositor_prereqs : Skipping optional solar_zenith_angle: Unknown dataset solar_zenith_angle
[2019-07-26 09:19:50,636] : PID    136 : DEBUG    : ahi_hsd : _read_header : Band number = 1
[2019-07-26 09:19:50,637] : PID    136 : DEBUG    : ahi_hsd : _read_header : Time_interval: 2018-10-08 02:02:37.548255 - 2018-10-08 02:04:03.821116
[2019-07-26 09:19:50,640] : PID    136 : DEBUG    : ahi_hsd : read_band : Reading time 0:00:00.004231
[2019-07-26 09:19:50,693] : PID    136 : DEBUG    : ahi_hsd : calibrate : Calibration time 0:00:00.052319
[2019-07-26 09:19:50,730] : PID    136 : WARNING  : satpy.readers.yaml_reader : _get_file_handlers : Required file type 'hsd_b03' not found or loaded for 'B03'
[2019-07-26 09:19:50,731] : PID    136 : DEBUG    : satpy.composites : __call__ : Applying sun zen correction
[2019-07-26 09:19:50,732] : PID    136 : DEBUG    : satpy.composites : __call__ : Computing sun zenith angles.
[2019-07-26 09:19:50,740] : PID    136 : DEBUG    : satpy.composites : _apply_correction : Apply the standard sun-zenith correction [1/cos(sunz)]
[2019-07-26 09:19:50,748] : PID    136 : DEBUG    : satpy.composites : __call__ : Sun-zenith correction applied. Computation time:   0.0 (sec)
[2019-07-26 09:19:50,749] : PID    136 : DEBUG    : satpy.scene : _get_prereq_datasets : Missing prerequisite for 'DatasetID(name='B03', wavelength=(0.62, 0.64, 0.66), resolution=500, polarization=None, calibration='reflectance', level=None, modifiers=('sunz_corrected',))': 'DatasetID(name='B03', wavelength=(0.62, 0.64, 0.66), resolution=500, polarization=None, calibration='reflectance', level=None, modifiers=())'
[2019-07-26 09:19:50,750] : PID    136 : DEBUG    : satpy.scene : _get_prereq_datasets : Missing prerequisite for 'DatasetID(name='B01', wavelength=(0.45, 0.47, 0.49), resolution=1000, polarization=None, calibration='reflectance', level=None, modifiers=('sunz_corrected', 'rayleigh_corrected'))': 'DatasetID(name='B03', wavelength=(0.62, 0.64, 0.66), resolution=500, polarization=None, calibration='reflectance', level=None, modifiers=('sunz_corrected',))'
[2019-07-26 09:19:50,750] : PID    136 : DEBUG    : satpy.scene : _get_prereq_datasets : Missing prerequisite for 'DatasetID(name='B01_corrected', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None)': 'DatasetID(name='B01', wavelength=(0.45, 0.47, 0.49), resolution=1000, polarization=None, calibration='reflectance', level=None, modifiers=('sunz_corrected', 'rayleigh_corrected'))'
[2019-07-26 09:19:50,750] : PID    136 : DEBUG    : satpy.scene : _get_prereq_datasets : Missing prerequisite for 'DatasetID(name='B03', wavelength=(0.62, 0.64, 0.66), resolution=500, polarization=None, calibration='reflectance', level=None, modifiers=('sunz_corrected',))': 'DatasetID(name='B03', wavelength=(0.62, 0.64, 0.66), resolution=500, polarization=None, calibration='reflectance', level=None, modifiers=())'
[2019-07-26 09:19:50,751] : PID    136 : DEBUG    : satpy.scene : _get_prereq_datasets : Missing prerequisite for 'DatasetID(name='B01', wavelength=(0.45, 0.47, 0.49), resolution=1000, polarization=None, calibration='reflectance', level=None, modifiers=('sunz_corrected', 'rayleigh_corrected'))': 'DatasetID(name='B03', wavelength=(0.62, 0.64, 0.66), resolution=500, polarization=None, calibration='reflectance', level=None, modifiers=('sunz_corrected',))'
[2019-07-26 09:19:50,751] : PID    136 : DEBUG    : satpy.scene : _get_prereq_datasets : Missing prerequisite for 'DatasetID(name='B03', wavelength=(0.62, 0.64, 0.66), resolution=500, polarization=None, calibration='reflectance', level=None, modifiers=('sunz_corrected',))': 'DatasetID(name='B03', wavelength=(0.62, 0.64, 0.66), resolution=500, polarization=None, calibration='reflectance', level=None, modifiers=())'
[2019-07-26 09:19:50,752] : PID    136 : WARNING  : satpy.scene : load : The following datasets were not created and may require resampling to be generated: DatasetID(name='B01_corrected', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None)
[2019-07-26 09:19:50,752] : PID    136 : DEBUG    : satpy.scene : unload : Unloading dataset: DatasetID(name='B01', wavelength=(0.45, 0.47, 0.49), resolution=1000, polarization=None, calibration='reflectance', level=None, modifiers=())
[2019-07-26 09:19:50,753] : PID    136 : DEBUG    : satpy.scene : unload : Unloading dataset: DatasetID(name='B01', wavelength=(0.45, 0.47, 0.49), resolution=1000, polarization=None, calibration='reflectance', level=None, modifiers=('sunz_corrected',))
[2019-07-26 09:19:50,874] : PID    136 : DEBUG    : polar2grid.grids.manager : __init__ : Using default grid configuration: 'grids.conf' 
[2019-07-26 09:19:50,875] : PID    136 : DEBUG    : polar2grid.grids.manager : __init__ : Loading grid configuration 'grids.conf'
[2019-07-26 09:19:50,882] : PID    136 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid 'dwd_germany' origin from (lon: -2.000000, lat: 56.000000) to (x: -755570.370463, y: -3554679.114710)
[2019-07-26 09:19:50,882] : PID    136 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid 'australia' origin from (lon: 105.000000, lat: 5.000000) to (x: -3896182.177765, y: 557305.257275)
[2019-07-26 09:19:50,882] : PID    136 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid 'australia2' origin from (lon: 105.000000, lat: 5.000000) to (x: -3896182.177765, y: 557305.257275)
[2019-07-26 09:19:50,883] : PID    136 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '211e' origin from (lon: -123.044000, lat: 59.844000) to (x: -1950510.636800, y: 4368587.226913)
[2019-07-26 09:19:50,883] : PID    136 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '211e_10km' origin from (lon: -123.044000, lat: 59.844000) to (x: -1950510.636800, y: 4368587.226913)
[2019-07-26 09:19:50,883] : PID    136 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '211e_hi' origin from (lon: -123.044000, lat: 59.844000) to (x: -1950510.636800, y: 4368587.226913)
[2019-07-26 09:19:50,884] : PID    136 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '211w' origin from (lon: -152.855000, lat: 54.536000) to (x: -4226045.730436, y: 4368634.765713)
[2019-07-26 09:19:50,884] : PID    136 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '211w_10km' origin from (lon: -152.855000, lat: 54.536000) to (x: -4226045.730436, y: 4368634.765713)
[2019-07-26 09:19:50,884] : PID    136 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '203' origin from (lon: 115.601000, lat: 44.646000) to (x: -4952960.856044, y: 381022.610378)
[2019-07-26 09:19:50,884] : PID    136 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '203_10km' origin from (lon: 115.601000, lat: 44.646000) to (x: -4952960.856044, y: 381022.610378)
[2019-07-26 09:19:50,885] : PID    136 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '204' origin from (lon: 110.000000, lat: 60.644000) to (x: -7314463.252793, y: 8020503.279488)
[2019-07-26 09:19:50,885] : PID    136 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '204_10km' origin from (lon: 110.000000, lat: 60.644000) to (x: -7314463.252793, y: 8020503.279488)
[2019-07-26 09:19:50,885] : PID    136 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '205' origin from (lon: -115.304000, lat: 36.257000) to (x: -4952924.911284, y: -3429056.260492)
[2019-07-26 09:19:50,886] : PID    136 : DEBUG    : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '210' origin from (lon: -77.000000, lat: 26.422000) to (x: 0.000000, y: 2864298.891608)
[2019-07-26 09:19:50,886] : PID    136 : DEBUG    : ahi_hsd_geotiff : main : Products to preserve resolution for: set()
[2019-07-26 09:19:50,886] : PID    136 : DEBUG    : ahi_hsd_geotiff : main : Products to use new resolution for: set()
[2019-07-26 09:19:50,886] : PID    136 : ERROR    : ahi_hsd_geotiff : exc_handler : Unexpected error. Enable debug messages (-vvv) or see log file for details.
[2019-07-26 09:19:50,887] : PID    136 : DEBUG    : ahi_hsd_geotiff : exc_handler : Unexpected error exception: 
Traceback (most recent call last):
  File "/mnt/c/Users/BlueCARES/Downloads/geo2grid_v_1_0_0_0/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/mnt/c/Users/BlueCARES/Downloads/geo2grid_v_1_0_0_0/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/mnt/c/Users/BlueCARES/Downloads/geo2grid_v_1_0_0_0/lib/python3.6/site-packages/polar2grid/glue.py", line 388, in <module>
    sys.exit(main())
  File "/mnt/c/Users/BlueCARES/Downloads/geo2grid_v_1_0_0_0/lib/python3.6/site-packages/polar2grid/glue.py", line 344, in main
    area_def = scn.max_area()
  File "/mnt/c/Users/BlueCARES/Downloads/geo2grid_v_1_0_0_0/lib/python3.6/site-packages/satpy/scene.py", line 277, in max_area
    return self._compare_areas(datasets=datasets, compare_func=max)
  File "/mnt/c/Users/BlueCARES/Downloads/geo2grid_v_1_0_0_0/lib/python3.6/site-packages/satpy/scene.py", line 247, in _compare_areas
    raise ValueError("No dataset areas available")
ValueError: No dataset areas available
[2019-07-26 09:19:50,892] : PID    136 : ERROR    : traceback : exc_handler : No dataset areas available
Traceback (most recent call last):
  File "/mnt/c/Users/BlueCARES/Downloads/geo2grid_v_1_0_0_0/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/mnt/c/Users/BlueCARES/Downloads/geo2grid_v_1_0_0_0/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/mnt/c/Users/BlueCARES/Downloads/geo2grid_v_1_0_0_0/lib/python3.6/site-packages/polar2grid/glue.py", line 388, in <module>
    sys.exit(main())
  File "/mnt/c/Users/BlueCARES/Downloads/geo2grid_v_1_0_0_0/lib/python3.6/site-packages/polar2grid/glue.py", line 344, in main
    area_def = scn.max_area()
  File "/mnt/c/Users/BlueCARES/Downloads/geo2grid_v_1_0_0_0/lib/python3.6/site-packages/satpy/scene.py", line 277, in max_area
    return self._compare_areas(datasets=datasets, compare_func=max)
  File "/mnt/c/Users/BlueCARES/Downloads/geo2grid_v_1_0_0_0/lib/python3.6/site-packages/satpy/scene.py", line 247, in _compare_areas
    raise ValueError("No dataset areas available")
ValueError: No dataset areas available

ahi_yml

Code: Select all

sensor_name: visir/ahi

composites:
  true_color_night:
    compositor: !!python/name:satpy.composites.DayNightCompositor
    prerequisites:
      - true_color
      - B14
    standard_name: day_night_mix
  B01_corrected:
    compositor: !!python/name:satpy.composites.GenericCompositor
    prerequisites:
      - name: B01
        modifiers: [sunz_corrected, rayleigh_corrected]
  B02_corrected:
    compositor: !!python/name:satpy.composites.GenericCompositor
    prerequisites:
      - name: B02
        modifiers: [sunz_corrected, rayleigh_corrected]
  B03_corrected:
    compositor: !!python/name:satpy.composites.GenericCompositor
    prerequisites:
      - name: B03
        modifiers: [sunz_corrected, rayleigh_corrected]
  B04_corrected:
    compositor: !!python/name:satpy.composites.GenericCompositor
    prerequisites:
      - name: B04
        modifiers: [sunz_corrected, rayleigh_corrected]
  B05_corrected:
    compositor: !!python/name:satpy.composites.GenericCompositor
    prerequisites:
      - name: B05
        modifiers: [sunz_corrected, rayleigh_corrected]

davidh · Post by **davidh** » Fri Jul 26, 2019 10:45 am

The B03 (0.64um) channel is used to perform rayleigh correction so you'll have to provide it when generating the corrected versions of the bands.

Is there a reason you are running the commands separately? If you provide all the necessary files and ask for all of the corrected versions and non-corrected versions it should all work with one command:

Code: Select all

geo2grid.sh -r ahi_hsd -w geotiff -p B01_corrected B02_corrected B03_corrected B04_corrected B05_corrected B01 B02 B03 B04 B05 -f *.DAT

getentrance18 · Post by **getentrance18** » Sat Jul 27, 2019 3:20 am

I also facing same issue...

torresrodyvito · Post by **torresrodyvito** » Sun Jul 28, 2019 8:11 pm

It works now, thanks! However, I don't think B04 and B05 were processed properly as there was a warning for both B04 and B05, namely:

Warning for B04 and B05:

Code: Select all

[2019-07-29 08:39:23,912] : PID     30 : WARNING  : pyspectral.rayleigh : get_reflectance : Effective wavelength for band B04 outside 400-800 nm range!
[2019-07-29 08:39:23,986] : PID     30 : WARNING  : pyspectral.rayleigh : get_reflectance : Effective wavelength for band B05 outside 400-800 nm range!

Moreover, a warning requiring resampling was present in the process.

Warning for resampling:

Code: Select all

[2019-07-29 08:39:23,179] : PID     30 : WARNING  : satpy.scene : load : The following datasets were not created and may require resampling to be generated: DatasetID(name='B01_corrected', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None), DatasetID(name='B02_corrected', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None), DatasetID(name='B04_corrected', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None), DatasetID(name='B05_corrected', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None)

Do I need to resample in order for it to be processed properly? If yes, is there a resampling technique in geo2grid for HSD files?
For B04 and B05, does the warning affects the processed products? Moreover, when I compared the reflectances/DN of water pixels of the corrected images, B04 and B05 have higher values than the other bands, which is unlikely for water pixels.

SSEC forums

No filenames found for reader ahi_hsd

Re: No filenames found for reader ahi_hsd

Re: No filenames found for reader ahi_hsd

Re: No filenames found for reader ahi_hsd

Re: No filenames found for reader ahi_hsd

Re: No filenames found for reader ahi_hsd

Re: No filenames found for reader ahi_hsd

Re: No filenames found for reader ahi_hsd

Re: No filenames found for reader ahi_hsd

Re: No filenames found for reader ahi_hsd

Re: No filenames found for reader ahi_hsd