Page 1 of 1

Fail to run polar2gridv2.3 nucaps and viirs_edr_active_fires

Posted: Fri Oct 15, 2021 3:18 am
by suwei
Dear all:

I installed polar2grid2.3 on centos7.8 and passed the test example. I have also tested several other readers, and they are running very well.
But when I use nucaps or viirs_edr_active_fires as the reader, I encountered some problems. I attached the error log below:

1. viirs_edr_active_fires:
The command I used is:
polar2grid.sh viirs_edr_active_fires gtiff -p confidence_pct T13 -f ../../active_fire/AFMOD_j01_d20210908_t0644280_e0645526_b19719_c20210909034222777833_cspp_dev.nc

result is:
INFO : Initializing reader...
INFO : Initializing remapping...
INFO : Initializing backend...
INFO : Initializing compositor objects...
INFO : Extracting swaths from data files available...
INFO : Writing longitude data to disk cache...
ERROR : Frontend data extraction failed (see log for details)

logfile is:
[2021-10-15 14:41:42,109] : PID 103198 : DEBUG : viirs_edr_active_fires2gtiff : main : Starting script with arguments: /home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/site-packages/polar2grid/glue_legacy.py viirs_edr_active_fires gtiff -p confidence_pct T13 -f ../../active_fire/AFMOD_j01_d20210908_t0644280_e0645526_b19719_c20210909034222777833_cspp_dev.nc -vv
[2021-10-15 14:41:42,109] : PID 103198 : INFO : viirs_edr_active_fires2gtiff : main : Initializing reader...
[2021-10-15 14:41:42,110] : PID 103198 : DEBUG : satpy.scene : __init__ : Setting 'PPP_CONFIG_DIR' to '/home/qsw/program/p2g/polar2grid_v_2_3/etc/satpy'
[2021-10-15 14:41:42,112] : PID 103198 : DEBUG : satpy.readers : read_reader_config : Reading ['/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/site-packages/satpy/etc/readers/viirs_edr_active_fires.yaml']
[2021-10-15 14:41:42,360] : PID 103198 : DEBUG : satpy.readers.yaml_reader : create_filehandlers : Assigning to viirs_edr_active_fires: ['/mnt/hgfs/p2g_test/active_fire/AFMOD_j01_d20210908_t0644280_e0645526_b19719_c20210909034222777833_cspp_dev.nc']
[2021-10-15 14:41:42,382] : PID 103198 : DEBUG : satpy.composites : load_sensor_composites : Looking for composites config file viirs.yaml
[2021-10-15 14:41:42,515] : PID 103198 : DEBUG : satpy.composites : load_sensor_composites : Looking for composites config file visir.yaml
[2021-10-15 14:41:42,593] : PID 103198 : DEBUG : polar2grid.core.script_utils : rename_log_file : Log renamed from '/mnt/hgfs/p2g_test/output/active_fire/viirs_edr_active_fires2gtiff_fail.log' to 'viirs_edr_active_fires2gtiff_20210908_064428.log'
[2021-10-15 14:41:42,594] : PID 103198 : INFO : viirs_edr_active_fires2gtiff : main : Initializing remapping...
[2021-10-15 14:41:42,594] : PID 103198 : DEBUG : polar2grid.grids.manager : __init__ : Using default grid configuration: 'grids.conf'
[2021-10-15 14:41:42,594] : PID 103198 : DEBUG : polar2grid.grids.manager : __init__ : Loading grid configuration 'grids.conf'
[2021-10-15 14:41:42,685] : PID 103198 : DEBUG : polar2grid.grids.manager : parse_proj4_config_line : Converted grid 'dwd_germany' origin from (lon: -2.000000, lat: 56.000000) to (x: -755570.370463, y: -3554679.114710)
[2021-10-15 14:41:42,713] : PID 103198 : DEBUG : polar2grid.grids.manager : parse_proj4_config_line : Converted grid 'australia' origin from (lon: 105.000000, lat: 5.000000) to (x: -3896182.177765, y: 557305.257275)
[2021-10-15 14:41:42,715] : PID 103198 : DEBUG : polar2grid.grids.manager : parse_proj4_config_line : Converted grid 'australia2' origin from (lon: 105.000000, lat: 5.000000) to (x: -3896182.177765, y: 557305.257275)
[2021-10-15 14:41:42,717] : PID 103198 : DEBUG : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '211e' origin from (lon: -123.044000, lat: 59.844000) to (x: -1950510.636800, y: 4368587.226913)
[2021-10-15 14:41:42,719] : PID 103198 : DEBUG : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '211e_10km' origin from (lon: -123.044000, lat: 59.844000) to (x: -1950510.636800, y: 4368587.226913)
[2021-10-15 14:41:42,721] : PID 103198 : DEBUG : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '211e_hi' origin from (lon: -123.044000, lat: 59.844000) to (x: -1950510.636800, y: 4368587.226913)
[2021-10-15 14:41:42,723] : PID 103198 : DEBUG : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '211w' origin from (lon: -152.855000, lat: 54.536000) to (x: -4226045.730436, y: 4368634.765713)
[2021-10-15 14:41:42,724] : PID 103198 : DEBUG : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '211w_10km' origin from (lon: -152.855000, lat: 54.536000) to (x: -4226045.730436, y: 4368634.765713)
[2021-10-15 14:41:42,726] : PID 103198 : DEBUG : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '203' origin from (lon: 115.601000, lat: 44.646000) to (x: -4952960.856044, y: 381022.610378)
[2021-10-15 14:41:42,728] : PID 103198 : DEBUG : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '203_10km' origin from (lon: 115.601000, lat: 44.646000) to (x: -4952960.856044, y: 381022.610378)
[2021-10-15 14:41:42,730] : PID 103198 : DEBUG : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '204' origin from (lon: 110.000000, lat: 60.644000) to (x: -7314463.252793, y: 8020503.279488)
[2021-10-15 14:41:42,732] : PID 103198 : DEBUG : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '204_10km' origin from (lon: 110.000000, lat: 60.644000) to (x: -7314463.252793, y: 8020503.279488)
[2021-10-15 14:41:42,734] : PID 103198 : DEBUG : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '205' origin from (lon: -115.304000, lat: 36.257000) to (x: -4952924.911284, y: -3429056.260492)
[2021-10-15 14:41:42,735] : PID 103198 : DEBUG : polar2grid.grids.manager : parse_proj4_config_line : Converted grid '210' origin from (lon: -77.000000, lat: 26.422000) to (x: 0.000000, y: 2864298.891608)
[2021-10-15 14:41:42,736] : PID 103198 : INFO : viirs_edr_active_fires2gtiff : main : Initializing backend...
[2021-10-15 14:41:42,737] : PID 103198 : DEBUG : polar2grid.core.rescale : __init__ : Loading rescale configuration files:
polar2grid.core:rescale_configs/rescale.ini
[2021-10-15 14:41:42,739] : PID 103198 : DEBUG : polar2grid.core.roles : open_config_file : Loading package provided configuration file: 'polar2grid.core:rescale_configs/rescale.ini'
[2021-10-15 14:41:42,775] : PID 103198 : INFO : viirs_edr_active_fires2gtiff : main : Initializing compositor objects...
[2021-10-15 14:41:42,777] : PID 103198 : INFO : viirs_edr_active_fires2gtiff : main : Extracting swaths from data files available...
[2021-10-15 14:41:42,780] : PID 103198 : DEBUG : polar2grid.readers : create_scene : Loading scene data...
[2021-10-15 14:41:42,781] : PID 103198 : DEBUG : satpy.readers.yaml_reader : _load_dataset_area : No coordinates found for DatasetID(name='latitude', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=())
[2021-10-15 14:41:42,820] : PID 103198 : DEBUG : satpy.readers.yaml_reader : _load_dataset_area : No coordinates found for DatasetID(name='longitude', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=())
[2021-10-15 14:41:43,032] : PID 103198 : INFO : polar2grid.readers : area_to_swath_def : Writing longitude data to disk cache...
[2021-10-15 14:41:43,035] : PID 103198 : DEBUG : viirs_edr_active_fires2gtiff : main : Frontend data extraction exception:
Traceback (most recent call last):
File "/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/site-packages/polar2grid/glue_legacy.py", line 338, in main
scene = convert_satpy_to_p2g_swath(f, scene)
File "/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/site-packages/polar2grid/readers/__init__.py", line 317, in convert_satpy_to_p2g_swath
overwrite_existing=overwrite_existing)
File "/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/site-packages/polar2grid/readers/__init__.py", line 113, in area_to_swath_def
lon_arr = np.memmap(filename, mode="w+", dtype=lons.dtype, shape=lons.shape)
File "/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/site-packages/numpy/core/memmap.py", line 264, in __new__
mm = mmap.mmap(fid.fileno(), bytes, access=acc, offset=start)
ValueError: cannot mmap an empty file
[2021-10-15 14:41:43,037] : PID 103198 : ERROR : viirs_edr_active_fires2gtiff : main : Frontend data extraction failed (see log for details)





2. nucaps:
My data name is different from the data name on the nucaps website(https://www.ssec.wisc.edu/software/pola ... ucaps.html). The name format required on the website is: NUCAPS-EDR_v1r0_npp_s201603011158009_e201603011158307_c201603011222270.nc. But the data name format I got is NUCAPS-AGG-EDR_v2r0_m02_s202107270107170_e202107270119160_c2022107nc. And the data cannot be recognized by polar2gird. Then I changed the name of my data to NUCAPS-EDR_v2r0_m02_s202107270107170_e202107270119160_c202107270133220.nc. I tried to list products, but an error was reported. The result shows that there was no “mask surface” in the data.

The command I used is:
polar2grid.sh nucaps gtiff --list-products -f ../../nucaps/m02.202107270929test/NUCAPS-EDR_v2r0_m02_s202107270107170_e202107270119160_c202107270133220.nc

result is:
INFO : Initializing reader...
ERROR : Unexpected error. Enable debug messages (-vvv) or see log file for details.

logfile is:
[2021-10-15 15:31:58,282] : PID 76403 : DEBUG : nucaps2gtiff : main : Starting script with arguments: /home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/site-packages/polar2grid/glue_legacy.py nucaps gtiff --list-products -f ../../nucaps/m02.202107270929test/NUCAPS-EDR_v2r0_m02_s202107270107170_e202107270119160_c202107270133220.nc -vv
[2021-10-15 15:31:58,283] : PID 76403 : INFO : nucaps2gtiff : main : Initializing reader...
[2021-10-15 15:31:58,296] : PID 76403 : DEBUG : satpy.scene : __init__ : Setting 'PPP_CONFIG_DIR' to '/home/qsw/program/p2g/polar2grid_v_2_3/etc/satpy'
[2021-10-15 15:31:58,301] : PID 76403 : DEBUG : satpy.readers : read_reader_config : Reading ['/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/site-packages/satpy/etc/readers/nucaps.yaml']
[2021-10-15 15:31:58,445] : PID 76403 : DEBUG : satpy.readers.yaml_reader : create_filehandlers : Assigning to nucaps: ['/mnt/hgfs/p2g_test/nucaps/m02.202107270929test/NUCAPS-EDR_v2r0_m02_s202107270107170_e202107270119160_c202107270133220.nc']
[2021-10-15 15:31:58,452] : PID 76403 : ERROR : nucaps2gtiff : exc_handler : Unexpected error. Enable debug messages (-vvv) or see log file for details.
[2021-10-15 15:31:58,452] : PID 76403 : DEBUG : nucaps2gtiff : exc_handler : Unexpected error exception:
Traceback (most recent call last):
File "/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/site-packages/polar2grid/glue_legacy.py", line 506, in <module>
sys.exit(main())
File "/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/site-packages/polar2grid/glue_legacy.py", line 285, in main
f = fcls(search_paths=args.data_files, **args.subgroup_args["Frontend Initialization"])
File "/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/site-packages/polar2grid/readers/nucaps.py", line 290, in __init__
super(Frontend, self).__init__(**kwargs)
File "/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/site-packages/polar2grid/readers/__init__.py", line 362, in __init__
self.scene = Scene(reader=self.reader, filenames=pathnames, reader_kwargs=kwargs)
File "/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/site-packages/satpy/scene.py", line 157, in __init__
reader_kwargs=reader_kwargs)
File "/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/site-packages/satpy/scene.py", line 202, in create_reader_instances
ppp_config_dir=self.ppp_config_dir)
File "/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/site-packages/satpy/readers/__init__.py", line 727, in load_readers
reader_instance.create_filehandlers(loadables, fh_kwargs=reader_kwargs_without_filter)
File "/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/site-packages/satpy/readers/yaml_reader.py", line 531, in create_filehandlers
fh_kwargs=fh_kwargs)
File "/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/site-packages/satpy/readers/yaml_reader.py", line 517, in new_filehandlers_for_filetype
return list(filtered_iter)
File "/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/site-packages/satpy/readers/yaml_reader.py", line 488, in filter_fh_by_metadata
for filehandler in filehandlers:
File "/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/site-packages/satpy/readers/yaml_reader.py", line 429, in new_filehandler_instances
yield filetype_cls(filename, filename_info, filetype_info, *req_fh, **fh_kwargs)
File "/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/site-packages/satpy/readers/nucaps.py", line 57, in __init__
super(NUCAPSFileHandler, self).__init__(*args, **kwargs)
TypeError: __init__() got an unexpected keyword argument 'mask_surface'
[2021-10-15 15:31:58,456] : PID 76403 : ERROR : traceback : exc_handler : __init__() got an unexpected keyword argument 'mask_surface'
Traceback (most recent call last):
File "/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/site-packages/polar2grid/glue_legacy.py", line 506, in <module>
sys.exit(main())
File "/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/site-packages/polar2grid/glue_legacy.py", line 285, in main
f = fcls(search_paths=args.data_files, **args.subgroup_args["Frontend Initialization"])
File "/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/site-packages/polar2grid/readers/nucaps.py", line 290, in __init__
super(Frontend, self).__init__(**kwargs)
File "/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/site-packages/polar2grid/readers/__init__.py", line 362, in __init__
self.scene = Scene(reader=self.reader, filenames=pathnames, reader_kwargs=kwargs)
File "/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/site-packages/satpy/scene.py", line 157, in __init__
reader_kwargs=reader_kwargs)
File "/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/site-packages/satpy/scene.py", line 202, in create_reader_instances
ppp_config_dir=self.ppp_config_dir)
File "/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/site-packages/satpy/readers/__init__.py", line 727, in load_readers
reader_instance.create_filehandlers(loadables, fh_kwargs=reader_kwargs_without_filter)
File "/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/site-packages/satpy/readers/yaml_reader.py", line 531, in create_filehandlers
fh_kwargs=fh_kwargs)
File "/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/site-packages/satpy/readers/yaml_reader.py", line 517, in new_filehandlers_for_filetype
return list(filtered_iter)
File "/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/site-packages/satpy/readers/yaml_reader.py", line 488, in filter_fh_by_metadata
for filehandler in filehandlers:
File "/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/site-packages/satpy/readers/yaml_reader.py", line 429, in new_filehandler_instances
yield filetype_cls(filename, filename_info, filetype_info, *req_fh, **fh_kwargs)
File "/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/site-packages/satpy/readers/nucaps.py", line 57, in __init__
super(NUCAPSFileHandler, self).__init__(*args, **kwargs)
TypeError: __init__() got an unexpected keyword argument 'mask_surface'

Maybe it's a problem with the data? If it is a problem of my data, where can I get the test data of nucaps and viirs_edr_active_fires.

Thanks.
Suwei

Re: Fail to run polar2gridv2.3 nucaps and viirs_edr_active_f

Posted: Fri Oct 15, 2021 2:22 pm
by kathys
Hello,

We believe the reason that the viirs_edr_active_fire reader is failing is because there are no fires in your dataset.

The data file that you are providing to the NUCAPS reader is a NUCAPS-AGG-EDR file. These are pass aggregated files that have a different internal structure from the NUCAPS-EDR granule base files. The Polar2Grid reader will not work on these files. Can you please try executing the software on a file with this prefix: NUCAPS-EDR*.nc

Can you please stage the AFMOD data file for us to investigate? If you upload the file:

AFMOD_j01_d20210908_t0644280_e0645526_b19719_c20210909034222777833_cspp_dev.nc

to our anonymous ftp site at: ftp://ftp.ssec.wisc.edu/pub/incoming

we will take a close look at it and determine exactly why the reader failed.

Cheers,

Kathy

Re: Fail to run polar2gridv2.3 nucaps and viirs_edr_active_f

Posted: Sun Oct 17, 2021 9:59 am
by suwei
Thank you. I have uploaded the file to FTP:
AFMOD_j01_d20210908_t0644280_e0645526_b19719_c20210909034222777833_cspp_dev.nc
AFMOD_j01_d20210908_t0651350_e0652578_b19720_c20210909034222777870_cspp_dev.nc

Re: Fail to run polar2gridv2.3 nucaps and viirs_edr_active_f

Posted: Mon Oct 18, 2021 8:26 am
by suwei
Hello Kathy,

I found some NUCAPS-EDR files today. I tried to use polar2grid projecting the files but failed. Log file said " super(NUCAPSFileHandler, self).__init__(*args, **kwargs)TypeError: __init__() got an unexpected keyword argument 'mask_surface'".

My command is :
polar2grid.sh nucaps gtiff -f ../../nucaps_edr/NUCAPS-EDR_v2r0_m01_s202107261352090_e202107261355120_c202107261511510.nc

Maybe my command is wrong?

Log file :
[2021-10-18 21:11:49,691] : PID 101594 : DEBUG : nucaps2gtiff : main : Starting script with arguments: /home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/site-packages/polar2grid/glue_legacy.py nucaps gtiff -f ../../nucaps_edr/NUCAPS-EDR_v2r0_m01_s202107261352090_e202107261355120_c202107261511510.nc -vv
[2021-10-18 21:11:49,692] : PID 101594 : INFO : nucaps2gtiff : main : Initializing reader...
[2021-10-18 21:11:49,693] : PID 101594 : DEBUG : satpy.scene : __init__ : Setting 'PPP_CONFIG_DIR' to '/home/qsw/program/p2g/polar2grid_v_2_3/etc/satpy'
[2021-10-18 21:11:49,696] : PID 101594 : DEBUG : satpy.readers : read_reader_config : Reading ['/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/site-packages/satpy/etc/readers/nucaps.yaml']
[2021-10-18 21:11:49,836] : PID 101594 : DEBUG : satpy.readers.yaml_reader : create_filehandlers : Assigning to nucaps: ['/mnt/hgfs/p2g_test/nucaps_edr/NUCAPS-EDR_v2r0_m01_s202107261352090_e202107261355120_c202107261511510.nc']
[2021-10-18 21:11:49,841] : PID 101594 : ERROR : nucaps2gtiff : exc_handler : Unexpected error. Enable debug messages (-vvv) or see log file for details.
[2021-10-18 21:11:49,842] : PID 101594 : DEBUG : nucaps2gtiff : exc_handler : Unexpected error exception:
Traceback (most recent call last):
File "/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/site-packages/polar2grid/glue_legacy.py", line 506, in <module>
sys.exit(main())
File "/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/site-packages/polar2grid/glue_legacy.py", line 285, in main
f = fcls(search_paths=args.data_files, **args.subgroup_args["Frontend Initialization"])
File "/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/site-packages/polar2grid/readers/nucaps.py", line 290, in __init__
super(Frontend, self).__init__(**kwargs)
File "/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/site-packages/polar2grid/readers/__init__.py", line 362, in __init__
self.scene = Scene(reader=self.reader, filenames=pathnames, reader_kwargs=kwargs)
File "/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/site-packages/satpy/scene.py", line 157, in __init__
reader_kwargs=reader_kwargs)
File "/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/site-packages/satpy/scene.py", line 202, in create_reader_instances
ppp_config_dir=self.ppp_config_dir)
File "/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/site-packages/satpy/readers/__init__.py", line 727, in load_readers
reader_instance.create_filehandlers(loadables, fh_kwargs=reader_kwargs_without_filter)
File "/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/site-packages/satpy/readers/yaml_reader.py", line 532, in create_filehandlers
fh_kwargs=fh_kwargs)
File "/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/site-packages/satpy/readers/yaml_reader.py", line 518, in new_filehandlers_for_filetype
return list(filtered_iter)
File "/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/site-packages/satpy/readers/yaml_reader.py", line 489, in filter_fh_by_metadata
for filehandler in filehandlers:
File "/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/site-packages/satpy/readers/yaml_reader.py", line 429, in new_filehandler_instances
yield filetype_cls(filename, filename_info, filetype_info, *req_fh, **fh_kwargs)
File "/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/site-packages/satpy/readers/nucaps.py", line 58, in __init__
super(NUCAPSFileHandler, self).__init__(*args, **kwargs)
TypeError: __init__() got an unexpected keyword argument 'mask_surface'
[2021-10-18 21:11:49,846] : PID 101594 : ERROR : traceback : exc_handler : __init__() got an unexpected keyword argument 'mask_surface'
Traceback (most recent call last):
File "/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/site-packages/polar2grid/glue_legacy.py", line 506, in <module>
sys.exit(main())
File "/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/site-packages/polar2grid/glue_legacy.py", line 285, in main
f = fcls(search_paths=args.data_files, **args.subgroup_args["Frontend Initialization"])
File "/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/site-packages/polar2grid/readers/nucaps.py", line 290, in __init__
super(Frontend, self).__init__(**kwargs)
File "/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/site-packages/polar2grid/readers/__init__.py", line 362, in __init__
self.scene = Scene(reader=self.reader, filenames=pathnames, reader_kwargs=kwargs)
File "/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/site-packages/satpy/scene.py", line 157, in __init__
reader_kwargs=reader_kwargs)
File "/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/site-packages/satpy/scene.py", line 202, in create_reader_instances
ppp_config_dir=self.ppp_config_dir)
File "/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/site-packages/satpy/readers/__init__.py", line 727, in load_readers
reader_instance.create_filehandlers(loadables, fh_kwargs=reader_kwargs_without_filter)
File "/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/site-packages/satpy/readers/yaml_reader.py", line 532, in create_filehandlers
fh_kwargs=fh_kwargs)
File "/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/site-packages/satpy/readers/yaml_reader.py", line 518, in new_filehandlers_for_filetype
return list(filtered_iter)
File "/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/site-packages/satpy/readers/yaml_reader.py", line 489, in filter_fh_by_metadata
for filehandler in filehandlers:
File "/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/site-packages/satpy/readers/yaml_reader.py", line 429, in new_filehandler_instances
yield filetype_cls(filename, filename_info, filetype_info, *req_fh, **fh_kwargs)
File "/home/qsw/program/p2g/polar2grid_v_2_3/lib/python3.7/site-packages/satpy/readers/nucaps.py", line 58, in __init__
super(NUCAPSFileHandler, self).__init__(*args, **kwargs)
TypeError: __init__() got an unexpected keyword argument 'mask_surface'

I uploaded the file to FTP.

Thank you for your help!

suwei

Re: Fail to run polar2gridv2.3 nucaps and viirs_edr_active_f

Posted: Tue Oct 19, 2021 10:43 am
by kathys
Hello Suwei,

Thank you for providing your data files to us.

Active Fires
The reason that you are not able to create images using your VIIRS EDR Active Fires files is because the files that your are providing do not contain any data, meaning there were no fires found in the granules. When I dump out the contents of your files, I see this:

ncdump -h AFMOD_j01_d20210908_t0644280_e0645526_b19719_c20210909034222777833_cspp_dev.nc | grep "nfire ="
nfire = UNLIMITED ; // (0 currently)
ncdump -h AFMOD_j01_d20210908_t0651350_e0652578_b19720_c20210909034222777870_cspp_dev.nc | grep "nfire ="
nfire = UNLIMITED ; // (0 currently)

Polar2Grid is not providing a very useful message when this happens. We will try and add better text when when it fails because of no fires found.

NUCAPS EDR

We have examined the Polar2Grid reader and the NUCAPS EDR version 2.0 files, and you are correct. The new files do not seem to be supported by Polar2Grid Version 2.3. We are investigating the best advice we can give you concerning these files. I will write a response then.

Cheers,

Kathy

Re: Fail to run polar2gridv2.3 nucaps and viirs_edr_active_f

Posted: Thu Oct 21, 2021 12:30 am
by suwei
Dear Kathy:

Thank you for your reply! It helps me a lot.

When I tried the eample, I found a problem in the website(https://www.ssec.wisc.edu/software/pola ... avhrr.html). The product_name should be band4_bt and band5_bt instead of band4_vis band5_vis.

Thank you again for your help!

Sincerely,

suwei

Re: Fail to run polar2gridv2.3 nucaps and viirs_edr_active_f

Posted: Thu Oct 21, 2021 10:20 am
by kathys
Hello Suwei,

The problem that your found with the Polar2Grid AVHRR reader has been corrected and will be updated in the next release.

Thank you,

Kathy

Re: Fail to run polar2gridv2.3 nucaps and viirs_edr_active_f

Posted: Thu Oct 21, 2021 4:19 pm
by kathys
Hello again Suwei,

Polar2Grid Version 2.3 definitely does not work on NUCAPS EDR files. This will be fixed for the next version of Polar2Grid

In the meantime, the CSPP Sounder Quicklook package can create both reprojected pressure level plots of temperature and moisture as well as Skew-T Thermodynamic diagrams from both CSPP NUCAPS EDR v1r0 and v2r0 NetCDF input files. You can find out more about that software here: https://cimss.ssec.wisc.edu/cspp/sounder_ql_v1.0.shtml

If you have any more questions, or if the Quicklook software does not meet your needs, please let us know.

Cheers,

Kathy