Page 1 of 1

crefl2gtiff issue

Posted: Tue Apr 08, 2014 8:33 am
by junzhao
Hi colleagues,
I tried to run crefl2gtiff with sample data from ftp://ftp.ssec.wisc.edu/pub/gumley/viirs_crefl/data/.
I followed the steps described at viewtopic.php?f=42&t=304
install_crefl.bash and run_viirs_crefl.bash ran well and generated CREFL*.hdf files. However, when I ran crefl2gtiff.sh, it failed. The error message is attached here:

INFO : Using default grid configuration: 'grids.conf'
INFO : Loading grid configuration 'grids.conf'
INFO : Loading package provided rescale config: 'rescale_configs/rescale_inc.8bit.conf'
INFO : Extracting swaths...
INFO : Loading meta data from /home/junzhao/Documents/VIIRS_CSPP/CREFLM_npp_d20130331_t1802568_e1804210.hdf
INFO : Writing swath binary files for band kind crefl band 01
INFO : Writing swath binary files for band kind crefl band 04
INFO : Writing swath binary files for band kind crefl band 03
INFO : Writing navigation binary files for nav. set m_nav
INFO : Determining what grids the data fits in...
INFO : Running ll2cr for grid wgs84_fit and bands [('crefl', '01'), ('crefl', '04'), ('crefl', '03')]
INFO : row and column calculation complete
ERROR : Couldn't find 'fornav' command in PATH
WARNING : fornav failed for m_nav band, grid wgs84_fit, remapping as corrected_reflectance
WARNING : Cleaning up for this job...
ERROR : Removing crefl01 because of bad fornav execution
ERROR : Removing crefl04 because of bad fornav execution
ERROR : Removing crefl03 because of bad fornav execution
ERROR : The last grid job for grid wgs84_fit was removed
ERROR : Fornav was not able to complete any remapping for navigation set m_nav
ERROR : Remapping data failed

Any suggestions would be greatly appreciated.
Jun

Re: crefl2gtiff issue

Posted: Tue Apr 08, 2014 9:14 am
by davidh
Hi,

Are you using the polar2grid software bundle provided through the CSPP website? The error messages you posted are saying they can't find the "fornav" command. If you are using the software bundle on a RHEL5+ system then it should already be provided. Try running the following to see if it gives a list of possible command line flags:

/path/to/bundle/bin/fornav -h

Let me know what happens.

Re: crefl2gtiff issue

Posted: Thu Apr 10, 2014 12:22 am
by junzhao
Hi David,
Thanks for you reply
Running the command got the following message:

fornav: Parameter chan_count is invalid.
$Revision: 1.32 $
usage: fornav chan_count
[-v] [-m]
[-s chan_scan_first colrow_scan_first]
defaults: 0 0
[-S grid_col_start grid_row_start]
defaults: 0 0 0
[-t swath_data_type_1 ... swath_data_type_chan_count]
defaults: s2 s2
[-T grid_data_type_1 ... grid_data_type_chan_count]
defaults: swath_data_type_1 swath_data_type_chan_count]
[-f swath_fill_1 ... swath_fill_chan_count]
defaults: 0 0
[-F grid_fill_1 ... grid_fill_chan_count]
defaults: swath_fill_1 swath_fill_chan_count
[-r col_row_min]
defaults: 0
[-c weight_count] [-w weight_min] [-d weight_distance_max]
defaults: 10000 .01 1.0
[-D weight_delta_max] [-W weight_sum_min]
defaults: 10.0 weight_min
[-C swath_col_offset -R swath_row_offset
defaults: 0 0
swath_cols swath_scans swath_rows_per_scan
swath_col_file swath_row_file
swath_chan_file_1 ... swath_chan_file_chan_count
grid_cols grid_rows
grid_chan_file_1 ... grid_chan_file_chan_count

input : chan_count: number of input and output channel files. This parameter
must precede any specified options.
swath_cols: number of columns in each input swath file.
swath_scans: number of scans in each input swath file.
swath_rows_per_scan: number of swath rows constituting a scan.
Must be at least 2.
swath_col_file: file containing the projected column number of each
swath cell and consisting of swatch_cols x swath_rows of 4 byte
floating-point numbers.
swath_row_file: file containing the projected row number of each
swath cell and consisting of swatch_cols x swath_rows of 4 byte
floating-point numbers.
swath_chan_file_1 ... swath_chan_file_chan_count: swath channel files
1 through chan_count. Each file consists of swath_cols x swath_rows
cells as indicated by swath_data_type (see below).
grid_cols: number of columns in each output grid file.
grid_rows: number of rows in each output grid file.

output: grid_chan_file_1 ... grid_chan_file_chan_count: grid channel files
1 through chan_count. Each file consists of grid_cols x grid_rows
cells as indicated by grid_type (see below).

option: v: verbose (may be repeated).
m: maximum weight mode. If -m is not present, a weighted average of
all swath cells that map to a particular grid cell is used.
If -m is present, the swath cell having the maximum weight of all
swath cells that map to a particular grid cell is used. The -m
option should be used for coded data, i.e. snow cover.
s chan_scan_first colrow_scan_first: the first scan number to process
in the swath channel files and column and row files, respectively.
Default is 0 for both.
S grid_col_start grid_row_start: starting grid column number and row
number to write to each output grid file. The defaults are 0.
t swath_data_type_1 ... swath_data_type_chan_count: specifies the type
of each swath cell for each channel as follows:
u1: unsigned 8-bit integer.
u2: unsigned 16-bit integer.
s2: signed 16-bit integer (default).
u4: unsigned 32-bit integer.
s4: signed 32-bit integer.
f4: 32-bit floating-point.
T grid_data_type_1 ... grid_data_type_chan_count: specifies the type
of each grid cell for each channel as in the -t option. If the
default value is the corresponding swath data type value.
f swath_fill_1 ... swath_fill_chan_count: specifies fill value to use
for detecting any missing cells in each swath file. Missing swath
cells are ignored. The default value is 0.
F grid_fill_1 ... grid_fill_chan_count: specifies fill value to use
for any unmapped cells in each grid file. The default value is the
corresponding swath fill value.
r col_row_min: specifies minimum value to use for detecting any
missing cells in each column and row file. Missing swath cells
have column and row values less than this value and are
ignored. The default value is 0.
c weight_count: number of elements to create in the gaussian weight
table. Default is 10000. Must be at least 2.
w weight_min: the minimum value to store in the last position of the
weight table. Default is 0.01, which, with a weight_distance_max
of 1.0 produces a weight of 0.01 at a grid cell distance of 1.0.
Must be greater than 0.
d weight_distance_max: distance in grid cell units at which to apply a
weight of weight_min. Default is 1.0. Must be greater than 0.
D weight_delta_max: maximum distance in grid cells in each grid
dimension over which to distribute a single swath cell.
Default is 10.0.
W weight_sum_min: minimum weight sum value. Cells whose weight sums
are less than weight_sum_min are set to the grid fill value.
Default is weight_sum_min.
C swath_col_offset: value to be subtracted from each value in
swath_col_file. Default is 0.
R swath_row_offset: value to be subtracted from each value in
swath_row_file. Default is 0.

Please suggest.
Jun

Re: crefl2gtiff issue

Posted: Thu Apr 10, 2014 7:37 am
by davidh
Thanks for doing that. This is showing conflicting results. When you run "crefl2gtiff.sh" it can't find "fornav", but when you run "fornav -h" it is finding the command. Did you follow the installation instructions provided by the CSPP group that go with the ".tar.gz" file or did you install things yourself?

-Dave

Re: crefl2gtiff issue

Posted: Thu Apr 10, 2014 1:47 pm
by junzhao
Hi Dave,
Yes. I followed the installation instructions and installed the package by myself.
What should I do then?
Jun

Re: crefl2gtiff issue

Posted: Thu Apr 10, 2014 3:29 pm
by davidh
I'm having trouble debugging this. Could you provide me with the exact "crefl2gtiff.sh" command you are running? Include everything you are entering on the command line.

Can you also run "which fornav" on the command line and tell me the output?

Have you modified any of the files provided in the polar2grid software bundle (specifically the crefl2gtiff.sh or polar2grid_env.sh files)? Have you modified/moved/renamed any of the files or directories in the polar2grid software bundle? Thanks.

Re: crefl2gtiff issue

Posted: Mon Apr 14, 2014 12:14 am
by junzhao
Hi Dave,
Here is the crefl2gtiff.sh.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
#!/usr/bin/env bash
### VIIRS2GTIFF WRAPPER ###
#
# Copyright (C) 2013 Space Science and Engineering Center (SSEC),
# University of Wisconsin-Madison.
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
# This file is part of the polar2grid software package. Polar2grid takes
# satellite observation data, remaps it, and writes it to a file format for
# input into another program.
# Documentation: http://www.ssec.wisc.edu/software/polar2grid/
#
# Written by David Hoese January 2013
# University of Wisconsin-Madison
# Space Science and Engineering Center
# 1225 West Dayton Street
# Madison, WI 53706
# david.hoese@ssec.wisc.edu

if [ -z "$POLAR2GRID_HOME" ]; then
export POLAR2GRID_HOME="$( cd -P "$( dirname "${BASH_SOURCE[0]}" )" && cd .. && pwd )"
fi

# Setup necessary environments
source $POLAR2GRID_HOME/bin/polar2grid_env.sh

# Call the python module to do the processing, passing all arguments
$POLAR2GRID_HOME/ShellB3/bin/python -m polar2grid.crefl2gtiff -vv $@
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

The command I ran was
crefl2gtiff.sh -f ./CREFLM*.hdf

As a non-root user, the command 'which fornav' gave the following output:
$POLAR2GRID_HOME/bin/fornav
However, as a root user, the command 'which fornav' output nothing.

I did not change anything.

Jun