I am in the process of making the Aerospace ADL configuration more DMS like; instead of a input template containing a specific set of input LUTs, I am going to better leverage multiple input directories specified in the execution XML file. For example, I will have a directory for all valid versions of the VIIRS-SDR-F-LUT installed in the system. By default, ADL will use the DMS rules to select the correct version of that file. If the user desires to override the DMS selection and use a specific file, that directory will not be included as an input directory, and a link to the specific file will be placed into the "primary" input directory along with other this-specific-run files, like the unpacked RDR.
From a using ADL perspective, this is an important change as it allows us to mimic the input files that would have been used on the IDSD by default, whereas with the previous configuration, this was not possible.
Unfortunately, this is going to make it more difficult to gather all of the inputs for a given ADL run into a single tarball that we can submit to DPE for the LUT update process much more difficult. For the submission to be valid, we need to tar up every single input file used (minus the tile data), and thus it is essential that I be able to figure out which of each of those inputs ADL selected.
I can get part of this information by looking at the metadata file for one of the outputs. However, the information for the different files appears to be inconsistently specified.
Consider the following two files placed in an input directory for the ADL run:
Code: Select all
parmentier:inputs$ ls -l 4ed4e84c-7098d-dd828c19-8bdc5952* 4f06fd13-d1690-dd828c19-0f65f9bc*
lrwxrwxrwx 1 sh27576 viirs 83 Feb 8 11:15 4ed4e84c-7098d-dd828c19-8bdc5952.asc -> /tcs/ago/group/viirs/adl/lut/I1.5.06J/unpacked/4ed4e84c-7098d-dd828c19-8bdc5952.asc*
lrwxrwxrwx 1 sh27576 viirs 212 Feb 8 11:15 4ed4e84c-7098d-dd828c19-8bdc5952.VIIRS-SDR-DELTA-C-LUT_npp_20110822010100Z_20111101010000Z_ee00000000000000Z_PS-1-N-CCR-11-117-JPSS-DPA-001-PE-_noaa_all_all-_all.bin -> /tcs/ago/group/viirs/adl/lut/I1.5.06J/unpacked/4ed4e84c-7098d-dd828c19-8bdc5952.VIIRS-SDR-DELTA-C-LUT_npp_20110822010100Z_20111101010000Z_ee00000000000000Z_PS-1-N-CCR-11-117-JPSS-DPA-001-PE-_noaa_all_all-_all.bin
lrwxrwxrwx 1 sh27576 viirs 83 Feb 8 11:15 4f06fd13-d1690-dd828c19-0f65f9bc.asc -> /tcs/ago/group/viirs/adl/lut/I1.5.06J/unpacked/4f06fd13-d1690-dd828c19-0f65f9bc.asc*
lrwxrwxrwx 1 sh27576 viirs 219 Feb 8 11:15 4f06fd13-d1690-dd828c19-0f65f9bc.VIIRS-SDR-BB-TEMP-COEFFS-LUT_npp_20111109010000Z_20111101010000Z_ee00000000000000Z_PS-1-N-CCR-11-221-JPSS-DPA-001-PE-_noaa_all_all-_all.bin -> /tcs/ago/group/viirs/adl/lut/I1.5.06J/unpacked/4f06fd13-d1690-dd828c19-0f65f9bc.VIIRS-SDR-BB-TEMP-COEFFS-LUT_npp_20111109010000Z_20111101010000Z_ee00000000000000Z_PS-1-N-CCR-11-221-JPSS-DPA-001-PE-_noaa_all_all-_all.bin
parmentier:inputs$ ls -l 4f32c9be-28378-dd828c19-3102698f.*
-rwxrwxrwx 1 sh27576 viirs 1108 Feb 8 11:15 4f32c9be-28378-dd828c19-3102698f.asc*
lrwxrwxrwx 1 sh27576 viirs 47 Feb 8 11:15 4f32c9be-28378-dd828c19-3102698f.VIIRS-SDR-F-LUT -> /tcs/ago/home/sh27576/VIIRS-SDR-F-LUT-SN003.bin
parmentier:inputs$ ls -l /tcs/ago/group/viirs/adl/lut/I1.5.06J/unpacked/4ed4e84c-7098d-dd828c19-8bdc5952.asc*
-rwxrwxrwx 1 sh27576 viirs 1376 Nov 29 06:12 /tcs/ago/group/viirs/adl/lut/I1.5.06J/unpacked/4ed4e84c-7098d-dd828c19-8bdc5952.asc*
parmentier:inputs$ ls -l /tcs/ago/group/viirs/adl/lut/I1.5.06J/unpacked/4ed4e84c-7098d-dd828c19-8bdc5952.VIIRS-SDR-DELTA-C-LUT_npp_20110822010100Z_20111101010000Z_ee00000000000000Z_PS-1-N-CCR-11-117-JPSS-DPA-001-PE-_noaa_all_all-_all.bin
-rw-rw---- 1 sh27576 viirs 4505920 Feb 8 10:37 /tcs/ago/group/viirs/adl/lut/I1.5.06J/unpacked/4ed4e84c-7098d-dd828c19-8bdc5952.VIIRS-SDR-DELTA-C-LUT_npp_20110822010100Z_20111101010000Z_ee00000000000000Z_PS-1-N-CCR-11-117-JPSS-DPA-001-PE-_noaa_all_all-_all.bin
parmentier:inputs$ ls -l /tcs/ago/group/viirs/adl/lut/I1.5.06J/unpacked/4f06fd13-d1690-dd828c19-0f65f9bc.asc*
-rwxrwxrwx 1 sh27576 viirs 1467 Jan 6 05:54 /tcs/ago/group/viirs/adl/lut/I1.5.06J/unpacked/4f06fd13-d1690-dd828c19-0f65f9bc.asc*
parmentier:inputs$ ls -l /tcs/ago/group/viirs/adl/lut/I1.5.06J/unpacked/4f06fd13-d1690-dd828c19-0f65f9bc.VIIRS-SDR-BB-TEMP-COEFFS-LUT_npp_20111109010000Z_20111101010000Z_ee00000000000000Z_PS-1-N-CCR-11-221-JPSS-DPA-001-PE-_noaa_all_all-_all.bin
-rw-rw---- 1 sh27576 viirs 240 Feb 8 10:37 /tcs/ago/group/viirs/adl/lut/I1.5.06J/unpacked/4f06fd13-d1690-dd828c19-0f65f9bc.VIIRS-SDR-BB-TEMP-COEFFS-LUT_npp_20111109010000Z_20111101010000Z_ee00000000000000Z_PS-1-N-CCR-11-221-JPSS-DPA-001-PE-_noaa_all_all-_all.bin
- The Delta C and BB-TEMPS binary files and metadata files are symbolic links using absolute paths to the target
- The targets of the Delta C and BB-TEMPS binary and metadata files are real files
- The F-LUT binary file is a symbolic link to an absolute path, and the target of that link is again a real file
- The F-LUT metadata file in the input directory is a real file (the ADL front end "unpacks" the binary LUT, creating the metadata file as part of staging that file)
Code: Select all
("N_Aux_Filename" STRING EQ "../unpacked/4ed4e84c-7098d-dd828c19-8bdc5952.VIIRS-SDR-DELTA-C-LUT_npp_20110822010100Z_20111101010000Z_ee00000000000000Z_PS-1-N-CCR-11-117-JPSS-DPA-001-PE-_noaa_all_all-_all.bin")
... snip ...
("N_Aux_Filename" STRING EQ "/tcs/ago/group/viirs/adl/lut/I1.5.06J/unpacked/4f06fd13-d1690-dd828c19-0f65f9bc.VIIRS-SDR-BB-TEMP-COEFFS-LUT_npp_20111109010000Z_20111101010000Z_ee00000000000000Z_PS-1-N-CCR-11-221-JPSS-DPA-001-PE-_noaa_all_all-_all.bin")
("N_Aux_Filename" STRING EQ "VIIRS-SDR-F-LUT-SN003.bin")
This makes it very difficult to find the specific files. Is there a reason they are different, and if not (or the reason isn't important), can we get an ADL update that writes absolute paths for all ancillary and auxiliary files listed in the output metadata.
We are heading toward a near term weekly update of the F and H LUTs, and a slightly longer-term daily update cycle, so it is important to be able to communicate clearly for a given output exactly the input files that were used.