Version 1 (modified by mark1, 13 years ago) (diff)

--

APL User Guide - Level 1 to Level 3 processing

A guide to processing ARSF delivered level 1 hyperspectral data to a mapped image. Please note that this guide is available as a pdf document from here.

Summary of the Mapping Procedure

The hyperspectral data delivered by the Natural Environment Research Council (NERC) Airborne Research Survey Facility (ARSF) will almost certainly be in level-1 format. This is data which has had radiometric calibration routines applied and navigation data appended. To get level-3 geocorrected data you need to perform a few extra steps. These steps can be summarised as:

  • Creating an Input Geometry (IGM) file. This is a file which relates each level-1 pixel to a Geographic Latitude and Longitude position.
  • Reprojecting. Since most people do not want to map their data in Geographic Latitude and Longitude, the IGM file must be reprojected into a suitable projection.
  • Gridding. Once the IGM file has been reprojected into the desired projection system the level-1 data can be gridded into a regular map grid.

This document will help you proceed to generate level-3 products from your ARSF data, using the available software. Please ensure you have the most up to date version of this document, which will be available from the ARSF Data Analysis Node website, along with the most up to date versions of the mapping software.

Creating an IGM file

To create an IGM file from your level-1 data you need to use the aplcorr package. This can be downloaded from the ARSF-DAN website downloads page here.

aplcorr is designed to take in the calibrated level-1 binary BIL (Band Interleaved by Line) file and create a new binary 3 band BIL file. The bands are defined as:

  • Longitude
  • Latitude
  • Height

This new BIL file will have the same row and column dimensions as the original level-1 data BIL file, meaning that for each pixel of the level-1 data there is a corresponding longitude, latitude, height triple. Once you have this file you will be ready to proceed to the next step of the geocorrection procedure.

To create the most accurate IGM possible it is essential to use a Digital Elevation Model (DEM). This will allow the height of the imaged ground scene to be taken into account. It is possible to create an IGM file without a DEM in aplcorr ; in this case it will use the WGS-84 ellipsoid model as a height reference. This will result in errors in the geolocation, the magnitude of which will depend on the elevation difference between the ellipsoid surface and the true ground surface.

Currently aplcorr DEMs have to be in the WGS-84 datum Geographic Latitude/Longitude projection. If your DEM is in another coordinate system it will have to be reprojected prior to using it in the aplcorr program. This can be done using a GIS program such as GRASS. Alternatively the scripts available to download from here can be used.

Digital Elevation Model format for aplcorr

This section describes the format of the Digital Elevation Models that aplcorr accepts. Currently there is only a single format and projection that is accepted but in future releases this may change.

Projections

  • Geographic Longitude/Latitude in the WGS-84 datum. Data values should be in decimal degrees, positive to North and East with zero at the equator and Greenwich Meridian.

File Formats

  • Band Interleaved by Line (BIL) 1 band file with data type 8-bit, 16/32-bit unsigned integer, 16/32-bit signed integer or 32/64-bit floating point. This file format is associated with the ENVI image processing package.
  • Band Sequential (BSQ) 1 band file with data type 8-bit, 16/32-bit unsigned integer, 16/32-bit signed integer or 32/64-bit floating point. This file format is associated with the ENVI image processing package.

Reprojecting a DEM into Geographic Longitude/Latitude

Most Digital Elevation Models are not going to be projected in latitude/longitude and so will require transformation into this projection. This can be done using a GIS package. There are also scripts available from ARSF-DAN (see the downloads page) that make use of the open source GRASS GIS to reproject DEM data. A description on the use of these scripts is given further below. A suitable DEM, produced from freely available sources such as ASTER or SRTM data, should also be included in your data delivery.

If you do not have access to a GIS then the open source GDAL utilities can be used to reproject a DEM and convert the file format. The gdalwarp program can warp a DEM from one projection to another but currently only supports horizontal reprojection, i.e. the elevation data values are not adjusted for vertical datum shifts. So this can be used for DEMs which are already in the WGS-84 vertical datum, for example, a DEM in WGS-84 UTM Eastings and Northings.

The gdal_translate utility can be used to convert between different file formats. For a full list of supported file formats and help on the GDAL package see the ​GDAL website. For example, to convert a GeoTiff DEM called dem.tif into an ENVI style DEM for aplcorr named dem.bsq: gdal_translate -of ENVI dem.tif dem.bsq

If you do not have a DEM covering the area of your data then you can download ASTER elevation data and construct a DEM for your flight line coverage. More information about the dataset is available at ​http://www.ersdac.or.jp/GDEM/E/2.html, the data itself can be downloaded from ​http://asterweb.jpl.nasa.gov/gdem-wist.asp.

It should be noted that aplcorr will not accept a DEM that contains NULL data in the required region. If it finds a NULL data cell then it will exit at that point.

Converting DEM data using the ARSF scripts

This section describes how to use the ARSF DEM transformation scripts to convert a DEM that is in a GDAL supported format or GRASS ASCII raster format into an aplcorr format. The procedure is slightly different for data in UK Ordnance Survey National Grid projection than for other projections and so both methods are described. For non-UK re-projections, no extra files are needed assuming that the projection and datum transformations are implemented ‘built-in’ by PROJ4. PROJ4 is an open source library of coordinate transformations. More information about these libraries can be found on the web at ​http://trac.osgeo.org/proj/. If grid shift files are required to do the transformation for your DEM then a different approach may be needed.

Note that the scripts will apply an interpolation routine to remove any NULL values before outputting the new DEM. This could be unsuitable for DEMs which contain large regions of NULLs (such as a large body of water) or very large DEMs. In these cases a better approach would be to patch the DEM with another suitable surface to remove the NULLs prior to conversion.

Please follow this link to see the guide for using these scripts?

Example aplcorr commands

A few example commands to create an IGM file follow. To see example commands specific to your dataset please refer to the read-me document provided with your ARSF data delivery.

Process an Eagle line using a DEM

aplcorr -navfile line1_navigation.bil -lev1file line1_calibrated.bil -vvfile eagle_fov_fullccd_vectors.bil -dem myarea_dem.bil -igmfile output.igm

Process an Eagle line using the WGS-84 ellipsoid surface

aplcorr -navfile line1_navigation.bil -lev1file line1_calibrated.bil -vvfile eagle_fov_fullccd_vectors.bil -igmfile output.igm

Process a Hawk line using the WGS-84 ellipsoid surface with additional height of 50m

aplcorr -navfile line1_navigation.bil -lev1file line1_calibrated.bil -vvfile hawk_fov_fullccd_vectors.bil -heightoffset 50 -igmfile output.igm

Process a Hawk line and output parameters for use in external atmospheric algorithms

aplcorr -navfile line1_navigation.bil -lev1file line1_calibrated.bil -vvfile hawk_fov_fullccd_vectors.bil -dem myarea_dem.bil -igmfile output.igm -atmosfile line1_atmospheric_parameters.bil

Reprojecting the IGM Geolocation Information

Often it will be the case where you do not want to create a map in Geographic Longitude/Latitude. In these cases you will require the IGM file created from the previous stage to be re-projected into a more usable projection. This should be easily performed using a GIS system - the IGM file is a 3-band BIL file (ENVI format) which most GIS systems can handle. A package is provided called apltran for users who do not wish to use a GIS to transform the IGM. This is really just a wrapper program for the open source ​PROJ.4 library.

Using apltran

The usage of apltran is fairly straight forward and should be no problem if you are familiar with using the PROJ.4 libraries. If you have not used PROJ.4 before then this section could be of interest. PROJ.4 uses strings to describe the projections that it is transforming to and from. apltran has some projection strings built into it to make it easier to use.

To reproject to the Ordnance Survey National Grid (OSTN02) you need to download a grid shift file from:

​http://www.ordnancesurvey.co.uk/oswebsite/gps/osnetfreeservices/furtherinfo/ostn02_ntv2.html

This file needs to be unzipped and the one of interest for apltran is the one with .gsb extension.

Example apltran commands

Transform to OS National Grid

If you want to map your data to Ordnance Survey National Grid coordinates then you need to use a command line like:

apltran -igm line01.igm -output line01 OSNG.igm -outproj osng gridfile.gsb

where the gridfile.gsb is the NTv2 format file that performs the OSTN02 grid shift.

Transform to UTM

If you want to map your data into the UTM North zone 30 projection system (on the WGS84 ellipsoid) then you would use a command like:

apltran -igm line01.igm -output line01 UTM30.igm -outproj utm wgs84N 30