APL User Guide - Level 1 to Level 3 processing
A guide to processing NERC-ARF delivered level 1 hyperspectral data to a mapped image. Please note that this guide is available as a pdf document from here.
A quick start guide for command line usage can be found here: Quick start guide
Or for the graphical user interface here.
Summary of the Mapping Procedure
The hyperspectral data delivered by the Natural Environment Research Council (NERC) Airborne Research Facility (NERC-ARF) will almost certainly be in level-1 format. This is data which has had radiometric calibration routines applied. For newer deliveries level-3 geocoded data is also supplied but it is expected you will want to map yourself, using parameters most appropriate to your application. The steps to convert from level-1 to level-3 data can be summarised as: as:
- Creating an Input Geometry (IGM) file. This is a file which relates each level-1 pixel to a Geographic Latitude and Longitude position.
- Reprojecting. Since most people do not want to map their data in Geographic Latitude and Longitude, the IGM file must be reprojected into a suitable projection.
- Gridding. Once the IGM file has been reprojected into the desired projection system the level-1 data can be gridded into a regular map grid.
This document will help you proceed to generate level-3 products from your NERC-ARF data, using the available software. Please ensure you have the most up to date version of this document, which will be available from the NERC-ARF Data Analysis Node website, along with the most up to date versions of the mapping software.
Creating an IGM file
To create an IGM file from your level-1 data you need to use the aplcorr package. This can be downloaded from the NERC-ARF-DAN website downloads page here.
aplcorr is designed to take in the calibrated level-1 binary BIL (Band Interleaved by Line) file and create a new binary 3 band BIL file. The bands are defined as:
- Longitude
- Latitude
- Height
This new BIL file will have the same row and column dimensions as the original level-1 data BIL file, meaning that for each pixel of the level-1 data there is a corresponding longitude, latitude, height triple. Once you have this file you will be ready to proceed to the next step of the geocorrection procedure.
To create the most accurate IGM possible it is essential to use a Digital Elevation Model (DEM). This will allow the height of the imaged ground scene to be taken into account. It is possible to create an IGM file without a DEM in aplcorr ; in this case it will use the WGS-84 ellipsoid model as a height reference. This will result in errors in the geolocation, the magnitude of which will depend on the elevation difference between the ellipsoid surface and the true ground surface.
Currently aplcorr DEMs have to be in the WGS-84 datum Geographic Latitude/Longitude projection. If your DEM is in another coordinate system it will have to be reprojected prior to using it in the aplcorr program.
Digital Elevation Model format for aplcorr
This section describes the format of the Digital Elevation Models that aplcorr accepts. Currently there is only a single format and projection that is accepted:
Projections
- Geographic Longitude/Latitude in the WGS-84 datum. Data values should be in decimal degrees, positive to North and East with zero at the equator and Greenwich Meridian.
File Formats
- Band Interleaved by Line (BIL) 1 band file with data type 8-bit, 16/32-bit unsigned integer, 16/32-bit signed integer or 32/64-bit floating point. This file format is associated with the ENVI image processing package.
- Band Sequential (BSQ) 1 band file with data type 8-bit, 16/32-bit unsigned integer, 16/32-bit signed integer or 32/64-bit floating point. This file format is associated with the ENVI image processing package.
A DEM in the correct format for using with aplcorr is supplied with NERC-ARF data. For hyperspectral deliveries freely available data are used (normally ASTER or SRTM). If LiDAR was also flown a DEM is generated from the LiDAR point clouds and patched with ASTER or SRTM to fill in any gaps and extend the coverage.
Converting a DEM using the NERC-ARF-DEM Scripts
If you wish to convert your own DEM for use in APL we make our scripts, which are based on GRASS available from: https://github.com/pmlrsg/arsf_dem_scripts/.
We have a guide for using these scripts on GitHub.
Example aplcorr commands
A few example commands to create an IGM file follow. To see example commands specific to your dataset please refer to the read-me document provided with your NERC-ARF data delivery.
Process an Eagle line using a DEM
aplcorr -navfile line1_navigation.bil -lev1file line1_calibrated.bil -vvfile eagle_fov_fullccd_vectors.bil -dem myarea_dem.bil -igmfile output.igm
Process an Eagle line using the WGS-84 ellipsoid surface
aplcorr -navfile line1_navigation.bil -lev1file line1_calibrated.bil -vvfile eagle_fov_fullccd_vectors.bil -igmfile output.igm
Process a Hawk line using the WGS-84 ellipsoid surface with additional height of 50m
aplcorr -navfile line1_navigation.bil -lev1file line1_calibrated.bil -vvfile hawk_fov_fullccd_vectors.bil -heightoffset 50 -igmfile output.igm
Process a Hawk line and output parameters for use in external atmospheric algorithms
aplcorr -navfile line1_navigation.bil -lev1file line1_calibrated.bil -vvfile hawk_fov_fullccd_vectors.bil -dem myarea_dem.bil -igmfile output.igm -atmosfile line1_atmospheric_parameters.bil
Reprojecting the IGM Geolocation Information
Often it will be the case where you do not want to create a map in Geographic Longitude/Latitude. In these cases you will require the IGM file created from the previous stage to be re-projected into a more usable projection. This should be easily performed using a GIS system - the IGM file is a 3-band BIL file (ENVI format) which most GIS systems can handle. A package is provided called apltran for users who do not wish to use a GIS to transform the IGM. This is really just a wrapper program for the open source PROJ.4 library.
Using apltran
The usage of apltran is fairly straight forward and should be no problem if you are familiar with using the PROJ.4 libraries. If you have not used PROJ.4 before then this section could be of interest. PROJ.4 uses strings to describe the projections that it is transforming to and from. apltran has some projection strings built into it to make it easier to use.
To reproject to the Ordnance Survey National Grid (OSTN02) you need to download a grid shift file from:
This file needs to be unzipped and the one of interest for apltran is the one with .gsb extension.
Example apltran commands
Transform to OS National Grid
If you want to map your data to Ordnance Survey National Grid coordinates then you need to use a command line like:
apltran -igm line01.igm -output line01_OSNG.igm -outproj osng gridfile.gsb
where the gridfile.gsb is the NTv2 format file that performs the OSTN02 grid shift.
Transform to UTM
If you want to map your data into the UTM North zone 30 projection system (on the WGS84 ellipsoid) then you would use a command like:
apltran -igm line01.igm -output line01_UTM30.igm -outproj utm wgs84N 30
Gridding and Mapping the Data
Now that you have an IGM file in the required projection it is time to map the level-1 data to a regular grid. To do this you can use a GIS, some alternative mapping software or the ARSF aplmap package. This will take in the IGM file and level-1 data file and output a map of the level-1 data in the IGM projection.
Using aplmap
The aplmap software is capable of mapping a number of bands of the level-1 data to the projection system of the given IGM file, using the selected interpolation method.
Information on interpolation methods
This section briefly describes the algorithms used for the interpolation of the data within the aplmap software.
Nearest Neighbour
Mapping with nearest neighbour searches the neighbourhood around the grid cell and finds the closest point to this from the IGM file. The corresponding value from the level-1 file is then used for this pixel. This option is selected using -interpolation nearest on the command line.
Inverse Distance Weighted
The inverse distance weighted algorithm first searches for, at most, the X nearest points within the maximum allowed search distance. The number X is specified from the command line in the -interpolation option. Then when these points have been found it calculates a weight for each point and applies this to the value of the data from the level-1 file. The weight for point i is calculated as (distance−2) / (Sum (distance−2)) . This option is selected using -interpolation idw maxpoints where maxpoints is the maximum number of points you want to use.
Bilinear
For the bilinear interpolation algorithm, 4 neighbouring points are required. These are identified by using the level-3 position as an initial search position and finding the 4 nearest points that form a quadrilateral containing this level-3 position. The level-3 point that is being interpolated is found in terms of a proportional distance, U and V, between these 4 quadrilateral vertex points. The values of U and V are calculated using the following formulae (in level-3 space):
P = A + U ∗ (B − A)
Q = D + U ∗ (C − D)
X = P + V ∗ (Q − P)
The values of U and V, which are within the range 0-1, are then used to weight the level-1 pixel values in the interpolation formula:
f(X) = f(A) ∗ (1 − V ) ∗ (1 − U ) + f(B) ∗ (1 − V ) ∗ U + f(D) ∗ (1 − U ) ∗ V + f(C) ∗ U ∗ V
where f(x) is the data value of cell x.
Bilinear interpolation can be selected using -interpolation bilinear on the command line.
Example aplmap commands
For examples specific to your data set please see the read-me document supplied with your ARSF delivery.
To map bands 100, 60 and 30 of the level-1 file to a square pixel size of 5 metres using nearest neighbour interpolation:
aplmap -igm line01_UTM30.igm -lev1 line01_calibrated.bil -mapname line01 mapped.bil -bandlist 100 60 30 -pixelsize 5 5 -interpolation nearest
To map all bands of the level-1 file to a square pixel size of 5m using inverse distance weighted interpolation with 5 points and a maximum interpolation distance of 20m:
aplmap -igm line01_UTM30.igm -lev1 line01_calibrated.bil -mapname line01 mapped.bil -bandlist ALL -pixelsize 5 5 -interpolation idw 5 -maxinterpdistance 20
To map bands 10 to 20 and 50 to 79 inclusive, to a pixel resolution of 1m, using bilinear interpolation:
aplmap -igm line01_OSNG.igm -lev1 line01_calibrated.bil -mapname line01 mapped.bil -bandlist 10-20 50-79 -pixelsize 1 1 -interpolation bilinear
If you are interested in