Changes between Version 10 and Version 11 of Processing/AplUserGuide


Ignore:
Timestamp:
Mar 2, 2017, 10:45:22 AM (8 years ago)
Author:
dac
Comment:

Updated from ARSF to NERC-ARF and added links to DEM scripts (this page is linked from web processor).

Legend:

Unmodified
Added
Removed
Modified

  • Processing/AplUserGuide

    v10 v11  
    11= APL User Guide - Level 1 to Level 3 processing =
    22
    3 A guide to processing ARSF delivered level 1 hyperspectral data to a mapped image. Please note that this guide is available as a pdf document from [wiki:Downloads/doc here].
     3A guide to processing NERC-ARF delivered level 1 hyperspectral data to a mapped image. Please note that this guide is available as a pdf document from [wiki:Downloads/doc here].
    44
    55A quick start guide for command line usage can be found here: [wiki:Processing/aplquickstart Quick start guide]
    … …  
    1111
    1212The hyperspectral data delivered by the Natural Environment Research Council (NERC) Airborne
    13 Research Survey Facility (ARSF) will almost certainly be in level-1 format. This is data
    14 which has had radiometric calibration routines applied and navigation data appended. To get
    15 level-3 geocorrected data you need to perform a few extra steps. These steps can be summarised
     13Research  Facility (NERC-ARF) will almost certainly be in level-1 format. This is data
     14which has had radiometric calibration routines applied. For newer deliveries level-3 geocoded data is also supplied but
     15it is expected you will want to map yourself, using parameters most appropriate to your application. The steps to convert from level-1 to level-3 data can be summarised as:
    1616as:
    1717
    … …  
    2222* '''Gridding.''' Once the IGM file has been reprojected into the desired projection system the level-1 data can be gridded into a regular map grid.
    2323
    24 This document will help you proceed to generate level-3 products from your ARSF data, using
     24This document will help you proceed to generate level-3 products from your NERC-ARF data, using
    2525the available software. Please ensure you have the most up to date version of this document,
    26 which will be available from the ARSF Data Analysis Node website, along with the most up to
     26which will be available from the NERC-ARF Data Analysis Node website, along with the most up to
    2727date versions of the mapping software.
    2828----------------------------------
    … …  
    3030== Creating an IGM file ==
    3131To create an IGM file from your level-1 data you need to use the aplcorr package. This can be
    32 downloaded from the ARSF-DAN website downloads page [wiki:Downloads here].
     32downloaded from the NERC-ARF-DAN website downloads page [wiki:Downloads here].
    3333
    3434aplcorr is designed to take in the calibrated level-1 binary BIL (Band Interleaved by Line) file and create a new binary 3 band BIL file. The bands are defined as:
    … …  
    4242To create the most accurate IGM possible it is essential to use a Digital Elevation Model (DEM). This will allow the height of the imaged ground scene to be taken into account. It is possible to create an IGM file without a DEM in aplcorr ; in this case it will use the WGS-84 ellipsoid model as a height reference. This will result in errors in the geolocation, the magnitude of which will depend on the elevation difference between the ellipsoid surface and the true ground surface.
    4343
    44 Currently aplcorr DEMs have to be in the WGS-84 datum Geographic Latitude/Longitude projection. If your DEM is in another coordinate system it will have to be reprojected prior to using it in the aplcorr program. This can be done using a GIS program such as GRASS. Alternatively the scripts available to download from here can be used.
     44Currently aplcorr DEMs have to be in the WGS-84 datum Geographic Latitude/Longitude projection. If your DEM is in another coordinate system it will have to be reprojected prior to using it in the aplcorr program.
    4545
    4646=== Digital Elevation Model format for aplcorr ===
    47 This section describes the format of the Digital Elevation Models that aplcorr accepts. Currently there is only a single format and projection that is accepted but in future releases this may change.
     47This section describes the format of the Digital Elevation Models that aplcorr accepts. Currently there is only a single format and projection that is accepted:
    4848
    4949'''Projections'''
    … …  
    5656 * '''Band Sequential (BSQ)''' 1 band file with data type 8-bit, 16/32-bit unsigned integer, 16/32-bit signed integer or 32/64-bit floating point. This file format is associated with the ENVI image processing package.
    5757
    58 === Reprojecting a DEM into Geographic Longitude/Latitude ===
     58A DEM in the correct format for using with aplcorr is supplied with NERC-ARF data. For hyperspectral deliveries freely available data are used (normally ASTER or SRTM). If LiDAR was also flown a DEM is generated from the LiDAR point clouds and patched with ASTER or SRTM to fill in any gaps and extend the coverage.
    5959
    60 Most Digital Elevation Models are not going to be projected in latitude/longitude and so will require transformation into this projection. This can be done using a GIS package. There are also scripts available from ARSF-DAN (see the downloads page) that make use of the open source GRASS GIS to reproject DEM data. A description on the use of these scripts is given further below. A suitable DEM, produced from freely available sources such as ASTER or SRTM data, should also be included in your data delivery.
     60=== Converting a DEM using the NERC-ARF-DEM Scripts ===
    6161
    62 If you do not have access to a GIS then the open source GDAL utilities can be used to reproject a DEM and convert the file format. The gdalwarp program can warp a DEM from one projection to another but currently only supports horizontal reprojection, i.e. the elevation data values are not adjusted for vertical datum shifts. So this can be used for DEMs which are already in the WGS-84 vertical datum, for example, a DEM in WGS-84 UTM Eastings and Northings.
     62If you wish to convert your own DEM for use in APL we make our scripts, which are based on GRASS available from: https://github.com/pmlrsg/arsf_dem_scripts/.
    6363
    64 The gdal_translate utility can be used to convert between different file formats. For a full list of supported file formats and help on the GDAL package see the [http://www.gdal.org GDAL] website. For example, to convert a GeoTiff DEM called dem.tif into an ENVI style DEM for aplcorr named dem.bsq:
    65 {{{gdal_translate -of ENVI dem.tif dem.bsq}}}
    66 
    67 If you do not have a DEM covering the area of your data then you can download ASTER elevation data and construct a DEM for your flight line coverage. More information about the dataset is available at http://www.ersdac.or.jp/GDEM/E/2.html, the data itself can be downloaded from http://asterweb.jpl.nasa.gov/gdem-wist.asp.
    68 
    69 It should be noted that aplcorr will not accept a DEM that contains NULL data in the required region. If it finds a NULL data cell then it will exit at that point.
    70 
    71 === Converting DEM data using the ARSF scripts ===
    72 
    73 This section describes how to use the ARSF DEM transformation scripts to convert a DEM that is in a GDAL supported format or GRASS ASCII raster format into an aplcorr format. The procedure is slightly different for data in UK Ordnance Survey National Grid projection than for other projections and so both methods are described. For non-UK re-projections, no extra files are needed assuming that the projection and datum transformations are implemented ‘built-in’ by PROJ4. PROJ4 is an open source library of coordinate transformations. More information about these libraries can be found on the web at http://trac.osgeo.org/proj/. If grid shift files are required to do the transformation for your DEM then a different approach may be needed.
    74 
    75 Note that the scripts will apply an interpolation routine to remove any NULL values before outputting the new DEM. This could be unsuitable for DEMs which contain large regions of NULLs (such as a large body of water) or very large DEMs. In these cases a better approach would be to patch the DEM with another suitable surface to remove the NULLs prior to conversion.
    76 
    77 [wiki:Processing/Scripts/demconversionscript Please follow this link to see the guide for using these scripts]
     64We have a guide for using these scripts [https://github.com/pmlrsg/arsf_dem_scripts/blob/master/doc/source/tutorial_lidar.md#create-a-lidar--aster-dsm-for-use-in-apl on GitHub].
    7865
    7966=== Example aplcorr commands ===
    8067
    81 A few example commands to create an IGM file follow. To see example commands specific to your dataset please refer to the read-me document provided with your ARSF data delivery.
     68A few example commands to create an IGM file follow. To see example commands specific to your dataset please refer to the read-me document provided with your NERC-ARF data delivery.
    8269
    8370'''Process an Eagle line using a DEM'''
    … …  
    10996To reproject to the Ordnance Survey National Grid (OSTN02) you need to download a grid shift file from:
    11097
    111 [http://www.ordnancesurvey.co.uk/oswebsite/gps/osnetfreeservices/furtherinfo/ostn02_ntv2.html]
     98[https://www.ordnancesurvey.co.uk/business-and-government/help-and-support/navigation-technology/os-net/ostn02-ntv2-format.html]
    11299
    113100This file needs to be unzipped and the one of interest for apltran is the one with .gsb extension.
    … …  
    185172{{{aplmap -igm line01_OSNG.igm -lev1 line01_calibrated.bil -mapname line01 mapped.bil -bandlist 10-20 50-79 -pixelsize 1 1 -interpolation bilinear}}}
    186173
     174If you are interested in