= Leica Full Waveform Digitisation = == Concepts and Hardware == The full waveform digitiser (FWD) is supplied as an upgrade to the existing ALS50v2 (and up) sensors, and essentially consists of an upgraded data logger and an intensity digitiser (the FWD). The discrete point recording subsystem is unchanged. In implementation, the FWD is added as a spur after the automatic gain controller (AGC). There is a trigger from the discrete system to the FWD that tells the FWD when to start recording. {{{ +---> Full waveform digitiser -->--+ | ^ | Laser receiver -> Automatic Gain Controller -->| | |--> Data logger | | | +---> Discrete point subsystem -->--+ }}} The FWD is implemented as two sampling boards The FWD digitises the incoming intensity level and nothing else. It records either 64, 128 or 256 samples at 1 or 2 nanosecond resolution (1ns is approximately 15cm). The samples are recorded from the time of the first return, with a small buffer before that (the "pre-trigger samples"). This buffer allows the capture of the lead-in to the first pulse (otherwise it would start at the first detected peak) and is specified in terms of meters to the operating software (default 5m). Processing note: the trigger from the discrete system appears to have a small delay (order of 7ns?), that is different for the two possible laser pulse rates and also may be dependent on temperature (unproven). This delay manifests as an offset between the discrete returns and the matching waveform peaks. At the time of writing, it is corrected by a manual / statistical approach (see below). Some other important points: - normal lidar returns based on threshold, then look for peaks - requires a non-peak gap after a peak is found, so double peaks not spottable without fwd laser pulse is 4ns pulse duration or 9ns (<100khz digitisation), so must have a <2ns intensity digitisation to avoid aliasing (nyquist), normally prefer 1ns 1ns = 15 cm intensity is digitised to 8bit - no range gate yet but may be added (phil) - "pre-trigger" samples = buffer before first return peak to ensure you get lead in to pulse, defaults to 5m - especially important if there's low signal before first real peak, e.g. sparse tree top range data is measured independently to achieve 1.5cm range (100 picosec). - fwd data goes to RawWFD, structure identical to RawLaser, files named WFDYYMMDD_HHMMSS_XXXXXXXXXXX{AB}.LWV - A or B indicates which board digitised the signal .LWV files - are always up to 62501KB - always in A&B pairs - can't link single scn files to lwv files within a flightline flightline log file says if WFD on/off can turn processing of waveform data off always produces las 1.3 file (despite currently saying 1.2) - filename ends _4.las O_o - unless you choose las 1.0 from the output options, but you must also check the output with timestamp option trigger delay settings - TPR = transition pulse rate, when the lidar changes from long pulses to short ones - for our system, >100KHz = 4ns pulse, <100kHz = 9ns pulse - each must be calibrated separately.. - ie. calibration must include one above and one below TPR - possibly also temperature dependent. shonk. - should be able to do this with bit mode data? - possible hardware issue preventing this, definite software issue (fcms doesn't record fwd in bitmode) - definite killer: timing known to verify within a flightline processing time - discrete only, maybe 30secs per line - waveform data = ~10x slower - started 1023, finished 1030 - approx 10x larger file (46MB without fwd and 450MB with) waveform viewer - gps time = time of first return / when fwd started recording (but note it'll also have some pretrigger buffering) - retpoint loc = same time but in terms of fwd timing (?) - no way to accurately get second, third, etc times calibration procedure - delay between first return and trigger pulse getting to waveform digitiser - histogram viewer (lashistoviewer) - can see AGC histogram. Looking for dataset with agc somewhere in the middle range (150 ish) and without large variations - look for data point with intensity 60-80 ish / 1V, single return - find peak time for first return visually (e.g. 22.5ns into fwd sample window) - use ret point loc (e.g. 32.766ns) - calculate delay ;p - put into trigger delay in options, see if things look better - repeat N times - repeat for dataset above and below TPR - known to vary within a flightline by a small amount (+/- 100 ps) viewing with terrascan - alspp, create trj trajectory files - provide to terrascan.. next to helicopter icon under /// icon in toolbox ;p - then select file and import - terrabox, flightline -> deduce using time * possibly need to add trajectory to las 1.3? - view single point's waveform - draw in 3d draws a coloured ray, blue = low value, red = high - have to delete ray manually O-o choose selection tool, select line, delete - spectacularly poo - extract echoes (cf. calculate returns from waveform data) - place fence around, say, a tree - extract echoes - analyses waveform data and picks out a bunch of new returns, adds them - can extract data to ascii for a single point or a fence (EEK! one text file per point! NO, per {only or first+last} return!) - not sure there's a way to tie the waveform info to the return time? particularly for not-first-returns - spectacularly poo - terrascan user group meeting - 10-17 February 9th Terrasolid European Users Event, Levi Summit, Kittilä, Finland, Internet: www.terrasolid.fi/en/events/terrasolid_european_users_event_2010_agenda_an - check out other software, fullanalyze? RCD changes - uses full CCD instead of cut down area - 7162x5389 -> 7212x5408, 7012x5208 after geometric enhancement - see presentation for depressing readout location on old setup - change to SSD - new workflow software - file renaming to flightline/project naming from fcms - distortion removal, shutter correction - DTM support, for displaying in workflow manager = big change, but we don't use this * however, we probably should as this seems to be the bit that does distortion fixups..? - not ready yet.. maybe next week - post processing sw -> 1.2 - no breaker changes - minimised L/R imbalance, improved radiometric using ccd border area (hrm) AGC setting constant for a single pulse (ie. only need the one recorded in SCN) - confusing slide on intensity scale relating to voltage.. not quite sure what value is recorded (raw or agc scaled) - note intensity scales are different for standard returns and fwd things one can detect.. maybe.. - pulse stretching (return pulse longer than laser pulse - slopes - low vegetation (ie. may need to adjust "ground" peak downwards a little) - indication of biomass by evaluating area under peak - better vertical resolution/distinction of objects angle of laser may make a difference (flat ground at nadir is a slope at edge of fov, or cutting through vegetation) hardware changes - new data logger (cope with higher io) - hdd -> 160GB SSD, works up to 7620m altitude - should be ok to use spinning disk for bigger datasets @ low altitude - can't swap disks in flight without full shutdown of logger, but can keep ipas going without recording - fwd boards, 2 of them at 60kHz each software changes - hardware done, most software beta or not done yet - various operation changes - fcms now creates proper structure for file outputs, but only for the laser - rcd isn't done! - may change some of our naming ("WebCamImages" or something like that) - waveform viewer s/w - alspp outputting in las 1.3 type 4 (option to output 1.0 if needed for compat) - version list in presentation if pulse rate > 120kHz, then FWD on every second pulse.. e.g. 150kHz SCN file = 75kHz FWD where did gain come from in waveform viewer - volts= digitized value * gain + offset - not agc gain, this is a fixed amplication on the waveform board - value is 0.017290626 - offset is also hardcoded, value is 0 - system digitises from 0 to 4.something volts - gives you volts as received at the waveform board, after AGC applied, etc - probably only useful for engineering team bit mode - actually could process it! - interesting underflow spike - engineering think trigger delay shouldn't change with time, but may with temperature? - not sure if bit mode vs lase mode will have a different delay, not tested - fcms will record waveform bit mode data, will in version approximately 3.2.2? probably 6 months - should ensure you have full nav solution for maximum gps timing accuracy (ie. don't just switch on ipas and record, wait a few mins first) - need two recordings for bitmode, at pulse rates above and below TPR gps time difference between line time in alspp summary and waveform viewer times - Yuji checking lashisto viewer - can look at las files with it, gives various useful info - drag and drop a log file into it to get nicely parsed info data rate / height stuff - 19.2 / 38.4 / 76.8m for various heights - 1h04 recording at max rate on 160GB SSD terrascan - displays waveform as colour coded line along angle of ray - can set thresholds and ranges for colour coding future processing - gaussian fitting to fwd data - working with vienna uni guys ops changes: various.. relevantish ones to us are - hardware is now licensed (ie. turns into a paperweight if expires) - closing fcms shuts the system down cleanly - need to use fpes10 for planning - automatic bit mode at start and end of fcms session - moving to ethernet comms rather than rs232 processing changes - require alspp 2.69 or grater - GPS leverarms refer now to PAV80 rotation center (stabilised mount for camera, we don't have one) - PAV80 would be underneath the scanner * see slides for update formula * update wiki - alspp takes account of this if .sup file has a flag saying it was run with new fcms (sup comes from modern ipas pro) - lever arm measurement is the same, but you must include the offset to the virtual pav80 - note this doesn't apply to IMU - this is a fixed offset and entered into fcms * check it's right ;p - fixed offset is apparently x=-0.411, y=0.206, z=-0.192 - fcms also has a user defined offset currently set that is the offset from the pav80 back to the mirror - this is x=0.142, y=0.001, z=-0.188 * this possibly should be zeroed - Yuji checking if having it set will cause a double offset to be applied - check lever arm error (should be small, a few cm) - new option in alspp to process waveform - trigger delay in pico secs (= splitter circuit delay) - one for 4ns one for 9ns pulse width (!) - but no difference for A & B boards.. hrm - order of 10x slower processing IMU discussion - some discussion of IMU sharing - should be possible to mount lidar rigidly by removing shocks (or add massive stiffener!) and then applanix / ipas IMUs should be the same - not a risk of damage - interesting talk about harmonics between mirror vibration and engine vibration - IMU can pick this up and it'll show as banding alongtrack (for pitch errors) in the data * Nothing has changed for the AGC/mainboard of the ALS so the raw laser .SCN files will be the same as before. * Extra files will be created that contain the waveform data. Waveform files (.LWV) will be in the raw_wfd directory, SCN files in the raw_laser directory. * Waveform files have A or B in filename depending on which data board (A or B) acquired it. When processing both files are taken together. * The digitiser starts recording from the first returned peak for the number of required samples (256/128 @ 1ns or 256/128/64 @ 2ns) * If first return is noise (cloud/seagull) then the digitiser will start recording there and not at the target. * measured heights / distances for different sampling rates: || Sample rate || equivalent height || || 256 @ 1ns || 38.4m || || 128 @ 1ns || 19.2m || || 256 @ 2ns || 76.8m || || 128 @ 2ns || 38.4m || || 64 @ 2ns || 19.2m || * Benefits of full waveform * Better idea of the nature of the return e.g. tree canopy height is more accurate than from 1 discrete point * Extraction of points from the waveform missed by the discrete measuring * (a future benefit) pulse stretching at swath edges compared to nadir: indicating sloped terrain, improved classification * Maximum rate of acquiring digitised full waveform data is 120KHz. If a higher rate is used then the waveform data will record for every other emitted pulse. But the SCN files will still be recorded at the high rate. * Pulse length (sampling rate) should be chosen based on the maximum height of targets in the area. E.G. if all low lying targets then there is no point using 256 samples at 1 ns - better to use 128 at 1 ns. * No changes to the calibration flights except to collect FW data. Only change to the processing is to find the timing offset between FW and discrete points. * Pre-trigger - like a buffer zone to record digitised samples before the first return. Default is 5m. * Waveform and discrete data use different scalers for the intensity values and so will result in different intensities. * AGC is constant for a pulse - so the same value is used for the entire waveform * There is a slight delay between SCN files and waveform files being recorded (< 1 second). This may result in first few points not having waveform data. == Software == * ALSPP outputs in LAS 1.3 format which supports full waveform data (and also LAS 1.0) * To output in LAS v1.0 you must also select "LAS file with time stamp" in output options else no files will be output * ALSPP takes 10 times as long to process the data to produce a LAS file when using FWD * Includes a simple waveform viewer * Displays waveform and discrete points * There is a "constant" timing offset between the waveform and discrete points - thought to be due to processing time. This should be removed at calibration * In waveviewer "Ret point loc" is the discrete return time (from emit to receive) for the first return * intensity is the value for each discrete return * LASHistoViewer is useful for viewing stats of LAS file. Can also "drag and drop" Flightline_log into it to see info and stats together. * New version of Terrascan capable of displaying full waveform data and "creating extra" points from the waveform == Something == * When processing data collected after using FCMS v3.15 (anything after Jan 2010 ?) * ALS system refers to a virtual rotation axis (assumes a PAV80 stabilised mount even if you do not have one) * we need to update GPS lever arms in FCMS v3.15. * .SUP file will automatically offset to the mirror centre in ALSPP processing (Need IPAS version 1.35 for this) * Unsure as to whether leverarms need to be updated in IPAS or not - also how do we know IMU offsets to apply? == Changes to processing == * To process full waveform data in ALSPP - Select "process waveform data" from the inputs menu and tick the box. Enter the "Trigger Delay" - this is the time offset between discrete and waveform data. * ALSPP looks for the "!RawWfd" folder - this may imply that renaming the directory structure could cause problems. * ALSPP can limit the number of points it puts in a LAS file. This allows long flight lines to be split up into more manageable files. * Trigger delay needs to be derived - hopefully it remains constant throughout each flight line and whole flying season - this needs to be checked and monitored. Temperature may also affect it. * Will have 2 values - one for 4ns pulse width ("over TPR") and one for 9ns pulse width ("under TPR") * Use waveviewer tool with lines at 4ns to get the 4ns offset, and lines at 9ns to get the 9ns offset * Use points throughout flight line not just at start to derive the offset. * Procedure * Check LAS file is suitable by loading into LASHistoViewer and checking AGC is fairly constant (should be a tight/narrow histogram in the centre) * Select view by time and volt in waveviewer and zoom into the graph * Find a discrete point with ~ 1V intensity value * Measure the distance between discrete and full wave peak by zooming into the image and counting the time difference along the x-axis. * Can enter the offset value into waveviewer to check discrete and peak align (enter the value in picoseconds) * Repeat this for a few samples throughout the file and take an average. * Terrascan new options - waveform viewer and point extraction * Need to import trajectory * then flightline -> deduce from time * Should be able to view waveforms now * Waveform display settings * sample height - changes y scale * maximum value - changes x scale * 3D drawing * draws a coloured vector onto the design file depending on the waveform intensity. To remove it, you need to select the line and delete it. * ambient noise - change the threshold level of noise for colour coding the waveform vector * extract data - allows new points (returns) to be created from the waveform data * can export the waveforms as txt files for further analysis in other software == RCD105 == * Enhancements * CCD format is now full frame - allows improved radiometric calibration * software fixes * solid state disks * Now use the entire CCD area for radiometric calibration for better left/right balancing. The boundary is trimmed off in post-processing. * Creates a distortion free image (tries to model lens distortion and uses a DTM) * Filename convention change - files automatically renamed * RCD workflow manager software looks useful - includes ORIMA dll file for running APM (?) * New software should handle pixel saturation better and supports 2 CCD sizes