Fenix Calibration, 2016

[lah]

Data location: ~arsf/arsf_data/2016/misc/2016_cal_feb_cambridge

[lah]

Wavelength scale

Processing at Cambridge suggested an offset of approximately -6nm for the SWIR for the first 2 data sets, but not for the 3rd.

Day 1: -0.788086 and -6.872559 4 SWIR anchors pass
Day 2: -0.488281 and -6.652344 4 SWIR anchors pass
Day 4-1 -0.125000 and -5.734375 3 pass, 1 fail

Have redone them all and got the same results. There are 7 SWIR anchors to match, the unmatched ones are off by +7, -6 and -7 for d1.

I'm concerned about the credibility of the spectral lines used and the files have no recorded source, so I will investigate these next (they don't all match up to the booklets). e.g. 1473.44 for Kr is not in the lamp booklet and is also one of the unmatched anchors. The are also many other SWIR lines that could potentially be used to match against which we don't have. I will create a new spectral_lines file to test these other SWIR spectral lines.

[lah]

Wavelength Scale

I've updated the spectral lines file (new_spectral_lines_feb16.csv) to identify most of the spectral lines, from the Oriel booklets we have and ​http://www.nist.gov/pml/data/handbook/index.cfm. I can't find an Hg-Ar peak to match the 1480 nm data line, so we will always have that peak unmatched. The Kr 1533.4 nm peak is slightly outside the acceptable range for each day, so we may have to increase the pass threshold for the SWIR. This seems reasonable considering the spectral resolution is so much lower than the VNIR. Best results for the 3 days are:

Day 1: -0.783447 and -5.263965 8 pass, 1 fail (VNIR changed because O file named wrong initially)
Day 2: -0.488281 and -4.894000 4 pass, 2 fail
Day 4-1 -0.125000 and -5.484375 5 pass, 1 fail

Day 1 seems by far the best day to continue with. Remember these differences are on the original wavelength scales, not last years.

[lah]

Wavelength scale

Best results using the new lamp lines and after removing problem peaks from peak files:

day1: -0.695312 and -6.703125 (8 SWIR anchors)
day2: -0.550110 and -7.041016 (9 SWIR anchors)

day 4-1: VNIR = -0.473633 There is no possible way to produce a successful SWIR wavelength scale for this dataset. The very strong O 1130 nm line and the He 2058.1 line cannot be matched in a linear scale. There are no contributions from other sources or other lines, so it would be incorrect to delete one of these lines. (Occasionally a line from another lamp contributes to a line it shouldn't in the final anchor matching - an aspect to improve alongside automatically removing large FWHMs).

Proceeding with day 2 wavelength scale as it matches more anchors.

VNIR start: 380.370
SWIR start: 980.381

[lah]

Wavelength scale (2015)

Tested new lamp file with 2015 calibration. 1533.4 Kr line appears generally unreliable, so have decided to remove it from new lines file.

Produces more reliable wavelength scale with 8 anchors rather than the previous 3. However, the results are different:

-1.237305 and -9.078125

VNIR start: 378.852
SWIR start: 978.639

Previous results were based on a ~5 nm jump in the data, but we now know that the wavelength scale shifted about during the Malaysia campaign so this may not have been a reliable input.

Results saved in: arsf/2015/misc/2015_cal_feb_gloucester/processed2016

[lah]

Radiometric Calibration

Have tested the radiometric calibration on 2015 234c and compared with previous results (moved previous data to flightlines_2015cal). There is no spike in the former overlap region, so no need to mask out this region when using this calibration.

We may also want to switch to this calibration sooner rather than later.

[lah]

Vegetation spectrum, 2015 cal left, 2016 cal right

[lah]

Bad pixel file

Created bad pixel file with same settings as 2015. No SWIR spike or overlap, but doesn't mean that the same bands are not subject to the same noise.

VNIR:

A_thresh = 10
B_spectralwidth = 3
B_spatialwidth = 10
B_thresh = 10
C_thresh = 0.9825
D_spectralwidth = 1
D_spatialwidth = 18
D_thresh = 3
F_thresh = 0.1

SWIR:
A_thresh = 10
B_spectralwidth = 3
B_spatialwidth = 10
B_thresh = 20
C_thresh = 0.9825
D_spectralwidth = 1
D_spatialwidth = 18
D_thresh = 3
F_thresh = 0.3

[lah]

Data Quality Report

Created new 2016 data quality report. Created one with comparisons from existing 2015 calibration (0.31% shift) and also with the new 2015 wavescale (0.12% shift). (The new 2015 wavescale does not solve the SWIR spike; perhaps the sensor was still suffering from Malaysia during the 2015 calibration?)

[lah]

Calibration

Redoing with old lamp file for consistency. New lines can be part of new method to be later implemented and applied to all old calibrations.

Final data in processed_final

Day: 2&3 (d2_d3)
VNIR offset: -0.538086
SWIR offset: -7.062500

VNIR start: 380.382
SWIR start: 970.170

Differences in wavelength scale using the same offsets are surprising. Suggest that this is another thing to check when implementing the new calibration method.

[lah]

DQ report

Created new plots for data quality report.
Median percentage difference for band median: -0.40%
Median percentage difference for whole CCD: -0.42%