Eagle radiometric calibration mismatch

[mggr]

This ticket is spawned from #74 (support ticket for Rachel Gaulton, who found this problem). See that ticket for the initial work, summarised here.

Eagle's calibrated spectral profile on flights appears to differ substantially from CASI in terms of the measured radiance. The shape is the same but the numbers are different by a factor of approximately 4-8, depending on the pixel chosen.

Short history to date:

• 16/Oct Initial problem report from Rachel (2006 dataset)
• 18/Oct AKW recommended investigating on a known dataset
• 22/Oct Picked 166/2007 boresight flight and confirmed we're also seeing this issue
• 29/Oct Demonstrated to Gary and agreed routes for investigation

[mggr]

Routes of investigation:

1. Chris from FSF is visiting in early Dec to calibrate some instruments for 2008. While there, get him to do a quick and dirty calibration on the Specim too, so we have a calibration made in the same conditions as the CASI one to compare against (giving us a ground truth). This will establish if there's a problem with Specim's calibration procedure.
2. As soon as Bill is back from holiday (~10th Nov?), ask him to check on how he handles the Specim calibration in azspec, particularly with non 1-1 binnings (all 2007 Eagle flights are 2-1 or higher and the calibration is 1-1).
3. compare level 0 vs level 1b data to ensure the calibration is being applied
   • done - the level0 looks a bit like a CCD response curve and the level 1b looks like a ground-spectrum and matches the CASI spectra, so this appears to be working, albeit with an offset/factor
4. check the 2007 flights for various binnings to compare against CASI
   • done - no 2007 flights are 1/1, they vary from 2/1 to 8/1 and 8/4
5. compare CASI and Eagle spectra in multiple flights to see if there's a common factor, bearing in mind the binning rates
   • only the 166 boresight and some of the IPY flights have CASI and Eagle run together

[mggr]

Tested L3 vs L1b to ensure it isn't a pixel/sampling size issue (appears it isn't).

[mggr]

Flights with CASI and Eagle together (inc binning rates):

213-07/casi/
213-07/eagle/
 - binning 2, 1

214abcd-07/casi/
214abcd-07/eagle/
 - binning 2, 1

217b-07/casi/
217b-07/eagle/
 - binning 2, 1

219a-07/casi/
219a-07/eagle/
 - binning 2, 1

219b-07/casi/
219b-07/eagle/
 - binning 2, 1

boresight-2007_166/
 - binning = {8, 2}

boresight-2007_254b/
 - binning = {4, 1}

Conclusion: use boresight_166 (8,2), boresight_254 (4,1) and one of the IPY flights (2,1)

[mggr]

These are very initial results and purely based on eyeballing pixels:

Flight day	Binning	Notes
boresight 166, line 1	binning = {8, 2}	Variable scaling difference, around 5-7x
IPY 219, line 1	binning = {2, 1}	Very similar shape but peaks a little lower and troughs a little higher?
boresight 254, line 1	binning = {4, 1}	Similar-ish numbers (within a factor of 2 at worst), but the high end of the spectrum looks different

Binning = {spectral, spatial}

[mggr]

Boresight 166 pixel comparison

[mggr]

Boresight 254 pixel comparison

[mggr]

IPY 219 pixel comparison

[mggr]

binning 8,2

binning 2,1

binning 4,1

[mggr]

41 including pixel taken

[mggr]

254 (41) roof

[mggr]

Looks like the 254 with the strange shaped spectral profile was being corrupted by nearby pixels - doing the other tests on a larger surface (roof of a building) gives spectra looking very similar in shape and value.

The 4,1 and 2,1 spectra seem ok - only the 8,2 appears to be funny. Checking Rachel's flight (2006/195c), that was 2,2. Tentatively, it may be a problem with spatial binning > 1. Going to need to check a bit further, but Bill should know about this - I remember he was very unkeen on particular binning rates.

[mggr]

Dropped Bill an email explaining the above to see if he knows about it.

[mggr]

Spoke to Bill on the phone, who says that some of the Specim code for doing the calibration looks a little odd (sums pixels with dividing through after). One would expect the error to be closer to a factor of 2, but it's hard to pin down.

Actions:

• Bill will have a look at the source and see if anything looks really wrong
• Mike to make sure Bill has a copy of the 166 boresight flight to play with
• Mike to go through and try and pick out better comparison pixels, in larger numbers, and see if we can get an accurate overall difference between the spectra

[mggr]

Spreadsheet of pixel comparisons

[mggr]

PDF version of graphs in spreadsheet

[mggr]

Attached a spreadsheet with detailed comparisons on selected pixels. There appears to be (very) approximately a factor of 6 between Eagle and CASI numbers.

[mggr]

Bill has been going through the code with Jukka at Specim - it seems likely, though not yet confirmed, that Caligeo uses different code to that which Bill was originally given by Specim. This may account for the error.

Date of Eagle radiometric calibration is now the week of the 17th Dec.

[mggr]

Specim-processed Eagle vs. AZ-processed Eagle (vs CASI)

[mggr]

Jukka@Specim has processed the same eagle file with Specim software.

The file is a radiometrically corrected BIL, sourced from the raw data and calibration we're using (so should be identical) but produced with the AISATools software.

I've had a peek at it with ENVI and compared against the CASI flight again and Jukka's Eagle numbers are definitely in the right range and the spectral shape fits the CASI. Interestingly, the lower end of the spectral range is better shaped too.

I've attached a picture of the comparison - I'm pretty sure I'm hitting the same pixel in both Eagles. I'll do a more quantative comparison later, but thought this would be useful food for thought.

[mggr]

FWHM (​http://en.wikipedia.org/wiki/Full_width_at_half_maximum) changes around 750nm (a little) and 550nm (more substantially).. possibly this could account for the difference in the lower end on AZ output, but it's not very well aligned..

[mggr]

Concluding this is problem in azspec's handling of spatial binning > 1. The cause appears to be incorrect code being supplied to Bill by Specim in 2005/6.

As far as data delivery goes, we should probably hold onto Eagle spatial binning = 1 files for a short while longer - these don't seem to be greatly affected but Bill has a suspicion that the variance of bandwidth across the CCD isn't handled correctly, so there may be a minor variation at the blue end of the spectrum.

[mggr]

Confirmed issue on calibrated light source at Kidlington, including the issue at the blue end.

Two ways forward, both should be pursued:

• fix azspec calibration (this requires support from Specim)
• attempt workaround using caligeo to do the radiometric calibration, then reimport back into azgcorr compatible form

[mggr]

(belated updates)

The steering committee has been informed about this (10/11 Dec) and supports giving Specim a nudge.

Jukka and Bill have subsequently communicated a little more, with Jukka sending a binary copy of the AISATools package and re-examining source code (no outcome). This has allowed some additional testing by Bill, but we still really need a fuller explanation of the camera processing or the source code of the AISATools correction (which Jukka has promised). Possibly forthcoming in the new year..

[mggr]

8-10/Jan: Jukka mailed Bill relevant source and this enabled Bill to find the major problem in azspec (a line of code had moved outside a loop). azspec now produces output of the correct magnitude. However, the shape change at one end (red? blue) of the spectrum is still present, as above. Bill is continuing to investigate this.

[anee]

AZ Eagle vs. CaliGeo Eagle vs. AZ CASI

[anee]

Using caligeo for radiometric correction of eagle data produces a spectral shape which is far closer to the casi shape than the az eagle. The overall shape is similar but the actual values are still a bitt off (although a great improvement over the az values).

[anee]

AZ CASI vs. AZ Eagle vs. Jukka Eagle - using updated azspec

[anee]

Boresight 166 pond spectral shapes:

[anee]

AZ CASI vs. Caligeo Eagle vs. New AZ Eagle vs. Old AZ eagle boresight flight 254

[anee]

Boresight 254 pond spectral shapes:

[anee]

jpg version

[anee]

jpg version

[anee]

jpg version

[mggr]

AISA Tools on day 254 (4 spectral / 1 spatial). Spectral shape looks compressed and magnitudes are too low.

[mggr]

Missing entry from first week of Feb. The newest azspec now has a fix to the smear correction. Here's a summary of the problem from an email sent to AKW (by mggr):

Once that was solved, the second problem was that the signal via azspec drifted off at one end of the spectrum.  This appears to be due to a problem in the smear correction (as rows of data are shifted down the CCD, you have to remove a proportion of the value due to light accumulating on the CCD as the pixels are shifted).

There was an unusual multiplier in Jukka's smear correction that he'd not told Bill about (due to the spectral binning being done in the camera?), so Bill's result would drift off as one got towards the end of the CCD and the error accumulated.  Bill eventually discovered this by enabling and disabling features in AISATools (same algorithm as caligeo) and demonstrating the difference was in the smear correction, then discovering it was a fixed multiplier difference (spectral binning - 1?), whereupon Jukka was able to confirm..

[mggr]

Looking like this may be fixed.. Ben has sent Rachel her 195a (& soon c) data processed with azspec, so if she's happy with it, we're probably good :)

[anee]

AisaTools Eagle vs. Caligeo Eagle

[anee]

Tested AisaTools again with new version - also used separate dark file. Seems fine.

[mggr]

After some further comparisons and discussion, we're calling this fixed.

Eagle & Hawk processing can now be completed. Gary will notify PIs.