Opened 12 years ago

Closed 11 years ago

#81 closed bug (fixed)

Eagle radiometric calibration mismatch

Reported by: mggr Owned by: mggr
Priority: immediate Milestone: 2007 data processing completion
Component: az* programs Keywords:
Cc: Other processors:

Description (last modified by mggr)

This ticket is spawned from #74 (support ticket for Rachel Gaulton, who found this problem). See that ticket for the initial work, summarised here.

Eagle's calibrated spectral profile on flights appears to differ substantially from CASI in terms of the measured radiance. The shape is the same but the numbers are different by a factor of approximately 4-8, depending on the pixel chosen.

Short history to date:

  • 16/Oct Initial problem report from Rachel (2006 dataset)
  • 18/Oct AKW recommended investigating on a known dataset
  • 22/Oct Picked 166/2007 boresight flight and confirmed we're also seeing this issue
  • 29/Oct Demonstrated to Gary and agreed routes for investigation

Original L0 and L1b tests

Attachments (15)

boresight166.png (16.1 KB) - added by mggr 12 years ago.
Boresight 166 pixel comparison
boresight254.png (19.0 KB) - added by mggr 12 years ago.
Boresight 254 pixel comparison
ipy.png (22.3 KB) - added by mggr 12 years ago.
IPY 219 pixel comparison
41withpixel.png (824.6 KB) - added by mggr 12 years ago.
41 including pixel taken
254roof.png (795.1 KB) - added by mggr 12 years ago.
254 (41) roof
pixel_profiles_166.xls (175.0 KB) - added by mggr 12 years ago.
Spreadsheet of pixel comparisons
pixel_profiles_166_graphs.pdf (457.9 KB) - added by mggr 12 years ago.
PDF version of graphs in spreadsheet
comparison.jpg (244.5 KB) - added by mggr 12 years ago.
Specim-processed Eagle vs. AZ-processed Eagle (vs CASI)
comparison_eagle_caligeo.jpg (715.3 KB) - added by anee 11 years ago.
AZ Eagle vs. CaliGeo Eagle vs. AZ CASI
comparison_eagle_new_azspec.png (991.6 KB) - added by anee 11 years ago.
AZ CASI vs. AZ Eagle vs. Jukka Eagle - using updated azspec
comparison_eagle_caligeo_newaz.png (996.4 KB) - added by anee 11 years ago.
AZ CASI vs. Caligeo Eagle vs. New AZ Eagle vs. Old AZ eagle boresight flight 254
comparison_aisatools_caligeo.jpg (129.1 KB) - added by anee 11 years ago.
jpg version
comparison_eagle_new_azspec.jpg (216.5 KB) - added by anee 11 years ago.
jpg version
comparison_eagle_caligeo_newaz.jpg (259.8 KB) - added by anee 11 years ago.
jpg version
comparison_aisatools_caligeo2.jpg (552.6 KB) - added by anee 11 years ago.
AisaTools Eagle vs. Caligeo Eagle

Change History (41)

comment:1 Changed 12 years ago by mggr

  • Status changed from new to assigned

Routes of investigation:

  1. Chris from FSF is visiting in early Dec to calibrate some instruments for 2008. While there, get him to do a quick and dirty calibration on the Specim too, so we have a calibration made in the same conditions as the CASI one to compare against (giving us a ground truth). This will establish if there's a problem with Specim's calibration procedure.
  2. As soon as Bill is back from holiday (~10th Nov?), ask him to check on how he handles the Specim calibration in azspec, particularly with non 1-1 binnings (all 2007 Eagle flights are 2-1 or higher and the calibration is 1-1).
  3. compare level 0 vs level 1b data to ensure the calibration is being applied
    • done - the level0 looks a bit like a CCD response curve and the level 1b looks like a ground-spectrum and matches the CASI spectra, so this appears to be working, albeit with an offset/factor
  4. check the 2007 flights for various binnings to compare against CASI
    • done - no 2007 flights are 1/1, they vary from 2/1 to 8/1 and 8/4
  5. compare CASI and Eagle spectra in multiple flights to see if there's a common factor, bearing in mind the binning rates
    • only the 166 boresight and some of the IPY flights have CASI and Eagle run together

comment:2 Changed 12 years ago by mggr

  • Description modified (diff)

Tested L3 vs L1b to ensure it isn't a pixel/sampling size issue (appears it isn't).

L3 tests

L3 tests (big graphs)

comment:3 Changed 12 years ago by mggr

Flights with CASI and Eagle together (inc binning rates):

213-07/casi/
213-07/eagle/
 - binning 2, 1

214abcd-07/casi/
214abcd-07/eagle/
 - binning 2, 1

217b-07/casi/
217b-07/eagle/
 - binning 2, 1

219a-07/casi/
219a-07/eagle/
 - binning 2, 1

219b-07/casi/
219b-07/eagle/
 - binning 2, 1

boresight-2007_166/
 - binning = {8, 2}

boresight-2007_254b/
 - binning = {4, 1}

Conclusion: use boresight_166 (8,2), boresight_254 (4,1) and one of the IPY flights (2,1)

comment:4 Changed 12 years ago by mggr

These are very initial results and purely based on eyeballing pixels:

Flight day Binning Notes
boresight 166, line 1 binning = {8, 2} Variable scaling difference, around 5-7x
IPY 219, line 1 binning = {2, 1} Very similar shape but peaks a little lower and troughs a little higher?
boresight 254, line 1 binning = {4, 1} Similar-ish numbers (within a factor of 2 at worst), but the high end of the spectrum looks different

Binning = {spectral, spatial}

Changed 12 years ago by mggr

Boresight 166 pixel comparison

Changed 12 years ago by mggr

Boresight 254 pixel comparison

Changed 12 years ago by mggr

IPY 219 pixel comparison

comment:5 Changed 12 years ago by mggr

binning 8,2
Boresight 166 pixel comparison

binning 2,1
IPY 219 pixel comparison

binning 4,1
Boresight 254 pixel comparison

Changed 12 years ago by mggr

41 including pixel taken

comment:6 Changed 12 years ago by mggr

41 including pixel taken

Changed 12 years ago by mggr

254 (41) roof

comment:7 Changed 12 years ago by mggr

254 (41) roof

comment:8 Changed 12 years ago by mggr

Looks like the 254 with the strange shaped spectral profile was being corrupted by nearby pixels - doing the other tests on a larger surface (roof of a building) gives spectra looking very similar in shape and value.

The 4,1 and 2,1 spectra seem ok - only the 8,2 appears to be funny. Checking Rachel's flight (2006/195c), that was 2,2. Tentatively, it may be a problem with spatial binning > 1. Going to need to check a bit further, but Bill should know about this - I remember he was very unkeen on particular binning rates.

comment:9 Changed 12 years ago by mggr

Dropped Bill an email explaining the above to see if he knows about it.

comment:10 Changed 12 years ago by mggr

Spoke to Bill on the phone, who says that some of the Specim code for doing the calibration looks a little odd (sums pixels with dividing through after). One would expect the error to be closer to a factor of 2, but it's hard to pin down.

Actions:

  • Bill will have a look at the source and see if anything looks really wrong
  • Mike to make sure Bill has a copy of the 166 boresight flight to play with
  • Mike to go through and try and pick out better comparison pixels, in larger numbers, and see if we can get an accurate overall difference between the spectra

Changed 12 years ago by mggr

Spreadsheet of pixel comparisons

Changed 12 years ago by mggr

PDF version of graphs in spreadsheet

comment:11 Changed 12 years ago by mggr

Attached a spreadsheet with detailed comparisons on selected pixels. There appears to be (very) approximately a factor of 6 between Eagle and CASI numbers.

comment:12 Changed 12 years ago by mggr

Bill has been going through the code with Jukka at Specim - it seems likely, though not yet confirmed, that Caligeo uses different code to that which Bill was originally given by Specim. This may account for the error.

Date of Eagle radiometric calibration is now the week of the 17th Dec.

Changed 12 years ago by mggr

Specim-processed Eagle vs. AZ-processed Eagle (vs CASI)

comment:13 Changed 12 years ago by mggr

Jukka@Specim has processed the same eagle file with Specim software.

The file is a radiometrically corrected BIL, sourced from the raw data and calibration we're using (so should be identical) but produced with the AISATools software.

I've had a peek at it with ENVI and compared against the CASI flight again and Jukka's Eagle numbers are definitely in the right range and the spectral shape fits the CASI. Interestingly, the lower end of the spectral range is better shaped too.

I've attached a picture of the comparison - I'm pretty sure I'm hitting the same pixel in both Eagles. I'll do a more quantative comparison later, but thought this would be useful food for thought.

Specim-processed Eagle vs. AZ-processed Eagle (vs CASI)

comment:14 Changed 12 years ago by mggr

FWHM (http://en.wikipedia.org/wiki/Full_width_at_half_maximum) changes around 750nm (a little) and 550nm (more substantially).. possibly this could account for the difference in the lower end on AZ output, but it's not very well aligned..

comment:15 Changed 12 years ago by mggr

  • Component changed from Processing: Eagle to az* programs
  • Summary changed from Eagle radiometric calibration investigation to Eagle radiometric calibration mismatch
  • Type changed from task to bug

Concluding this is problem in azspec's handling of spatial binning > 1. The cause appears to be incorrect code being supplied to Bill by Specim in 2005/6.

As far as data delivery goes, we should probably hold onto Eagle spatial binning = 1 files for a short while longer - these don't seem to be greatly affected but Bill has a suspicion that the variance of bandwidth across the CCD isn't handled correctly, so there may be a minor variation at the blue end of the spectrum.

comment:16 Changed 11 years ago by mggr

Confirmed issue on calibrated light source at Kidlington, including the issue at the blue end.

Two ways forward, both should be pursued:

  • fix azspec calibration (this requires support from Specim)
  • attempt workaround using caligeo to do the radiometric calibration, then reimport back into azgcorr compatible form

comment:17 Changed 11 years ago by mggr

(belated updates)

The steering committee has been informed about this (10/11 Dec) and supports giving Specim a nudge.

Jukka and Bill have subsequently communicated a little more, with Jukka sending a binary copy of the AISATools package and re-examining source code (no outcome). This has allowed some additional testing by Bill, but we still really need a fuller explanation of the camera processing or the source code of the AISATools correction (which Jukka has promised). Possibly forthcoming in the new year..

comment:18 Changed 11 years ago by mggr

8-10/Jan: Jukka mailed Bill relevant source and this enabled Bill to find the major problem in azspec (a line of code had moved outside a loop). azspec now produces output of the correct magnitude. However, the shape change at one end (red? blue) of the spectrum is still present, as above. Bill is continuing to investigate this.

Changed 11 years ago by anee

AZ Eagle vs. CaliGeo Eagle vs. AZ CASI

comment:19 Changed 11 years ago by anee

Using caligeo for radiometric correction of eagle data produces a spectral shape which is far closer to the casi shape than the az eagle. The overall shape is similar but the actual values are still a bitt off (although a great improvement over the az values).

 AZ Eagle vs. CaliGeo Eagle vs. AZ CASI

Changed 11 years ago by anee

AZ CASI vs. AZ Eagle vs. Jukka Eagle - using updated azspec

comment:20 Changed 11 years ago by anee

Boresight 166 pond spectral shapes:

AZ CASI vs. AZ Eagle vs. Jukka Eagle - using updated azspec

Changed 11 years ago by anee

AZ CASI vs. Caligeo Eagle vs. New AZ Eagle vs. Old AZ eagle boresight flight 254

comment:21 Changed 11 years ago by anee

Boresight 254 pond spectral shapes:

AZ CASI vs. Caligeo Eagle vs. New AZ Eagle vs. Old AZ eagle boresight flight 254

Changed 11 years ago by anee

jpg version

Changed 11 years ago by anee

jpg version

Changed 11 years ago by anee

jpg version

comment:22 Changed 11 years ago by mggr

jpg version

AISA Tools on day 254 (4 spectral / 1 spatial). Spectral shape looks compressed and magnitudes are too low.

comment:23 Changed 11 years ago by mggr

Missing entry from first week of Feb. The newest azspec now has a fix to the smear correction. Here's a summary of the problem from an email sent to AKW (by mggr):

Once that was solved, the second problem was that the signal via azspec drifted off at one end of the spectrum.  This appears to be due to a problem in the smear correction (as rows of data are shifted down the CCD, you have to remove a proportion of the value due to light accumulating on the CCD as the pixels are shifted).

There was an unusual multiplier in Jukka's smear correction that he'd not told Bill about (due to the spectral binning being done in the camera?), so Bill's result would drift off as one got towards the end of the CCD and the error accumulated.  Bill eventually discovered this by enabling and disabling features in AISATools (same algorithm as caligeo) and demonstrating the difference was in the smear correction, then discovering it was a fixed multiplier difference (spectral binning - 1?), whereupon Jukka was able to confirm..

comment:24 Changed 11 years ago by mggr

Looking like this may be fixed.. Ben has sent Rachel her 195a (& soon c) data processed with azspec, so if she's happy with it, we're probably good :)

Changed 11 years ago by anee

AisaTools Eagle vs. Caligeo Eagle

comment:25 Changed 11 years ago by anee

Tested AisaTools again with new version - also used separate dark file. Seems fine.

AisaTools Eagle vs. Caligeo Eagle

comment:26 Changed 11 years ago by mggr

  • Resolution set to fixed
  • Status changed from assigned to closed

After some further comparisons and discussion, we're calling this fixed.

Eagle & Hawk processing can now be completed. Gary will notify PIs.

Note: See TracTickets for help on using tickets.