Opened 10 years ago

Closed 10 years ago

Last modified 9 years ago

#249 closed bug (invalid)

Rescaling dark frames causes calibration errors

Reported by: benj Owned by: benj
Priority: alpha 5 Milestone:
Component: az* programs Keywords:
Cc: mggr, jatt Other processors:

Description (last modified by mggr)

Processing data in azspec and using the -dIc option to rescale dark frames seems to cause the data to be incorrectly radiometrically calibrated.

I've attached three screenshots. dIc_ex1.png shows spectral profiles for the rescaled flightline from both azspec (top) and CaliGeo (bottom). You can see that the azspec spectrum goes up at both ends, where the CaliGeo spectrum doesn't. The azspec spectrum also has values that are consistently about 600 higher than CaliGeo (discounting the edges of the spectrum, which as mentioned are much higher).

dIc_ex2.png shows spectral profiles from the second flightline, which did not require the dark frames to be rescaled. Azspec and CaliGeo are in very close agreement for this line.

odd_one_out.png is just for illustration - it shows all of the Eagle lines from that flight (as processed through azspec) overlaid. The first flightline (with the rescaled dark frames) is clearly visible as different - much whiter than the other lines.

This may affect any datasets from 2006, and some VOCALS data.

Related tickets: #218 #250

Attachments (4)

dIc_ex1.png (602.7 KB) - added by benj 10 years ago.
Spectral profiles from Eagle line with rescaled dark frames (GB04-06 line 1)
dIc_ex2.png (493.8 KB) - added by benj 10 years ago.
Spectral profiles from Eagle line with separate but not rescaled dark frames (GB04-06 line 2)
odd_one_out.png (684.6 KB) - added by benj 10 years ago.
Image for illustration showing difference of rescaled line (far left, whitish)
hawk_dic_working.png (217.5 KB) - added by benj 10 years ago.
Image showing synthetic dataset hawk with and without rescaling

Download all attachments as: .zip

Change History (11)

Changed 10 years ago by benj

Spectral profiles from Eagle line with rescaled dark frames (GB04-06 line 1)

Changed 10 years ago by benj

Spectral profiles from Eagle line with separate but not rescaled dark frames (GB04-06 line 2)

comment:1 Changed 10 years ago by benj

Have reported this issue to Bill for investigation.

comment:2 Changed 10 years ago by benj

VOCALS data affected by this issue are as follows (thanks Jane):

eagle:   line 04
         line 05

hawk:    line 04
         line 05


Vocals day 04

hawk:    line 05
         line 06
         line 07
         line 08

Vocals day 05

hawk:    line 02
         line 03
         line 04
         line 05
         line 06
         line 07
         line 08
         line 09
         line 10
         line 11


Vocals day 06

eagle:   line 03

hawk:    line 03

comment:3 Changed 10 years ago by benj

Top lines in the previous comment (without a day number) are for day 3 - copy/paste error on my part.

Changed 10 years ago by benj

Image for illustration showing difference of rescaled line (far left, whitish)

comment:4 Changed 10 years ago by benj

  • Resolution set to wontfix
  • Status changed from new to closed

Not doing anything about this, going to produce a program to rescale dark frames as appropriate external to azspec (see #250). Closing this as wontfix unless there turns out to be some problem with the other approach.

Changed 10 years ago by benj

Image showing synthetic dataset hawk with and without rescaling

comment:5 Changed 10 years ago by benj

  • Resolution wontfix deleted
  • Status changed from closed to reopened

-dIc option does actually appear to be working. It goes wrong for Eagle but that's because Eagle darkframe values don't appear to vary with integration time (see comments in #250), so its use for Eagle is inappropriate. Seems to work fine for Hawk.

Image shows synthetic data processed with -dIc and with dark frames for correct integration time. Values don't match exactly, but that's due to a combination of the fact that the dark frames were produced using an average value (and dark values for Hawk vary with band) and also the rescaling also necessarily uses only an average scaling value. Values are close enough to say it's behaving correctly given the limitations inherent in the method.

Image showing synthetic dataset hawk with and without rescaling

Going to reopen ticket and reclose as invalid rather than wontfix.

comment:6 Changed 10 years ago by benj

  • Resolution set to invalid
  • Status changed from reopened to closed

comment:7 Changed 9 years ago by mggr

  • Description modified (diff)

Added convenience links for later memory assistance ;)

Note: See TracTickets for help on using tickets.