Opened 16 years ago
Last modified 14 years ago
#172 closed support
Support: 26/June/2008, Chris Hecker, WM06/06 + ET07/05 — at Version 3
Reported by: | mggr | Owned by: | mggr |
---|---|---|---|
Priority: | immediate | Milestone: | |
Component: | Support | Keywords: | |
Cc: | Other processors: |
Description (last modified by mggr)
Chris contacted us with a lot of complex questions. Going to link full emails here rather than precis.
Hi Gary I am currently working on the Eagle / Hawk data from the 2006 Campaign to Mojacar, Spain (WM06-06, PI: Teeuw, 2 June 2006), also as a testcase for the Ethiopia data. I am using ATCOR-4 for airborne data for a model based atmospheric correction. while doing so I noticed artifacts that give troubles. I remember that the first year's data gave lots of headaches and was wondering if you knew about the symptoms I am getting and if you may have a workaround / solution for them. I put some screendumps in a *.ppt file for you. Sensor Saturation In some of the brighter areas of flightline 13, I notice some strange behaviour (it may be in other places as well, but I first noticed it in flightline 13, since our calibration sites are in that line). The Eagle spectra look OK as of about 0.7 micron. wavelengths shorter than that show a strong trough in their reflectance spectra. Slide 1 shows the spectra we recorded on the ground (in gree) and the Eagle spectra (in yellow and orange). It looks like the detector saturated and the data wrapped around when it reached the end of the ?sensitivit or long integer range. I checked the hdf file but couldnt really find any indication on which pixels had saturated during acquisition. Here some questions: - Was sensor saturation a problem that year? - Can we get a mask that shows pixels that reached the saturation level (or close to saturation level) so we can mask them out. Or alternatively, the original DN values before they were changed to scaled radiance values. Wavelength calilbration The eagle and hawk hdf files come with central wavelength and width of each band. When I use that for the atmospheric correction, I get clear artifacts that show that there is a wavelength accuracy problem. slide2 shows a hawk spectrum with derivative looking artifact at 1.14 and also a general shift of the image spectrum towards longer wavelengths as compared to fieldspectra (green; for example alunite spectral feature at 2.2 looks shifted). - are there any spectral response curves for individual detectors? - is anything known about the shape of the response curves? - can we assume that the widths of bands as listed in the hdf file is a fwhm for a gaussian shaped response curve (as an approximation) - is there a newer wavelength calibration file than the one that was used then? Spikes or steps spectra often show a step around 0.688 microns and some spikes / noise at 0.860 (see slide 1). - Is that related to an internal change to a different detector array? - anything that can be done to correct for it? Radiometric accuracy radiance values in general look a bit lower than ground spectra with appropriate looking atmospheric correction. That's more an observation than a problem, since we can correct for that easily. Actually, if I look at the figure 2c in the data_quality-2007.pdf , it looks to me like there is not only a radiometric shift between Eagle and Hawk but also a 50 nm wavelength shift. Sounds like a lot though, so not sure if that is possible. Greets from Holland, Chris
Summary of actions required:
- generate 2006 overflow info and masks for Chris' projects (requires 2006 data)
- notify with results of 2008 July calibration (wavelength and radiance) info when they become available
info on changes made to the setup of the two sensors between 2006 and 2008 (fov and nr of pixels)(speculated on this)- pass on copies of relevant CCD datasets if and when we get them from Specim (#173)
Change History (3)
comment:1 Changed 16 years ago by mggr
comment:2 Changed 16 years ago by mggr
Reply on 30/June:
Hi Mike As usual a great anser with lots of detail but still right to the point! Thanks for that. Here some follow up thoughts: The spain 2006 data is more of a testcase for the ethiopia 2008jan data that has/will be coming in. If the data from 2006 is re-processed with the improved saturation detection, I would be happy to get the mask of saturated pixels (and an estimation of how many neighbouring pixels spatially / spectrally may have to be buffered out as well) when the re-prosessing is done. No need for the raw data there. > > We're happy to either generate a mask for you or pass you the > > raw data so you can generate your own. The 2006 data will > > take some time though > > - currently it's held in a fire safe at the airport. Gary > > was going to ship a copy down to us sometime in the next few > > weeks, whereupon we can make it available to you. If you > > want copies of 2007 data, we have that here and can make it > > available immediately. I wasn't involved in any 2007 acquisitions but the 2008 ET07-05 (Mojo) data was delivered a few weeks ago. Any changes / saturation issues there would be of great interest to us. > > We don't have any information on the response curves at this > > time :/ We've recently asked Specim for copies of the CCD > > datasets - I can pass those on to you when we get them. We'd > > currently use the gaussian curve approximation with the listed fwhms. OK will stick to gaussian curves at the moment but response curves are appreciated when they become available. > > I can provide the Feb 2007 calibration files if that'd be > > useful to you? I think the Ethiopian data still uses the same calibration and I have extracted the info from the hdf file. So that should be OK, I just havent had the chance yet to look at the data. Will keep you up-to-date on what I find. I also noticed that the 2008 data for Eagle and for Hawk shows different wavelength positions (I guess it was re-calibrated in 2007, as you mentioned) and even the number of bands and pixels per scan line have changed. Eagle now uses a different total fov, of which the port and starboard section seem to be asymmetric. Is the sensor set up now different, and where do the extra pixels come from? > > With regard to the steps, one thing that does come to mind is > > this might be where the overflow regions tend to begin. If > > an overflow occurs, the raw value will stay fixed at the > > maximum and the result after calibration will only reflect > > the changes in the dark values and the calibration multiplier > > [calibration procedure is basically reflectance = (DN - dark > > value) * multiplier] > > > > The other less likely possibility is there may be filters > > inside the detector that catch second- or higher-order light > > that'd otherwise affect measurements of the first order > > light. I know some of the Specim instruments incorporate > > these filters, but presume this would have been accounted > > for. I'd have to ask Specim for the wavelengths where this > > filter begins, if it's present. I think that it could be due to the saturation or even due to spectral shifts which may cause the artifact during atm. Correction. Will have a look at the 2008 data and let you know what I find. >> > > *Radiometric accuracy* >> > > radiance values in general look a bit lower than ground > > spectra with >> > > appropriate looking atmospheric correction. That's more an > > observation >> > > than a problem, since we can correct for that easily. > > > > Yup, again, we're pretty limited on the sources of > > verification we have at this time - for the most part we > > don't even have ground spectra we can confirm against. Gary > > has been working on this and has made an arrangement that'll > > give us some large spectrally calibrated targets near to the > > airport and we'll incorporate regular checks when these are in place. Great! We do have simultaneous ground measurements for Spain (2006) and Ethiopia (2008) although they were taken on reasonably homogeneous natural targets rather than on atrificial tarps. May give an indication, though. Our bright target on the 2006 Mojacar flight is useless, since it was a bit too bright for the sensor! ;-) Our dark target of the 2006 Mojacar flight (Coal stock pile) lloks very resonable compared to the airborne data. Richard Teeuw had made some measurement on the opposite end of the study area and we may find some good spectra of bright targets in his collection to compare with. > > Let me know if you'd like any of the above offers of data. > > Obviously we'll inform you on the progress of the overflow > > issue once we have a proper idea of the extent of the > > problem, and will let you know the results of the radiometric > > calibration at the end of July. Thanks Mike. For right now, I will focus on the Ethiopia data (mojo) and see what comes out of that. Looking forward to - 2006 overflow info and masks - 2008July calibration (wavelength and radiance)info when they become available. - info on changes made to the setup of the two sensors between 2006 and 2008 (fov and nr of pixels) Cheers, Chris
and counter reply:
Hi Chris, Chris Hecker wrote: > As usual a great anser with lots of detail but still right to the point! Albeit 10KB long! This reply should strain the mail server less though :) > The spain 2006 data is more of a testcase for the ethiopia 2008jan data > that has/will be coming in. If the data from 2006 is re-processed with > the improved saturation detection, I would be happy to get the mask of > saturated pixels (and an estimation of how many neighbouring pixels > spatially / spectrally may have to be buffered out as well) when the > re-prosessing is done. No need for the raw data there. Ok, we'll let you know when we have the 2006 info available. I'd expect that'll be on the order of weeks-months (minimum 2-3 weeks before we get the data here). > I wasn't involved in any 2007 acquisitions but the 2008 ET07-05 (Mojo) > data was delivered a few weeks ago. Any changes / saturation issues > there would be of great interest to us. We could certainly make the Mojo raw data available now, which would let you determine any saturation there. We could also probably produce a mask if you want to wait a few days for us to produce the code to generate this (I suspect you can out-speed us on this if you're in a hurry!). Let me know what you'd prefer :) > OK will stick to gaussian curves at the moment but response curves are > appreciated when they become available. Will do :) > I also noticed that the 2008 data for Eagle and for Hawk shows different > wavelength positions (I guess it was re-calibrated in 2007, as you > mentioned) and even the number of bands and pixels per scan line have > changed. Eagle now uses a different total fov, of which the port and > starboard section seem to be asymmetric. Is the sensor set up now > different, and where do the extra pixels come from? Hmm, that's interesting. I'd speculate that the difference is due to the FODIS region being activated - I don't know if this was present in 2006. The FODIS is an incident light measurement which works by bringing light down from the top of the aircraft, via a fiber optic cable, direct to the CCD. The FODIS region is roughly the first 64 columns on the left side of the CCD. We don't have 2006 data to look at, but would this account for what you're seeing? See http://www.npm.ac.uk/rsg/projects/arsf/trac/ticket/133 for some example pics of the raw files including the FODIS region. > I think that it could be due to the saturation or even due to spectral > shifts which may cause the artifact during atm. Correction. Will have a > look at the 2008 data and let you know what I find. That'd be very helpful, thank you :) > Great! We do have simultaneous ground measurements for Spain (2006) and > Ethiopia (2008) although they were taken on reasonably homogeneous > natural targets rather than on atrificial tarps. May give an indication, > though. Our bright target on the 2006 Mojacar flight is useless, since > it was a bit too bright for the sensor! ;-) Our dark target of the 2006 > Mojacar flight (Coal stock pile) lloks very resonable compared to the > airborne data. Richard Teeuw had made some measurement on the opposite > end of the study area and we may find some good spectra of bright > targets in his collection to compare with. That's reassuring that the dark target sounds reasonable (and oops @ the bright target ;) ). We'd certainly be very interested if you find anything notable in other comparisons - perhaps we could ask Richard for a copy of his ground data if it turns out to be useful. Cheers, Mike.
comment:3 Changed 16 years ago by mggr
- Description modified (diff)
Added note on actions required for this ticket.
Note: See
TracTickets for help on using
tickets.
Reply: