TIFF tagger

[mggr]

We need a small tool for adding tags (small text/numerical elements) to TIFF image files.

The pre 2009 aerial photos contain metadata inserted by a scanning company. You can see what's in the 2008 photos by doing something like tiffinfo ~arsf/arsf_data/2008/flight_data/uk/GB08_18-2008_176a_River_Frome/photography/08_03_8190.tif | less.

Mostly they seem to have created an XML metadata section. Many of their tags relate to the scanning process and are not relevant. Ideally we should duplicate the XML metadata in addition to putting data into relevant tags.

---

There are a large number of defined tags and a mechanism for picking "free" tags. I'd suggest trying to conform to the EXIF standard where possible.

​http://www.awaresystems.be/imaging/tiff/tifftags.html

• for the exif bit, see private ifd tags -> exif, or look at ​http://www.exif.org/

For example, there are predefined tags for GPS location that we may wish to use as that'd mean the images could be correctly geolocated in a standard package.

• ​http://www.awaresystems.be/imaging/tiff/tifftags/privateifd/gps/gpsareainformation.html

There are also some bits for descriptions and comments, which could be used for the project info.. Alternatively, you could pick one of the "free" ranges and make up your own tag. Before making up your own tag, please a) check for a standard one and b) check what the 2008 photos did.

---

Suggested approach:

• review EXIF / TIFF tag documents to familarise yourself with the standard tags
• identify suitable tools / libraries for working with TIFF tags (google "python tiff tag" or "python exif"?)
• duplicate the XML metadata in a 2008 photo for a 2009 one
• insert the XML metadata into a 2009 TIFF
• insert EXIF standard additional tags (particularly geolocation ones) into the 2009 TIFF

[mggr]

The linux command line tool "tiffset" may be of use.

[chrfi]

using exiv2 program command line tool for tiff editing

using gps idf to store gps data

output of the exiv2 tif print is

u@pmpc974:~/scratch/tifftag<1016>~/usr/bin/exiv2 pr -pas 18204753100197G3.tif Exif.Image.NewSubfileType                    Long        1  Primary image
Exif.Image.ImageWidth                        Short       1  7162
Exif.Image.ImageLength                       Short       1  5389
Exif.Image.BitsPerSample                     Short       3  16 16 16
Exif.Image.Compression                       Short       1  Uncompressed
Exif.Image.PhotometricInterpretation         Short       1  RGB
Exif.Image.ImageDescription                  Ascii     644  <?xml version="1.0" encoding="ISO-8859-1"?>
<Camera_Image>
   <File_info>
         <Original_filename>18204753100197G3.raw</Original_filename>
   </File_info>
   <Project_info>
      <Project_code>this is a project code</Project_code>
      <Flight_day>176a</Flight_day>
      <Location_name>bobbington</Location_name>
   </Project_info>
   <Plane_info>
      <Position>
         <Latitude> 51.877316</Latitude>
         <Longitude>--1.697540</Longitude>
         <Height>1616.127032</Height>
      </Position>
      <Attitude>
         <Pitch>2.882941</Pitch>
         <Roll>-0.516225</Roll>
      </Attitude>
   </Plane_info>
</Camera_Image>
Exif.Image.StripOffsets                      Long        1  1238
Exif.Image.SamplesPerPixel                   Short       1  3
Exif.Image.RowsPerStrip                      Short       1  5389
Exif.Image.StripByteCounts                   Long        1  231576108
Exif.Image.XResolution                       Rational    1  300
Exif.Image.YResolution                       Rational    1  300
Exif.Image.PlanarConfiguration               Short       1  1
Exif.Image.ResolutionUnit                    Short       1  inch
Exif.Image.GPSTag                            Long        1  922
Exif.GPSInfo.GPSLatitudeRef                  Ascii       2  North
Exif.GPSInfo.GPSLatitude                     Rational    3  51deg 52' 38.338" 
Exif.GPSInfo.GPSLongitudeRef                 Ascii       2  West
Exif.GPSInfo.GPSLongitude                    Rational    3  1deg 41' 51.144" 
Exif.GPSInfo.GPSAltitude                     Rational    1  1616.1 m
Exif.GPSInfo.GPSTimeStamp                    Rational    3  09:43:15
Exif.GPSInfo.GPSMapDatum                     Ascii       6  WGS84
Exif.GPSInfo.GPSDestLatitudeRef              Ascii       2  (E)
Exif.GPSInfo.GPSDestLatitude                 Rational    3  90deg 
Exif.GPSInfo.GPSDestLongitude                Rational    3  0deg 
Exif.GPSInfo.GPSDestBearingRef               Ascii       2  True direction
Exif.GPSInfo.GPSDestBearing                  Rational    1  179019253/1000000
Exif.GPSInfo.GPSDateStamp                    Ascii      11  2008:09:18
Exif.Image.UniqueCameraModel                 Ascii      25  Leica RDC digital camera
Exif.Image.CameraSerialNumber                Ascii       3  21

[mggr]

NEODC are happy with this. We just need to get it into subversion, etc and use it now!

[mggr]

Should consider adding some GeoTIFF tags to indicate it's an uncorrected photo

mggr:
http://www.remotesensing.org/geotiff/spec/geotiff3.html#3.2.1. ?  I don't see tags for time and attitude, so I guess it'd go in a text field (GeogCitationGeoKey?). 

billm:
What you do is put in a comment in ENVI style that can be parsed....

so like  ?Citation? = "owner = NERC-ARSF photoID = 271634 date = 04/09/2009 gpstime = 12:34:24.34567 attitude = { 34.567, 12.234, 124.500 }"

[mggr]

Some suggestions from Bill:

• ensure lat/longs are 7-9 decimal places (cm accuracy)
• including heading (done)
• add a means of telling the orientation of the photo (ie. it the top of the photo "forwards" on the plane?)
  • there may be a standard TIFF tag for orientation.. investigate

[mggr]

Comment about decimal place accuracy referred to the ASCII XML.

[chrfi]

xml lat/longs are as accurate as possible, the lat long is pulled out of the events file as a string and what you see in the xml is all there is, the accuracy in the tags is a slightly fuzzier affair as they have to be saved as a (rational number / rational number), but both R numbers have a limited size so its a question of the most accurate possible within the restrictions.

So far I have not found a tag for orientation, for now I will include orientation in the xml sometime soon.

using geotif tags has been looked into several times and no suitable program for manipulation has been found yet, it is time prohibitive to create our own

[mggr]

When you add the orientation xml, let's also put in the exact GPS timestamp from the CSV file. That'll allow referencing against the processed navigation for more precise positioning at a later point (extension: optionally do this using Ben's python sbet handler, should be a quick job).

[mggr]

Sample output from exiftool arsf_sample_photo.tif:

--------------------------------------------------
mggr@pmpc889:~<1007>exiftool
/users/rsg/arsf/workspace/IPY09_11-2009_219_Helheim/leica/rcd/images-processed/07120958100001G4.tif
ExifTool Version Number         : 7.67
File Name                       : 07120958100001G4.tif
Directory                       : 
/users/rsg/arsf/workspace/IPY09_11-2009_219_Helheim/leica/rcd/images-processed
File Size                       : 221 MB
File Modification Date/Time     : 2009:11:05 12:19:15+00:00
File Type                       : TIFF
MIME Type                       : image/tiff
Exif Byte Order                 : Little-endian (Intel, II)
Subfile Type                    : Full-resolution Image
Image Width                     : 7162
Image Height                    : 5389
Bits Per Sample                 : 16 16 16
Compression                     : Uncompressed
Photometric Interpretation      : RGB
Image Description               : 
<?xml version="1.0" encoding="ISO-8859-1"?>
<Camera_Image>
   <File_info>
         <Original_filename>07120958100001G4.raw</Original_filename>
   </File_info>
   <Project_info>
      <Project_code>IPY09/11</Project_code>
      <Flight_day>219</Flight_day>
      <Location_name>Helheim</Location_name>
   </Project_info>
   <Plane_info>
      <Position>
         <Latitude> 65.681986</Latitude>
         <Longitude>-37.251825</Longitude>
         <Height>1493.593631</Height>
      </Position>
      <Attitude>
         <Pitch>5.475196</Pitch>
         <Roll>-0.188219</Roll>
         <Heading>349.280971</Heading>
      </Attitude>
   </Plane_info>
</Camera_Image>
Strip Offsets                   : 1260
Samples Per Pixel               : 3
Rows Per Strip                  : 5389
Strip Byte Counts               : 231576108
X Resolution                    : 300
Y Resolution                    : 300
Planar Configuration            : Chunky
Resolution Unit                 : inches
GPS Latitude Ref                : North
GPS Longitude Ref               : West
GPS Altitude                    : 1493.593631 m
GPS Time Stamp                  : 11:24:00
GPS Map Datum                   : WGS84
GPS Dest Latitude Ref           : Unknown (E)
GPS Dest Latitude               : 90 deg 0' 0.00"
GPS Dest Longitude              : 0 deg 0' 0.00"
GPS Dest Bearing Ref            : True North
GPS Dest Bearing                : 349.280971
GPS Date Stamp                  : 2009:08:07
Unique Camera Model             : Leica RCD digital camera
Camera Serial Number            : 21
GPS Date/Time                   : 2009:08:07 11:24:00
GPS Latitude                    : 65 deg 40' 55.15" N
GPS Longitude                   : 37 deg 15' 6.57" W
GPS Position                    : 65 deg 40' 55.15" N, 37 deg 15' 6.57" W
Image Size                      : 7162x5389
-------------------------------------------

[mggr]

Required before we can deliver photos:

1. add orientation item to XML
2. add flight year to XML
3. add exact (fractions of second) GPS timestamp to XML

Extras that'd be nice:

1. acquire postprocessed position & attitude from supplied sbet file ; use instead of CSV posatt (CSV should be a backup if no sbet specified)
2. add appropriate geotiff tags (yeah, this one is harder)

[mggr]

Top of image = plane forwards

[chrfi]

Exif.Image.NewSubfileType                    Long        1  Primary image
Exif.Image.ImageWidth                        Short       1  7162
Exif.Image.ImageLength                       Short       1  5389
Exif.Image.BitsPerSample                     Short       3  16 16 16
Exif.Image.Compression                       Short       1  Uncompressed
Exif.Image.PhotometricInterpretation         Short       1  RGB
Exif.Image.ImageDescription                  Ascii     838  <?xml version="1.0" encoding="ISO-8859-1"?>
<Camera_Image>
   <File_info>
         <Original_filename>18204753100197G3.raw</Original_filename>
   </File_info>
   <Project_info>
      <Project_code>IPY09/11</Project_code>
       Year>2008</Year>
      <Flight_day_of_year>219</Flight_day_of_year>
      <GPStime_of_week>466980.393639</GPStime_of_week>
      <Location_name>bobbington</Location_name>
   </Project_info>
   <Plane_info>
      <Position>
         <Latitude> 51.877316</Latitude>
         <Longitude>-1.697540</Longitude>
         <Height>1616.127032</Height>
      </Position>
      <Attitude>
         <Pitch>2.882941</Pitch>
         <Roll>-0.516225</Roll>
         <Heading>179.019253</Heading>
      </Attitude>
   </Plane_info>
   <Image>
      <Orientation>top = plane fowards</Orientation>
   </Image>
</Camera_Image>
Exif.Image.StripOffsets                      Long        1  1440
Exif.Image.SamplesPerPixel                   Short       1  3
Exif.Image.RowsPerStrip                      Short       1  5389
Exif.Image.StripByteCounts                   Long        1  231576108
Exif.Image.XResolution                       Rational    1  300
Exif.Image.YResolution                       Rational    1  300
Exif.Image.PlanarConfiguration               Short       1  1
Exif.Image.ResolutionUnit                    Short       1  inch
Exif.Image.GPSTag                            Long        1  1124
Exif.GPSInfo.GPSLatitudeRef                  Ascii       2  North
Exif.GPSInfo.GPSLatitude                     Rational    3  51deg 52' 38.338" 
Exif.GPSInfo.GPSLongitudeRef                 Ascii       2  West
Exif.GPSInfo.GPSLongitude                    Rational    3  1deg 41' 51.144" 
Exif.GPSInfo.GPSAltitude                     Rational    1  1616.1 m
Exif.GPSInfo.GPSTimeStamp                    Rational    3  09:43:15.4
Exif.GPSInfo.GPSMapDatum                     Ascii       6  WGS84
Exif.GPSInfo.GPSDestLatitudeRef              Ascii       2  (E)
Exif.GPSInfo.GPSDestLatitude                 Rational    3  90deg 
Exif.GPSInfo.GPSDestLongitude                Rational    3  0deg 
Exif.GPSInfo.GPSDestBearingRef               Ascii       2  True direction
Exif.GPSInfo.GPSDestBearing                  Rational    1  179019253/1000000
Exif.GPSInfo.GPSDateStamp                    Ascii      11  2008:09:18
Exif.Image.UniqueCameraModel                 Ascii      25  Leica RCD digital camera
Exif.Image.CameraSerialNumber                Ascii       3  21

added the new data, improved accuracy of GPS time stamp

[mggr]

("forwards" typo was fixed)

[mggr]

Failed to validate XML (used ​http://www.w3schools.com/XML/xml_validator.asp) - Year tag was missing a < (ie. it says "Year>2009</Year>"). Fixed chrfi's live version but will need pushing to other versions and subversion.

[chrfi]

year typo fixed, exiftool version is now on svn

[chrfi]

this script has been actively used for some time now without problems so i am closing this ticket, further bug or feature requests will each have their own ticket