2000.01.28 15:33 "Libtiff inplace update and overviews", by Frank Warmerdam

Folks,

I have checked in some changes to libtiff in support of "inplace update". In particular, the ability to read, and update a TIFF file in place (rather than doing as part of a file copy operation). There are many limitations but I have now successfully done the following after opening a file with an "r+" access mode.

• Write new data to existing strips/tiles (uncompressed only!)
• Create a new directory, and close without writing data. The stripoffsets are all written as zero.
• Open a file with strip offset values of zero, and actually write the missing strips, with the strip offsets being appropriately updated.

The culmination (and immediate objective of all this) was to support adding overviews to an existing file. Previously I would read the file, writing out the overviews to temporary files as I go. Then reopen the file for append, and add the overviews to the original file.

Now I open the file with "r+" mode, create a set of new overview directories (zeroed strip offset arrays) and then read through the file, occationally writing out a strip of data to each of the overviews as it is completed. This involves alot of switching directories during access, and switching between reading and writing on the different directories.

While it is gratifying that I have this working there are a number of issues. The library was not written with this sort of random update in place in mind. There were and are a number of associated problems with different internal buffer semantics between modes. I have tried to employ a light touch in fiddling with the code, but there is some danger that my changes have broken the library for some types of usage.

Furthermore, you can't rewrite strips/tiles that are compressed. The current logic doesn't attempt to move strips/tiles to the end of the file if the new set of compressed data is larger than the reserved space. Therefore, it's use is only appropriate for uncompressed data.

I have also not yet tried reading, and writing within the same directory without rereading the directory (which cleans up some mode dependent state). I don't know whether it would work, though it is clearly desirable.

Is there anyone interested in "inplace" update for TIFF files? I would be happy to have some other folks try out the code, and report problems, preferably before the next official cut of libtiff.

---------------------------------------+--------------------------------------
I set the clouds in motion - turned up | Frank Warmerdam, warmerda@home.com
light and sound - activate the windows | http://members.home.com/warmerda
and watch the world go round - Rush    | Geospatial Programmer for Rent