2004.04.26 14:50 "Re: [Tiff] Large TIFF files", by Marco Schmidt

[...]

The IFD already allows for you to do things like that (solution a). Yes, the current approach leaves a hole and adds to the end. But you rarely end up with too large a hole.

Besides, an additional utility could analyze a TIFF file and report how much space is wasted. Another one could write a new file that copies from the first, of course without the holes. Maybe even "linearizing" the TIFF - if somebody finds that useful.

[memory managment]

OK - that's much too complex.

Now you're trying to implement a file system within a file. That's been tried before - and it was a nightmare.

But isn't adding to the end and leaving a hole already the beginning of a "file system within a file"? In-place editing of files with complex binary headers always has the potential to become a nightmare. Know-how from managing database files could be reused. With those files, the authors of the DBMS code have to deal with holes, deletion markers, overflow areas, periodic reorganization to reclaim space etc. all the time. However, such a file memory management project is probably the exact opposite of "a small extension of TIFF".

A lot of work could be saved if writing back image data into a TIFF file would remain reserved to compression type 1 (uncompressed). The amount of data for a strip / tile would never expand compared to the previous size of the same strip / tile. Thus, "only" metadata from IFDs would have to be moved around.

Another approach (for a completely new file format) could save metadata in one file and each strip / tile in a file of its own. This hands over all the work of managing changing strip sizes to the file system, where it belongs. Obviously, having a number of files for one image instead of a single file makes it less convenient to handle those images. But it's not that hard either, and the necessity to copy these huge images between places may be smaller compared to normal-sized image files.

By the way, I agree with the notion that any file format that breaks TIFF compatibility should get a file extension of its own. Also a name that doesn't have TIFF in it, to avoid confusion. Java and JavaScript still suffer from people thinking that those two languages are somewhat related (understandable given the name similarity). With end users of imaging software (instead of developers) it's likely to be even worse.

[memory managed file format]

The most complex I'd go is something like the ZIP archive format. And that's only really useful if you're dealing with several independent chunks of data.

I don't think archiver programs really do in-place editing of ZIPs. They read from the original and write to a new file.

Regards,
Marco