2007.01.16 09:25 "Re: [Tiff] bigtiff", by Joris Van Damme

Albert,

The waste is less than a millionth of the file size, or 0.0001 %. At "ten times", it's still only 1/100000 of the file size, or 0.001 %.

Ah, is it?

Say an image is 32x32 pixels. You have those, in TIFF. Say there are ten

Yes, TIFF. There is no reason to break TIFF6 compatibility for such an image. Putting little icon files in a less-compatible format that already wastes space on extra-big IFD offsets is not sensible. Doing this for no gain, and breaking all the old TIFF readers, should be very strongly discouraged.

Once you get into the gigabytes, a few kilobytes here and there just isn't an issue.

Whatch it! You very nearly admitted that BigTIFF, as is, scales very well, relieving applications and users alike from the burden to pick the suitable flavour depending on needs they need to be able to predict before writing, even if they're the lower layer and unaware of what the upper layers are going to call for next.

Truth is there's more to it then simple good scaling. BigTIFF files, even if each of the main images is gigabytes, will even more likely then TIFF contain thumbnails in SubIFDs. A common thumbnail size might be 200*100, compressed as JPEG. In your scheme, that'll cost ten times more then required. No programmer is going to write that. We'd see a dozen more private tags with thumbnails in them, Photoshop tag style, instead. A dozen more, or ten dozen if you enable having private tags with registration procedure as is part of your proposal. For practical purposes, thumbnail interchange will be lost.

Same will happen with private IFDs. No programmer is going to waste 50K, to code his private IFD with 10 small values. So we'll see a dozen more proprietary additional key-value pair pile encodings in byte array private tags, instead. The fundamental functionality of TIFF as a hierarchy of directories of key-value pairs, will simply rightly be regarded broken, for all practical purposes.

As to the other of your mutually conflicting proposals, pulled between compatibility and sanity, the bitshifting of offsets... Are you aware the codec level in TIFF/BigTIFF doesn't even always know what is an offset, and what is not? The LONG/LONG8 datatypes are just as valid for pointing to private IFDs, as IFD/IFD8.

In your original post, I read 'The first IFD must go in the low 4 GB of course...'. Are you aware that applications that 'edit' IFDs, adding tags to it, can be forced to rewrite an IFD in a new location, appending it to the existing file, and updating the pointer to it? Are you aware that a page sorting application can choose to just relink IFDs in different orders, and certainly Unlink/Relink functions are a proper part of a TIFF codec?

These are just a few of the obstacles you might be able to deal with in your proposals, if you change more, and add more, to overcome some of the problems you've newly created. The point I'm trying to make though, is that TIFF is simple as is, but its application is huge and wide, and it's not easy to foresee all consequences of changing any one rule. It's much more easy to break things. That's why we choose to design BigTIFF as close as possible to TIFF, changing offset bitdepths only. We've something that works, there, cumulated decades of proper evolution without cumulating scruff, and we don't want to have to restart that process.

You may think it's ugly scruff, but frankly I think it's rather beautiful, simple and straightforward, and I think that's part of the reason why it works. It certainly doesn't waste 4 kilobyte on a 10 byte value, to name one thing. We preserve the properties of the existing file format, and the thing is guaranteed to work, as whole, in all applications out there even if any of us is able to foresee only half of them.

I regard it too late to change BigTIFF, but in any case, there's nothing that need to be changed. There's nothing, even, that can safely be changed without breakage. That's my point, and I do hope that puts an end to this. Did I mention there's a codec out there that implements this already? Did I point out we are starting BigTIFF works in LibTiff on March 1st?

Best regards,

Joris Van Damme
info@awaresystems.be
http://www.awaresystems.be/
Download your free TIFF tag viewer for windows here:
http://www.awaresystems.be/imaging/tiff/astifftagviewer.html