2005.09.27 02:06 "Re: [Tiff] Additional Lossless Compression Schemes", by Joris Van Damme

Ron,

Some pretty useful stuff you're saying. Here's my two cents adding to your two dollars.

Personally, if libtiff has good generic support for plugging in alternate compression schemes on demand, I don't really mind which are actually implemented at this stage. So long as its easy to add the ones that people really want (ie. enough to submit a patch for cvs now), or the one I really need to get at some wacko non-standard format that one of my clients apps has used... which we may or may not want to ever publish.

I think there is 'good generic support for pluggin in alternate compression schemes'. The only thing that's missing, is proper documentation. Currently, you need to study the existing code to figure out what goes where and why. But aside from that, the actual plugin functionality is there in the sense that it's enough to write a single code file and a single line outside that file registring your new compression codec in the library.

You seem to be concerned with something like 'locally valid compression schemes'. There is no such thing in TIFF. There are 'locally valid tag codes'. You can use any tag code from 65000 and upward to mean something only in your local setup, and you don't have to register. The downside is that these tags cannot be attributed any meaning outside of your local system. But there is no such thing for compression schemes, you do need to get yourself a registered code for that even if you don't plan interchange of files with the world 'out there', so it'll never be as easy as merely getting the plug-in into CVS.

If you do plan interchange, then simply registring and writing the plugin is not going to be enough. You'll have to get the word out, too, and/or wait long enough for most software to upgrade to newer versions of LibTiff that include the plugin.

Last, but certainly not least, many vendors seem unaware of the basic properties of TIFF and/or simply do not care about anything but their own reader part reading whatever crap it is their own writing part is writing. We've seen such things as offsets in data blocks to other data blocks, for example in the Makernote data, the datatype LONG being used for fundamentally mixed datatype stuff, respectable specs build by respectable people ignoring the fact that data blocks are and have to be relocatable for TIFF to work, and that tag scope is limited to a single IFD, etc. Such stuff fundamentally breaks TIFF. Hey, perhaps some people don't like to be reminded, but we've seen the TIFF 6.0 spec itself fundamentally break TIFF, with the very bad specification of what is now refered to as 'old-style jpeg-in-tiff' but still makes life hard and still is generated by some of today's software. Too much freedom and ease of pluging in new stuff, and we may very well soon end up with TIFF getting more broken then not, and basic stuff like concattenating pages and transcoding TIFF files and all not working anymore as a consequence...

I think we probably will want something "better" as the size and number of images that people manipulate grows, so this is good to think about. Otherwise a shop somewhere will surely implement their own brand of better, perhaps without that vital step...

I personally don't believe any 5% better but essentially still flate kinda compression scheme is worth the trouble. You've noted quite correctly, that every now and then you read good stuff about any of the twelve dozen such flate on steroids schemes, but that none actually prove worthy and break through.

It's a whole other story for fundamentally new compression modes that we firmly believe are here to stay. I believe we should try and agree on integration for JPEG2000, and old and new JBIG.

Joris Van Damme
info@awaresystems.be
http://www.awaresystems.be/
Download your free TIFF tag viewer for windows here:
http://www.awaresystems.be/imaging/tiff/astifftagviewer.html