2010.06.12 17:25 "Re: [Tiff] Class Y (YCbCr) tiffs with 420 subsampling", by Thomas Richter

Sure enough, however, I'm not quite clear on the semantics of TIFFScanlineSize(). Actually, if you do have subsampling, what is the the "size" of "a" scan line given that every other scanline holds only a subset of all samples? This is not at all clear.

Given this problem, I'm not sure whether the bug is in TIFFScanlineSize() or in the directory reader.

It may be that using scanline access is quite wrong for your subsampled files. It is not possible to support vertical sampling in a scanline. The data is stored using strips (which support multiple scanlines and can support vertical sampling) so it seems that TIFFStripSize() should be used rather than TIFFScanlineSize() and the strip should be read as a whole.

Thanks, Bob, I certainly don't disagree that line-by-line decoding isn't the right way to go, but that wasn't quite the question. The trouble is that the directory reader *does* use TIFFScanlineSize() to compute the number of bytes it expects, thus what should it do?

There are a couple of options:

*) patch TIFFScanlineSize() such that it is consistent with the subsampling, and the directory scanner works as intended. Drawback is that this way programs could still try using a by-line decode, which is unsuitable.

*) patch TIFFScanlineSize() such that it returns the number of bytes in a pair (or n-tupel) of scanlines, where n is given by the subsampling in Y direction, and patch the directory scanner such that it takes care of dividing the height by the subsampling. Still, compatibility problems, but it would at least be a consistent interface, though old programs likely fall over,

*) leave the TIFFScanlineSize() alone, remove it from the directory scanner, and compute there the size of bytes manually with a proper formula, hoping that TIFFScanlineSize() returns at least a buffer size that is "too long". Maybe that is good enough for some programs, maybe it isn't.

*) as above, but install a patch into TIFFScanlineSize() such that it generates a fatal error if someone is trying to use it even though the image contains subsampling. This is safe, but probably too picky.

I'm not competent enough in libTIFF usage to decide what is the proper way of addressing this bug. I can only tell you that there is one.

So long,

Thomas