2005.10.28 03:18 "Re: [Tiff] How located pixels with BitsPerSample not bound to byte?", by David McKenzie

Hello Frank and Eugene:

I am including my code for arbitrary unpacking of odd bit data into words though it may be a bit opaque.I believe Bob uses a more efficient solution.

>... <code excerpt> (see below)

It so happens that while testing a viewer application I'm working on, two problems were tracked to this section of GDAL code (in geotiff.cpp). The first is apparently a simple coding error. Notice that the local variable iPixelBitSkip in the code excerpt is initialized but never used. I saw this first in a separate section that handles 12-bit samples as a special case. There I fixed the problem by simply replacing the statement "iBitOffset += 12" with "iBitOffset += iPixelBitSkip". In the section here that handles other 1-32 bit samples a similar fix is necessary. I haven't tried the very latest version of OpenEV, but I believe for the above reason it can't correctly display many of Bob Friesenhahn's example TIFFs (see below) with "contig" in the name.

The second problem involves this same section of code (1-32 bit sample unpacking) but it's really a compatibility issue instead of a code bug. It may take more effort to explain than it's worth given the almost nonexistent support for 24-bps TIFF files in applications. It does, however, touch upon Eugene's question. Anyway, there's a conflict between GDAL's and libtiff's interpretation of files with a 24-bit sample size. Both GDAL and GraphicsMagick apparently make the assumption that the byte order of 24-bps data isn't processor dependent. I suspect this is because Adobe's baseline TIFF specification, TIFF6.PDF (page 30), asserts that CPU-dependent byte order comes into play for sample sizes of 16 and 32 bits, and that uncompressed samples of other sizes are stored packed into an array of BYTEs, not SHORTs or LONGs. The document then says: "If the image data is stored as an array of SHORTs or LONGs, the byte ordering must be consistent with that specified in bytes 0 and 1 of the TIFF file header."

I realize this can't be taken as the final word on the TIFF format, but at least GDAL's driver code is compatible in that it expects libtiff's TIFFReadEncodedStrip() to return 24-bit samples in the same format it does other sizes that are less than 32 and not 16. When the driver unpacks a sequence of such samples it assumes that when a sample crosses a byte boundary, the most significant bits of the sample appear first, in the lowest addressed byte. Also conforming to this interpretation are the 24-bps versions of images Bob Friesenhahn created using GraphicsMagick v1.2:

ftp://ftp.remotesensing.org/pub/libtiff/pics-3.7.2.tar.gz ftp://ftp.graphicsmagick.org/pub/outgoing/sample-tiffs.zip

ftp://ftp.graphicsmagick.org/pub/outgoing/sample-tiffs-le.zip

So given this evidence the problem seems to be with libtiff, which treats 24-bit samples like it does 16- and 32-bit samples and byte-swaps them, say, if the file prefix is "MM" and the code is running on Windows. As a result, GDAL is unpacking 24-bps data in LS-to-MS byte order when it's expecting the opposite. OpenEV displays the "MM" version of flower-minisblack-strip-24.tif, for example, as a scrambled pattern.

If this is judged to be a libtiff error, I believe all that's necessary is to delete a couple of lines (an "else" clause) that references _TIFFSwab24BitData() in file tiff_dir.c. This is what I've done for the time being. The alternative is to assume the example 24-bps TIFFs are bad and treat 24-bps as a special case in GDAL's driver.

Whatever you folks decide, I'll certainly adapt to it. Thanks for all the great work!

--David

Frank Warmerdam wrote:

On 10/27/05, Шебеко Евгений <shebeko@mail.ru> wrote:

If we talking about testing exotic images.

Look at the flower-rgb-contig-12.tif from the test images. It have RGB,Uncompressed, PlanarConfig=Continue, and BitsPerSample=12

How the pixels are located inside a strip in such case? I really cann't understand :)

Is it correct?

  Bytes:       0   |    1    |    2   |   3    |   4
  Bits :   76543210|7654|3210|76543210|76543210|7654|3210
  Pixels:  rrrrrrrr|rrrr|gggg|gggggggg|bbbbbbbb|bbbb|rrrr|
  Samples:   hiR   |lowR|hiG |  lowG  |  hiB   |lowB| hiR

Eugene,

I thought it should work as you have shown, but when I review my code it seems the first byte will be the low order 8bits of red, not the high (and so forth). I am including my code for arbitrary unpacking of odd bit data into words though it may be a bit opaque. I believe Bob uses a more efficient solution.

/*  --------------------------------------------------------------------  */
/*      Handle 1-32 bit integer data.                                    */
/*  --------------------------------------------------------------------  */

    else
    {

        int     iBit, iPixel, iBitOffset = 0;
        int     iPixelBitSkip,  iBandBitOffset, iX, iY, nBitsPerLine;

        if( poGDS->nPlanarConfig == PLANARCONFIG_CONTIG )
        {
            iPixelBitSkip = poGDS->nBands * poGDS->nBitsPerSample;
            iBandBitOffset = (nBand-1) * poGDS->nBitsPerSample;
        }
        else
        {
            iPixelBitSkip = poGDS->nBitsPerSample;
            iBandBitOffset = 0;
        }

        // bits per line rounds up to next byte boundary.
        nBitsPerLine = nBlockXSize * poGDS->nBitsPerSample;
        if( (nBitsPerLine & 7) != 0 )
            nBitsPerLine = (nBitsPerLine + 7) & (~7);

        iPixel = 0;
        for( iY = 0; iY < nBlockYSize; iY++ )
        {

            iBitOffset = iBandBitOffset + iY * nBitsPerLine;

            for( iX = 0; iX < nBlockXSize; iX++ )
            {
                int nOutWord = 0;

                for( iBit = 0; iBit < poGDS->nBitsPerSample; iBit++ )
                {
                    if( poGDS->pabyBlockBuf[iBitOffset>>3]
                        & (0x80 >>(iBitOffset & 7)) )
                        nOutWord |= (1 << (poGDS->nBitsPerSample - 1 - iBit));
                    iBitOffset++;
                }

                if( eDataType == GDT_Byte )
                    ((GByte *) pImage)[iPixel++] = nOutWord;
                else if( eDataType == GDT_UInt16 )
                    ((GUInt16 *) pImage)[iPixel++] = nOutWord;
                else if( eDataType == GDT_UInt32 )
                    ((GUInt32 *) pImage)[iPixel++] = nOutWord;

            }
        }
    }