TIFF and LibTiff Mail List Archive 2003.12.26 16:00 "[Tiff] [ANNOUNCE]: Libtiff 3.6.1 released", by Andrey Kiselev
[...] 2004.01.14 19:48 "Re: [Tiff] COLORMAP and byte padding", by Andy Cave 2004.01.15 17:24 "Re: [Tiff] COLORMAP and byte padding", by 2004.01.15 17:37 "Re: [Tiff] COLORMAP and byte padding", by Andy Cave
[...]

# 2004.01.15 17:24 "Re: [Tiff] COLORMAP and byte padding", by

Hi,

Your code will produce false results. Although I'd divide by 256, I would not multiple by 256 (which is where your errors come in). You'd still need to multiple by 257 so as to re-map the range back correctly, so a16_by256 should use the same calculation as a16_by257.

Ok, I have replaced the multiplication by

a16_by256 = ((unsigned short) (a8_by256 << 8) | a8_by256 );

And the results are still:

Max error: by 256 = 255, by 257 = 128
Mean error: by 256 = 85.3333, by 257 = 64.249

I was proposing the macro:

#define RGB_16_TO_8(rgb) (BYTE) ((((rgb)*65281+8388608)>>24)&0xFF)

Because this *does* produce same results as floor(rgb / 257.0 + 0.5)

Max error: by macro = 128, by double = 128
Mean error: by macro = 64.249, by double = 64.249

Check it by yourself :-)

Regards,
Marti.

#include <stdio.h>
#include <stdlib.h>
#include <math.h>

#define RGB_16_TO_8(rgb) (unsigned char) ((((rgb)*65281+8388608)>>24)&0xFF) #define RGB_8_TO_16(rgb) (unsigned short)((((unsigned short)(rgb))<<8)|rgb))

int main()
{
int i;
unsigned char byMacro_8, byDouble_8;
unsigned short byMacro_16, byDouble_16;
int errorMacro, errorDouble;
int maxMacro = 0, maxDouble = 0;
double sumMacro=0., sumDouble=0.;

for (i=0; i < 0xFFFF; i++) {

byMacro_8 = (unsigned char) RGB_16_TO_8(i);
byDouble_8 = (unsigned char) (double) floor(i/257.+.5);

byMacro_16 = (unsigned short) RGB_8_TO_16(byMacro_8);
byDouble_16 = (unsigned short) (double) floor(byDouble_8*257.0+0.5);

errorMacro = abs(byMacro_16 - i);
errorDouble = abs(byDouble_16 - i);

if (errorMacro > maxMacro) maxMacro = errorMacro;
if (errorDouble > maxDouble) maxDouble = errorDouble;

sumMacro += errorMacro;
sumDouble += errorDouble;
}

printf("Max error: by macro = %d, by double = %d\n", maxMacro,
maxDouble);
printf("Mean error: by macro = %g, by double = %g\n", sumMacro / 65535.,
sumDouble / 65535.);

return 0;
}