-
2004.01.14 15:07 "Re: [Tiff] COLORMAP and byte padding", by Frank Warmerdam
-
2004.01.14 16:18 "Re: [Tiff] COLORMAP and byte padding", by Gerben Vos
- 2004.01.14 16:43 "Re: [Tiff] COLORMAP and byte padding", by Joris Van Damme
- 2004.01.14 18:18 "Re: [Tiff] COLORMAP and byte padding", by Marti Maria
- 2004.01.15 00:05 "Re: [Tiff] COLORMAP and byte padding", by Chris Cox
-
2004.01.14 16:18 "Re: [Tiff] COLORMAP and byte padding", by Gerben Vos
2004.01.14 19:36 "Re: [Tiff] COLORMAP and byte padding", by Marti Maria
Hi,
Here is a small program to check the error by using both methods, shifting 8, (same as *, / by 256) and the "good" method, which is by 257. But this should be not so surprising. If you are scaling a value with range 0.. 255 to a range 0..65535, the factor would be 65535 / 255, and that equals to 257 :-)
--- Cut ---
#include <stdio.h>
#include <stdlib.h>
int main()
{
int i;
unsigned char a8_by256, a8_by257;
unsigned short a16_by256, a16_by257;
int error256, error257;
int max256 = 0, max257 = 0;
double sum256=0., sum257=0.;
for (i=0; i < 0xFFFF; i++) {
a8_by256 = (unsigned char) (i >> 8);
a8_by257 = (unsigned char) (double) floor(i / 257. + .5);
a16_by256 = ((unsigned short) a8_by256 << 8);
a16_by257 = ((unsigned short) a8_by257 << 8) | a8_by257;
error256 = abs(a16_by256 - i);
error257 = abs(a16_by257 - i);
if (error256 > max256) max256 = error256;
if (error257 > max257) max257 = error257;
sum256 += error256;
sum257 += error257;
}
printf("Max error: by 256 = %d, by 257 = %d\n", max256, max257);
printf("Mean error: by 256 = %g, by 257 = %g\n", sum256 / 65535., sum257 / 65535.);
return 0;
}
--- Cut --