2005.01.08 16:36 "Re: [Tiff] JPEG (T.30-E, fax) in TIFF", by Lee Howard

I'd say you need to make sure PhotometricInterpretation is set to 10 (PHOTOMETRIC_ITULAB), and Compression is set to 7 (COMPRESSION_JPEG).

I had been testing with COMPRESSION_JPEG and PHOTOMETRIC_RGB, with no luck, as I mentioned. Ed Grissom suggested that I try PHOTOMETRIC_YCBCR with YCBCRSUBSAMPLING left at the 2:2 default, and immediately after I tried that I was successful in being able to pass the data through tiffcp and tiff2ps with the exception of two warning messages:

fax000003467.tif: Warning, incorrect count for field "StripOffsets" (3, expecting 1); tag trimmed.
fax000003467.tif: Warning, incorrect count for field "StripByteCounts" (3, expecting 1); tag trimmed.

(Well, to be perfectly honest, I had been using libtiff-3.6.1 up until last night when I upgraded to 3.7.1. The message above is from 3.7.1. In 3.6.1 it said "tag ignored" instead of "tag trimmed".) However, viewing the JPEG-in-TIFF image there was a very noticeable blue tint, much like viewing the JPEG I linked before directly in Mozilla. I'm not completely certain if the blue tint indicates that the fax machine that I'm using to send the image is broken, if the faxed JPEG uses a different color pallate than is being expected, or if the fax machine is buggy and isn't generating the proper JPEG tables. In any case, the images as-is are not preferrable to usual black-and-white faxes due to the blue shift; so it can't be used that way.

You also need to set ImageWidth and ImageLength tags, of course. You should have one single value of StripOffsets and StripByteCounts,

Which is the cause for the warning messages I noted above: the TIFFs I'm generating have three values in each of those tags. But I don't understand why they do. It seems to be affected by the SAMPLESPERPIXEL tag. If I change the SAMPLESPERPIXEL tag to 1 I only get one single value of StripOffsets and StripByteCounts. If I just change the SAMPLESPERPIXEL tag to 3 then I get three values in each of StripOffsets and StripByteCounts. I don't know if I'm doing something wrong or if there is a libtiff bug in this.

and rows per strip needs to equal ImageLength.

I can't use unlimited (-1)? I'm accustomed to using unlimited rows per strip since I use but one strip per page with these kinds (ECM) of faxes.

BitsPerSample needs to be 8 (8,8,8, to be more exact, but LibTiff should take care of that for you), SamplesPerPixel needs to be 3.

I had these before, although as I mentioned SamplesPerPixel was a bit troublesome, as I mentioned.

IIRC, LibTiff also requires you to set the PlanarConfiguration tag to 1 (PLANARCONFIG_CONTIG), even though the spec says that's the default value and thus the tag should not be strictly requires.

It looks like libtiff sets this tag to 1 on its own.

Now, for the less obvious. You might need to set the Decode tag (http://www.awaresystems.be/imaging/tiff/tifftags/decode.html), depending on whether the ITULAB data in the JPEG has default range or not.

Aha! Well, this probably explains why my image appears blue. How do I know what the correct values are for this tag? I've tried the said defaults (0, 100, -21760/255, 21590/255, -19200/255, 31800/255), but there's really no way for me to view the image right now due to the issues I mention below. Furthermore, the tag doesn't seem to be getting set, despite doing:

   TIFFSetField(tif, 433, 0, 100, -21760/255, 21590/255, -19200/255, 31800/255);

You'll also need to set the YCbCrSubSampling tag (http://www.awaresystems.be/imaging/tiff/tifftags/ycbcrsubsampling.html), since your subsampling values are not the default. Note that the tag's name incorrectly seems to imply that it is for YCbCr data only, but it should also apply in your case.

Unfortunately, the YCBCRSUBSAMPLING tag appears to be ignored by tiffcp when the photometric tag is not set for YCbCr. Observe:

[root@gollum recvq]# tiffdump fax000003468.tif

fax000003468.tif:
Magic: 0x4949 <little-endian> Version: 0x2a
Directory 0: offset 236712 (0x39ca8) next 0 (0)
SubFileType (254) LONG (4) 1 <2>
ImageWidth (256) SHORT (3) 1 <1728>
ImageLength (257) SHORT (3) 1 <2128>
BitsPerSample (258) SHORT (3) 3 <8 8 8>
Compression (259) SHORT (3) 1 <7>
Photometric (262) SHORT (3) 1 <10>
FillOrder (266) SHORT (3) 1 <2>
ImageDescription (270) ASCII (2) 11 <3604278160\0>
Make (271) ASCII (2) 62 <LT V.92 1.0 MT5634ZBA-V9 ...>
Model (272) ASCII (2) 62 <LT V.92 1.0 MT5634ZBA-V9 ...>
StripOffsets (273) LONG (4) 3 <118360 0 0>
Orientation (274) SHORT (3) 1 <1>
SamplesPerPixel (277) SHORT (3) 1 <3>
RowsPerStrip (278) LONG (4) 1 <4294967295>
StripByteCounts (279) LONG (4) 3 <118352 0 0>
XResolution (282) RATIONAL (5) 1 <204>
YResolution (283) RATIONAL (5) 1 <196>
PlanarConfig (284) SHORT (3) 1 <1>
ResolutionUnit (296) SHORT (3) 1 <2>
Software (305) ASCII (2) 27 <HylaFAX (tm) Version 4.2 ...>
DateTime (306) ASCII (2) 20 <2005:01:08 07:26:05\0>
HostComputer (316) ASCII (2) 16 <gollum.x101.com\0>
YCbCrSubsampling (530) SHORT (3) 2 <1 1>

[root@gollum recvq]# tiffcp fax000003468.tif foo.tif
fax000003468.tif: Warning, incorrect count for field "StripOffsets" (3, expecting 1); tag trimmed.
fax000003468.tif: Warning, incorrect count for field "StripByteCounts" (3, expecting 1); tag trimmed.
JPEGPreDecode: Warning, Improper JPEG sampling factors 2,2
Apparently should be 1,1..
JPEGPreDecode: Warning, Decompressor will try reading with sampling 2,2..
JPEGLib: Warning, Application transferred too many scanlines.

Perhaps we might be able to spot a problem if you post a code snippet. Mentioning the exact errors and warnings might also ring a bell.

The errors and warnings are mentioned above. Here is a synopsis of the code (most of which I didn't write - most of it probably originated from Sam):

     TIFFSetField(tif, TIFFTAG_SUBFILETYPE,     FILETYPE_PAGE);
     TIFFSetField(tif, TIFFTAG_IMAGEWIDTH,      (uint32) params.pageWidth());
     TIFFSetField(tif, TIFFTAG_BITSPERSAMPLE, 8);
     TIFFSetField(tif, TIFFTAG_PHOTOMETRIC,     PHOTOMETRIC_ITULAB);
     TIFFSetField(tif, TIFFTAG_YCBCRSUBSAMPLING, 1, 1);
     TIFFSetField(tif, 433, 0, 100, -21760/255, 21590/255, -19200/255, 31800/255); // Decode tag
     TIFFSetField(tif, TIFFTAG_ORIENTATION,     ORIENTATION_TOPLEFT);
     TIFFSetField(tif, TIFFTAG_SAMPLESPERPIXEL, 3);
     TIFFSetField(tif, TIFFTAG_ROWSPERSTRIP,    (uint32) -1);
     TIFFSetField(tif, TIFFTAG_PLANARCONFIG,    PLANARCONFIG_CONTIG);
     TIFFSetField(tif, TIFFTAG_FILLORDER,       (uint16) fillOrder);
     TIFFSetField(tif, TIFFTAG_XRESOLUTION,     (float) params.horizontalRes());
     TIFFSetField(tif, TIFFTAG_YRESOLUTION,     (float) params.verticalRes());
     TIFFSetField(tif, TIFFTAG_RESOLUTIONUNIT,  RESUNIT_INCH);
     TIFFSetField(tif, TIFFTAG_SOFTWARE, HYLAFAX_VERSION);
     TIFFSetField(tif, TIFFTAG_IMAGEDESCRIPTION,        (const char*) id);
     char dateTime[24];
     time_t now = Sys::now();
     strftime(dateTime, sizeof (dateTime), "%Y:%m:%d %H:%M:%S", localtime(&now));
     TIFFSetField(tif, TIFFTAG_DATETIME,            dateTime);
     TIFFSetField(tif, TIFFTAG_MAKE,        (const char*) getManufacturer());
     TIFFSetField(tif, TIFFTAG_MODEL,       (const char*) getModel());
     TIFFSetField(tif, TIFFTAG_HOSTCOMPUTER, (const char*) server.hostname);
     TIFFSetField(tif, TIFFTAG_COMPRESSION, COMPRESSION_JPEG);
/* this appears to be done in order to collect the StripOffsets value */
     u_char null[1];
     (void) TIFFWriteRawStrip(tif, 0, null, 0);
/* this is done repeatedly until all of the data is collected */
     TIFFWriteRawStrip(tif, strip, (tdata_t)buf, cc);
/* this is done to change the reliance on the DNL marker recvEOLCount
is not a count of EOLs, in this case it is a value that we set from
reading the DNL marker */
     uint32* stripbytecount;
     (void) TIFFGetField(tif, TIFFTAG_STRIPBYTECOUNTS, &stripbytecount);
     uint32 sbc = stripbytecount[0];
     u_char* stripdata = new u_char[sbc];
     TIFFReadRawStrip(tif, 0, stripdata, sbc);
     for (uint32 i = 0; i < sbc-6; i++) {
         if (stripdata[i] == 0xFF && stripdata[i+1] == 0xC0 && stripdata[i+5] == 0x00 && stripdata[i+6] == 0x00) {
             stripdata[i+5] = recvEOLCount >> 8;
             stripdata[i+6] = recvEOLCount & 0xFF;
             protoTrace("RECV: fixing zero image frame length in SOF marker at byte %lu to %lu", i, recvEOLCount);
         }
     }
     TIFFSetWriteOffset(tif, 0);
     TIFFWriteRawStrip(tif, 0, (tdata_t) stripdata, sbc);
/* these tags are owned by SGI specifically for HylaFAX */
     TIFFSetField(tif, TIFFTAG_FAXRECVPARAMS, (uint32) params.encode());
     TIFFSetField(tif, TIFFTAG_FAXRECVTIME, (uint32) server.getPageTransferTime());

Another problem that I'll mention is that TIFFTAG_FAXRECVPARAMS and TIFFTAG_FAXRECVTIME are not being set as expected. They get set when using traditional (black-and-white) compression methods, but not with JPEG compression. I don't know if this is a hard-coded limitation in libtiff or not, I haven't looked into it.

Note that few TIFF readers are ITULAB aware. I don't know if tiffcp can or cannot handle such a TIFF you're trying to build.

Well, at first looks it appears that there are some hurdles, indeed, but maybe I'm doing something wrong. However, if this fax machine is not incorrect or broken (blue-shift), then most JPEG viewers are not going to work with JPEG files as I've produced them before, either.

In fact, it may be a challenge to find a reader that you can use to crosstest your results, and some code failing to handle your results might not be sufficient to indicate that you haven't yet succeeded perfectly fine...

For the most part I'm only really interested in libtiff being able to support these files. I assume that if libtiff supports them that tiff2ps will produce a properly-colored PostScript document that can then be used in whatever way the user needs. In other words, most people who use HylaFAX that I work with will not use the TIFF directly - instead, they will post-process the fax image into PDF. As for other HylaFAX users that may choose to use the TIFFs directly - for example in a TIFF viewer - well, unless libtiff can post-process the TIFF from an unusable format to a usable one for them, then they'll just need to leave the to-be "color fax" option in HylaFAX disabled. I don't think that color faxing is done very frequently, and when it is done with any regularity I suspect that JBIG compression is used with greater frequency than JPEG. But, since libtiff doesn't support JBIG at the moment, I thought that I would have an easier time of getting JPEG support implemented, especially since I don't have a working JBIG-compressed fax sender always available to me.

Please do keep us posted on the outcome.

Will do. As I've indicated, I'm also planning on getting JBIG support going, via JBIG-KIT. I received your last message on that subject, and I've looked into doing it, but unfortunately I don't understand libtiff well enough and neither do I understand JBIG-KIT well enough to be able to accomplish that right now.

Are you a HylaFAX contributor?

Yes, I'm one of regulars. The main direction in HylaFAX development that I try to work involves anything with fax protocol and modem support, although I have done some other things. Other people tend to be more affluent at working with the HylaFAX job scheduler and such.

The HylaFAX links page still links to libtiff.org. Can you change that, or point me to someone who can? The correct LibTiff URL is, instead, http://www.remotesensing.org/libtiff/.

Oh, and yes, I am one of the webmasters. I've updated the link on the HylaFAX HOWTO, which I believe is the only relevant one. It may take a while to propagate to the live-side, though, so you may not see a change immediately.

What happened to libtiff.org, may I ask?

Thanks.

Lee.