2020.04.19 17:01 "[Tiff] tiffsplit from FIFO", by Levi Wilson

2020.04.19 17:44 "Re: [Tiff] tiffsplit from FIFO", by Olivier Paquet

My next question would be this; is it possible with the TIFF format to say "give me this page" by requesting content ranges of bytes and to ask for one at a time?

Yes, but to know where the pages are, you have to read the directory, starting at the end of the file.

The directory could be anywhere actually. And there could be several. As I recall, the location of the first one is indicated in the file header. Then each one can point to another one. Or several with subIFDs.

To achieve what you want, I think you will need to use TIFFClientOpen and provide your own IO routines which will fetch from S3. That would involve recompiling the utilities you want to use. You may also need to enlarge some of the requests and cache the results as libtiff can make a lot of smaller reads. I think each tag in a directory involves a read and you probably don't want to make dozens of S3 requests for a few dozen bytes each. It would work as a prototype but will slow things down and cost you (S3 bills each request). Once you get to actually reading tiles or strips, the request sizes should be more reasonable, assuming well structured TIFF files.

Bottom line: do this only if you really can't fix the problem upstream and avoid getting random, too large TIFF files.

Olivier