2019.03.30 13:47 "Re: [Tiff] combining TIFF images, not multi-page", by Bob Friesenhahn

Thanks Bob.

As to the "low RAM footprint way" - in fact I am hoping to use the hard-drive instead of RAM. This is why I have a folder full of small partial images in the first place.

Even with a pyramidal representation, if I'm talking about a final image that may be several hundred gigabytes (or larger) in size, I could never assemble it all in memory in order to write the bottom layer of that pyramid. Can each resolution level be written in "tile by tile" or even line by line? Does this require a different API (like the ImageMagick API?) I assume as well that BigTIFF can be a tiled pyramid?

The libtiff API allows a suitably skilled software developer to create an efficient program which works as you describe. Huge files (greater than 4GB) can be supported using BigTIFF. Only software supporting BigTIFF can then read such files.

You may find that GraphicsMagick or ImageMagick will succeed given use of temporary disk files and quite a lot of time. This is generic software which is not optimized for your requirement.

The access time for traditional rotating media is pretty long (e.g. 10ms) so it might take 10k times longer to perform an operation with image data on hard disks vs RAM. Using SSDs rather than a hard drive will help quite a lot.

If you want to produce reduced forms of your images, then the trick is to select a sampling algorithm which accesses image pixels in an efficient way while producing the quality needed. Given tiles, it may work ok (perhaps with some tile boundary artifacts) to start with a large tile size and resize each tile to a smaller size. If multiple tiles are selected then resizing could avoid tile boundary artifacts.

Lastly, I should mention that modern server-class hardware can be fitted with a terrabyte of RAM. Depending on your objective, it may cost less to buy the hardware than it would cost to pay a skilled software developer to write the specialized software you need on a lesser system.

Bob
--
Bob Friesenhahn
bfriesen@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer, http://www.GraphicsMagick.org/

Public Key, http://www.simplesystems.org/users/bfriesen/public-key.txt