2021.01.01 19:48 "Re: [Tiff] Multithreading support?", by Roger Leigh

I think that the minimal step forward would be to expose a stateless/reentrant TIFF{Encode,Decode}{Strip,Tile}, which would be a simple way to access libtiff's implementations of the compression codecs (e.g. TIFFReadEncodedStrip would be equivalent to TIFFReadRawStrip followed immediately by TIFFDecodeStrip) -- that would allow apps to do all the thread policy stuff by using libtiff to serially perform the I/O and then

While that's a good idea in theory, I don't think it could be entirely stateless. Some compression codecs have a significant setup time and we need to reuse an initialized state several times to get good performance. The general idea is still valid but the implementation will be more difficult. I should probably give it a try at some point as I've also had to write my own multithread support which also only handles a handful of cases.

Hi Olivier,

Another point to consider is error handling. Due to libtiff using an error handler routine rather than a returned error code, this complicates error handling.

I can imagine you can use e.g. thread-local storage to have a single global handler which can handle errors in multiple threads. But it’s already messy to tie synchronous callbacks to the specific call triggering the failure in a multithreaded system.

For example this sentry <https://gitlab.com/codelibre/ome/ome-files-cpp/-/blob/master/lib/ome/files/tiff/Sentry.h#L148> which requires going through contortions (in the handler <https://gitlab.com/codelibre/ome/ome-files-cpp/-/blob/master/lib/ome/files/tiff/Sentry.cpp#L104> ) and wrapping every use of the C API (like this <https://gitlab.com/codelibre/ome/ome-files-cpp/-/blob/master/lib/ome/files/tiff/TIFF.cpp#L175> ). Note the contortions it goes through to re-register the global handler and use a mutex to avoid races with other threads, at the expense of contention. It would be much nicer if the error handling was simple, straightforward, and thread-safe by default. And not relying upon global state which could be ripped out from underneath you by other libtiff users in different threads. A per-thread handler would be perhaps something to consider; though getting rid of it entirely would be even nicer (though would likely be too breaking since the existing error handling is entrenched).

Maybe there’s already a better way of doing things; the above is something I cooked up 6 years back when doing some libtiff C++ wrappers, and I haven’t had a chance to revisit it to refine it further. Useful parts of this library code could be put into libtiffxx if there was a demand for something more usable for C++.

Kind regards,

Roger