2018.01.16 07:31 "Re: [Tiff] Strategies for multi-core speedups", by Paavo Helde
On 16.01.2018 2:58, scott ribe wrote:
On Jan 15, 2018, at 5:24 PM, Bob Friesenhahn <email@example.com> wrote:
The I/O is not necessarily serialized. When using mmap, pread/pwrite, or asynchronous I/O, the I/O can also benefit from multi-threading given that the underlying storage is a multi-disk array or SSDs using PCIe NVMe (capable of up to 1024 simultaneous I/Os).
Also, I/O calls are likely only to RAM, not to disk, thanks to caching...
With TIFFClientOpen() it is pretty easy to set up one's own I/O so that everything happens in RAM definitely, not only likely. All our libtiff reading happens via memory, either mmap-ped or obtained otherwise.
That does not mean RAM is always super fast. If the files are large enough and there are many frames then traversing them takes a lot of time because they do not fit in L3 caches. The main culprit in libtiff is TIFFSetDirectory() which always traverses the whole file. This becomes pretty slow when you have something like 500 frames. In our app we are trying to cache the known IFD offsets externally and (ab)using TIFFSetSubDirectory() for frame hopping instead.