2018.01.16 19:57 "Re: [Tiff] Strategies for multi-core speedups", by

These laptops have SSDs but they're some of the worst I have ever seen,

less than 100Mb read or write, it comes close single threaded but there

was always a hitch where you could tell it was loading new tiles into

memory. I basically have tiles in view being drawn, then i have a

perimeter of tiles around those either leaving/entering memory based on

their proximity to the viewing area. I was actually designing a pyramid

template(I'm building this tool out for various other filetypes) without

knowing it was a thing, but instead I opted to just generate a few

downsampled examples of the image when zoomed out, and then move into

tiled mode once the user zoomed in to where you can see actual pixels

because we validate literally on a pixel by pixel to basis.

Joseph Maniaci
Staff Software Engineer
RPPS

Subject:        Re: [Tiff] Strategies for multi-core speedups [EXTERNAL]
Sent by:        tiff-bounces@lists.maptools.org

On 16.01.2018 20:36, Joe.Maniaci@ricoh-usa.com wrote:

I am a complete newbie to digital imagery and TIFFs in general, so please
forgive any ignorance.

I'm an automated test tool developer and I am actually going to start
working on a viewer for the TIFFs that my company generates, to
potentially include relying on the bigtiff format and this is the thing
I've spent most of my time thinking about. I've come to the conclusion
that for the images we generate, up to 10GB uncompressed(soon to be 40GB),
that I will need to preprocess a single TIFF into multiple TIFFs
containing intelligently placed tiles just to get around the I/O issue. By
intelligently placed, I mean that if a user were to scroll their view
window to the right, and five tiles(vertically aligned) had to be loaded
into memory, I could theoretically have those files intelligently located
in different TIFF files(subTIFFs?) that would allow multiple threads to
operate on. So each thread only had to find one tile. So for my tool it
would be a staggering amount of pre-processing, but real time manipulation
is extremely fluid for the user which is basically my number one
requirement.

So for a case like mine, thread overhead is nothing to worry about. That
and the more I read about the C++11 thread library, the less I worry. I
can just leave it to the library to determine how many threads should
operate in my 4-core environments, or my 16-core environments. At least
that's the impression I've garnered.

If all you want is a viewer then you are limited by the screen sizes which
are typically not more than ca 2000x2000 pixels. And this is pretty small
image nowadays, there is no need to multithread anything, you could just
read the needed tiles from the TIFF file in real time and maybe add a
small cache of the read tiles so you don't need to read them again if the
user pans left 1 pixel.

For zooming out the TIFF file should have different frames of the same
image stored in decreasing resolutions so that a suitable resolution can
be chosen for display - these are called pyramidal tiffs. The TIFF
standard suggests that such varying resolution images should be stored as
sub-IFD-s but as many viewers do not support sub-IFD-s they are typically
stored just in standard IFD-s, and some XML piece in the first IFD
provides the details. I am sure there are multiple variants of such
pyramidal tiff formats, one from our company as well, and of course there
are multiple readers supporting such tiff files.

hth
Paavo

https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.maptools.org_mailman_listinfo_tiff&d=DwICAg&c=5hYF0Zu0Yz-C6S-kaHDItw&r=HnVw9YJBS4ST0tOBucT3NvWUJRRGhDxXu4zHnS5AkgY&m=RcsXrpPDw-KXo-5NeLU-puXNVyvQfx7dLHdgzA4wxTw&s=4-nAkbKaJaJMlatde8ffh_QxF23k8hdM9UgRBbWwIsM&e=

https://urldefense.proofpoint.com/v2/url?u=http-3A__www.remotesensing.org_libtiff_&d=DwICAg&c=5hYF0Zu0Yz-C6S-kaHDItw&r=HnVw9YJBS4ST0tOBucT3NvWUJRRGhDxXu4zHnS5AkgY&m=RcsXrpPDw-KXo-5NeLU-puXNVyvQfx7dLHdgzA4wxTw&s=2t33zDcteNgQ1Ihq6G5nrh669iJFqB535PFOMpJ1VAE&e=