2018.01.16 20:18 "Re: [Tiff] Strategies for multi-core speedups", by Kemp Watson
We found that we were getting very reasonable interactivity in web browsers, with no 'fancy' WebGL, WebSockets, WebRTC, etc, just raw XMLHTTP, at about 0.5-0.6 Mbps per user. That's for viewing at screen pixel data sizes.
On Tue, Jan 16, 2018 at 3:08 PM, <Joe.Maniaci@ricoh-usa.com> wrote:
I did experiment with HTML5 + webGL in the past and I actually ran into draw call limitations by the browsers. But looking back I'm sure I did things quite incorrectly setting images up. But even then the thought did cross our minds and we basically came to the same conclusions. We may be getting some 10Gb network capabilities soon, but I'm not banking on it so I went the desktop application route.
Staff Software Engineer
From: Kemp Watson <email@example.com>
To: Paavo Helde <firstname.lastname@example.org>
Date: 01/16/2018 01:02 PM
Subject: Re: [Tiff] Strategies for multi-core speedups [EXTERNAL]
Sent by: email@example.com
Also Joseph, you might find yourself limited by network bandwidth or BDP long before you are limited by application threading, good idea to think about network simulation (if relevant) before detailed design.
> On Tue, Jan 16, 2018 at 2:34 PM, Paavo Helde <*firstname.lastname@example.org* > <email@example.com>> wrote:
> On 16.01.2018 20:36, *Joe.Maniaci@ricoh-usa.com*
> <Joe.Maniaci@ricoh-usa.com> wrote:
> I am a complete newbie to digital imagery and TIFFs in general, so please
> forgive any ignorance.
> I'm an automated test tool developer and I am actually going to start
> working on a viewer for the TIFFs that my company generates, to potentially
> include relying on the bigtiff format and this is the thing I've spent most
> of my time thinking about. I've come to the conclusion that for the images
> we generate, up to 10GB uncompressed(soon to be 40GB), that I will need to
> preprocess a single TIFF into multiple TIFFs containing intelligently
> placed tiles just to get around the I/O issue. By intelligently placed, I
> mean that if a user were to scroll their view window to the right, and five
> tiles(vertically aligned) had to be loaded into memory, I could
> theoretically have those files intelligently located in different TIFF
> files(subTIFFs?) that would allow multiple threads to operate on. So each
> thread only had to find one tile. So for my tool it would be a staggering
> amount of pre-processing, but real time manipulation is extremely fluid for
> the user which is basically my number one requirement.
> So for a case like mine, thread overhead is nothing to worry about. That
> and the more I read about the C++11 thread library, the less I worry. I can
> just leave it to the library to determine how many threads should operate
> in my 4-core environments, or my 16-core environments. At least that's the
> impression I've garnered.
If all you want is a viewer then you are limited by the screen sizes which are typically not more than ca 2000x2000 pixels. And this is pretty small image nowadays, there is no need to multithread anything, you could just read the needed tiles from the TIFF file in real time and maybe add a small cache of the read tiles so you don't need to read them again if the user pans left 1 pixel.
For zooming out the TIFF file should have different frames of the same image stored in decreasing resolutions so that a suitable resolution can be chosen for display - these are called pyramidal tiffs. The TIFF standard suggests that such varying resolution images should be stored as sub-IFD-s but as many viewers do not support sub-IFD-s they are typically stored just in standard IFD-s, and some XML piece in the first IFD provides the details. I am sure there are multiple variants of such pyramidal tiff formats, one from our company as well, and