- 2007.07.03 19:04 "Re: [Tiff] BigTIFF extension?", by Stephen Carlsen
-
2007.07.03 19:09 "Re: [Tiff] BigTIFF extension?", by Bob Friesenhahn
-
2007.07.04 15:58 "Re: [Tiff] BigTIFF extension?", by Phil Harvey
- 2007.07.04 11:37 "Re: [Tiff] BigTIFF extension?", by Phil Harvey
-
2007.07.04 17:13 "[Tiff] 16-Bit-Per-Channel Lossless Compression", by Stephen Carlsen
-
2007.07.04 17:24 "Re: [Tiff] 16-Bit-Per-Channel Lossless Compression", by Andy Cave
-
2007.07.04 17:26 "Re: [Tiff] 16-Bit-Per-Channel Lossless Compression", by Andy Cave
-
2007.07.04 17:49 "Re: [Tiff] 16-Bit-Per-Channel Lossless Compression", by Stephen Carlsen
- 2007.07.04 18:00 "Re: [Tiff] 16-Bit-Per-Channel Lossless Compression", by Joris Van Damme
- 2007.07.04 20:37 "Re: [Tiff] 16-Bit-Per-Channel Lossless Compression", by Andy Cave
-
2007.07.05 10:56 "Re: [Tiff] 16-Bit-Per-Channel Lossless Compression", by Andrey Kiselev
- 2007.07.05 12:03 "Re: [Tiff] 16-Bit-Per-Channel Lossless Compression", by Andy Cave
-
2007.07.05 17:58 "Re: [Tiff] 16-Bit-Per-Channel Lossless Compression", by Stephen Carlsen
- 2007.07.04 18:02 "Re: [Tiff] 16-Bit-Per-Channel Lossless Compression", by Joris Van Damme
- 2007.07.05 18:30 "Re: [Tiff] 16-Bit-Per-Channel Lossless Compression", by Joris Van Damme
- 2007.07.06 01:46 "Re: [Tiff] 16-Bit-Per-Channel Lossless Compression", by Chris Cox
-
2007.07.04 17:49 "Re: [Tiff] 16-Bit-Per-Channel Lossless Compression", by Stephen Carlsen
-
2007.07.04 17:26 "Re: [Tiff] 16-Bit-Per-Channel Lossless Compression", by Andy Cave
- 2007.07.04 17:28 "Re: [Tiff] 16-Bit-Per-Channel Lossless Compression", by Michael Wolf
- 2007.07.04 17:36 "Re: [Tiff] 16-Bit-Per-Channel Lossless Compression", by Andrew Brooks
-
2007.07.04 17:24 "Re: [Tiff] 16-Bit-Per-Channel Lossless Compression", by Andy Cave
-
2007.07.04 15:58 "Re: [Tiff] BigTIFF extension?", by Phil Harvey
- 2007.07.03 19:17 "Re: [Tiff] BigTIFF extension?", by Andy Cave
- 2007.07.04 11:53 "Re: [Tiff] BigTIFF extension?", by Phil Harvey
-
2007.07.04 20:52 "Re: [Tiff] BigTIFF extension", by Joris Van Damme
-
2007.07.04 21:04 "Re: [Tiff] BigTIFF extension", by Toby Thain
-
2007.07.05 05:07 "Re: [Tiff] BigTIFF extension", by Joris Van Damme
-
2007.07.05 08:09 "Re: [Tiff] BigTIFF extension", by Andy Cave
-
2007.07.05 13:40 "Re: [Tiff] BigTIFF extension", by Joris Van Damme
-
2007.07.05 21:00 "Re: [Tiff] BigTIFF extension", by Chris Cox
- 2007.07.05 21:19 "Re: [Tiff] BigTIFF extension", by Joris Van Damme
-
2007.07.05 22:13 "Re: [Tiff] BigTIFF extension", by Bob Friesenhahn
-
2007.08.06 22:48 "RE: [Tiff] Transparency ... associated or unassociated alpha is morecompatible?", by Chris Cox
- 2007.08.06 22:32 "RE: [Tiff] Transparency ... associated or unassociated alpha is morecompatible?", by Chris Cox
-
2007.08.06 22:48 "RE: [Tiff] Transparency ... associated or unassociated alpha is morecompatible?", by Chris Cox
- 2007.07.06 12:36 "Re: [Tiff] BigTIFF extension", by Gary McGath
-
2007.07.05 21:00 "Re: [Tiff] BigTIFF extension", by Chris Cox
-
2007.07.05 13:40 "Re: [Tiff] BigTIFF extension", by Joris Van Damme
-
2007.07.05 08:09 "Re: [Tiff] BigTIFF extension", by Andy Cave
-
2007.07.05 16:26 "Re: [Tiff] BigTIFF extension", by Joris Van Damme
-
2007.07.05 04:19 "RE: [Tiff] BigTIFF extension", by Kemp Watson
-
2007.07.04 16:56 "[Tiff] BigTIFF extension", by Kemp Watson
-
2007.07.04 17:35 "Re: [Tiff] BigTIFF extension", by Andrew Brooks
- 2007.07.04 18:13 "Re: [Tiff] BigTIFF extension", by John Aldridge
-
2007.07.04 18:25 "Re: [Tiff] BigTIFF extension", by Phil Harvey
- 2007.07.04 18:52 "Re: [Tiff] BigTIFF extension", by Toby Thain
- 2007.07.04 19:17 "Re: [Tiff] BigTIFF extension", by Frank Warmerdam
- 2007.07.05 20:31 "Re: [Tiff] BigTIFF extension", by Chris Cox
- 2007.07.05 15:14 "[Tiff] BigTIFF - Aware Aperio version diff", by Kemp Watson
-
2007.07.04 17:35 "Re: [Tiff] BigTIFF extension", by Andrew Brooks
- 2007.07.05 04:54 "RE: [Tiff] BigTIFF extension", by Stephen Carlsen
- 2007.07.05 16:32 "RE: [Tiff] BigTIFF extension", by Kemp Watson
-
2007.07.04 16:56 "[Tiff] BigTIFF extension", by Kemp Watson
- 2007.07.05 18:56 "Re: [Tiff] BigTIFF extension", by Toby Thain
-
2007.07.05 04:19 "RE: [Tiff] BigTIFF extension", by Kemp Watson
-
2007.07.05 05:07 "Re: [Tiff] BigTIFF extension", by Joris Van Damme
-
2007.07.05 10:06 "Re: [Tiff] BigTIFF extension", by John Aldridge
-
2007.07.05 13:23 "[Tiff] Re: Tiff Digest, Vol 38, Issue 12", by Gary McGath
-
2007.07.05 13:56 "Re: [Tiff] Re: Tiff Digest, Vol 38, Issue 12", by Joris Van Damme
-
2007.07.05 14:14 "Re: [Tiff] Re: Tiff Digest, Vol 38, Issue 12", by Gary McGath
- 2007.07.05 14:39 "Re: [Tiff] Re: Tiff Digest, Vol 38, Issue 12", by Joris Van Damme
- 2007.07.05 15:26 "Re: [Tiff] Re: Tiff Digest, Vol 38, Issue 12", by Andrey Kiselev
-
2007.07.05 14:14 "Re: [Tiff] Re: Tiff Digest, Vol 38, Issue 12", by Gary McGath
-
2007.07.05 13:56 "Re: [Tiff] Re: Tiff Digest, Vol 38, Issue 12", by Joris Van Damme
-
2007.07.05 13:23 "[Tiff] Re: Tiff Digest, Vol 38, Issue 12", by Gary McGath
- 2007.07.05 16:51 "Re: [Tiff] BigTIFF extension", by Andrew Brooks
- 2007.07.09 16:11 "Re: [Tiff] BigTIFF extension", by Stephen Carlsen
-
2007.07.04 21:04 "Re: [Tiff] BigTIFF extension", by Toby Thain
2007.07.05 16:30 "Re: [Tiff] 16-Bit-Per-Channel Lossless Compression", by Joris Van Damme
Bob,
As I don't speak gdalese, what size do tiles have in your test? It's my opinion ideally efficient tiles over a broad range of compression modes, that fit workflow on today's machines best, are in the order of about 1 to 3 megabyte, not the legacy 8 kilobyte (uncompressed, of course). I believe that for some compression modes, tile size does have a considerable impact on efficienty.
You make many assumptions in the above statement, and particularly
about how applications store/access their pixels. I would not
encourage anyone to write such huge tiles.There are many major factors which influence performance:
- Application pixel storage organization
- CPU cache
- Disk cache
- Main memory bandwidth
- Disk filesystem implementation
- Operating system I/O implementation
- Network filesystem implementation
I have done considerable testing and benchmarking in various scenarios
and results usually show that data access sizes of 8K or 16K are best,
with performance diminishing past 32K.Your huge tiles have these problems:
- They are larger than the cache on anything but the most exotic RISC systems. Regardless, the first level cache is usually fairly small.
- They are larger than many disk drive caches.
- They don't mesh well with the typical 8K or 16K filesystem block sizes.
- They don't mesh well with network filesystem buffering ("block") sizes.
- The application is blocked on large I/O when it could be simultaneously processing data (parsing/storing pixels).
- They require the application to allocate more memory.
- They defeat the operating system's built in "read ahead" logic for sequential file access. The operating system is only going to be willing to buffer a limited amount of unrequested data in anticipation that it might be used. Only a rather foolish OS will gamble on reading 3MB of data in advance with the hope that it will be used. If the OS is only willing to do read ahead by 8K or 16K, then performance is best if the application does not need more than that in order to proceed.
Though a number of your points are based on LibTiff's design, which is not ideal in that it buffers compressed and decompressed striles (=strips or tiles, whatever applies), I do think you make sense given that LibTiff design and a number of your points probably apply to AsTiff design as well. Likely my estimate of 1 to 3 megabyte for ideal strile size is a bit much.
Best regards,
Joris Van Damme
info@awaresystems.be
http://www.awaresystems.be/
Download your free TIFF tag viewer for windows here:
http://www.awaresystems.be/imaging/tiff/astifftagviewer.html