2008.08.22 15:21 "Re: [Tiff] creating sparse files......", by Toby Thain

On 22-Aug-08, at 11:44 AM, Bob Friesenhahn wrote:

> On Fri, 22 Aug 2008, Rogier Wolff wrote:
>>
>> I'm stitching kind of large panoramas. This results in big

>> intermediate files. On my last run, which took overnight to stitch, >> I thought 42 Gb of free disk space would be enough. Wrong!

>>
>> I got over thrity files of over 1.2Gbytes, filling up the disk.
>>

>> It turns out that most of the files contain lots of zeroes. On Unix >> this can be stored effciently by not issueing a "write" with a buffer

>> full of zeroes, but by seeking over the area. The operating system >> will act as if the area was filled with zeroes.

>
> This is an interesting issue. While holey files seem like a
> panacea, there can be some drawbacks. They are best for files

> which are written just once (like core files) and not so good for > files which are expected to be updated in place. For files which

> are updated in place, the updated hole is quite likely to increase > disk fragmentation since now it takes more space and the space will

> need to be from some other place on disk. Fragmentation behavior > is quite filesystem dependent.

>

> It is worth considering enabling filesystem compression, or using > whole-file compression. Perhaps even just enabling normal TIFF

> compression (e.g. LZW) is sufficient to eliminate the long spans of > zeros.

Agreed, LZW (or even RLE) will be effective here.

--Toby

Bob
======================================
Bob Friesenhahn

> bfriesen@simple.dallas.tx.us, http://www.simplesystems.org/users/
> bfriesen/