2020.04.19 17:01 "[Tiff] tiffsplit from FIFO", by Levi Wilson

2020.04.19 17:01 "[Tiff] tiffsplit from FIFO", by Levi Wilson

Curious if anyone has any experience processing larger documents with named pipes. The context that I'm working in is that I'd like to process a TIFF file from S3, but start processing the file as it is downloading. This could be useful in a lambda execution environment where space is limited.

I was thinking that I could do the following:

mkfifo fifo.tif
tiffsplit fifo.tif &
aws s3 cp s3://my-bucket/some/path/to/my_image.tif fifo.tif

When I do this, I start up tiffsplit and it seems to be waiting, but when I fire off the aws s3 cp command I get this:

$ tiffsplit fifo.tif
TIFFFetchDirectory: fifo.tif: Seek error accessing TIFF directory.
TIFFReadDirectory: Failed to read directory at offset 15278.

And on the writing side I get this:

$ aws s3 cp s3://my-bucket/some/path/to/my_image.tif fifo.tif
download failed: s3://traverse-levi/multi_image_content_items/016/aa2/08-/016aa208-003a-41af-968b-2caeb6c718e0_files to ./fifo.tif [Errno 32] Broken pipe

I'm probably doing something wrong with this, but curious if anyone else has attempted something like this.