2021.04.24 20:00 "Re: [Tiff] Tiff Digest, Vol 29, Issue 10", by Richard Nolde

It seems like the files are being read while they are still being updated. It may be that the client thinks it is done (because the operating system told it so) but the operating system used by the client is still sending data to the server. Meanwhile, the reading client is depending on the implementation of the network share to access the file and the client likely uses caching, which may cause it to not know the correct answer.

What type of network share is being used? What are the server and client operating systems involved?

If this is Linux and using something like inotify(7) to notice when files are available, then it is possible to be notified about a new file available while the file is still being updated or even "settling" in the OS (e.g. file is "there" but OS has not yet actually done everything to make it completely so).

The rename() system call within the same filesystem is normally expected to be atomic. This still does not mean that two clients of a server can not have a differing opinion of the current state.

Some sort of locking beyond whether a file appears to exist or not is likely to be required.

I have written and used in a production environment a utility to watch a directory for new files uploaded from up to 50 high speed networked copier scanners to a Linux server. The utility, reads a configuration file that includes the delay for which the file must remain unchanged before it is considered complete at which point a sub process is forked to run a bar code reader script to identify the bar codes in the files and create a series of files from the multi-page TIFF originals which could have 500 or more pages per input file. If you are interested in the c source code for the watchdir program, contact me directly. I wrote the same kind of program for Windows and it took ten times as much code and could not be run without administrative privileges. It did however, perform much better and more reliably than the commercial packages the scanner vendors wanted to sell us.

This may or may not be of use to you since my files were received and processed on a single server unless you can configure your server to receive the files in one directory and then move them to another directory or perhaps create a symlink to the completed file where the clients can pick them up.

Richard Nolde