We transfer files from a VAN provider at 15 minute intervals using rsync over ssh. The setup is somewhat complicated in that the VAN will not permit direct rsync access and so we establish the link via sshfs and then mount remote location as local.
My question is, given the above conditions and the following rsync command:
/usr/bin/rsync --chmod=o+r --chmod=g+w --itemize-changes --remove-sent-files --times /var/spool/imanet/pick_up/* /var/spool/imanet/drop_off
Under what circumstances would a file containing data at the remote end (/var/spool/imanet/pick_up/) arrive at our end (/var/spool/imanet/drop_off) as an empty file? No transmission errors were logged and multiple files were transferred during the same session. All but one arrived with their contents intact.
-----Original Message----- From: CentOS [mailto:centos-bounces@centos.org] On Behalf Of James B. Byrne Sent: Friday, June 30, 2017 2:43 PM To: centos@centos.org Subject: [CentOS] rsync and cause/source of an empty file
We transfer files from a VAN provider at 15 minute intervals using rsync over ssh. The setup is somewhat complicated in that the VAN will not permit direct rsync access and so we establish the link via sshfs and then mount remote location as local.
My question is, given the above conditions and the following rsync command:
/usr/bin/rsync --chmod=o+r --chmod=g+w --itemize-changes --remove-sent-files --times /var/spool/imanet/pick_up/* /var/spool/imanet/drop_off
Under what circumstances would a file containing data at the remote end (/var/spool/imanet/pick_up/) arrive at our end (/var/spool/imanet/drop_off) as an empty file? No transmission errors were logged and multiple files were transferred during the same session. All but one arrived with their contents intact.
First guess is you had the misfortune of syncing a file that was being written to at the same time on the source end.
Not that it helps, but FFS, an EDI provider exposing rsync? What kind of SP does that? This problem has been so concretely solved ages ago?
On 06/30/2017 01:42 PM, James B. Byrne wrote:
The setup is somewhat complicated in that the VAN will not permit direct rsync access and so we establish the link via sshfs and then mount remote location as local.
sshfs doesn't seem to preserve/transmit ctime, so rsync won't be able to reliably determine whether a file has changed based on times. There's a small chance that you'll miss updates in this configuration. That's probably not related to your problem, just a note on that setup.
Regarding the empty file, perhaps the source had the file locked for reading. Do you know what type of system is providing the data?
-
Gordon Messmer -
James B. Byrne -
Joseph L. Casale