On Fri, Mar 21, 2014 at 5:31 PM, SilverTip257 silvertip257@gmail.com wrote:
Before very recent versions of rsync (not sure exactly when it changed), it would load the entire tree listing from both sides into memory before walking them for the comparison. What's the destination side look like? Maybe you aren't doing a --delete and a lot of cruft has accumulated.
I avoid this problem by having the cronjob run more often and the script checks that syncs are running (which means on average it should sync less files and re-sync as soon as the previous sync is finished).
The number of files being sync'd isn't the big memory consumer - it is the whole directory trees being traversed that are loaded into memory for the comparison. There is additional overhead it you use -H to propagate hardlinks.