I'm trying to back up our svn repositories, and I found a nice little backup command line bzip's the backup and creates the md5 hash all in one:
svnadmin dump --deltas /repo |bzip2 |tee dump.bz2 | md5sum >dump.md5
The problem is I need to split the backups, so this doesn't really work. Is there perhaps another way of piping things to allow for splitting of the backups? Currently I'm doing something like this
svnadmin dump --deltas /repo |bzip2 |split - -b 64m cat *.bz2* | md5sum >dump.md5
Is there a way to do this all in one step?
Russ
On Fri, Nov 09, 2007 at 12:53:00PM -0500, Ruslan Sivak wrote:
I'm trying to back up our svn repositories, and I found a nice little backup command line bzip's the backup and creates the md5 hash all in one:
svnadmin dump --deltas /repo |bzip2 |tee dump.bz2 | md5sum >dump.md5
The problem is I need to split the backups, so this doesn't really work. Is there perhaps another way of piping things to allow for splitting of the backups? Currently I'm doing something like this
svnadmin dump --deltas /repo |bzip2 |split - -b 64m cat *.bz2* | md5sum >dump.md5
Is there a way to do this all in one step?
Would something as simple as:
svnadmin dump --deltas /repo | bzip2 | split - -b 64m && cat *.bz2* | md5sum >dump.md5
Work for you? :)
Ray
Ray Van Dolson wrote:
On Fri, Nov 09, 2007 at 12:53:00PM -0500, Ruslan Sivak wrote:
I'm trying to back up our svn repositories, and I found a nice little backup command line bzip's the backup and creates the md5 hash all in one:
svnadmin dump --deltas /repo |bzip2 |tee dump.bz2 | md5sum >dump.md5
The problem is I need to split the backups, so this doesn't really work. Is there perhaps another way of piping things to allow for splitting of the backups? Currently I'm doing something like this
svnadmin dump --deltas /repo |bzip2 |split - -b 64m cat *.bz2* | md5sum >dump.md5
Is there a way to do this all in one step?
Would something as simple as:
svnadmin dump --deltas /repo | bzip2 | split - -b 64m && cat *.bz2* | md5sum >dump.md5
Work for you? :)
Ray
Good idea, but no. The idea is to minimize disk IO, so I don't want to read the files that I just wrote. I wish tee had a split option...
Russ
Good idea, but no. The idea is to minimize disk IO, so I don't want to read the files that I just wrote. I wish tee had a split option...
Well we could get really creative perhaps...
mkfifo temp.fifo split temp.fifo -b 64m & svnadmin dump --deltas /repo | bzip2 | tee temp.fifo | md5sum rm -f temp.fifo
Ray
I'm trying to back up our svn repositories, and I found a nice little backup command line bzip's the backup and creates the md5 hash all in one:
svnadmin dump --deltas /repo |bzip2 |tee dump.bz2 | md5sum >dump.md5
The problem is I need to split the backups, so this doesn't really work. Is there perhaps another way of piping things to allow for splitting of the backups? Currently I'm doing something like this
svnadmin dump --deltas /repo |bzip2 |split - -b 64m cat *.bz2* | md5sum >dump.md5
Is there a way to do this all in one step?
What about:
svnadmin dump --deltas /repo | bzip2 | tee >(split -b 64m - dump.bz2.) | md5sum > dump.md5
-Shad
Shad L. Lords wrote:
I'm trying to back up our svn repositories, and I found a nice little backup command line bzip's the backup and creates the md5 hash all in one:
svnadmin dump --deltas /repo |bzip2 |tee dump.bz2 | md5sum >dump.md5
The problem is I need to split the backups, so this doesn't really work. Is there perhaps another way of piping things to allow for splitting of the backups? Currently I'm doing something like this
svnadmin dump --deltas /repo |bzip2 |split - -b 64m cat *.bz2* | md5sum >dump.md5
Is there a way to do this all in one step?
What about:
svnadmin dump --deltas /repo | bzip2 | tee >(split -b 64m - dump.bz2.) | md5sum > dump.md5
-Shad _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
This seems to work well, but I have no idea what it's doing. Can someone walk me through what tee >(split -b 64m - dump.bz2.) does and why?
Russ
Ruslan Sivak wrote:
Shad L. Lords wrote:
I'm trying to back up our svn repositories, and I found a nice little backup command line bzip's the backup and creates the md5 hash all in one:
svnadmin dump --deltas /repo |bzip2 |tee dump.bz2 | md5sum >dump.md5
The problem is I need to split the backups, so this doesn't really work. Is there perhaps another way of piping things to allow for splitting of the backups? Currently I'm doing something like this
svnadmin dump --deltas /repo |bzip2 |split - -b 64m cat *.bz2* | md5sum >dump.md5
Is there a way to do this all in one step?
What about:
svnadmin dump --deltas /repo | bzip2 | tee >(split -b 64m - dump.bz2.) | md5sum > dump.md5
-Shad _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Sorry to resurrect an old thread, but even though this works fine from the command line, it doesn't seem to work from my perl script. I get sh: -c: line 1: syntax error near unexpected token `(' sh: -c: line 1: `svnadmin dump --deltas /svn/russ 2>>/backup/russ/2007/12/full.4.log | bzip2 | tee >(split -b 1888m - /backup/russ/2007/12/full.4.bz2.) | md5sum > /backup/russ/2007/12/full.4.md5'
Looks like it's running sh instead of bash? Is there a way to change the shell that executes the command? I'm using backticks to execute the command in perl.
Russ
On Tue, 2007-12-04 at 16:35 -0500, Ruslan Sivak wrote:
Ruslan Sivak wrote:
Shad L. Lords wrote:
<snip>
Sorry to resurrect an old thread, but even though this works fine from the command line, it doesn't seem to work from my perl script. I get sh: -c: line 1: syntax error near unexpected token `(' sh: -c: line 1: `svnadmin dump --deltas /svn/russ 2>>/backup/russ/2007/12/full.4.log | bzip2 | tee >(split -b 1888m -
AA || Is this a valid construct in bash now? I'm unsure--++ I've not RTFM recently, but doesn't this say to redirect standard output to a sub-shell? AFAIK, that's not valid? IIRC, the operand of the ?>? needs to be a "file" (which in the old-time *IX semantics includes devices, FIFOs, etc.).
And "tee" wants a filename to write to, no?
If you're not "tee"ing to a file, can't you drop tee and just pipe to the sub-shell?
/backup/russ/2007/12/full.4.bz2.) | md5sum > /backup/russ/2007/12/full.4.md5'
Looks like it's running sh instead of bash? Is there a way to change the shell that executes the command? I'm using backticks to execute the command in perl.
I only dabbled in Perl briefly long ago. I would dare comment about that. I know the effects of the back-ticks in shells though. Same in Perl?
Russ
<snip sig stuff>
I hope my questions sparked a clue and wasn't just a band-width waste.
-- Bill
William L. Maltby wrote:
On Tue, 2007-12-04 at 16:35 -0500, Ruslan Sivak wrote:
Ruslan Sivak wrote:
Shad L. Lords wrote:
<snip>
Sorry to resurrect an old thread, but even though this works fine from the command line, it doesn't seem to work from my perl script. I get sh: -c: line 1: syntax error near unexpected token `(' sh: -c: line 1: `svnadmin dump --deltas /svn/russ 2>>/backup/russ/2007/12/full.4.log | bzip2 | tee >(split -b 1888m -
AA ||Is this a valid construct in bash now? I'm unsure--++ I've not RTFM recently, but doesn't this say to redirect standard output to a sub-shell? AFAIK, that's not valid? IIRC, the operand of the ?>? needs to be a "file" (which in the old-time *IX semantics includes devices, FIFOs, etc.).
And "tee" wants a filename to write to, no?
If you're not "tee"ing to a file, can't you drop tee and just pipe to the sub-shell?
Someone helped me with this syntax a few weeks back, and it works perfectly from the shell. It just refuses to work from inside perl's backticks.
/backup/russ/2007/12/full.4.bz2.) | md5sum > /backup/russ/2007/12/full.4.md5'
Looks like it's running sh instead of bash? Is there a way to change the shell that executes the command? I'm using backticks to execute the command in perl.
I only dabbled in Perl briefly long ago. I would dare comment about that. I know the effects of the back-ticks in shells though. Same in Perl?
Russ
<snip sig stuff>
I hope my questions sparked a clue and wasn't just a band-width waste.
-- Bill
At least it's a reply... Hopefully someone who understand this a bit more will see the thread.
Russ
On Fri, 9 Nov 2007, Ruslan Sivak wrote:
I'm trying to back up our svn repositories, and I found a nice little backup command line bzip's the backup and creates the md5 hash all in one:
If you have a newer Subversion, svnsync is great for this, albeit without the md5 sums. The destination repository is an exact replica, revision properties and all, of the source repo.
E.g.,
svnadmin create /srv/svn/myrepo.bak echo '#!/bin/sh' > /srv/svn/myrepo.bak/hooks/pre-revprop-change chmod +x /srv/svn/myrepo.bak/hooks/pre-revprop-change svnsync init file:///srv/svn/myrepo.bak http://src.me.com/svn/myrepo svnsync sync file:///srv/svn/myrepo.bak
Then, somewhat regularly,
svnsync sync file:///srv/svn/myrepo.bak
Paul Heinlein wrote:
On Fri, 9 Nov 2007, Ruslan Sivak wrote:
I'm trying to back up our svn repositories, and I found a nice little backup command line bzip's the backup and creates the md5 hash all in one:
If you have a newer Subversion, svnsync is great for this, albeit without the md5 sums. The destination repository is an exact replica, revision properties and all, of the source repo.
E.g.,
svnadmin create /srv/svn/myrepo.bak echo '#!/bin/sh' > /srv/svn/myrepo.bak/hooks/pre-revprop-change chmod +x /srv/svn/myrepo.bak/hooks/pre-revprop-change svnsync init file:///srv/svn/myrepo.bak http://src.me.com/svn/myrepo svnsync sync file:///srv/svn/myrepo.bak
Then, somewhat regularly,
svnsync sync file:///srv/svn/myrepo.bak
One problem with this... my destination is on a mounted S3 drive, and for some reason I can't set up a working SVN repo on there. Speaking of which, is anyone running svn on S3? Are there plans for an S3 backend? Perhaps I should make this a new thread...
Russ
-
Paul Heinlein -
Ray Van Dolson -
Ruslan Sivak -
Shad L. Lords -
William L. Maltby