Can someone explain why this: find . -depth -print0 | cpio --null -pmd /tmp/test will copy all files in and below the current directory -and- this: find . -depth -print | grep -v .iso$ | wc -l will count all the non-iso files -and- this: find . -depth -print | grep .iso$ | wc -l will count *only* the iso files -but- this: find . -depth -print0 | grep -v .iso$ | cpio --null -pmd /tmp/test doesn't copy *anything*? Any suggestions for a work-around would also be most welcome.
On Thu, 22 Jun 2006, Robert wrote:
Can someone explain why this: find . -depth -print0 | cpio --null -pmd /tmp/test will copy all files in and below the current directory -and- this: find . -depth -print | grep -v .iso$ | wc -l will count all the non-iso files -and- this: find . -depth -print | grep .iso$ | wc -l will count *only* the iso files -but- this: find . -depth -print0 | grep -v .iso$ | cpio --null -pmd /tmp/test doesn't copy *anything*? Any suggestions for a work-around would also be most welcome.
I think that passing -print0 output to grep is the culprit. How about using find to identify the files rather than grep:
find . -depth ! -name '*.iso' -print0 | cpio ...
-- Paul Heinlein heinlein@madboa.com
On Thu, 2006-06-22 at 05:30 -0700, Paul Heinlein wrote:
On Thu, 22 Jun 2006, Robert wrote:
<snip>
I think that passing -print0 output to grep is the culprit. How about using find to identify the files rather than grep:
find . -depth ! -name '*.iso' -print0 | cpio ...
Yes. That is a much better way. Only suggestion I make is "-iname" incase some of those files come from a well-known environment that tends towards a "capitalist society" :-).
<snip>
Robert wrote:
Can someone explain why this: find . -depth -print0 | cpio --null -pmd /tmp/test will copy all files in and below the current directory -and- this: find . -depth -print | grep -v .iso$ | wc -l will count all the non-iso files -and- this: find . -depth -print | grep .iso$ | wc -l will count *only* the iso files -but- this: find . -depth -print0 | grep -v .iso$ | cpio --null -pmd /tmp/test doesn't copy *anything*? Any suggestions for a work-around would also be most welcome.
Because -print0 generates a NULL separated list - so grep will match .iso\0 in the string generated by find and hence not output any list for cpio to read ...
One workaround:
find . -depth -print | grep -v .iso$ | cpio -pmd /tmp/test
James Pearson
On Thu, 22 Jun 2006, James Pearson wrote:
Robert wrote:
Can someone explain why this: find . -depth -print0 | cpio --null -pmd /tmp/test will copy all files in and below the current directory -and- this: find . -depth -print | grep -v .iso$ | wc -l will count all the non-iso files -and- this: find . -depth -print | grep .iso$ | wc -l will count *only* the iso files -but- this: find . -depth -print0 | grep -v .iso$ | cpio --null -pmd /tmp/test doesn't copy *anything*? Any suggestions for a work-around would also be most welcome.
Because -print0 generates a NULL separated list - so grep will match .iso\0 in the string generated by find and hence not output any list for cpio to read ...
One workaround:
find . -depth -print | grep -v .iso$ | cpio -pmd /tmp/test
The -print0 command is typically used when find is passing filenames with spaces or other characters that require a shell escape.
If the filenames are all shell-safe, then -print0 is unnecessary. If there's a chance that even one name contains a space, however, then -print0 is your friend.
-- Paul Heinlein heinlein@madboa.com
On Thu, 2006-06-22 at 05:40 -0700, Paul Heinlein wrote:
On Thu, 22 Jun 2006, James Pearson wrote:
Robert wrote:
<snip>
The -print0 command is typically used when find is passing filenames with spaces or other characters that require a shell escape.
If the filenames are all shell-safe, then -print0 is unnecessary. If there's a chance that even one name contains a space, however, then -print0 is your friend.
And, as a reminder for the OP, if none of the file names are being seen by the shell, as is the case for this problem, there is no concern at all.
<snip sig stuff>
On Thu, 2006-06-22 at 07:21 -0500, Robert wrote:
Can someone explain why this: find . -depth -print0 | cpio --null -pmd /tmp/test will copy all files in and below the current directory -and- this: find . -depth -print | grep -v .iso$ | wc -l will count all the non-iso files -and- this: find . -depth -print | grep .iso$ | wc -l will count *only* the iso files -but- this: find . -depth -print0 | grep -v .iso$ | cpio --null -pmd /tmp/test doesn't copy *anything*? Any suggestions for a work-around would also be most welcome.
Replace the print0 with print. If you need to know why, redirect your find print0 output to a file & then do an od -c on it.
Hint: cpio expects one entry per line (in spite of what it did for you) and grep operates on lines of input. If it's not clear after doing the od -c, call again. :-)
BTW, this is not a wise-acre reply. I've always been victimized by the unexpected behavior of the utilities (my fault, not their's... *usually*) and have become somewhat adept at discovery of underlying cause. The method I post above is just sharing my experience.
Oh! Also, a vi on the two files (made by print0 and print) will give an indication.
<snip>
HTH
On Thu, 22 Jun 2006, William L. Maltby wrote:
On Thu, 2006-06-22 at 07:21 -0500, Robert wrote:
Can someone explain why this: find . -depth -print0 | cpio --null -pmd /tmp/test will copy all files in and below the current directory -and- this: find . -depth -print | grep -v .iso$ | wc -l will count all the non-iso files -and- this: find . -depth -print | grep .iso$ | wc -l will count *only* the iso files -but- this: find . -depth -print0 | grep -v .iso$ | cpio --null -pmd /tmp/test doesn't copy *anything*? Any suggestions for a work-around would also be most welcome.
Replace the print0 with print. If you need to know why, redirect your find print0 output to a file & then do an od -c on it.
Hint: cpio expects one entry per line (in spite of what it did for you) and grep operates on lines of input. If it's not clear after doing the od -c, call again. :-)
cpio, like xargs, can accept an argument list terminated by a null character rather than whitespace. That what the OP was doing with the --null switch (aka -0, also possible with xargs). It's very helpful with unpredictable filenames.
-- Paul Heinlein heinlein@madboa.com
On Thu, 2006-06-22 at 05:46 -0700, Paul Heinlein wrote:
On Thu, 22 Jun 2006, William L. Maltby wrote:
On Thu, 2006-06-22 at 07:21 -0500, Robert wrote:
<snip>
Hint: cpio expects one entry per line (in spite of what it did for you) and grep operates on lines of input. If it's not clear after doing the od -c, call again. :-)
cpio, like xargs, can accept an argument list terminated by a null character rather than whitespace.
Drat! I keep forgetting that things have changed, improvements been made,... to some of this stuff over the decades! I skipped right over the "--null" in cpio. Thanks for reminding me to... "man xyz" even old friends!
That what the OP was doing with the --null switch (aka -0, also possible with xargs). It's very helpful with unpredictable filenames.
He'd have been OK if grep could have done the same for him, I guess.
Hmm... in line with my new found policy of "man old_friends", grep came close, but "no cigar" with the "-Z/--null" param. However, if the "-a/-- text" param is added, it seems to do OK.
<cranky bitter old man mode> Why do the GNUrus keep fixing things that aren't broken! I know in this environ, a series (pipeline) of modules that "do one thing and do it well" is not as good as a single command to "do all things and none well". </cranky bitter old man mode> :-))
<snip sig stuff>
Well, enough. Soon the Perlateers will come out and attack!
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Thu, Jun 22, 2006 at 07:21:22AM -0500, Robert wrote:
Can someone explain why this: find . -depth -print0 | cpio --null -pmd /tmp/test will copy all files in and below the current directory -and- this: find . -depth -print | grep -v .iso$ | wc -l will count all the non-iso files -and- this: find . -depth -print | grep .iso$ | wc -l will count *only* the iso files -but- this: find . -depth -print0 | grep -v .iso$ | cpio --null -pmd /tmp/test doesn't copy *anything*?
Because the "grep" in the middle will mess up with the "print0/--null" trick.
- -- Rodrigo Barbosa "Quid quid Latine dictum sit, altum viditur" "Be excellent to each other ..." - Bill & Ted (Wyld Stallyns)
On Thu, 22 Jun 2006, Robert wrote:
Can someone explain why this: find . -depth -print0 | cpio --null -pmd /tmp/test will copy all files in and below the current directory -and- this: find . -depth -print | grep -v .iso$ | wc -l will count all the non-iso files -and- this: find . -depth -print | grep .iso$ | wc -l will count *only* the iso files -but- this: find . -depth -print0 | grep -v .iso$ | cpio --null -pmd /tmp/test doesn't copy *anything*? Any suggestions for a work-around would also be most welcome.
Never seen print0 before, but on my system it has the effect of putting all the filenames in single line. So grep -v eliminates the one line that it finds leaving nothing to pass to cpio.
man find -print True; print the full file name on the standard output, followed by a newline.
-print0 True; print the full file name on the standard output, followed by a null character. This allows file names that contain new- lines to be correctly interpreted by programs that process the find output.
cpio will understand the output of -print just fine. No need to use print0
------------------------------------------------------------------------ Jim Wildman, CISSP, RHCE jim@rossberry.com http://www.rossberry.com "Society in every state is a blessing, but Government, even in its best state, is a necessary evil; in its worst state, an intolerable one." Thomas Paine
Robert wrote:
Can someone explain why this: find . -depth -print0 | cpio --null -pmd /tmp/test will copy all files in and below the current directory -and- this: find . -depth -print | grep -v .iso$ | wc -l will count all the non-iso files -and- this: find . -depth -print | grep .iso$ | wc -l will count *only* the iso files -but- this:
find . -depth -print0 | grep -v .iso$ | cpio --null -pmd /tmp/test doesn't copy *anything*? Any suggestions for a work-around would also be most welcome.
What does this give you? find . -depth -print0 | grep -v .iso$
The print0 option causes all the file names to be on a single line. So when you grep out .ios$ you're losing the whole line.
Here's a tar alternative to the other cpio based solutions: # mkdir /tmp/test # tar cf - . --exclude '*.iso' | (cd /tmp/test; tar xvf -)
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
On Thu, 2006-06-22 at 08:45 -0400, Mark Belanger wrote:
Robert wrote:
<snip>
Here's a tar alternative to the other cpio based solutions: # mkdir /tmp/test # tar cf - . --exclude '*.iso' | (cd /tmp/test; tar xvf -)
Like it says on the box "Micr... or better". Use cpio to make the tar files! >:->> LOL I *do* remember that it has been able to do that for a long time.
<snip>
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Thu, Jun 22, 2006 at 07:21:22AM -0500, Robert wrote:
Can someone explain why this: find . -depth -print0 | cpio --null -pmd /tmp/test will copy all files in and below the current directory -and- this: find . -depth -print | grep -v .iso$ | wc -l will count all the non-iso files -and- this: find . -depth -print | grep .iso$ | wc -l will count *only* the iso files -but- this: find . -depth -print0 | grep -v .iso$ | cpio --null -pmd /tmp/test doesn't copy *anything*? Any suggestions for a work-around would also be most welcome.
Okey, a new options for you:
find . -depth -print0 | grep -vZ .iso$ | cpio --null -pmd /tmp/test
From grep manpage:
-Z, --null Output a zero byte (the ASCII NUL character) instead of the character that normally follows a file name. For example, grep -lZ outputs a zero byte after each file name instead of the usual newline. This option makes the output unambiguous, even in the presence of file names containing unusual characters like newlines. This option can be used with commands like find -print0, perl -0, sort -z, and xargs -0 to process arbitrary file names, even those that contain newline characters.
- -- Rodrigo Barbosa "Quid quid Latine dictum sit, altum viditur" "Be excellent to each other ..." - Bill & Ted (Wyld Stallyns)
On Thu, 2006-06-22 at 07:21, Robert wrote:
Can someone explain why this: find . -depth -print0 | cpio --null -pmd /tmp/test will copy all files in and below the current directory -and- this: find . -depth -print | grep -v .iso$ | wc -l will count all the non-iso files -and- this: find . -depth -print | grep .iso$ | wc -l will count *only* the iso files -but- this: find . -depth -print0 | grep -v .iso$ | cpio --null -pmd /tmp/test doesn't copy *anything*? Any suggestions for a work-around would also be most welcome.
This doesn't have much to do with bash except that you could easily see the answer if you left off the last element of that last pipeline so you could see the output from grep. You are feeding grep one long line with null terminated filenames when it wants things one separate lines. You can let find do the selection:
find . -depth ! -name '*.iso' -print0 | cpio ... but I'd probably: rsync -av --exclude '*.iso' . /tmp/test instead.
Les Mikesell wrote:
On Thu, 2006-06-22 at 07:21, Robert wrote:
Can someone explain why this: find . -depth -print0 | cpio --null -pmd /tmp/test will copy all files in and below the current directory -and- this: find . -depth -print | grep -v .iso$ | wc -l will count all the non-iso files -and- this: find . -depth -print | grep .iso$ | wc -l will count *only* the iso files -but- this: find . -depth -print0 | grep -v .iso$ | cpio --null -pmd /tmp/test doesn't copy *anything*? Any suggestions for a work-around would also be most welcome.
This doesn't have much to do with bash except that you could easily see the answer if you left off the last element of that last pipeline so you could see the output from grep. You are feeding grep one long line with null terminated filenames when it wants things one separate lines. You can let find do the selection:
find . -depth ! -name '*.iso' -print0 | cpio ... but I'd probably: rsync -av --exclude '*.iso' . /tmp/test instead.
Thanks to all who replied and an apology for not acknowledging them earlier. I had to leave almost immediately after posing the problem. Aside from answering my questions, I'd say that 14 responses within 49 minutes of my original post is a testament to the helpful attitude that characterizes this list.
Best regards
-
James Pearson -
Jim Wildman -
Les Mikesell -
Mark Belanger -
Paul Heinlein -
Robert -
Rodrigo Barbosa -
William L. Maltby