I am trying to move a group of sendmail queue files into a special area and am developing s script to assist. The manual steps are:
# Identify which messages to move mailq -qR<domain> > file1
# Select only lines with message ID strings grep '^[[:alpha:]][[:alnum:]]{13}' file1 > file2
# extract only the messages ID cut -b -14 file2 > file3
# prepend '*' to message IDs sed "s/^/*/" file3 > file4
All of this works the way that I expect. What I now want to do is to mv all of the related files listed in file4 in the form "*messageid" to another directory. Using xargs I expected (naively) that the following construction would work:
cat file4 | xargs mv /var/spool/mqueue/'{}' \ /var/spool/mqueue/offline
(note that in the original this is all one line.)
However, when I do this I get the error: mv: when moving multiple files, last argument must be a directory Try `mv --help' for more information.
There is obviously something about xargs that I do not understand. In my imagination I see this xargs construction expanding to this:
mv /var/spool/mqueue/*messageid1 /var/spool/mqueue/offline mv /var/spool/mqueue/*messageid2 /var/spool/mqueue/offline . . . mv /var/spool/mqueue/*messageidn /var/spool/mqueue/offline
so that the qf and df files for each message are moved into the subdirectory offline. But this is obviously incorrect. Can anyone here point out to me what my misunderstanding is and how to get this to work? If this is not the forum for this kind of question then can someone with more experience point me to a mailing list that would be more suitable?
Regards, JIm
-- *** e-mail is not a secure channel *** mailto:byrnejb.<token>@harte-lyne.ca James B. Byrne Harte & Lyne Limited vox: +1 905 561 1241 9 Brockley Drive fax: +1 905 561 0757 Hamilton, Ontario <token> = hal Canada L8E 3C3
On Tue, 8 Feb 2005, James B. Byrne wrote:
I am trying to move a group of sendmail queue files into a special area and am developing s script to assist. The manual steps are:
# Identify which messages to move mailq -qR<domain> > file1
# Select only lines with message ID strings grep '^[[:alpha:]][[:alnum:]]{13}' file1 > file2
# extract only the messages ID cut -b -14 file2 > file3
# prepend '*' to message IDs sed "s/^/*/" file3 > file4
All of this works the way that I expect. What I now want to do is to mv all of the related files listed in file4 in the form "*messageid" to another directory. Using xargs I expected (naively) that the following construction would work:
cat file4 | xargs mv /var/spool/mqueue/'{}' \ /var/spool/mqueue/offline
(note that in the original this is all one line.)
However, when I do this I get the error: mv: when moving multiple files, last argument must be a directory Try `mv --help' for more information.
There is obviously something about xargs that I do not understand. In my imagination I see this xargs construction expanding to this:
mv /var/spool/mqueue/*messageid1 /var/spool/mqueue/offline mv /var/spool/mqueue/*messageid2 /var/spool/mqueue/offline . . . mv /var/spool/mqueue/*messageidn /var/spool/mqueue/offline
so that the qf and df files for each message are moved into the subdirectory offline. But this is obviously incorrect. Can anyone here point out to me what my misunderstanding is and how to get this to work? If this is not the forum for this kind of question then can someone with more experience point me to a mailing list that would be more suitable?
Compare this:
echo -e "file1\nfile2\nfile3" | xargs echo '{}' blah
with:
echo -e "file1\nfile2\nfile3" | xargs -i echo '{}' blah
What you require is -i to make '{}' work. xargs by default appends the input as a list of arguments. The manpage says:
--replace[=replace-str], -I replace-str, -i[replace-str] Replace occurences of replace-str in the initial arguments with names read from standard input. Also, unquoted blanks do not terminate argu- ments. If replace-str is omitted, it defaults to "{}" (like for 'find -exec'). Implies -x and -L 1.
Kind regards, -- dag wieers, dag@wieers.com, http://dag.wieers.com/ -- [all I want is a warm bed and a kind word and unlimited power]
On 8 Feb 2005 at 23:14, Dag Wieers wrote:
Compare this:
echo -e "file1\nfile2\nfile3" | xargs echo '{}' blah
with:
echo -e "file1\nfile2\nfile3" | xargs -i echo '{}' blah
Thank you for the advice on the -i switch. I had tried this earlier. With the -i '{}' argument added then this is the result:
# ll -d ./mqueue/offline drwxr-xr-x 2 root root 4096 Feb 8 14:03 ./mqueue/offline # # cat workmv | xargs -i '{}' mv ./mqueue/'{}' ./mqueue/offline xargs: {}: No such file or directory # # cat workmv | xargs -i '{}' mv ./mqueue/'{}' ./mqueue/offline/ xargs: {}: No such file or directory # # cat workmv | xargs -i '{}' mv --Target-Directory=./mqueue/offline ./mqueue/'{}' xargs: {}: No such file or directory # # cat workmv | xargs -i '{}' mv --Target-Directory=./mqueue/offline/ ./mqueue/'{}' xargs: {}: No such file or directory #
P.S. Also, thank you very much for the depository that you run. I use several of your packages.
Regards, Jim
-- *** e-mail is not a secure channel *** mailto:byrnejb.<token>@harte-lyne.ca James B. Byrne Harte & Lyne Limited vox: +1 905 561 1241 9 Brockley Drive fax: +1 905 561 0757 Hamilton, Ontario <token> = hal Canada L8E 3C3
On Wed, 9 Feb 2005, James B. Byrne wrote:
On 8 Feb 2005 at 23:14, Dag Wieers wrote:
Compare this:
echo -e "file1\nfile2\nfile3" | xargs echo '{}' blah
with:
echo -e "file1\nfile2\nfile3" | xargs -i echo '{}' blah
Thank you for the advice on the -i switch. I had tried this earlier. With the -i '{}' argument added then this is the result:
# ll -d ./mqueue/offline drwxr-xr-x 2 root root 4096 Feb 8 14:03 ./mqueue/offline # # cat workmv | xargs -i '{}' mv ./mqueue/'{}' ./mqueue/offline xargs: {}: No such file or directory # # cat workmv | xargs -i '{}' mv ./mqueue/'{}' ./mqueue/offline/ xargs: {}: No such file or directory # # cat workmv | xargs -i '{}' mv --Target-Directory=./mqueue/offline ./mqueue/'{}' xargs: {}: No such file or directory # # cat workmv | xargs -i '{}' mv --Target-Directory=./mqueue/offline/ ./mqueue/'{}' xargs: {}: No such file or directory #
P.S. Also, thank you very much for the depository that you run. I use several of your packages.
Let me show you the manual entry again:
--replace[=replace-str], -I replace-str, -i[replace-str] Replace occurences of replace-str in the initial arguments with names read from standard input. Also, unquoted blanks do not terminate arguments. If replace-str is omitted, it defaults to "{}" (like for 'find -exec'). Implies -x and -L 1.
So you either do:
echo -e "file1\nfile2\nfile3" | xargs -i echo '{}' blah or echo -e "file1\nfile2\nfile3" | xargs -i echo '{}' blah/'{}' or echo -e "file1\nfile2\nfile3" | xargs -i'**' echo '**' blah or echo -e "file1\nfile2\nfile3" | xargs -I '**' echo '**' blah
This is correct:
xargs -i'{}' or xargs -I '{}'
This is not:
xargs -i '{}'
Compare this to the manual entry to verify. The distinction is necessary and important to allow for an '{}' default replace-str. Otherwise xargs wouldn't know whether the next argument is in fact the replace-str or the command to run !
-- dag wieers, dag@wieers.com, http://dag.wieers.com/ -- [all I want is a warm bed and a kind word and unlimited power]
On Wed, 9 Feb 2005, Dag Wieers wrote:
On Wed, 9 Feb 2005, James B. Byrne wrote:
On 8 Feb 2005 at 23:14, Dag Wieers wrote:
Compare this:
echo -e "file1\nfile2\nfile3" | xargs echo '{}' blah
with:
echo -e "file1\nfile2\nfile3" | xargs -i echo '{}' blah
Thank you for the advice on the -i switch. I had tried this earlier. With the -i '{}' argument added then this is the result:
# ll -d ./mqueue/offline drwxr-xr-x 2 root root 4096 Feb 8 14:03 ./mqueue/offline # # cat workmv | xargs -i '{}' mv ./mqueue/'{}' ./mqueue/offline xargs: {}: No such file or directory # # cat workmv | xargs -i '{}' mv ./mqueue/'{}' ./mqueue/offline/ xargs: {}: No such file or directory # # cat workmv | xargs -i '{}' mv --Target-Directory=./mqueue/offline ./mqueue/'{}' xargs: {}: No such file or directory # # cat workmv | xargs -i '{}' mv --Target-Directory=./mqueue/offline/ ./mqueue/'{}' xargs: {}: No such file or directory #
P.S. Also, thank you very much for the depository that you run. I use several of your packages.
Let me show you the manual entry again:
--replace[=replace-str], -I replace-str, -i[replace-str] Replace occurences of replace-str in the initial arguments with names read from standard input. Also, unquoted blanks do not terminate arguments. If replace-str is omitted, it defaults to "{}" (like for 'find -exec'). Implies -x and -L 1.So you either do:
echo -e "file1\nfile2\nfile3" | xargs -i echo '{}' blah or echo -e "file1\nfile2\nfile3" | xargs -i echo '{}' blah/'{}' or echo -e "file1\nfile2\nfile3" | xargs -i'**' echo '**' blah or echo -e "file1\nfile2\nfile3" | xargs -I '**' echo '**' blah
This is correct:
xargs -i'{}' or xargs -I '{}'
This is not:
xargs -i '{}'
Compare this to the manual entry to verify. The distinction is necessary and important to allow for an '{}' default replace-str. Otherwise xargs wouldn't know whether the next argument is in fact the replace-str or the command to run !
And apparently, on RHEL3 the -I option does not exist:
[dag@horsea dag]$ echo -e "file1\nfile2\nfile3" | xargs -I '**' echo '**' blah xargs: invalid option -- I Usage: xargs [-0prtx] [-e[eof-str]] [-i[replace-str]] [-l[max-lines]] [-n max-args] [-s max-chars] [-P max-procs] [--null] [--eof[=eof-str]] [--replace[=replace-str]] [--max-lines[=max-lines]] [--interactive] [--max-chars=max-chars] [--verbose] [--exit] [--max-procs=max-procs] [--max-args=max-args] [--no-run-if-empty] [--version] [--help] [command [initial-arguments]] Report bugs to bug-findutils@gnu.org.
While on FC3, it should exist, but doesn't work:
[root@emyn ~]# echo -e "file1\nfile2\nfile3" | xargs -I '**' echo '**' blah xargs: invalid option -- I Usage: xargs [-0prtx] [-E eof-str] [-e[eof-str]] [-I replace-str] [-i[replace-str]] [-L max-lines] [-l[max-lines]] [-n max-args] [-s max-chars] [-P max-procs] [--null] [--eof[=eof-str]] [--replace[=replace-str]] [--max-lines[=max-lines]] [--interactive] [--max-chars=max-chars] [--verbose] [--exit] [--max-procs=max-procs] [--max-args=max-args] [--no-run-if-empty] [--version] [--help] [command [initial-arguments]]
So you can forget about -I altogether on RHEL3. And I'll report is as a bug for FC3.
-- dag wieers, dag@wieers.com, http://dag.wieers.com/ -- [all I want is a warm bed and a kind word and unlimited power]
Thank you for that clarification.
Now what happens is this:
#cat workmv | xargs -i mv ./mqueue/'{}' ./mqueue/offline mv: cannot stat `./mqueue/*j17KeeL7025621': No such file or directory mv: cannot stat ... one for each entry in workmv
# # ll ./mqueue/*j17KeeL7025621 -rw------- 1 root root 19397 Feb 7 15:41 ./mqueue/dfj17KeeL7025621 -rw------- 1 root smmsp 1812 Feb 8 13:41 ./mqueue/qfj17KeeL7025621
Do I need to escape the leading * in the workfile?
Regards, Jim
-- *** e-mail is not a secure channel *** mailto:byrnejb.<token>@harte-lyne.ca James B. Byrne Harte & Lyne Limited vox: +1 905 561 1241 9 Brockley Drive fax: +1 905 561 0757 Hamilton, Ontario <token> = hal Canada L8E 3C3
On Wed, 9 Feb 2005, James B. Byrne wrote:
Thank you for that clarification.
Now what happens is this:
#cat workmv | xargs -i mv ./mqueue/'{}' ./mqueue/offline mv: cannot stat `./mqueue/*j17KeeL7025621': No such file or directory mv: cannot stat ... one for each entry in workmv
# # ll ./mqueue/*j17KeeL7025621 -rw------- 1 root root 19397 Feb 7 15:41 ./mqueue/dfj17KeeL7025621 -rw------- 1 root smmsp 1812 Feb 8 13:41 ./mqueue/qfj17KeeL7025621
Do I need to escape the leading * in the workfile?
Please do not add the * in workmv, but add it to the cmdline. Like:
xargs -i mv ./mqueue/*'{}' ./mqueue/offline
The quotes around {} escapes the wildcard, so it loses its special meaning. I would used ?? (or {df,qf}) instead of *, but it makes no big difference.
Kind regards, -- dag wieers, dag@wieers.com, http://dag.wieers.com/ -- [all I want is a warm bed and a kind word and unlimited power]
On 9 Feb 2005 at 19:59, Dag Wieers wrote:
Please do not add the * in workmv, but add it to the cmdline. Like:
xargs -i mv ./mqueue/*'{}' ./mqueue/offline
The quotes around {} escapes the wildcard, so it loses its special meaning. I would used ?? (or {df,qf}) instead of *, but it makes no big difference.
Ok, I cut out the leading '*' from the work file and used the following command:
# cat workmid | xargs -i mv ./mqueue/*'{}' ./mqueue/offline mv: cannot stat `./mqueue/*j18IrbIn002131': No such file or directory
#ll ./mqueue/*j18IrbIn002131 -rw------- 1 root root 1150 Feb 8 13:53 ./mqueue/dfj18IrbIn002131 -rw------- 1 root smmsp 1817 Feb 8 13:54 ./mqueue/qfj18IrbIn002131 #
Now what is wrong?
Regards, Jim
-- *** e-mail is not a secure channel *** mailto:byrnejb.<token>@harte-lyne.ca James B. Byrne Harte & Lyne Limited vox: +1 905 561 1241 9 Brockley Drive fax: +1 905 561 0757 Hamilton, Ontario <token> = hal Canada L8E 3C3
On Wed, 9 Feb 2005, James B. Byrne wrote:
On 9 Feb 2005 at 19:59, Dag Wieers wrote:
Please do not add the * in workmv, but add it to the cmdline. Like:
xargs -i mv ./mqueue/*'{}' ./mqueue/offline
The quotes around {} escapes the wildcard, so it loses its special meaning. I would used ?? (or {df,qf}) instead of *, but it makes no big difference.
Ok, I cut out the leading '*' from the work file and used the following command:
# cat workmid | xargs -i mv ./mqueue/*'{}' ./mqueue/offline mv: cannot stat `./mqueue/*j18IrbIn002131': No such file or directory
#ll ./mqueue/*j18IrbIn002131 -rw------- 1 root root 1150 Feb 8 13:53 ./mqueue/dfj18IrbIn002131 -rw------- 1 root smmsp 1817 Feb 8 13:54 ./mqueue/qfj18IrbIn002131 #
Now what is wrong?
Well, normally glob expansion is the shells work and I would have guessed (hoped) xargs would perform that for you, but it does not.
This is a good example why I usually don't write (even simple) scripts in bash. There are too many things you have to know and too many pitfalls and it often is more like trial-and-error programming :) There are other ways you can make it work like:
cat workmid | xargs -i mv ./mqueue/df'{}' ./mqueue/qf'{}' ./mqueue/offline
But either you understand what's going on and adapt to that when it's necessary or shell scripting is simply not your thing. It's not my thing.
-- dag wieers, dag@wieers.com, http://dag.wieers.com/ -- [all I want is a warm bed and a kind word and unlimited power]
It appears that xargs really, really does not like a wildcard in the command construction.
#cat workmid | xargs -i mv ./mqueue/df'{}' ./mqueue/offline #cat workmid | xargs -i mv ./mqueue/qf'{}' ./mqueue/offline
work and move the files, but neither of:
#cat workmid | xargs -i mv ./mqueue/*'{}' ./mqueue/offline #cat workmid | xargs -i mv ./mqueue/*'{}' ./mqueue/offline
work. Bizarre.
Thanks for all of the assistance. This was really helpful.
Regards, Jim
-- *** e-mail is not a secure channel *** mailto:byrnejb.<token>@harte-lyne.ca James B. Byrne Harte & Lyne Limited vox: +1 905 561 1241 9 Brockley Drive fax: +1 905 561 0757 Hamilton, Ontario <token> = hal Canada L8E 3C3
James B. Byrne wrote:
cat file4 | xargs mv /var/spool/mqueue/'{}' \ /var/spool/mqueue/offline
To provide an alternate way (thanks Dag, learned a new xargs switch :) ) I would instintively write this bash command as:
while read filename && [[ "$filename" != end ]] do mv -f "/var/spool/mqueue/$filename" /var/spool/mqueue/offline/ done < "/path/to/file4"
hth, -te
On 8 Feb 2005 at 15:15, James B. Byrne wrote:
I am trying to move a group of sendmail queue files into a special area and am developing s script to assist. The manual steps are:
This doesn't answer your question directly, and it's a bit late given the rest of the thread, but for future reference you might want to have a look at "qtool.pl" (included in the contrib/ directory with the Sendmail source).
It pretty much does everything you need, and has the advantage of using Sendmail's file locking method to avoid conflicts with a running sendmail queue-running process.
Something like the following will do what you need:
#!/bin/bash OLQUEUE=/var/spool/mqueue/offline QUEUE=/var/spool/mqueue qtool.pl -e '$msg{recipient} ~= "<domain>"' $OLQUEUE $QUEUE
---- Nels Lindquist <*> Information Systems Manager Morningstar Air Express Inc.
On 10 Feb 2005 at 11:37, Nels Lindquist wrote:
This doesn't answer your question directly, and it's a bit late given the rest of the thread, but for future reference you might want to have a look at "qtool.pl" (included in the contrib/ directory with the Sendmail source).
That is exactly what I was trying to find before I gave up and decided to roll my own. Now the odd thing is, I googled 'sendmail queue manager script' and similar constructs and came up empty on anything like this. As I do not have the sendmail src packages installed I will see if I can locate this tool on the web somewhere.
Thank you very much for the info. This will make future occurrences much easier to deal with. I find it remarkable that this tool is not mentioned in the Bat book however.
Regards, Jim -- *** e-mail is not a secure channel *** mailto:byrnejb.<token>@harte-lyne.ca James B. Byrne Harte & Lyne Limited vox: +1 905 561 1241 9 Brockley Drive fax: +1 905 561 0757 Hamilton, Ontario <token> = hal Canada L8E 3C3
-
Dag Wieers -
James B. Byrne -
Nels Lindquist -
Troy Engel