I have two CentOS5 systems server1 and server2. There is user peter on server1 who can ssh to server2 using public ssh keys and no password is needed.
What I noticed is that running remote ssh commands in bash script breaks while loops.
====== #!/bin/sh for i in server2 server2; do echo "--> Start" ssh peter@$i ls echo "--> END" done
echo " server2 server2" | \ while read confLine; do echo "--> $confLine" ssh peter@$confLine ls echo "--> END $confLine" done ====
The "for" loop in the script above will run twice but the "while" loop below it will run only once.
This is very simple to test and I've tried it on different systems including CentOS6 and OpenSolaris with the same result.
Any idea what would cause the ssh command to break the while loop?
Thanks Peter
Am 01.02.2012 22:07, schrieb Peter Blajev:
I have two CentOS5 systems server1 and server2. There is user peter on server1 who can ssh to server2 using public ssh keys and no password is needed.
What I noticed is that running remote ssh commands in bash script breaks while loops.
====== #!/bin/sh for i in server2 server2; do echo "--> Start" ssh peter@$i ls echo "--> END" done
echo " server2 server2" | \ while read confLine; do echo "--> $confLine" ssh peter@$confLine ls echo "--> END $confLine" done ====
The "for" loop in the script above will run twice but the "while" loop below it will run only once.
This is very simple to test and I've tried it on different systems including CentOS6 and OpenSolaris with the same result.
Any idea what would cause the ssh command to break the while loop?
Thanks Peter
That has simply nothing to do with SSH. Compare following:
echo "foo bar" | while read LINE; do echo $LINE; done
and
echo -e "foo\nbar" | while read $LINE; do echo $LINE; done
Alexander
On Wed, Feb 1, 2012 at 3:46 PM, Alexander Dalloz ad+lists@uni-x.org wrote:
That has simply nothing to do with SSH. Compare following:
echo "foo bar" | while read LINE; do echo $LINE; done
and
echo -e "foo\nbar" | while read $LINE; do echo $LINE; done
No, (a) that read $LINE should be read LINE and (b) echo "foo bar" | something should preserve the quoted newline. Ssh does seem to be consuming stuff from the inherited piped stdin even though it isn't obvious what it does with it in a non-interactive scenario. But anything that reads stdin inside the loop would cause that (throw a 'cat >dev/null' in...).
On Wed, Feb 01, 2012 at 01:07:31PM -0800, Peter Blajev wrote:
echo " server2 server2" | \ while read confLine; do echo "--> $confLine" ssh peter@$confLine ls echo "--> END $confLine" done
The "for" loop in the script above will run twice but the "while" loop below it will run only once.
Any idea what would cause the ssh command to break the while loop?
"ssh" is reading from stdin and passing the data over to the remote machine. You can test this with ssh peter@$confLine 'read x ; echo we got $x'
To stop it doing this, use the "-n" flag ssh -n peter@$confLine ls
On Wed, Feb 1, 2012 at 2:53 PM, Stephen Harris lists@spuddy.org wrote:
On Wed, Feb 01, 2012 at 01:07:31PM -0800, Peter Blajev wrote:
echo " server2 server2" | \ while read confLine; do echo "--> $confLine" ssh peter@$confLine ls echo "--> END $confLine" done
The "for" loop in the script above will run twice but the "while" loop below it will run only once.
Any idea what would cause the ssh command to break the while loop?
"ssh" is reading from stdin and passing the data over to the remote machine. You can test this with ssh peter@$confLine 'read x ; echo we got $x'
To stop it doing this, use the "-n" flag ssh -n peter@$confLine ls
This is it. Right on Stephen. Thank you very much. I can't believe I've gone so long without knowing it.
This works for me. I still don't have full understanding of it but I'll do some more reading.
Unfortunately I can't always use the (-n) option. If I wan't to send data through the pipe then the (-n) won't work. For example (on top of my head): mysqldump dB | ssh peter@remoteServer "mysql dB"
In my script I ended up using "ssh -n" when I want to work on the output of remotely ran command and "ssh" without (-n) when I want to send data over ssh to a remote command.
This so far is not breaking the while loop and it seems to be working but it makes me nervous.
Any note will be appreciated.
Thanks again.
-- Peter
On Wed, Feb 01, 2012 at 07:03:33PM -0800, Peter Blajev wrote:
On Wed, Feb 1, 2012 at 2:53 PM, Stephen Harris lists@spuddy.org wrote:
On Wed, Feb 01, 2012 at 01:07:31PM -0800, Peter Blajev wrote:
echo " server2 server2" | \ while read confLine; do echo "--> $confLine" ssh peter@$confLine ls echo "--> END $confLine" done
Any idea what would cause the ssh command to break the while loop?
"ssh" is reading from stdin and passing the data over to the remote machine. You can test this with ssh peter@$confLine 'read x ; echo we got $x'
To stop it doing this, use the "-n" flag ssh -n peter@$confLine ls
Unfortunately I can't always use the (-n) option. If I wan't to send data through the pipe then the (-n) won't work. For example (on top of my head): mysqldump dB | ssh peter@remoteServer "mysql dB"
In this situation, ssh will read from the pipe and not from the "echo" statement and so it won't break your while loop.
The thing you need to understand is how redirection works. echo foo | while read do ... done
Everything from the "while" to the "done" will have stdin configured to read from the pipe (the output of "echo"). Here's an example: % echo "a b c d" | while read a do echo We have $a read b echo And $b done We have a And b We have c And d
You can see that the "read" statement inside the loop is reading from the pipe and so draining input.
An "ssh" on its own, in this situation, would drain _all_ the input. The "-n" flag tells ssh to not do this. But, equally, when you do sqldump | ssh then you've told ssh to take input from the output of sqldump, and so it's no longer reading the from outer loop.
Instead of "ssh -n" you could do "ssh < /dev/null" or "echo | ssh" or other options, and get the same sort of effect; you're setting up the stdin to ssh to be from somewhere _other_ than your main echo.
On Wed, Feb 1, 2012 at 7:13 PM, Stephen Harris lists@spuddy.org wrote:
On Wed, Feb 01, 2012 at 07:03:33PM -0800, Peter Blajev wrote:
On Wed, Feb 1, 2012 at 2:53 PM, Stephen Harris lists@spuddy.org wrote:
On Wed, Feb 01, 2012 at 01:07:31PM -0800, Peter Blajev wrote:
echo " server2 server2" | \ while read confLine; do echo "--> $confLine" ssh peter@$confLine ls echo "--> END $confLine" done
Any idea what would cause the ssh command to break the while loop?
"ssh" is reading from stdin and passing the data over to the remote machine. You can test this with ssh peter@$confLine 'read x ; echo we got $x'
To stop it doing this, use the "-n" flag ssh -n peter@$confLine ls
Unfortunately I can't always use the (-n) option. If I wan't to send data through the pipe then the (-n) won't work. For example (on top of my
head):
mysqldump dB | ssh peter@remoteServer "mysql dB"
In this situation, ssh will read from the pipe and not from the "echo" statement and so it won't break your while loop.
The thing you need to understand is how redirection works. echo foo | while read do ... done
Everything from the "while" to the "done" will have stdin configured to read from the pipe (the output of "echo"). Here's an example: % echo "a b c d" | while read a do echo We have $a read b echo And $b done We have a And b We have c And d
You can see that the "read" statement inside the loop is reading from the pipe and so draining input.
An "ssh" on its own, in this situation, would drain _all_ the input. The "-n" flag tells ssh to not do this. But, equally, when you do sqldump | ssh then you've told ssh to take input from the output of sqldump, and so it's no longer reading the from outer loop.
Instead of "ssh -n" you could do "ssh < /dev/null" or "echo | ssh" or other options, and get the same sort of effect; you're setting up the stdin to ssh to be from somewhere _other_ than your main echo.
Wow. Great lesson. Thank you Stephen. It makes more sense now.
I'll save this email for sure.
-- Peter
On Wednesday 01 February 2012 19:03:33 Peter Blajev wrote:
On Wed, Feb 1, 2012 at 2:53 PM, Stephen Harris lists@spuddy.org wrote:
On Wed, Feb 01, 2012 at 01:07:31PM -0800, Peter Blajev wrote:
echo " server2 server2" | \ while read confLine; do echo "--> $confLine" ssh peter@$confLine ls echo "--> END $confLine" done
The "for" loop in the script above will run twice but the "while" loop below it will run only once.
Any idea what would cause the ssh command to break the while loop?
"ssh" is reading from stdin and passing the data over to the remote machine. You can test this with ssh peter@$confLine 'read x ; echo we got $x'
To stop it doing this, use the "-n" flag ssh -n peter@$confLine ls
This is it. Right on Stephen. Thank you very much. I can't believe I've gone so long without knowing it.
This works for me. I still don't have full understanding of it but I'll do some more reading.
Unfortunately I can't always use the (-n) option. If I wan't to send data through the pipe then the (-n) won't work. For example (on top of my head): mysqldump dB | ssh peter@remoteServer "mysql dB"
In my script I ended up using "ssh -n" when I want to work on the output of remotely ran command and "ssh" without (-n) when I want to send data over ssh to a remote command.
This so far is not breaking the while loop and it seems to be working but it makes me nervous.
Curious I found this very same problem some days ago and wrote about it in my blog hehe :)
All the options you have described in the thread will usually work but I suggest you to think about another option. Make the while read from another file descriptor as in:
while read <&3 line; do ...... done 3< file
And why this? because not only ssh reads from standard input but others like ffmpeg do as well(and probably more...)
Regards,
Marc Deop
-
Alexander Dalloz -
Les Mikesell -
Marc Deop -
Peter Blajev -
Stephen Harris