[CentOS] Pipes (fifos) not working in concurrently

Sat Nov 26 13:56:52 UTC 2011
Timothy Madden <terminatorul at gmail.com>

On 26.11.2011 07:41, Bart Schaefer wrote:
> On Fri, Nov 25, 2011 at 6:34 PM, Bart Schaefer
> <barton.schaefer at gmail.com>  wrote:
>> Next you create wget #2, which (because it was forked from the parent
>> shell) shares all the file descriptors that the shell had open to wget
>> #1, e.g., including the input to the fifo.  Repeat for all the rest of
>> the wget.  By the time you have created the last one, each of them has
>> a set of descriptors shared with every other that was created ahead of
>> them.
>> Thus, even though you write to the fifo for wget #2 and close it from
>> the parent shell, it doesn't actually see EOF and begin processing the
>> input until the corresponding descriptor shared by wget #1 is closed
>> when wget #1 exits.
> I wrote that backwards.  Actually I think the *last* one (#20) exits
> first, and then #19, and so on down to #1 ... but the descriptor
> management issue is the same.

Wow ! You guys are so great !

That is exactly how my script behaves. The POSIX (actually  SuS) text on 
the matter said that the implementation may or may not inherit the 
descriptors in the forked child processes, and that a portable script 
should close the descriptors to be sure they are not inherited. Somehow 
I did not pay to much attention to the issue and I just assumed I need 
not worry about it.

But you are right, in my loop every new wget process inherits the file 
descriptors opened by the script for the previous childs (for their 
pipes, to be accurate), and as such keeps the files open even if my 
script later closes its copies of the descriptors. And I though I am the 
toughest programmer ever ... !

The solution was simple, no file descriptor wrangling: just use two 
loops: one to start all the wget processes and connect them to the 
pipes, and the other to open all those fd's. In this way the wgets have 
nothing special to inherit.

Sorry about the wrong group. After asking in a Unix shell group without 
much success, I suspected it must be CentOS doing something strange with 
the FIFOs at the system level.

Thank you,
Timothy Madden