[CentOS] Socket behavior change from 6.5 to 6.6

[I wish I knew how to get the mailing list to thread my replies properly in the
archives; I subscribe to the daily digest, and replying to that doesn't do it.]

Greg Lindahl wrote:
> On Thu, Jan 15, 2015 at 03:40:08PM -0300, Glenn Eychaner wrote:
> 
> > My only theory is that this has something to do with non-ephemeral ports and
> > socket reuse, but I'm not sure what.
> 
> If you want a quick detection that the link is dead, have the server
> occasionally send bytes to the dos box. You will get an immediate
> error if the dos box is up and knows that connection is kaput.

What if I am sending bytes to the DOS box, but it never reads the socket?
(Let us assume, for the sake of argument, that I can't change the DOS box
software. In fact, I can, but it's more difficult than changing the Linux end.)
Won't that either result in my detecting the socket as "dead" when it is not,
or eventually overflowing the socket buffering?

> Given that the port numbers of the new connection are the same, I'm
> kind of surprised that the behavior changed from 6.5 to 6.6, but, I
> always use defensive programming (sending those extra bytes).

I was super-surprised by the change, in that I fully tested the upgrade on
my simulator system before deploying, and still got bit on deployment.
Of course, the simulator doesn't have a real DOS box, just a simulation
process that sends the images. [And, I also recently got bit by this
http://www.macstadium.com/blog/osx-10-9-mavericks-bugs/
after upgrading some Macs. Sigh, network issues.]

Alex from Germany wrote:
> Since you always use the same local port -
> maybe you need to set SO_REUSEADDR option.

I assume I would have to set that on the client (DOS) side (the box which is
using the same local port 1025 each time); setting it on the bound-listener
socket on the Linux side doesn't seem like it would do anything to resolve
the issue, based on my reading of SO_REUSEADDR on the net:
http://www.unixguide.net/network/socketfaq/4.5.shtml
http://stackoverflow.com/questions/14388706/

-G.
--
Glenn Eychaner (geychaner at lco.cl)
Telescope Systems Programmer, Las Campanas Observatory