[CentOS] Could not complete SSL handshake to Amazon EC2 host

Fri May 1 16:45:24 UTC 2015
Tim Dunphy <bluethundr at gmail.com>

>
> Oh my mistake. I mean nrpe without parameters. It should say something
> about SSL/TLS aktiv or so.
> You could test nrpe without SSL. Use nrpe -n - H host



This is what I see about ssl if I just run nrpe on the client without any
flags:

[root at ops:~] #nrpe| head -8

NRPE - Nagios Remote Plugin Executor
Copyright (c) 1999-2008 Ethan Galstad (nagios at nagios.org)
Version: 2.15
Last Modified: 09-06-2013
License: GPL v2 with exemptions (-l for more info)
SSL/TLS Available: Anonymous DH Mode, OpenSSL 0.9.6 or higher required
TCP Wrappers Available

And if I go back to the monitoring host and try to run nrpe with the -n
flag, this is what I get:

[root at monitor1:~] #/usr/local/nagios/libexec/check_nrpe -n -H
ops.jokefire.com
*CHECK_NRPE: Error receiving data from daemon.*

And still getting the SSL error without the -n flag:

[root at monitor1:~] #/usr/local/nagios/libexec/check_nrpe -H ops.jokefire.com
*CHECK_NRPE: Error - Could not complete SSL handshake.*

Running nmap from the monitor host I can see that the nrpe port is open:

[root at monitor1:~] #nmap -p 5666 ops.jokefire.com

Starting Nmap 6.40 ( http://nmap.org ) at 2015-05-01 12:38 EDT
Nmap scan report for ops.jokefire.com (54.225.218.125)
Host is up (0.011s latency).
rDNS record for 54.225.218.125: ec2-54-225-218-125.compute-1.amazonaws.com
PORT     STATE SERVICE
*5666/tcp open  nrpe*

Nmap done: 1 IP address (1 host up) scanned in 0.14 seconds

Yet if I try telnetting to it, it connects, then closes the connection
immediately:

[root at monitor1:~] #telnet ops.jokefire.com 5666
Trying 54.225.218.125...
*Connected to ops.jokefire.com <http://ops.jokefire.com>.*
Escape character is '^]'.
*Connection closed by foreign host.*

Going back to the ops host that I want to monitor, I can verify that the
port is listening:

[root at ops:~] #lsof -i :5666
COMMAND  PID USER   FD   TYPE DEVICE SIZE NODE NAME
xinetd  1434 root    5u  IPv4   4063       TCP *:nrpe (LISTEN)


And I can verify that the nrpe conf is owned by the nagios user and group:

[root at ops:~] #ls -l /usr/local/nagios/etc/nrpe.cfg
-rw-r--r-- 1 nagios nagios 7988 May  1 00:37 /usr/local/nagios/etc/nrpe.cfg

I think that covers all your suggestions. Except for Eero's suggestion to
try running nrpe without xinetd. I can try to get to that later, but I may
not have time for that suggestion today. But as I demonstrate above, the
problem is not that nrpe isn't listening.

This remains a really odd situation. Does anyone else have any clues?

Thanks,
Tim



On Fri, May 1, 2015 at 7:43 AM, Eric Lehmann <e.lehmann88 at gmail.com> wrote:

> Oh my mistake. I mean nrpe without parameters. It should say something
> about SSL/TLS aktiv or so.
> You could test nrpe without SSL. Use nrpe -n - H host
> Am 01.05.2015 13:18 schrieb "Eero Volotinen" <eero.volotinen at iki.fi>:
>
> > well. how about trying default setting and running nrped without xinetd.
> >
> > --
> > Eero
> >
> > 2015-05-01 14:14 GMT+03:00 Tim Dunphy <bluethundr at gmail.com>:
> >
> > > > This is strange...
> > > > Do you have SSL aktive on both systems? Run nrpr localy without
> > > parameters
> > > > (this should return some nrpe stats) and check ldd for libssl.
> > >
> > >
> > > I don't seem to have that command.
> > >
> > >
> > > [root at monitor1:~] #find / -name "*nrpr" 2> /dev/null
> > > [root at monitor1:~] #
> > >
> > > And that's on either system.
> > >
> > >  And if I do an ldd on both, this is what I can tell:
> > >
> > > Server:
> > >
> > > [root at monitor1:~] #ldd /usr/local/nagios/libexec/check_nrpe
> > >         linux-vdso.so.1 =>  (0x00007fffd895d000)
> > >        * libssl.so.10 => /lib64/libssl.so.10 (0x00007fc61722a000)*
> > > *        libcrypto.so.10 => /lib64/libcrypto.so.10
> (0x00007fc616e43000)*
> > >         libnsl.so.1 => /lib64/libnsl.so.1 (0x00007fc616c29000)
> > >         libc.so.6 => /lib64/libc.so.6 (0x00007fc616868000)
> > >         libgssapi_krb5.so.2 => /lib64/libgssapi_krb5.so.2
> > > (0x00007fc61661c000)
> > >         libkrb5.so.3 => /lib64/libkrb5.so.3 (0x00007fc616338000)
> > >         libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00007fc616134000)
> > >         libk5crypto.so.3 => /lib64/libk5crypto.so.3
> (0x00007fc615f02000)
> > >         libdl.so.2 => /lib64/libdl.so.2 (0x00007fc615cfd000)
> > >         libz.so.1 => /lib64/libz.so.1 (0x00007fc615ae7000)
> > >         /lib64/ld-linux-x86-64.so.2 (0x00007fc6174a0000)
> > >         libkrb5support.so.0 => /lib64/libkrb5support.so.0
> > > (0x00007fc6158d8000)
> > >         libkeyutils.so.1 => /lib64/libkeyutils.so.1
> (0x00007fc6156d3000)
> > >         libresolv.so.2 => /lib64/libresolv.so.2 (0x00007fc6154b9000)
> > >         libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fc61529d000)
> > >         libselinux.so.1 => /lib64/libselinux.so.1 (0x00007fc615077000)
> > >         libpcre.so.1 => /lib64/libpcre.so.1 (0x00007fc614e16000)
> > >         liblzma.so.5 => /lib64/liblzma.so.5 (0x00007fc614bf1000)
> > >
> > >
> > > Client:
> > >
> > > [root at ops:~] #ldd /usr/local/nagios/libexec/check_nrpe
> > >        * libssl.so.6 => /lib64/libssl.so.6 (0x00002aaaaaaba000)*
> > > *        libcrypto.so.6 => /lib64/libcrypto.so.6 (0x00002aaaaad08000)*
> > >         libnsl.so.1 => /lib64/libnsl.so.1 (0x00002aaaab05a000)
> > >         libc.so.6 => /lib64/libc.so.6 (0x00002aaaab273000)
> > >         libgssapi_krb5.so.2 => /usr/lib64/libgssapi_krb5.so.2
> > > (0x00002aaaab5cc000)
> > >         libkrb5.so.3 => /usr/lib64/libkrb5.so.3 (0x00002aaaab7fa000)
> > >         libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00002aaaaba90000)
> > >         libk5crypto.so.3 => /usr/lib64/libk5crypto.so.3
> > > (0x00002aaaabc92000)
> > >         libdl.so.2 => /lib64/libdl.so.2 (0x00002aaaabeb7000)
> > >         libz.so.1 => /lib64/libz.so.1 (0x00002aaaac0bc000)
> > >         /lib64/ld-linux-x86-64.so.2 (0x0000555555554000)
> > >         libkrb5support.so.0 => /usr/lib64/libkrb5support.so.0 (0x0
> > > 0002aaaac2d0000)
> > >         libkeyutils.so.1 => /lib64/libkeyutils.so.1
> (0x00002aaaac4d8000)
> > >         libresolv.so.2 => /lib64/libresolv.so.2 (0x00002aaaac6db000)
> > >         libselinux.so.1 => /lib64/libselinux.so.1 (0x00002aaaac8f0000)
> > >         libsepol.so.1 => /lib64/libsepol.so.1 (0x00002aaaacb09000)
> > >
> > >
> > > So it looks like everything is OK from the SSL end of things. Any other
> > > ideas or suggestions?
> > >
> > > Thanks
> > > Tim
> > >
> > > On Fri, May 1, 2015 at 5:46 AM, Eric Lehmann <e.lehmann88 at gmail.com>
> > > wrote:
> > >
> > > > This is strange...
> > > > Do you have SSL aktive on both systems? Run nrpr localy without
> > > parameters
> > > > (this should return some nrpe stats) and check ldd for libssl.
> > > > Am 01.05.2015 07:32 schrieb "Tim Dunphy" <bluethundr at gmail.com>:
> > > >
> > > > > Hi Eric,
> > > > >
> > > > >  Thanks for your reply. I do have nrpe running under xinetd on the
> > host
> > > > I'm
> > > > > trying to monitor.
> > > > >
> > > > >  And running the nrpe checl locally:
> > > > >
> > > > > [root at ops:~] #/usr/local/nagios/libexec/check_nrpe -H localhost
> > > > > NRPE v2.15
> > > > >
> > > > > [root at ops:~] #grep only_from /etc/xinetd.d/nrpe
> > > > >         only_from       = 127.0.0.1 216.120.248.126
> > > > >
> > > > > And I do have port 5666 open on the security group for this host.
> > > > >
> > > > > And I made sure the local firewall was stopped, because I am
> blocking
> > > > ports
> > > > > with the security groups instead.
> > > > >
> > > > > [root at ops:~] #service iptables status
> > > > > Firewall is stopped.
> > > > >
> > > > > It's only when checking from the monitoring host that nrpe fails:
> > > > >
> > > > > [root at monitor1:~] #/usr/local/nagios/libexec/check_nrpe -H
> > > > > ops.jokefire.com
> > > > > CHECK_NRPE: Error - Could not complete SSL handshake.
> > > > >
> > > > > Really, really puzzling. This is driving me up a wall!! I hopeI can
> > > solve
> > > > > this soon....
> > > > >
> > > > > Thanks for any and all help with this one!!
> > > > > Tim
> > > > >
> > > > > On Fri, May 1, 2015 at 1:02 AM, Eric Lehmann <
> e.lehmann88 at gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Hi
> > > > > > Does the deamon run under xinetd? Then  you have to configure the
> > > > > only_from
> > > > > > in  */etc/**xinetd.d**/**nrpe* to.
> > > > > >
> > > > > > Regards
> > > > > > Eric
> > > > > > Am 01.05.2015 06:46 schrieb "Tim Dunphy" <bluethundr at gmail.com>:
> > > > > >
> > > > > > > Hello,
> > > > > > >
> > > > > > >  I am trying to monitor a host in the Amazon EC2 cloud.
> > > > > > >
> > > > > > > Yet when I try to check NRPE from the monitoring host I am
> > getting
> > > an
> > > > > SSL
> > > > > > > handshake error:
> > > > > > >
> > > > > > > [root at monitor1:~] #/usr/local/nagios/libexec/check_nrpe -H
> > > > > > > ops.jokefire.com
> > > > > > > CHECK_NRPE: Error - Could not complete SSL handshake.
> > > > > > >
> > > > > > > And if I telnet into the host on port 5666 to see if the FW
> port
> > is
> > > > > open,
> > > > > > > the connection closes right away:
> > > > > > >
> > > > > > > [root at monitor1:~] #telnet ops.somewhere.com 5666
> > > > > > > Trying 54.225.218.125...
> > > > > > > Connected to ops.somewhere.com.
> > > > > > > Escape character is '^]'.
> > > > > > > Connection closed by foreign host.
> > > > > > >
> > > > > > > You can see there it connects, but then it closes immediately
> > after
> > > > the
> > > > > > > connection.
> > > > > > >
> > > > > > >  I have NRPE running on the host I want to monitor:
> > > > > > >
> > > > > > > [root at ops:~] #lsof -i :5666
> > > > > > > COMMAND  PID USER   FD   TYPE DEVICE SIZE NODE NAME
> > > > > > > xinetd  1434 root    5u  IPv4   4063       TCP *:nrpe (LISTEN)
> > > > > > >
> > > > > > > And I have the IP of my nagios server listed in the xinetd conf
> > > file:
> > > > > > >
> > > > > > > [root at ops:~] #cat /etc/xinetd.d/nrpe
> > > > > > > # default: on
> > > > > > > # description: NRPE (Nagios Remote Plugin Executor)
> > > > > > > service nrpe
> > > > > > > {
> > > > > > >         flags           = REUSE
> > > > > > >         socket_type     = stream
> > > > > > >         port            = 5666
> > > > > > >         wait            = no
> > > > > > >         user            = nagios
> > > > > > >         group           = nagios
> > > > > > >         server          = /usr/local/nagios/bin/nrpe
> > > > > > >         server_args     = -c /usr/local/nagios/etc/nrpe.cfg
> > --inetd
> > > > > > >         log_on_failure  += USERID
> > > > > > >         disable         = no
> > > > > > >         only_from       = 127.0.0.1 xx.xx.xx.xx   # <-
> > representing
> > > > my
> > > > > > real
> > > > > > > nagios server IP
> > > > > > > }
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > And I have my default security group for that host open on port
> > > 5666
> > > > to
> > > > > > the
> > > > > > > world for this experiment.  I plan on locking that down again
> to
> > > the
> > > > > > single
> > > > > > > IP of my monitoring host once I get this resolved.
> > > > > > >
> > > > > > > Does anyone have any suggestions on how I can get that problem
> > > > solved?
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Tim
> > > > > > >
> > > > > > > --
> > > > > > > GPG me!!
> > > > > > >
> > > > > > > gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B
> > > > > > > _______________________________________________
> > > > > > > CentOS mailing list
> > > > > > > CentOS at centos.org
> > > > > > > http://lists.centos.org/mailman/listinfo/centos
> > > > > > >
> > > > > > _______________________________________________
> > > > > > CentOS mailing list
> > > > > > CentOS at centos.org
> > > > > > http://lists.centos.org/mailman/listinfo/centos
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > GPG me!!
> > > > >
> > > > > gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B
> > > > > _______________________________________________
> > > > > CentOS mailing list
> > > > > CentOS at centos.org
> > > > > http://lists.centos.org/mailman/listinfo/centos
> > > > >
> > > > _______________________________________________
> > > > CentOS mailing list
> > > > CentOS at centos.org
> > > > http://lists.centos.org/mailman/listinfo/centos
> > > >
> > >
> > >
> > >
> > > --
> > > GPG me!!
> > >
> > > gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B
> > > _______________________________________________
> > > CentOS mailing list
> > > CentOS at centos.org
> > > http://lists.centos.org/mailman/listinfo/centos
> > >
> > _______________________________________________
> > CentOS mailing list
> > CentOS at centos.org
> > http://lists.centos.org/mailman/listinfo/centos
> >
> _______________________________________________
> CentOS mailing list
> CentOS at centos.org
> http://lists.centos.org/mailman/listinfo/centos
>



-- 
GPG me!!

gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B