[CentOS] Could not complete SSL handshake to Amazon EC2 host

Fri May 1 17:31:11 UTC 2015
Tim Dunphy <bluethundr at gmail.com>

Hi Eric,


> NRPE: Error receiving data from daemon
> Seems as this is not a SSL Problem. Do you have a nagios user account? Cat
> /etc/passwd




Yep! Both hosts have nagios user accounts.


Demonstrating from the client:

[root at ops:~] #id nagios
uid=2002(nagios) gid=2002(nagios) groups=2002(nagios),2008(nagioscmd)


And this is from the monitoring server:

[root at monitor1:~] #id nagios
uid=1001(nagios) gid=1001(nagios) groups=1001(nagios),1002(nagcmd)

I do notice a slight difference in the user id and group id numbers.  But I
don't think that could be causing any issue. Does anyone else disagree?

I might want to standardize user accounts at some point howver.

Thanks!
Tim


On Fri, May 1, 2015 at 1:03 PM, Eric Lehmann <e.lehmann88 at gmail.com> wrote:

> Hi
>
> NRPE: Error receiving data from daemon
>
> Seems as this is not a SSL Problem. Do you have a nagios user account? Cat
> /etc/passwd
> Am 01.05.2015 18:45 schrieb "Tim Dunphy" <bluethundr at gmail.com>:
>
> > >
> > > Oh my mistake. I mean nrpe without parameters. It should say something
> > > about SSL/TLS aktiv or so.
> > > You could test nrpe without SSL. Use nrpe -n - H host
> >
> >
> >
> > This is what I see about ssl if I just run nrpe on the client without any
> > flags:
> >
> > [root at ops:~] #nrpe| head -8
> >
> > NRPE - Nagios Remote Plugin Executor
> > Copyright (c) 1999-2008 Ethan Galstad (nagios at nagios.org)
> > Version: 2.15
> > Last Modified: 09-06-2013
> > License: GPL v2 with exemptions (-l for more info)
> > SSL/TLS Available: Anonymous DH Mode, OpenSSL 0.9.6 or higher required
> > TCP Wrappers Available
> >
> > And if I go back to the monitoring host and try to run nrpe with the -n
> > flag, this is what I get:
> >
> > [root at monitor1:~] #/usr/local/nagios/libexec/check_nrpe -n -H
> > ops.jokefire.com
> > *CHECK_NRPE: Error receiving data from daemon.*
> >
> > And still getting the SSL error without the -n flag:
> >
> > [root at monitor1:~] #/usr/local/nagios/libexec/check_nrpe -H
> > ops.jokefire.com
> > *CHECK_NRPE: Error - Could not complete SSL handshake.*
> >
> > Running nmap from the monitor host I can see that the nrpe port is open:
> >
> > [root at monitor1:~] #nmap -p 5666 ops.jokefire.com
> >
> > Starting Nmap 6.40 ( http://nmap.org ) at 2015-05-01 12:38 EDT
> > Nmap scan report for ops.jokefire.com (54.225.218.125)
> > Host is up (0.011s latency).
> > rDNS record for 54.225.218.125:
> ec2-54-225-218-125.compute-1.amazonaws.com
> > PORT     STATE SERVICE
> > *5666/tcp open  nrpe*
> >
> > Nmap done: 1 IP address (1 host up) scanned in 0.14 seconds
> >
> > Yet if I try telnetting to it, it connects, then closes the connection
> > immediately:
> >
> > [root at monitor1:~] #telnet ops.jokefire.com 5666
> > Trying 54.225.218.125...
> > *Connected to ops.jokefire.com <http://ops.jokefire.com>.*
> > Escape character is '^]'.
> > *Connection closed by foreign host.*
> >
> > Going back to the ops host that I want to monitor, I can verify that the
> > port is listening:
> >
> > [root at ops:~] #lsof -i :5666
> > COMMAND  PID USER   FD   TYPE DEVICE SIZE NODE NAME
> > xinetd  1434 root    5u  IPv4   4063       TCP *:nrpe (LISTEN)
> >
> >
> > And I can verify that the nrpe conf is owned by the nagios user and
> group:
> >
> > [root at ops:~] #ls -l /usr/local/nagios/etc/nrpe.cfg
> > -rw-r--r-- 1 nagios nagios 7988 May  1 00:37
> /usr/local/nagios/etc/nrpe.cfg
> >
> > I think that covers all your suggestions. Except for Eero's suggestion to
> > try running nrpe without xinetd. I can try to get to that later, but I
> may
> > not have time for that suggestion today. But as I demonstrate above, the
> > problem is not that nrpe isn't listening.
> >
> > This remains a really odd situation. Does anyone else have any clues?
> >
> > Thanks,
> > Tim
> >
> >
> >
> > On Fri, May 1, 2015 at 7:43 AM, Eric Lehmann <e.lehmann88 at gmail.com>
> > wrote:
> >
> > > Oh my mistake. I mean nrpe without parameters. It should say something
> > > about SSL/TLS aktiv or so.
> > > You could test nrpe without SSL. Use nrpe -n - H host
> > > Am 01.05.2015 13:18 schrieb "Eero Volotinen" <eero.volotinen at iki.fi>:
> > >
> > > > well. how about trying default setting and running nrped without
> > xinetd.
> > > >
> > > > --
> > > > Eero
> > > >
> > > > 2015-05-01 14:14 GMT+03:00 Tim Dunphy <bluethundr at gmail.com>:
> > > >
> > > > > > This is strange...
> > > > > > Do you have SSL aktive on both systems? Run nrpr localy without
> > > > > parameters
> > > > > > (this should return some nrpe stats) and check ldd for libssl.
> > > > >
> > > > >
> > > > > I don't seem to have that command.
> > > > >
> > > > >
> > > > > [root at monitor1:~] #find / -name "*nrpr" 2> /dev/null
> > > > > [root at monitor1:~] #
> > > > >
> > > > > And that's on either system.
> > > > >
> > > > >  And if I do an ldd on both, this is what I can tell:
> > > > >
> > > > > Server:
> > > > >
> > > > > [root at monitor1:~] #ldd /usr/local/nagios/libexec/check_nrpe
> > > > >         linux-vdso.so.1 =>  (0x00007fffd895d000)
> > > > >        * libssl.so.10 => /lib64/libssl.so.10 (0x00007fc61722a000)*
> > > > > *        libcrypto.so.10 => /lib64/libcrypto.so.10
> > > (0x00007fc616e43000)*
> > > > >         libnsl.so.1 => /lib64/libnsl.so.1 (0x00007fc616c29000)
> > > > >         libc.so.6 => /lib64/libc.so.6 (0x00007fc616868000)
> > > > >         libgssapi_krb5.so.2 => /lib64/libgssapi_krb5.so.2
> > > > > (0x00007fc61661c000)
> > > > >         libkrb5.so.3 => /lib64/libkrb5.so.3 (0x00007fc616338000)
> > > > >         libcom_err.so.2 => /lib64/libcom_err.so.2
> > (0x00007fc616134000)
> > > > >         libk5crypto.so.3 => /lib64/libk5crypto.so.3
> > > (0x00007fc615f02000)
> > > > >         libdl.so.2 => /lib64/libdl.so.2 (0x00007fc615cfd000)
> > > > >         libz.so.1 => /lib64/libz.so.1 (0x00007fc615ae7000)
> > > > >         /lib64/ld-linux-x86-64.so.2 (0x00007fc6174a0000)
> > > > >         libkrb5support.so.0 => /lib64/libkrb5support.so.0
> > > > > (0x00007fc6158d8000)
> > > > >         libkeyutils.so.1 => /lib64/libkeyutils.so.1
> > > (0x00007fc6156d3000)
> > > > >         libresolv.so.2 => /lib64/libresolv.so.2
> (0x00007fc6154b9000)
> > > > >         libpthread.so.0 => /lib64/libpthread.so.0
> > (0x00007fc61529d000)
> > > > >         libselinux.so.1 => /lib64/libselinux.so.1
> > (0x00007fc615077000)
> > > > >         libpcre.so.1 => /lib64/libpcre.so.1 (0x00007fc614e16000)
> > > > >         liblzma.so.5 => /lib64/liblzma.so.5 (0x00007fc614bf1000)
> > > > >
> > > > >
> > > > > Client:
> > > > >
> > > > > [root at ops:~] #ldd /usr/local/nagios/libexec/check_nrpe
> > > > >        * libssl.so.6 => /lib64/libssl.so.6 (0x00002aaaaaaba000)*
> > > > > *        libcrypto.so.6 => /lib64/libcrypto.so.6
> > (0x00002aaaaad08000)*
> > > > >         libnsl.so.1 => /lib64/libnsl.so.1 (0x00002aaaab05a000)
> > > > >         libc.so.6 => /lib64/libc.so.6 (0x00002aaaab273000)
> > > > >         libgssapi_krb5.so.2 => /usr/lib64/libgssapi_krb5.so.2
> > > > > (0x00002aaaab5cc000)
> > > > >         libkrb5.so.3 => /usr/lib64/libkrb5.so.3
> (0x00002aaaab7fa000)
> > > > >         libcom_err.so.2 => /lib64/libcom_err.so.2
> > (0x00002aaaaba90000)
> > > > >         libk5crypto.so.3 => /usr/lib64/libk5crypto.so.3
> > > > > (0x00002aaaabc92000)
> > > > >         libdl.so.2 => /lib64/libdl.so.2 (0x00002aaaabeb7000)
> > > > >         libz.so.1 => /lib64/libz.so.1 (0x00002aaaac0bc000)
> > > > >         /lib64/ld-linux-x86-64.so.2 (0x0000555555554000)
> > > > >         libkrb5support.so.0 => /usr/lib64/libkrb5support.so.0 (0x0
> > > > > 0002aaaac2d0000)
> > > > >         libkeyutils.so.1 => /lib64/libkeyutils.so.1
> > > (0x00002aaaac4d8000)
> > > > >         libresolv.so.2 => /lib64/libresolv.so.2
> (0x00002aaaac6db000)
> > > > >         libselinux.so.1 => /lib64/libselinux.so.1
> > (0x00002aaaac8f0000)
> > > > >         libsepol.so.1 => /lib64/libsepol.so.1 (0x00002aaaacb09000)
> > > > >
> > > > >
> > > > > So it looks like everything is OK from the SSL end of things. Any
> > other
> > > > > ideas or suggestions?
> > > > >
> > > > > Thanks
> > > > > Tim
> > > > >
> > > > > On Fri, May 1, 2015 at 5:46 AM, Eric Lehmann <
> e.lehmann88 at gmail.com>
> > > > > wrote:
> > > > >
> > > > > > This is strange...
> > > > > > Do you have SSL aktive on both systems? Run nrpr localy without
> > > > > parameters
> > > > > > (this should return some nrpe stats) and check ldd for libssl.
> > > > > > Am 01.05.2015 07:32 schrieb "Tim Dunphy" <bluethundr at gmail.com>:
> > > > > >
> > > > > > > Hi Eric,
> > > > > > >
> > > > > > >  Thanks for your reply. I do have nrpe running under xinetd on
> > the
> > > > host
> > > > > > I'm
> > > > > > > trying to monitor.
> > > > > > >
> > > > > > >  And running the nrpe checl locally:
> > > > > > >
> > > > > > > [root at ops:~] #/usr/local/nagios/libexec/check_nrpe -H
> localhost
> > > > > > > NRPE v2.15
> > > > > > >
> > > > > > > [root at ops:~] #grep only_from /etc/xinetd.d/nrpe
> > > > > > >         only_from       = 127.0.0.1 216.120.248.126
> > > > > > >
> > > > > > > And I do have port 5666 open on the security group for this
> host.
> > > > > > >
> > > > > > > And I made sure the local firewall was stopped, because I am
> > > blocking
> > > > > > ports
> > > > > > > with the security groups instead.
> > > > > > >
> > > > > > > [root at ops:~] #service iptables status
> > > > > > > Firewall is stopped.
> > > > > > >
> > > > > > > It's only when checking from the monitoring host that nrpe
> fails:
> > > > > > >
> > > > > > > [root at monitor1:~] #/usr/local/nagios/libexec/check_nrpe -H
> > > > > > > ops.jokefire.com
> > > > > > > CHECK_NRPE: Error - Could not complete SSL handshake.
> > > > > > >
> > > > > > > Really, really puzzling. This is driving me up a wall!! I hopeI
> > can
> > > > > solve
> > > > > > > this soon....
> > > > > > >
> > > > > > > Thanks for any and all help with this one!!
> > > > > > > Tim
> > > > > > >
> > > > > > > On Fri, May 1, 2015 at 1:02 AM, Eric Lehmann <
> > > e.lehmann88 at gmail.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hi
> > > > > > > > Does the deamon run under xinetd? Then  you have to configure
> > the
> > > > > > > only_from
> > > > > > > > in  */etc/**xinetd.d**/**nrpe* to.
> > > > > > > >
> > > > > > > > Regards
> > > > > > > > Eric
> > > > > > > > Am 01.05.2015 06:46 schrieb "Tim Dunphy" <
> bluethundr at gmail.com
> > >:
> > > > > > > >
> > > > > > > > > Hello,
> > > > > > > > >
> > > > > > > > >  I am trying to monitor a host in the Amazon EC2 cloud.
> > > > > > > > >
> > > > > > > > > Yet when I try to check NRPE from the monitoring host I am
> > > > getting
> > > > > an
> > > > > > > SSL
> > > > > > > > > handshake error:
> > > > > > > > >
> > > > > > > > > [root at monitor1:~] #/usr/local/nagios/libexec/check_nrpe -H
> > > > > > > > > ops.jokefire.com
> > > > > > > > > CHECK_NRPE: Error - Could not complete SSL handshake.
> > > > > > > > >
> > > > > > > > > And if I telnet into the host on port 5666 to see if the FW
> > > port
> > > > is
> > > > > > > open,
> > > > > > > > > the connection closes right away:
> > > > > > > > >
> > > > > > > > > [root at monitor1:~] #telnet ops.somewhere.com 5666
> > > > > > > > > Trying 54.225.218.125...
> > > > > > > > > Connected to ops.somewhere.com.
> > > > > > > > > Escape character is '^]'.
> > > > > > > > > Connection closed by foreign host.
> > > > > > > > >
> > > > > > > > > You can see there it connects, but then it closes
> immediately
> > > > after
> > > > > > the
> > > > > > > > > connection.
> > > > > > > > >
> > > > > > > > >  I have NRPE running on the host I want to monitor:
> > > > > > > > >
> > > > > > > > > [root at ops:~] #lsof -i :5666
> > > > > > > > > COMMAND  PID USER   FD   TYPE DEVICE SIZE NODE NAME
> > > > > > > > > xinetd  1434 root    5u  IPv4   4063       TCP *:nrpe
> > (LISTEN)
> > > > > > > > >
> > > > > > > > > And I have the IP of my nagios server listed in the xinetd
> > conf
> > > > > file:
> > > > > > > > >
> > > > > > > > > [root at ops:~] #cat /etc/xinetd.d/nrpe
> > > > > > > > > # default: on
> > > > > > > > > # description: NRPE (Nagios Remote Plugin Executor)
> > > > > > > > > service nrpe
> > > > > > > > > {
> > > > > > > > >         flags           = REUSE
> > > > > > > > >         socket_type     = stream
> > > > > > > > >         port            = 5666
> > > > > > > > >         wait            = no
> > > > > > > > >         user            = nagios
> > > > > > > > >         group           = nagios
> > > > > > > > >         server          = /usr/local/nagios/bin/nrpe
> > > > > > > > >         server_args     = -c /usr/local/nagios/etc/nrpe.cfg
> > > > --inetd
> > > > > > > > >         log_on_failure  += USERID
> > > > > > > > >         disable         = no
> > > > > > > > >         only_from       = 127.0.0.1 xx.xx.xx.xx   # <-
> > > > representing
> > > > > > my
> > > > > > > > real
> > > > > > > > > nagios server IP
> > > > > > > > > }
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > And I have my default security group for that host open on
> > port
> > > > > 5666
> > > > > > to
> > > > > > > > the
> > > > > > > > > world for this experiment.  I plan on locking that down
> again
> > > to
> > > > > the
> > > > > > > > single
> > > > > > > > > IP of my monitoring host once I get this resolved.
> > > > > > > > >
> > > > > > > > > Does anyone have any suggestions on how I can get that
> > problem
> > > > > > solved?
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > > Tim
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > GPG me!!
> > > > > > > > >
> > > > > > > > > gpg --keyserver pool.sks-keyservers.net --recv-keys
> F186197B
> > > > > > > > > _______________________________________________
> > > > > > > > > CentOS mailing list
> > > > > > > > > CentOS at centos.org
> > > > > > > > > http://lists.centos.org/mailman/listinfo/centos
> > > > > > > > >
> > > > > > > > _______________________________________________
> > > > > > > > CentOS mailing list
> > > > > > > > CentOS at centos.org
> > > > > > > > http://lists.centos.org/mailman/listinfo/centos
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > GPG me!!
> > > > > > >
> > > > > > > gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B
> > > > > > > _______________________________________________
> > > > > > > CentOS mailing list
> > > > > > > CentOS at centos.org
> > > > > > > http://lists.centos.org/mailman/listinfo/centos
> > > > > > >
> > > > > > _______________________________________________
> > > > > > CentOS mailing list
> > > > > > CentOS at centos.org
> > > > > > http://lists.centos.org/mailman/listinfo/centos
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > GPG me!!
> > > > >
> > > > > gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B
> > > > > _______________________________________________
> > > > > CentOS mailing list
> > > > > CentOS at centos.org
> > > > > http://lists.centos.org/mailman/listinfo/centos
> > > > >
> > > > _______________________________________________
> > > > CentOS mailing list
> > > > CentOS at centos.org
> > > > http://lists.centos.org/mailman/listinfo/centos
> > > >
> > > _______________________________________________
> > > CentOS mailing list
> > > CentOS at centos.org
> > > http://lists.centos.org/mailman/listinfo/centos
> > >
> >
> >
> >
> > --
> > GPG me!!
> >
> > gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B
> > _______________________________________________
> > CentOS mailing list
> > CentOS at centos.org
> > http://lists.centos.org/mailman/listinfo/centos
> >
> _______________________________________________
> CentOS mailing list
> CentOS at centos.org
> http://lists.centos.org/mailman/listinfo/centos
>



-- 
GPG me!!

gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B