[CentOS] SEtroubleshootd Crashing

Wed Dec 3 10:36:26 UTC 2014
John Beranek <john at redux.org.uk>

Indeed, thanks Dan - it doesn't get us to a completely clean running that
would allow us to run our Node app as we are under Passenger with SELinux
enforcing, but it at least has stopped the excessive amount of AVCs we were
getting.

John

On 3 December 2014 at 10:01, Daniel J Walsh <dwalsh at redhat.com> wrote:

> Looks like turning on three booleans will solve most of the problem.
>
> httpd_execmem, httpd_run_stickshift, allow_httpd_anon_write
>
>
> On 12/03/2014 03:55 AM, John Beranek wrote:
> > Mark: Labels look OK, restorecon has nothing to do, and:
> >
> > -rwxr-xr-x. root root system_u:object_r:bin_t:s0       /bin/ps
> >
> > dr-xr-xr-x. root root system_u:object_r:proc_t:s0      /proc
> >
> > I'll send the audit log on to Dan.
> >
> > Cheers,
> >
> > John
> >
> > On 2 December 2014 at 16:10, Daniel J Walsh <dwalsh at redhat.com> wrote:
> >
> >> Could you send me a copy of your audit.log.
> >>
> >> You should not be getting hundreds of AVC's a day.
> >>
> >> ausearch -m avc,user_avc -ts today
> >>
> >> On 12/02/2014 05:08 AM, John Beranek wrote:
> >>> I'll jump in here to say we'll try your suggestion, but I guess what's
> >> not
> >>> been mentioned is that we get the setroubleshoot abrt's only a few
> times
> >> a
> >>> day, but we're getting 10000s of setroubleshoot messages in
> >>> /var/log/messages a day.
> >>>
> >>> e.g.
> >>>
> >>> Dec  2 10:03:55 server audispd: queue is full - dropping event
> >>> Dec  2 10:04:00 server audispd: last message repeated 199 times
> >>> Dec  2 10:04:00 server rsyslogd-2177: imuxsock begins to drop messages
> >> from
> >>> pid 5967 due to rate-limiting
> >>> Dec  2 10:04:01 server rsyslogd-2177: imuxsock lost 2 messages from pid
> >>> 5967 due to rate-limiting
> >>> Dec  2 10:04:01 server audispd: queue is full - dropping event
> >>> Dec  2 10:04:02 server audispd: last message repeated 134 times
> >>> Dec  2 10:04:02 server setroubleshoot: SELinux is preventing /bin/ps
> from
> >>> read access on the file /proc/<pid>/stat. For complete SELinux
> messages.
> >>> run sealert -l 2274b1c7-fd69-4fa8-8e67-cd7a9da9eff4
> >>> Dec  2 10:04:02 server audispd: queue is full - dropping event
> >>> Dec  2 10:04:03 server audispd: last message repeated 48 times
> >>> Dec  2 10:04:03 server setroubleshoot: SELinux is preventing /bin/ps
> from
> >>> getattr access on the directory /proc/<pid>. For complete SELinux
> >> messages.
> >>> run sealert -l 2d09d555-8834-4c27-976b-6647f8673286
> >>> Dec  2 10:04:03 server audispd: queue is full - dropping event
> >>> Dec  2 10:04:03 server audispd: last message repeated 15 times
> >>> Dec  2 10:04:03 server rsyslogd-2177: imuxsock begins to drop messages
> >> from
> >>> pid 5967 due to rate-limiting
> >>> Dec  2 10:04:03 server setroubleshoot: SELinux is preventing /bin/ps
> from
> >>> search access on the directory /proc/<pid>/stat. For complete SELinux
> >>> messages. run sealert -l 0ef0c7a1-acb2-433a-aaa2-361cc95b6069
> >>> Dec  2 10:04:04 server setroubleshoot: last message repeated 2 times
> >>> Dec  2 10:04:04 server setroubleshoot: SELinux is preventing /bin/ps
> from
> >>> getattr access on the directory /proc/<pid>. For complete SELinux
> >> messages.
> >>> run sealert -l 58f859b0-7382-428e-81f0-3e85f66d79fc
> >>> Dec  2 10:04:04 server setroubleshoot: SELinux is preventing /bin/ps
> from
> >>> search access on the directory /proc/<pid>/stat. For complete SELinux
> >>> messages. run sealert -l 2448a46d-5089-4f85-aae8-e9013341471f
> >>> Dec  2 10:04:05 server setroubleshoot: last message repeated 2 times
> >>> Dec  2 10:04:05 server setroubleshoot: SELinux is preventing /bin/ps
> from
> >>> getattr access on the directory /proc/<pid>. For complete SELinux
> >> messages.
> >>> run sealert -l f935416b-54fe-4bbd-b66c-2e1b2e6724be
> >>> Dec  2 10:04:06 server setroubleshoot: SELinux is preventing /bin/ps
> from
> >>> search access on the directory /proc/<pid>/stat. For complete SELinux
> >>> messages. run sealert -l d8dbf973-7bc2-4fd5-9540-18c4040be03c
> >>> Dec  2 10:04:06 server setroubleshoot: last message repeated 2 times
> >>> Dec  2 10:04:06 server sedispatch: AVC Message for setroubleshoot,
> >> dropping
> >>> message
> >>> Dec  2 10:04:06 server sedispatch: last message repeated 3 times
> >>>
> >>> Cheers,
> >>>
> >>> John
> >>>
> >>> On 1 December 2014 at 17:19, Daniel J Walsh <dwalsh at redhat.com> wrote:
> >>>
> >>>> On 12/01/2014 10:39 AM, Gary Smithson wrote:
> >>>>> We are currently running libxml2-2.7.6-14.el6_5.2.x86_64
> >>>>>
> >>>>> How far back would you suggest we go? would
> >>>> libxml2-2.7.6-14.el6_5.1.x86_64 be sufficient
> >>>> Ok might not be related.  One other suggestion would be to clear the
> >>>> database out.  And see if there
> >>>> was something in the database that was causing it problems.
> >>>>
> >>>> Make sure there is no setroubleshootd running and
> >>>>
> >>>>> /var/lib/setroubleshoot/setroubleshoot_database.xml
> >>>>> -----Original Message-----
> >>>>> From: centos-bounces at centos.org [mailto:centos-bounces at centos.org]
> On
> >>>> Behalf Of Daniel J Walsh
> >>>>> Sent: 01 December 2014 15:10
> >>>>> To: CentOS mailing list
> >>>>> Subject: Re: [CentOS] SEtroubleshootd Crashing
> >>>>>
> >>>>> I am not sure.  I was just seeing email on this today.  Could you try
> >> to
> >>>> downgrade the latest version of libxml to see if the problem goes
> away.
> >>>>> On 12/01/2014 10:01 AM, Gary Smithson wrote:
> >>>>>> Thanks
> >>>>>>
> >>>>>> Could you please clarify, which version libxml is broken and has
> there
> >>>> been a newer version released that will fix it.
> >>>>>> -----Original Message-----
> >>>>>> From: centos-bounces at centos.org [mailto:centos-bounces at centos.org]
> On
> >>>>>> Behalf Of Daniel J Walsh
> >>>>>> Sent: 01 December 2014 14:58
> >>>>>> To: CentOS mailing list
> >>>>>> Subject: Re: [CentOS] SEtroubleshootd Crashing
> >>>>>>
> >>>>>> This seems to be a problem with an updated version of libxml.
> >>>>>> On 11/28/2014 09:04 AM, Gary Smithson wrote:
> >>>>>>> When running Node.js through Phusion Passenger on Centos 6.5 (
> Linux
> >>>> 2.6.32-431.23.3.el6.x86_64 #1 SMP Thu Jul 31 17:20:51 UTC 2014 x86_64
> >>>> x86_64 x86_64 GNU/Linux), with SELinux enabled in permissive mode we
> >>>> receive a large number of entries in the audit.log and setroubleshootd
> >>>> randomly crashes with the following error, We have resolved the
> selinux
> >>>> alerts by following the troubleshooting steps recommend by running
> >>>> sealert,However we are concerned by setroubleshootd crashing and are
> >>>> concered that we may have masked the issue by fixing the entries in
> the
> >>>> audit.log.
> >>>>>>>
> >>>>>>> abrt_version:   2.0.8
> >>>>>>>
> >>>>>>> cmdline:        /usr/bin/python -Es /usr/sbin/setroubleshootd -f ''
> >>>>>>>
> >>>>>>> executable:     /usr/sbin/setroubleshootd
> >>>>>>>
> >>>>>>> kernel:         2.6.32-431.23.3.el6.x86_64
> >>>>>>>
> >>>>>>> last_occurrence: 1417101625
> >>>>>>>
> >>>>>>> time:           Thu 27 Nov 2014 03:20:25 PM UTC
> >>>>>>>
> >>>>>>> uid:            0
> >>>>>>>
> >>>>>>> username:       root
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> sosreport.tar.xz: Binary file, 3642240 bytes
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> backtrace:
> >>>>>>>
> >>>>>>> :analyze.py:426:lookup_signature:ProgramError: [Errno 1001]
> signature
> >>>>>>> not found
> >>>>>>>
> >>>>>>> :
> >>>>>>>
> >>>>>>> :Traceback (most recent call last):
> >>>>>>>
> >>>>>>> :  File
> >>>>>>> "/usr/lib64/python2.6/site-packages/setroubleshoot/analyze.py",
> line
> >>>>>>> 401, in auto_save_callback
> >>>>>>>
> >>>>>>> :    self.save()
> >>>>>>>
> >>>>>>> :  File
> >>>>>>> "/usr/lib64/python2.6/site-packages/setroubleshoot/analyze.py",
> line
> >>>>>>> 377, in save
> >>>>>>>
> >>>>>>> :    self.prune()
> >>>>>>>
> >>>>>>> :  File
> >>>>>>> "/usr/lib64/python2.6/site-packages/setroubleshoot/analyze.py",
> line
> >>>>>>> 340, in prune
> >>>>>>>
> >>>>>>> :    self.delete_signature(sig, prune=True)
> >>>>>>>
> >>>>>>> :  File
> >>>>>>> "/usr/lib64/python2.6/site-packages/setroubleshoot/analyze.py",
> line
> >>>>>>> 471, in delete_signature
> >>>>>>>
> >>>>>>> :    siginfo = self.lookup_signature(sig)
> >>>>>>>
> >>>>>>> :  File
> >>>>>>> "/usr/lib64/python2.6/site-packages/setroubleshoot/analyze.py",
> line
> >>>>>>> 426, in lookup_signature
> >>>>>>>
> >>>>>>> :    raise ProgramError(ERR_NO_SIGNATURE_MATCH)
> >>>>>>>
> >>>>>>> :ProgramError: [Errno 1001] signature not found
> >>>>>>>
> >>>>>>> :
> >>>>>>>
> >>>>>>> :Local variables in innermost frame:
> >>>>>>>
> >>>>>>> :matches: []
> >>>>>>>
> >>>>>>> :siginfo: None
> >>>>>>>
> >>>>>>> :self: <setroubleshoot.analyze.SETroubleshootDatabase object at
> >>>>>>> 0x151d590>
> >>>>>>>
> >>>>>>> :sig: <setroubleshoot.signature.SEFaultSignature object at
> 0x645a050>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> We are running the following versions Passenger/htttpd/node
> >>>>>>>
> >>>>>>>
> >>>>>>> passenger --version
> >>>>>>>
> >>>>>>> Phusion Passenger version 4.0.53
> >>>>>>>
> >>>>>>>
> >>>>>>> httpd -v
> >>>>>>> Server version: Apache/2.2.15 (Unix)
> >>>>>>> Server built:   Jul 23 2014 14:17:29
> >>>>>>>
> >>>>>>>
> >>>>>>> node -v
> >>>>>>> v0.10.32
> >>>>>>>
> >>>>>>> This email is from the Press Association. For more information, see
> >>>> www.pressassociation.com. This email may contain confidential
> >>>> information. Only the addressee is permitted to read, copy, distribute
> >> or
> >>>> otherwise use this email or any attachments. If you have received it
> in
> >>>> error, please contact the sender immediately. Any opinion expressed in
> >> this
> >>>> email is personal to the sender and may not reflect the opinion of the
> >>>> Press Association. Any email reply to this address may be subject to
> >>>> interception or monitoring for operational reasons or for lawful
> >> business
> >>>> practices.
> >>>>>>> _______________________________________________
> >>>>>>> CentOS mailing list
> >>>>>>> CentOS at centos.org
> >>>>>>> http://lists.centos.org/mailman/listinfo/centos
> >>>>>> _______________________________________________
> >>>>>> CentOS mailing list
> >>>>>> CentOS at centos.org
> >>>>>> http://lists.centos.org/mailman/listinfo/centos
> >>>>>>
> >>>>>> This email is from the Press Association. For more information, see
> >>>> www.pressassociation.com. This email may contain confidential
> >>>> information. Only the addressee is permitted to read, copy, distribute
> >> or
> >>>> otherwise use this email or any attachments. If you have received it
> in
> >>>> error, please contact the sender immediately. Any opinion expressed in
> >> this
> >>>> email is personal to the sender and may not reflect the opinion of the
> >>>> Press Association. Any email reply to this address may be subject to
> >>>> interception or monitoring for operational reasons or for lawful
> >> business
> >>>> practices.
> >>>>>> _______________________________________________
> >>>>>> CentOS mailing list
> >>>>>> CentOS at centos.org
> >>>>>> http://lists.centos.org/mailman/listinfo/centos
> >>>>> _______________________________________________
> >>>>> CentOS mailing list
> >>>>> CentOS at centos.org
> >>>>> http://lists.centos.org/mailman/listinfo/centos
> >>>>>
> >>>>> This email is from the Press Association. For more information, see
> >>>> www.pressassociation.com. This email may contain confidential
> >>>> information. Only the addressee is permitted to read, copy, distribute
> >> or
> >>>> otherwise use this email or any attachments. If you have received it
> in
> >>>> error, please contact the sender immediately. Any opinion expressed in
> >> this
> >>>> email is personal to the sender and may not reflect the opinion of the
> >>>> Press Association. Any email reply to this address may be subject to
> >>>> interception or monitoring for operational reasons or for lawful
> >> business
> >>>> practices.
> >>>>> _______________________________________________
> >>>>> CentOS mailing list
> >>>>> CentOS at centos.org
> >>>>> http://lists.centos.org/mailman/listinfo/centos
> >>>> _______________________________________________
> >>>> CentOS mailing list
> >>>> CentOS at centos.org
> >>>> http://lists.centos.org/mailman/listinfo/centos
> >>>>
> >>>
> >> _______________________________________________
> >> CentOS mailing list
> >> CentOS at centos.org
> >> http://lists.centos.org/mailman/listinfo/centos
> >>
> >
> >
>
> _______________________________________________
> CentOS mailing list
> CentOS at centos.org
> http://lists.centos.org/mailman/listinfo/centos
>



-- 
John Beranek                         To generalise is to be an idiot.
http://redux.org.uk/                                 -- William Blake