[CentOS] SEtroubleshootd Crashing

Thu Dec 4 12:35:26 UTC 2014
Daniel J Walsh <dwalsh at redhat.com>

Are you seeing other AVCs?

On 12/03/2014 05:36 AM, John Beranek wrote:
> Indeed, thanks Dan - it doesn't get us to a completely clean running that
> would allow us to run our Node app as we are under Passenger with SELinux
> enforcing, but it at least has stopped the excessive amount of AVCs we were
> getting.
>
> John
>
> On 3 December 2014 at 10:01, Daniel J Walsh <dwalsh at redhat.com> wrote:
>
>> Looks like turning on three booleans will solve most of the problem.
>>
>> httpd_execmem, httpd_run_stickshift, allow_httpd_anon_write
>>
>>
>> On 12/03/2014 03:55 AM, John Beranek wrote:
>>> Mark: Labels look OK, restorecon has nothing to do, and:
>>>
>>> -rwxr-xr-x. root root system_u:object_r:bin_t:s0       /bin/ps
>>>
>>> dr-xr-xr-x. root root system_u:object_r:proc_t:s0      /proc
>>>
>>> I'll send the audit log on to Dan.
>>>
>>> Cheers,
>>>
>>> John
>>>
>>> On 2 December 2014 at 16:10, Daniel J Walsh <dwalsh at redhat.com> wrote:
>>>
>>>> Could you send me a copy of your audit.log.
>>>>
>>>> You should not be getting hundreds of AVC's a day.
>>>>
>>>> ausearch -m avc,user_avc -ts today
>>>>
>>>> On 12/02/2014 05:08 AM, John Beranek wrote:
>>>>> I'll jump in here to say we'll try your suggestion, but I guess what's
>>>> not
>>>>> been mentioned is that we get the setroubleshoot abrt's only a few
>> times
>>>> a
>>>>> day, but we're getting 10000s of setroubleshoot messages in
>>>>> /var/log/messages a day.
>>>>>
>>>>> e.g.
>>>>>
>>>>> Dec  2 10:03:55 server audispd: queue is full - dropping event
>>>>> Dec  2 10:04:00 server audispd: last message repeated 199 times
>>>>> Dec  2 10:04:00 server rsyslogd-2177: imuxsock begins to drop messages
>>>> from
>>>>> pid 5967 due to rate-limiting
>>>>> Dec  2 10:04:01 server rsyslogd-2177: imuxsock lost 2 messages from pid
>>>>> 5967 due to rate-limiting
>>>>> Dec  2 10:04:01 server audispd: queue is full - dropping event
>>>>> Dec  2 10:04:02 server audispd: last message repeated 134 times
>>>>> Dec  2 10:04:02 server setroubleshoot: SELinux is preventing /bin/ps
>> from
>>>>> read access on the file /proc/<pid>/stat. For complete SELinux
>> messages.
>>>>> run sealert -l 2274b1c7-fd69-4fa8-8e67-cd7a9da9eff4
>>>>> Dec  2 10:04:02 server audispd: queue is full - dropping event
>>>>> Dec  2 10:04:03 server audispd: last message repeated 48 times
>>>>> Dec  2 10:04:03 server setroubleshoot: SELinux is preventing /bin/ps
>> from
>>>>> getattr access on the directory /proc/<pid>. For complete SELinux
>>>> messages.
>>>>> run sealert -l 2d09d555-8834-4c27-976b-6647f8673286
>>>>> Dec  2 10:04:03 server audispd: queue is full - dropping event
>>>>> Dec  2 10:04:03 server audispd: last message repeated 15 times
>>>>> Dec  2 10:04:03 server rsyslogd-2177: imuxsock begins to drop messages
>>>> from
>>>>> pid 5967 due to rate-limiting
>>>>> Dec  2 10:04:03 server setroubleshoot: SELinux is preventing /bin/ps
>> from
>>>>> search access on the directory /proc/<pid>/stat. For complete SELinux
>>>>> messages. run sealert -l 0ef0c7a1-acb2-433a-aaa2-361cc95b6069
>>>>> Dec  2 10:04:04 server setroubleshoot: last message repeated 2 times
>>>>> Dec  2 10:04:04 server setroubleshoot: SELinux is preventing /bin/ps
>> from
>>>>> getattr access on the directory /proc/<pid>. For complete SELinux
>>>> messages.
>>>>> run sealert -l 58f859b0-7382-428e-81f0-3e85f66d79fc
>>>>> Dec  2 10:04:04 server setroubleshoot: SELinux is preventing /bin/ps
>> from
>>>>> search access on the directory /proc/<pid>/stat. For complete SELinux
>>>>> messages. run sealert -l 2448a46d-5089-4f85-aae8-e9013341471f
>>>>> Dec  2 10:04:05 server setroubleshoot: last message repeated 2 times
>>>>> Dec  2 10:04:05 server setroubleshoot: SELinux is preventing /bin/ps
>> from
>>>>> getattr access on the directory /proc/<pid>. For complete SELinux
>>>> messages.
>>>>> run sealert -l f935416b-54fe-4bbd-b66c-2e1b2e6724be
>>>>> Dec  2 10:04:06 server setroubleshoot: SELinux is preventing /bin/ps
>> from
>>>>> search access on the directory /proc/<pid>/stat. For complete SELinux
>>>>> messages. run sealert -l d8dbf973-7bc2-4fd5-9540-18c4040be03c
>>>>> Dec  2 10:04:06 server setroubleshoot: last message repeated 2 times
>>>>> Dec  2 10:04:06 server sedispatch: AVC Message for setroubleshoot,
>>>> dropping
>>>>> message
>>>>> Dec  2 10:04:06 server sedispatch: last message repeated 3 times
>>>>>
>>>>> Cheers,
>>>>>
>>>>> John
>>>>>
>>>>> On 1 December 2014 at 17:19, Daniel J Walsh <dwalsh at redhat.com> wrote:
>>>>>
>>>>>> On 12/01/2014 10:39 AM, Gary Smithson wrote:
>>>>>>> We are currently running libxml2-2.7.6-14.el6_5.2.x86_64
>>>>>>>
>>>>>>> How far back would you suggest we go? would
>>>>>> libxml2-2.7.6-14.el6_5.1.x86_64 be sufficient
>>>>>> Ok might not be related.  One other suggestion would be to clear the
>>>>>> database out.  And see if there
>>>>>> was something in the database that was causing it problems.
>>>>>>
>>>>>> Make sure there is no setroubleshootd running and
>>>>>>
>>>>>>> /var/lib/setroubleshoot/setroubleshoot_database.xml
>>>>>>> -----Original Message-----
>>>>>>> From: centos-bounces at centos.org [mailto:centos-bounces at centos.org]
>> On
>>>>>> Behalf Of Daniel J Walsh
>>>>>>> Sent: 01 December 2014 15:10
>>>>>>> To: CentOS mailing list
>>>>>>> Subject: Re: [CentOS] SEtroubleshootd Crashing
>>>>>>>
>>>>>>> I am not sure.  I was just seeing email on this today.  Could you try
>>>> to
>>>>>> downgrade the latest version of libxml to see if the problem goes
>> away.
>>>>>>> On 12/01/2014 10:01 AM, Gary Smithson wrote:
>>>>>>>> Thanks
>>>>>>>>
>>>>>>>> Could you please clarify, which version libxml is broken and has
>> there
>>>>>> been a newer version released that will fix it.
>>>>>>>> -----Original Message-----
>>>>>>>> From: centos-bounces at centos.org [mailto:centos-bounces at centos.org]
>> On
>>>>>>>> Behalf Of Daniel J Walsh
>>>>>>>> Sent: 01 December 2014 14:58
>>>>>>>> To: CentOS mailing list
>>>>>>>> Subject: Re: [CentOS] SEtroubleshootd Crashing
>>>>>>>>
>>>>>>>> This seems to be a problem with an updated version of libxml.
>>>>>>>> On 11/28/2014 09:04 AM, Gary Smithson wrote:
>>>>>>>>> When running Node.js through Phusion Passenger on Centos 6.5 (
>> Linux
>>>>>> 2.6.32-431.23.3.el6.x86_64 #1 SMP Thu Jul 31 17:20:51 UTC 2014 x86_64
>>>>>> x86_64 x86_64 GNU/Linux), with SELinux enabled in permissive mode we
>>>>>> receive a large number of entries in the audit.log and setroubleshootd
>>>>>> randomly crashes with the following error, We have resolved the
>> selinux
>>>>>> alerts by following the troubleshooting steps recommend by running
>>>>>> sealert,However we are concerned by setroubleshootd crashing and are
>>>>>> concered that we may have masked the issue by fixing the entries in
>> the
>>>>>> audit.log.
>>>>>>>>> abrt_version:   2.0.8
>>>>>>>>>
>>>>>>>>> cmdline:        /usr/bin/python -Es /usr/sbin/setroubleshootd -f ''
>>>>>>>>>
>>>>>>>>> executable:     /usr/sbin/setroubleshootd
>>>>>>>>>
>>>>>>>>> kernel:         2.6.32-431.23.3.el6.x86_64
>>>>>>>>>
>>>>>>>>> last_occurrence: 1417101625
>>>>>>>>>
>>>>>>>>> time:           Thu 27 Nov 2014 03:20:25 PM UTC
>>>>>>>>>
>>>>>>>>> uid:            0
>>>>>>>>>
>>>>>>>>> username:       root
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> sosreport.tar.xz: Binary file, 3642240 bytes
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> backtrace:
>>>>>>>>>
>>>>>>>>> :analyze.py:426:lookup_signature:ProgramError: [Errno 1001]
>> signature
>>>>>>>>> not found
>>>>>>>>>
>>>>>>>>> :
>>>>>>>>>
>>>>>>>>> :Traceback (most recent call last):
>>>>>>>>>
>>>>>>>>> :  File
>>>>>>>>> "/usr/lib64/python2.6/site-packages/setroubleshoot/analyze.py",
>> line
>>>>>>>>> 401, in auto_save_callback
>>>>>>>>>
>>>>>>>>> :    self.save()
>>>>>>>>>
>>>>>>>>> :  File
>>>>>>>>> "/usr/lib64/python2.6/site-packages/setroubleshoot/analyze.py",
>> line
>>>>>>>>> 377, in save
>>>>>>>>>
>>>>>>>>> :    self.prune()
>>>>>>>>>
>>>>>>>>> :  File
>>>>>>>>> "/usr/lib64/python2.6/site-packages/setroubleshoot/analyze.py",
>> line
>>>>>>>>> 340, in prune
>>>>>>>>>
>>>>>>>>> :    self.delete_signature(sig, prune=True)
>>>>>>>>>
>>>>>>>>> :  File
>>>>>>>>> "/usr/lib64/python2.6/site-packages/setroubleshoot/analyze.py",
>> line
>>>>>>>>> 471, in delete_signature
>>>>>>>>>
>>>>>>>>> :    siginfo = self.lookup_signature(sig)
>>>>>>>>>
>>>>>>>>> :  File
>>>>>>>>> "/usr/lib64/python2.6/site-packages/setroubleshoot/analyze.py",
>> line
>>>>>>>>> 426, in lookup_signature
>>>>>>>>>
>>>>>>>>> :    raise ProgramError(ERR_NO_SIGNATURE_MATCH)
>>>>>>>>>
>>>>>>>>> :ProgramError: [Errno 1001] signature not found
>>>>>>>>>
>>>>>>>>> :
>>>>>>>>>
>>>>>>>>> :Local variables in innermost frame:
>>>>>>>>>
>>>>>>>>> :matches: []
>>>>>>>>>
>>>>>>>>> :siginfo: None
>>>>>>>>>
>>>>>>>>> :self: <setroubleshoot.analyze.SETroubleshootDatabase object at
>>>>>>>>> 0x151d590>
>>>>>>>>>
>>>>>>>>> :sig: <setroubleshoot.signature.SEFaultSignature object at
>> 0x645a050>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> We are running the following versions Passenger/htttpd/node
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> passenger --version
>>>>>>>>>
>>>>>>>>> Phusion Passenger version 4.0.53
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> httpd -v
>>>>>>>>> Server version: Apache/2.2.15 (Unix)
>>>>>>>>> Server built:   Jul 23 2014 14:17:29
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> node -v
>>>>>>>>> v0.10.32
>>>>>>>>>
>>>>>>>>> This email is from the Press Association. For more information, see
>>>>>> www.pressassociation.com. This email may contain confidential
>>>>>> information. Only the addressee is permitted to read, copy, distribute
>>>> or
>>>>>> otherwise use this email or any attachments. If you have received it
>> in
>>>>>> error, please contact the sender immediately. Any opinion expressed in
>>>> this
>>>>>> email is personal to the sender and may not reflect the opinion of the
>>>>>> Press Association. Any email reply to this address may be subject to
>>>>>> interception or monitoring for operational reasons or for lawful
>>>> business
>>>>>> practices.
>>>>>>>>> _______________________________________________
>>>>>>>>> CentOS mailing list
>>>>>>>>> CentOS at centos.org
>>>>>>>>> http://lists.centos.org/mailman/listinfo/centos
>>>>>>>> _______________________________________________
>>>>>>>> CentOS mailing list
>>>>>>>> CentOS at centos.org
>>>>>>>> http://lists.centos.org/mailman/listinfo/centos
>>>>>>>>
>>>>>>>> This email is from the Press Association. For more information, see
>>>>>> www.pressassociation.com. This email may contain confidential
>>>>>> information. Only the addressee is permitted to read, copy, distribute
>>>> or
>>>>>> otherwise use this email or any attachments. If you have received it
>> in
>>>>>> error, please contact the sender immediately. Any opinion expressed in
>>>> this
>>>>>> email is personal to the sender and may not reflect the opinion of the
>>>>>> Press Association. Any email reply to this address may be subject to
>>>>>> interception or monitoring for operational reasons or for lawful
>>>> business
>>>>>> practices.
>>>>>>>> _______________________________________________
>>>>>>>> CentOS mailing list
>>>>>>>> CentOS at centos.org
>>>>>>>> http://lists.centos.org/mailman/listinfo/centos
>>>>>>> _______________________________________________
>>>>>>> CentOS mailing list
>>>>>>> CentOS at centos.org
>>>>>>> http://lists.centos.org/mailman/listinfo/centos
>>>>>>>
>>>>>>> This email is from the Press Association. For more information, see
>>>>>> www.pressassociation.com. This email may contain confidential
>>>>>> information. Only the addressee is permitted to read, copy, distribute
>>>> or
>>>>>> otherwise use this email or any attachments. If you have received it
>> in
>>>>>> error, please contact the sender immediately. Any opinion expressed in
>>>> this
>>>>>> email is personal to the sender and may not reflect the opinion of the
>>>>>> Press Association. Any email reply to this address may be subject to
>>>>>> interception or monitoring for operational reasons or for lawful
>>>> business
>>>>>> practices.
>>>>>>> _______________________________________________
>>>>>>> CentOS mailing list
>>>>>>> CentOS at centos.org
>>>>>>> http://lists.centos.org/mailman/listinfo/centos
>>>>>> _______________________________________________
>>>>>> CentOS mailing list
>>>>>> CentOS at centos.org
>>>>>> http://lists.centos.org/mailman/listinfo/centos
>>>>>>
>>>> _______________________________________________
>>>> CentOS mailing list
>>>> CentOS at centos.org
>>>> http://lists.centos.org/mailman/listinfo/centos
>>>>
>>>
>> _______________________________________________
>> CentOS mailing list
>> CentOS at centos.org
>> http://lists.centos.org/mailman/listinfo/centos
>>
>
>