[CentOS] SEtroubleshootd Crashing

Wed Dec 3 10:01:44 UTC 2014
Daniel J Walsh <dwalsh at redhat.com>

Looks like turning on three booleans will solve most of the problem.

httpd_execmem, httpd_run_stickshift, allow_httpd_anon_write


On 12/03/2014 03:55 AM, John Beranek wrote:
> Mark: Labels look OK, restorecon has nothing to do, and:
>
> -rwxr-xr-x. root root system_u:object_r:bin_t:s0       /bin/ps
>
> dr-xr-xr-x. root root system_u:object_r:proc_t:s0      /proc
>
> I'll send the audit log on to Dan.
>
> Cheers,
>
> John
>
> On 2 December 2014 at 16:10, Daniel J Walsh <dwalsh at redhat.com> wrote:
>
>> Could you send me a copy of your audit.log.
>>
>> You should not be getting hundreds of AVC's a day.
>>
>> ausearch -m avc,user_avc -ts today
>>
>> On 12/02/2014 05:08 AM, John Beranek wrote:
>>> I'll jump in here to say we'll try your suggestion, but I guess what's
>> not
>>> been mentioned is that we get the setroubleshoot abrt's only a few times
>> a
>>> day, but we're getting 10000s of setroubleshoot messages in
>>> /var/log/messages a day.
>>>
>>> e.g.
>>>
>>> Dec  2 10:03:55 server audispd: queue is full - dropping event
>>> Dec  2 10:04:00 server audispd: last message repeated 199 times
>>> Dec  2 10:04:00 server rsyslogd-2177: imuxsock begins to drop messages
>> from
>>> pid 5967 due to rate-limiting
>>> Dec  2 10:04:01 server rsyslogd-2177: imuxsock lost 2 messages from pid
>>> 5967 due to rate-limiting
>>> Dec  2 10:04:01 server audispd: queue is full - dropping event
>>> Dec  2 10:04:02 server audispd: last message repeated 134 times
>>> Dec  2 10:04:02 server setroubleshoot: SELinux is preventing /bin/ps from
>>> read access on the file /proc/<pid>/stat. For complete SELinux messages.
>>> run sealert -l 2274b1c7-fd69-4fa8-8e67-cd7a9da9eff4
>>> Dec  2 10:04:02 server audispd: queue is full - dropping event
>>> Dec  2 10:04:03 server audispd: last message repeated 48 times
>>> Dec  2 10:04:03 server setroubleshoot: SELinux is preventing /bin/ps from
>>> getattr access on the directory /proc/<pid>. For complete SELinux
>> messages.
>>> run sealert -l 2d09d555-8834-4c27-976b-6647f8673286
>>> Dec  2 10:04:03 server audispd: queue is full - dropping event
>>> Dec  2 10:04:03 server audispd: last message repeated 15 times
>>> Dec  2 10:04:03 server rsyslogd-2177: imuxsock begins to drop messages
>> from
>>> pid 5967 due to rate-limiting
>>> Dec  2 10:04:03 server setroubleshoot: SELinux is preventing /bin/ps from
>>> search access on the directory /proc/<pid>/stat. For complete SELinux
>>> messages. run sealert -l 0ef0c7a1-acb2-433a-aaa2-361cc95b6069
>>> Dec  2 10:04:04 server setroubleshoot: last message repeated 2 times
>>> Dec  2 10:04:04 server setroubleshoot: SELinux is preventing /bin/ps from
>>> getattr access on the directory /proc/<pid>. For complete SELinux
>> messages.
>>> run sealert -l 58f859b0-7382-428e-81f0-3e85f66d79fc
>>> Dec  2 10:04:04 server setroubleshoot: SELinux is preventing /bin/ps from
>>> search access on the directory /proc/<pid>/stat. For complete SELinux
>>> messages. run sealert -l 2448a46d-5089-4f85-aae8-e9013341471f
>>> Dec  2 10:04:05 server setroubleshoot: last message repeated 2 times
>>> Dec  2 10:04:05 server setroubleshoot: SELinux is preventing /bin/ps from
>>> getattr access on the directory /proc/<pid>. For complete SELinux
>> messages.
>>> run sealert -l f935416b-54fe-4bbd-b66c-2e1b2e6724be
>>> Dec  2 10:04:06 server setroubleshoot: SELinux is preventing /bin/ps from
>>> search access on the directory /proc/<pid>/stat. For complete SELinux
>>> messages. run sealert -l d8dbf973-7bc2-4fd5-9540-18c4040be03c
>>> Dec  2 10:04:06 server setroubleshoot: last message repeated 2 times
>>> Dec  2 10:04:06 server sedispatch: AVC Message for setroubleshoot,
>> dropping
>>> message
>>> Dec  2 10:04:06 server sedispatch: last message repeated 3 times
>>>
>>> Cheers,
>>>
>>> John
>>>
>>> On 1 December 2014 at 17:19, Daniel J Walsh <dwalsh at redhat.com> wrote:
>>>
>>>> On 12/01/2014 10:39 AM, Gary Smithson wrote:
>>>>> We are currently running libxml2-2.7.6-14.el6_5.2.x86_64
>>>>>
>>>>> How far back would you suggest we go? would
>>>> libxml2-2.7.6-14.el6_5.1.x86_64 be sufficient
>>>> Ok might not be related.  One other suggestion would be to clear the
>>>> database out.  And see if there
>>>> was something in the database that was causing it problems.
>>>>
>>>> Make sure there is no setroubleshootd running and
>>>>
>>>>> /var/lib/setroubleshoot/setroubleshoot_database.xml
>>>>> -----Original Message-----
>>>>> From: centos-bounces at centos.org [mailto:centos-bounces at centos.org] On
>>>> Behalf Of Daniel J Walsh
>>>>> Sent: 01 December 2014 15:10
>>>>> To: CentOS mailing list
>>>>> Subject: Re: [CentOS] SEtroubleshootd Crashing
>>>>>
>>>>> I am not sure.  I was just seeing email on this today.  Could you try
>> to
>>>> downgrade the latest version of libxml to see if the problem goes away.
>>>>> On 12/01/2014 10:01 AM, Gary Smithson wrote:
>>>>>> Thanks
>>>>>>
>>>>>> Could you please clarify, which version libxml is broken and has there
>>>> been a newer version released that will fix it.
>>>>>> -----Original Message-----
>>>>>> From: centos-bounces at centos.org [mailto:centos-bounces at centos.org] On
>>>>>> Behalf Of Daniel J Walsh
>>>>>> Sent: 01 December 2014 14:58
>>>>>> To: CentOS mailing list
>>>>>> Subject: Re: [CentOS] SEtroubleshootd Crashing
>>>>>>
>>>>>> This seems to be a problem with an updated version of libxml.
>>>>>> On 11/28/2014 09:04 AM, Gary Smithson wrote:
>>>>>>> When running Node.js through Phusion Passenger on Centos 6.5 ( Linux
>>>> 2.6.32-431.23.3.el6.x86_64 #1 SMP Thu Jul 31 17:20:51 UTC 2014 x86_64
>>>> x86_64 x86_64 GNU/Linux), with SELinux enabled in permissive mode we
>>>> receive a large number of entries in the audit.log and setroubleshootd
>>>> randomly crashes with the following error, We have resolved the selinux
>>>> alerts by following the troubleshooting steps recommend by running
>>>> sealert,However we are concerned by setroubleshootd crashing and are
>>>> concered that we may have masked the issue by fixing the entries in the
>>>> audit.log.
>>>>>>>
>>>>>>> abrt_version:   2.0.8
>>>>>>>
>>>>>>> cmdline:        /usr/bin/python -Es /usr/sbin/setroubleshootd -f ''
>>>>>>>
>>>>>>> executable:     /usr/sbin/setroubleshootd
>>>>>>>
>>>>>>> kernel:         2.6.32-431.23.3.el6.x86_64
>>>>>>>
>>>>>>> last_occurrence: 1417101625
>>>>>>>
>>>>>>> time:           Thu 27 Nov 2014 03:20:25 PM UTC
>>>>>>>
>>>>>>> uid:            0
>>>>>>>
>>>>>>> username:       root
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> sosreport.tar.xz: Binary file, 3642240 bytes
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> backtrace:
>>>>>>>
>>>>>>> :analyze.py:426:lookup_signature:ProgramError: [Errno 1001] signature
>>>>>>> not found
>>>>>>>
>>>>>>> :
>>>>>>>
>>>>>>> :Traceback (most recent call last):
>>>>>>>
>>>>>>> :  File
>>>>>>> "/usr/lib64/python2.6/site-packages/setroubleshoot/analyze.py", line
>>>>>>> 401, in auto_save_callback
>>>>>>>
>>>>>>> :    self.save()
>>>>>>>
>>>>>>> :  File
>>>>>>> "/usr/lib64/python2.6/site-packages/setroubleshoot/analyze.py", line
>>>>>>> 377, in save
>>>>>>>
>>>>>>> :    self.prune()
>>>>>>>
>>>>>>> :  File
>>>>>>> "/usr/lib64/python2.6/site-packages/setroubleshoot/analyze.py", line
>>>>>>> 340, in prune
>>>>>>>
>>>>>>> :    self.delete_signature(sig, prune=True)
>>>>>>>
>>>>>>> :  File
>>>>>>> "/usr/lib64/python2.6/site-packages/setroubleshoot/analyze.py", line
>>>>>>> 471, in delete_signature
>>>>>>>
>>>>>>> :    siginfo = self.lookup_signature(sig)
>>>>>>>
>>>>>>> :  File
>>>>>>> "/usr/lib64/python2.6/site-packages/setroubleshoot/analyze.py", line
>>>>>>> 426, in lookup_signature
>>>>>>>
>>>>>>> :    raise ProgramError(ERR_NO_SIGNATURE_MATCH)
>>>>>>>
>>>>>>> :ProgramError: [Errno 1001] signature not found
>>>>>>>
>>>>>>> :
>>>>>>>
>>>>>>> :Local variables in innermost frame:
>>>>>>>
>>>>>>> :matches: []
>>>>>>>
>>>>>>> :siginfo: None
>>>>>>>
>>>>>>> :self: <setroubleshoot.analyze.SETroubleshootDatabase object at
>>>>>>> 0x151d590>
>>>>>>>
>>>>>>> :sig: <setroubleshoot.signature.SEFaultSignature object at 0x645a050>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> We are running the following versions Passenger/htttpd/node
>>>>>>>
>>>>>>>
>>>>>>> passenger --version
>>>>>>>
>>>>>>> Phusion Passenger version 4.0.53
>>>>>>>
>>>>>>>
>>>>>>> httpd -v
>>>>>>> Server version: Apache/2.2.15 (Unix)
>>>>>>> Server built:   Jul 23 2014 14:17:29
>>>>>>>
>>>>>>>
>>>>>>> node -v
>>>>>>> v0.10.32
>>>>>>>
>>>>>>> This email is from the Press Association. For more information, see
>>>> www.pressassociation.com. This email may contain confidential
>>>> information. Only the addressee is permitted to read, copy, distribute
>> or
>>>> otherwise use this email or any attachments. If you have received it in
>>>> error, please contact the sender immediately. Any opinion expressed in
>> this
>>>> email is personal to the sender and may not reflect the opinion of the
>>>> Press Association. Any email reply to this address may be subject to
>>>> interception or monitoring for operational reasons or for lawful
>> business
>>>> practices.
>>>>>>> _______________________________________________
>>>>>>> CentOS mailing list
>>>>>>> CentOS at centos.org
>>>>>>> http://lists.centos.org/mailman/listinfo/centos
>>>>>> _______________________________________________
>>>>>> CentOS mailing list
>>>>>> CentOS at centos.org
>>>>>> http://lists.centos.org/mailman/listinfo/centos
>>>>>>
>>>>>> This email is from the Press Association. For more information, see
>>>> www.pressassociation.com. This email may contain confidential
>>>> information. Only the addressee is permitted to read, copy, distribute
>> or
>>>> otherwise use this email or any attachments. If you have received it in
>>>> error, please contact the sender immediately. Any opinion expressed in
>> this
>>>> email is personal to the sender and may not reflect the opinion of the
>>>> Press Association. Any email reply to this address may be subject to
>>>> interception or monitoring for operational reasons or for lawful
>> business
>>>> practices.
>>>>>> _______________________________________________
>>>>>> CentOS mailing list
>>>>>> CentOS at centos.org
>>>>>> http://lists.centos.org/mailman/listinfo/centos
>>>>> _______________________________________________
>>>>> CentOS mailing list
>>>>> CentOS at centos.org
>>>>> http://lists.centos.org/mailman/listinfo/centos
>>>>>
>>>>> This email is from the Press Association. For more information, see
>>>> www.pressassociation.com. This email may contain confidential
>>>> information. Only the addressee is permitted to read, copy, distribute
>> or
>>>> otherwise use this email or any attachments. If you have received it in
>>>> error, please contact the sender immediately. Any opinion expressed in
>> this
>>>> email is personal to the sender and may not reflect the opinion of the
>>>> Press Association. Any email reply to this address may be subject to
>>>> interception or monitoring for operational reasons or for lawful
>> business
>>>> practices.
>>>>> _______________________________________________
>>>>> CentOS mailing list
>>>>> CentOS at centos.org
>>>>> http://lists.centos.org/mailman/listinfo/centos
>>>> _______________________________________________
>>>> CentOS mailing list
>>>> CentOS at centos.org
>>>> http://lists.centos.org/mailman/listinfo/centos
>>>>
>>>
>> _______________________________________________
>> CentOS mailing list
>> CentOS at centos.org
>> http://lists.centos.org/mailman/listinfo/centos
>>
>
>