[CentOS] puppet files denied by SELinux

[Daniel J Walsh]

I have no idea of the current dependency problem.  I think your original
problem was caused by mv'ing files from an nfs share to /etc which
maintained the context.  And SELinux prevented puppet from accessing
nfs_t type.  If you had just run restorecon on the object it would have
set it back to the correct/default context.

You might want to setup an alias mv "mv -Z"

This changes the way mv works to set the context after mv rather then
maintaining the source context.

On 06/21/2015 02:05 PM, Tim Dunphy wrote:
> Hey guys,
>
>  Quick update. I grepped through the output of getsebool -a to see that
> related to puppet. And I found this setting: puppetagent_manage_all_files.
>
>  So I tried running this command: setsebool -P puppetagent_manage_all_files
> 0
>
>  And did a restorecon on my modules directory: restorecon -R -v
> environments/production/moudles
>
>  So there's good news and bad news to report! It seems that now puppet on
> the client isn't complaining about not having access to the cert and key
> files anymore! That's the good news. The bad news is, when I do puppet runs
> on all the hosts now, I get the following errors:
>
> Notice: /File[/var/lib/puppet/lib/facter/concat_basedir.rb]: Dependency
> File[/var/lib/puppet/lib] has failures: true
> Warning: /File[/var/lib/puppet/lib/facter/concat_basedir.rb]: Skipping
> because of failed dependencies
> Notice: /File[/var/lib/puppet/lib/facter/ssldir.rb]: Dependency
> File[/var/lib/puppet/lib] has failures: true
> Warning: /File[/var/lib/puppet/lib/facter/ssldir.rb]: Skipping because of
> failed dependencies
> Notice:
> /File[/var/lib/puppet/lib/puppet/parser/functions/ensure_resource.rb]:
> Dependency File[/var/lib/puppet/lib] has failures: true
> Warning:
> /File[/var/lib/puppet/lib/puppet/parser/functions/ensure_resource.rb]:
> Skipping because of failed dependencies
> Notice: /File[/var/lib/puppet/lib/puppet/parser/functions/validate_re.rb]:
> Dependency File[/var/lib/puppet/lib] has failures: true
> Warning: /File[/var/lib/puppet/lib/puppet/parser/functions/validate_re.rb]:
> Skipping because of failed dependencies
> Notice: /File[/var/lib/puppet/lib/puppet/reports/datadog_reports.rb]:
> Dependency File[/var/lib/puppet/lib] has failures: true
> Warning: /File[/var/lib/puppet/lib/puppet/reports/datadog_reports.rb]:
> Skipping because of failed dependencies
> Notice:
> /File[/var/lib/puppet/lib/puppet/parser/functions/is_function_available.rb]:
> Dependency File[/var/lib/puppet/lib] has failures: true
> Warning:
> /File[/var/lib/puppet/lib/puppet/parser/functions/is_function_available.rb]:
> Skipping because of failed dependencies
> Notice:
> /File[/var/lib/puppet/lib/puppet/parser/functions/str2saltedsha512.rb]:
> Dependency File[/var/lib/puppet/lib] has failures: true
> Warning:
> /File[/var/lib/puppet/lib/puppet/parser/functions/str2saltedsha512.rb]:
> Skipping because of failed dependencies
> Notice:
> /File[/var/lib/puppet/lib/puppet/parser/functions/delete_undef_values.rb]:
> Dependency File[/var/lib/puppet/lib] has failures: true
> Warning:
> /File[/var/lib/puppet/lib/puppet/parser/functions/delete_undef_values.rb]:
> Skipping because of failed dependencies
> Notice: /File[/var/lib/puppet/lib/puppet/parser/functions/fqdn_rotate.rb]:
> Dependency File[/var/lib/puppet/lib] has failures: true
> Warning: /File[/var/lib/puppet/lib/puppet/parser/functions/fqdn_rotate.rb]:
> Skipping because of failed dependencies
> Notice: /File[/var/lib/puppet/lib/facter/gemhome.rb]: Dependency
> File[/var/lib/puppet/lib] has failures: true
> Warning: /File[/var/lib/puppet/lib/facter/gemhome.rb]: Skipping because of
> failed dependencies
> Notice: /File[/var/lib/puppet/lib/puppet/parser/functions/values_at.rb]:
> Dependency File[/var/lib/puppet/lib] has failures: true
> Warning: /File[/var/lib/puppet/lib/puppet/parser/functions/values_at.rb]:
> Skipping because of failed dependencies
> Notice: /File[/var/lib/puppet/lib/puppet/parser/functions/getvar.rb]:
> Dependency File[/var/lib/puppet/lib] has failures: true
> Warning: /File[/var/lib/puppet/lib/puppet/parser/functions/getvar.rb]:
> Skipping because of failed dependencies
> Notice: /File[/var/lib/puppet/lib/puppet/provider/vcsrepo/cvs.rb]:
> Dependency File[/var/lib/puppet/lib] has failures: true
> Warning: /File[/var/lib/puppet/lib/puppet/provider/vcsrepo/cvs.rb]:
> Skipping because of failed dependencies
> Notice: /File[/var/lib/puppet/lib/puppet/parser/functions/strftime.rb]:
> Dependency File[/var/lib/puppet/lib] has failures: true
> Warning: /File[/var/lib/puppet/lib/puppet/parser/functions/strftime.rb]:
> Skipping because of failed dependencies
> Notice: /File[/var/lib/puppet/lib/puppet/parser/functions/chop.rb]:
> Dependency File[/var/lib/puppet/lib] has failures: true
> Warning: /File[/var/lib/puppet/lib/puppet/parser/functions/chop.rb]:
> Skipping because of failed dependencies
> Notice: /File[/var/lib/puppet/lib/puppet/util/firewall.rb]: Dependency
> File[/var/lib/puppet/lib] has failures: true
> Warning: /File[/var/lib/puppet/lib/puppet/util/firewall.rb]: Skipping
> because of failed dependencies
> Notice: /File[/var/lib/puppet/lib/puppet/parser/functions/is_float.rb]:
> Dependency File[/var/lib/puppet/lib] has failures: true
> Warning: /File[/var/lib/puppet/lib/puppet/parser/functions/is_float.rb]:
> Skipping because of failed dependencies
> Notice: /File[/var/lib/puppet/lib/puppet/parser/functions/parsejson.rb]:
> Dependency File[/var/lib/puppet/lib] has failures: true
> Warning: /File[/var/lib/puppet/lib/puppet/parser/functions/parsejson.rb]:
> Skipping because of failed dependencies
> Notice: /File[/var/lib/puppet/lib/puppet/parser/functions/validate_cmd.rb]:
> Dependency File[/var/lib/puppet/lib] has failures: true
> Warning:
> /File[/var/lib/puppet/lib/puppet/parser/functions/validate_cmd.rb]:
> Skipping because of failed dependencies
>
> It's actually a long list of errors that's too long to reproduce here. It'd
> go on for a couple pages at least.
>
> However if I turn off SELinux on the puppet master, everything returns to
> normal. Goes from utter chaos to complete order in an instant!
>
> So I guess I've muffed up my SELinux config on this puppet host. I just
> hope it's repairable at this point! I'd hate to leave it off just so that
> puppet will be able to do it's job. And of all the hosts that would need
> SELinux protection I would think that a puppet host would be one of the
> most important if not 'the' most important to protect!
>
> I'm definitely open to suggestions at this point!
>
> Thanks,
> Tim
>
> On Sun, Jun 21, 2015 at 11:11 AM, Tim Dunphy <bluethundr at gmail.com> wrote:
>
>> Hi all,
>>
>> Thanks for all your suggestions. Here's where I'm at with this.
>>
>> Can you give details about your puppetmasterd setup ? it seems that
>>> you're using Foreman as puppet ENC.
>>>
>> Yes, I'm on foreman 1.7.4 and puppet 3.75. You are correct that I'm using
>> foreman, sorry I hadn't thought to mention it!
>>
>>
>>> Foreman works fine with selinux enabled : that's what we use for the
>>> centos.org infra :-)
>>> Which version of puppet/foreman are you using ? Note that foreman has
>>> the foreman-selinux package that is used to automatically tune
>>> contexts and booleans needed for this.
>>> You can still reapply those settings with
>>> /usr/sbin/foreman-selinux-{disable,enable,relabel}
>>> There is no need to recompile a custom selinux policy for
>>> foreman/puppet those days
>>>
>> I didn't recompile any custom selinux policies. All I did to try to
>> resolve the issue is to consult audit2allow and install the module it
>> suggested.
>> I did try running /usr/sbin/foreman-selinux-enable but that didn't seem
>> to have an effect.
>>
>> Knowing nothing of your scenario, look at the source and target context.
>>> Looks like you copied a crt from an nfs location and you don't have a
>>> file context defined to transition labels, maybe something like:
>>>
>>> semanage fcontext -a -t passenger_t "/etc/puppet/environments(/.*)?"
>>>
>>> However, I know nothing of puppets selinux infrastructure, you may need
>>> a more applicable  type.
>>>
>>> In these cases, audit2allow can't possibly guess the right thing and will
>>> certainly produce a rule that is either unsafe or simply wrong.
>>>
>> You are correct that I copied the key and cert from an NFS share! Both the
>> puppet server and the monitor1 client share the same /home directory via
>> NFS. Pretty cool that you picked up on that! I do suspect you're probably
>> right that this may be causing the problem. Just on a hunch, I tried
>> copying the certs and keys from the montior1 host over to the puppet host
>> to the /tmp directory on the puppet server. That leaves out NFS altogether.
>> And when I do that, my bacula puppet module WORKS!! Puppet doesn't complain
>> at all!
>>
>> But if I check out another host where I copied the cert and key from the
>> NFS home directory I still get the error:
>>
>> Error:
>> /Stage[main]/Bacula::Config/File[/etc/pki/tls/private/monitor2.mydomain.com.key]:
>> Could not evaluate: Could not retrieve information from environment
>> production source(s)
>> puppet:///modules/bacula/monitor2/monitor2.mydomain.com.key
>> Error:
>> /Stage[main]/Bacula::Config/File[/etc/pki/tls/certs/monitor2.mydomain.com.crt]:
>> Could not evaluate: Could not retrieve information from environment
>> production source(s)
>> puppet:///modules/bacula/monitor2/monitor2.mydomain.com.crt
>>
>> Also when I try to set context using the line you suggested I get an
>> error:
>>
>> #semanage fcontext -a -t passenger_t "/etc/puppet/environments(/.*)?"
>> ValueError: Type passenger_t is invalid, must be a file or device type
>>
>> So I googled around and found what seems to be the correct syntax:
>>
>> semanage fcontext -a -t passenger_exec_t "/etc/puppet/environments(/.*)?"
>>
>> Because when I applied that line, I didn't get any errors or complaints.
>> However the problem still existed on the monitor2 host which had the key
>> pair copied from the NFS share.
>>
>> So in summary it appears that there is some interaction between SELinux
>> and NFS that is causing the issue.
>>
>> Any thoughts?
>>
>> Thanks,
>> Tim
>>
>> On Sun, Jun 21, 2015 at 11:09 AM, Tim Dunphy <bluethundr at gmail.com> wrote:
>>
>>> Yes, you did when you used the audit2allow with the -M option argument
>>>> of "puppet", which is confirmed by the command you issued to try to load
>>>> it "semodule -i puppet.pp" (which you stated in your original message).
>>>> I'm okay with you asserting otherwise and not following my first
>>>> suggestion -- my second is to use a totally different name, e.g., "barf"
>>>> and thus "semodule -i barf.pp".
>>>
>>> Haha!! Ok man. I get you now. Thanks. Also I meant to send this to the
>>> list.. Whoops! I'll try doing it again with something like 'my' in the
>>> front. I remember having a similar problem with Zabbix last week that I
>>> solved this way.
>>>
>>> On Sun, Jun 21, 2015 at 12:19 AM, Mark Milhollan <mlm at pixelgate.net>
>>> wrote:
>>>
>>>> On Sat, 20 Jun 2015, Tim Dunphy wrote:
>>>>> I wrote:
>>>>>> That suggests there's already a module named puppet, and thus you are
>>>>>> replacing it with the one you made which does not supply the
>>>>>> puppet_var_lib_t type.  Always prefix your own modules with something
>>>>>> that makes them almost certain to be unique, e.g., yourdom_puppet.
>>>>>>
>>>>> No, actually I didn't compile my own selinux module. :) Not sure how you
>>>>> got that idea, but that is not the case.
>>>> Yes, you did when you used the audit2allow with the -M option argument
>>>> of "puppet", which is confirmed by the command you issued to try to load
>>>> it "semodule -i puppet.pp" (which you stated in your original message).
>>>> I'm okay with you asserting otherwise and not following my first
>>>> suggestion -- my second is to use a totally different name, e.g., "barf"
>>>> and thus "semodule -i barf.pp".
>>>>
>>>>
>>>> /mark
>>>>
>>>
>>>
>>> --
>>> GPG me!!
>>>
>>> gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B
>>>
>>>
>>
>> --
>> GPG me!!
>>
>> gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B
>>
>>
>