[CentOS] After reboot of web-server accessing website shows "Forbidden", restarting httpd all is fine

[Mark Milhollan]

On Sat, 7 Nov 2015, Fred Smith wrote:
>On Fri, Nov 06, 2015 at 07:23:59PM -0800, Gordon Messmer wrote:
>>On 11/06/2015 06:30 PM, Jobst Schmalenbach wrote:

>>>What troubles me that a simple restart of the daemon fixes everything but it does not come up on reboot.
>> 
>>Running the service script manually may not give you the same
>>selinux context as on boot.  Services should be started using
>>"run_init" to ensure they get the correct context.

Yet it isn't really documented anywhere that you can or should use it, 
certainly not in the RHEL Administration or SELinux documentation, not 
even as a footnote.  Only in a few bug reports and errata notices can 
you even find mention of the command.

>How long has this been the case? I have never heard of this before,
>it seems a very well-kept secret!

Always, i.e., since SELinux appeared in RHEL and thus CentOS.

The service command has never done what is expected of it -- properly 
(re)start a service.  This is noted here and there in the documentation.  
It usually does well enough for the other management tasks, reloading, 
stopping and providing status.  But start and restart is almost totally 
wrong.

It allows (even forces) a "dirty" environment to be provided to the 
service (which is seldom wanted or expected), does not ensure that the 
current tty cannot be the controlling tty for the service (which 
sometimes matters) and leaves the CWD unchanged instead of ensuring / is 
used (which sometimes matters).  No revision of the service command took 
place to cope with context when SELinux appeared and so the service 
inherits the current context, usually unconfined (which is wildly 
wrong).  Sometimes doing it this way is useful, but not often and when 
it is one can invoke the service's init script directly.

So even before SELinux (or with it disabled) a mere "/sbin/service 
whichever start" does not always suffice.  But building a boot 
environment is a bit tedious, you'd almost want some command to take 
care of that for you -- alas none exists.  However the boot environment 
is usually very simple, which can be approximated with something like:

  ( cd / && setsid env -i /sbin/service whichever start )

With SELinux it is critical to run it in a more useful context:

  ( cd / && /usr/sbin/run_init setsid env -i /sbin/service whichever start )

If run_init isn't installed the following gives a similar result:

  ( cd / && runcon -u system_u setsid env -i /sbin/service whichever start )

Also for restart, condrestart, try-restart or --full-restart.  And for 
some services reload or force-reload too, thus easiest to always use one 
of these for all service management needs.  (To be fair, I don't usually 
use it for status or stop, but I was bitten once.)  Alas the latter two 
fail if SELinux is disabled at which point you trim it back to the 
first.

Patching the service command locally is a hassle -- it is a shell script 
so not at all difficult to change, merely difficult to remember to 
re-patch after system updates lest you go back to being surprised 
(though a custom RPM with a trigger can do it) or go elsewhere where it 
hasn't ever been patched.  Or produce your own script that does all 
that, so that only when it isn't available do you need to type a long 
line to (re)start a service.  I leave the service command unpatched and 
didn't write my own script, instead I have the subshell things as muscle 
memory.

With systemd (CentOS 7) this changes.  If you know systemd handles the 
service -- which is most of the time -- you can use the bare service 
command (or systemctl) to (re)start the service, in an environment that 
is the same as at boot, i.e., as expected.  But not everything is 
controlled by systemd, and there you are back where this started.


/mark