[CentOS] Ulimit problem - CentOS 5.10

On Apr 28, 2014, at 6:40 PM, Stephen Harris <lists at spuddy.org> wrote:

> On Mon, Apr 28, 2014 at 04:20:25PM -0600, Nathan Duehr wrote:
>> Seems like the brokenness is the behavior of init ignoring /etc/security/limits.conf, to my way of thinking anyway.
> 
> Umm, no.  That's you not understanding what limits.conf is.
> 
> Limits are hard to grok.  I had to write a massive document at work
> explaining it.  And people still don't get it.
> 


Actually, I completely understand, but I still had to step through it in various forms and write myself a flowchart.  It's beyond non-intuitive at this point in Linux "design" history.  

For starters, how Linux is handling it is *really* FUBAR compared to some of the commericial *nix variants, but that's kinda normal... Linux loves to be obtuse and overly-complex at times.  It's an endearing quality.  :-)  Note that if you had to "write a massive document" it's probably overly-complex for no good design reason. 


> Basically:
>  init scripts inherit from init (pid 1), which gets defaults from
>  the kernel
> 
>  Processes initiated by a user will inherit limits from the the user's
>  environment.  For most users that will have involved a PAM session,
>  and most PAM configs call pam_limits and _that_ reads limits.conf.


Not processes started that change to a non-root user from a root/init/rc script. No session. At least not from what I was seeing in 5.10.  Intended or not, it wasn't behaving like PAM was ever involved. :-)


>  Doing a 'su' will involve PAM and that may cause pam_limits (and thus
>  limits.conf) to be read.
> 
> Remember that init processes started at boot time will run as root
> and so can increase limits.  You need to increase hard limits before
> you increase soft limits.
> 
> Processes started as a user can _not_ increase hard limits.  You need to
> "su" to root, or "su" to a user defined in limits.conf to change those
> values.
> 
> Bottom line: limits.conf is a PAM config setting for pam_limits.  It's
> not in the general path.  Other process _may_ use the file but they need
> to have root level privs to obey it properly.


This is all true, but doesn't apply to the scenario presented.

The PAM stuff is entertaining... since most new admins would think to themselves, "I'll just become that user and see what that user's ulimit is..." do an "su - [user]" and see something and assume it's what would happen to the user's limits inside that start script.

Of course, the best place to look in the end, is /proc/[PID] ... 

Here's the real fun... the asterisk guys (at least back in this old version), punted... LOL... from their safe_asterisk secondary start script that gets called from their main init.d script... 


# since we're going to change priority and open files limits, we need to be
# root. if running asterisk as other users, pass that to asterisk on the command
# line.
# if we're not root, fall back to standard everything.
if test `id -u` != 0 ; then
        echo "Oops. I'm not root. Falling back to standard prio and file max." >&2
        echo "This is NOT suitable for large systems." >&2
        PRIORITY=0

Even more fun... if your non-Production environment is KVM virtuals, and your Production systems are on bare-metal... don't get me started on that one... but that's a pretty common scenario these days, too.  (Limits are exceedingly horked on KVM VMs... haha...)

Anyway, we're all sorted here, and I had a hell week doing a lot of other stuff in-between, but it was a fun reminder of the whole ulimit mess.  I suspect this stuff will be even more entertaining as init goes away in lieu of upstart and similar.  Heh.

Appreciate the thought put into the reply.  Perhaps someone struggling with this down the road will find the thread via Google or similar and realize they have to think a lot harder than really should be necessary to get it set where they want it.  Most "basics" type docs out there just point newbies to limits.conf and do all examples from a user's shell that they got to from su... which isn't how much really starts/runs on Production servers.  :-)

--
Nate Duehr
denverpilot at me.com