[CentOS] [Fwd: Re: The directory that I am trying to clean up is huge]

Mon Jan 25 14:48:29 UTC 2010
James B. Byrne <byrnejb at harte-lyne.ca>


On Sat, January 23, 2010 20:21, Robert Nichols wrote:
> Robert Heller wrote:

>
> Gosh, then I guess the manpage for 'find' must be totally wrong
> where it
> says:
>
>         -exec command ;
>                ...
>                The specified command is run once for each matched
> file.
>

Not wrong, just not very explicit regarding process.  The man page
does not say that find acts upon each file as it is found, only that
it acts upon each file that is found.  Neither does the man page
speak to the limits of the kernel configuration (MAX_ARG_PAGES)
implicitly used by cp, find, ls, etc.

The problem you have is that the selection of all qualified files is
completed before any are acted on by find.  So, in your case of an
overlarge collection, the page limit is exceeded before any are
deleted.  Taking each file as it is found and piping it to an
external rm command explicitly defines the process as find, delete
and find again, and thereby avoids hitting the page limit.

CentOS-5.3 was supposed to address this issue:

> Previously, the MAX_ARG_PAGES limit that is set in the kernel
> was too low, and may have resulted in the following error:
>
> execve: Argument list too long
>
> In this update, this limit has been increased to 25 percent of
> the stack size, which resolves this issue.

So, perhaps if you update to 5.3+ the problem might go away?
Although, in my opinion, piping find results through xargs is far
more reliable and portable.

Regards,

-- 
***          E-Mail is NOT a SECURE channel          ***
James B. Byrne                mailto:ByrneJB at Harte-Lyne.ca
Harte & Lyne Limited          http://www.harte-lyne.ca
9 Brockley Drive              vox: +1 905 561 1241
Hamilton, Ontario             fax: +1 905 561 0757
Canada  L8E 3C3