[CentOS] lots of small files in a folder on Linux centos

[yonatan pingle]

On Sun, Jul 24, 2011 at 2:19 PM, Alexander Dalloz <ad+lists at uni-x.org> wrote:
> Am 24.07.2011 13:03, schrieb Eero Volotinen:
>> 2011/7/24 yonatan pingle <yonatan.pingle at gmail.com>:
>
>>> uploads]# ls | wc -l
>>> 3123
>
>> I assume that you are using ext3 or ext4 filesystems? Both ext3 and
>> ext4 slows down, if there is too much files in same directory.
>> XFS-fs is solution to fix this problem.
>
>> Eero
>
> Seriously, 3123 files in a single directory is not an issue for any of
> the extX filesystems. Though ext2 probably performs the worst, ext3 and
> particular ext4 should not have any problem with that small amount of
> file objects. Given that the filesystem is not already filled nearby 100%.
>
> An issue may be, how the code deals with the directory content. Horrible
> code for sure can impact the speed of the website, but should not affect
> the system globally.
>
> Yonatan, if you really are concerned about the uploads directory, then
> use vmstat, iostat or sar to check system parameters while the directory
> is accessed.
>
> Your problem is something else, I am pretty sure.
>
> Alexander
>
> _______________________________________________
> CentOS mailing list
> CentOS at centos.org
> http://lists.centos.org/mailman/listinfo/centos
>


Hi, Alexander
good suggestions, ill monitor I/O and mysql code, sounds like a code
related issue and not a centos issue after all.

it runs on ext3  ,i could only guess how to code deals with the dir,
as it seems to be the site builds the pages using php+mysql for each
visitor, with about 40K unique visitors a day, that is a lot of I/O.

This looks like an issue with MySQL after all.
Queries: 48.0M  qps:   66 Slow:    65.0

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
                     0.97    0.00    0.28   97.91    0.00    0.84

   runq-sz  plist-sz   ldavg-1   ldavg-5  ldavg-15

         0       102      5.30      3.13      2.06
         2       120      3.14      2.77      2.22

we wait and see,
 tail -f log-slow-queries.log
/usr/sbin/mysqld, Version: 5.0.67-community-log (MySQL Community
Edition (GPL)). started with:
Tcp port: 3306  Unix socket: /var/lib/mysql/mysql.sock
Time                 Id Command    Argument



thank you



-- 
Best Regards,
Yonatan Pingle
RHCT | RHCSA | CCNA1