[CentOS] lots of small files in a folder on Linux centos

[Ljubomir Ljubojevic]

yonatan pingle wrote:
> On Sun, Jul 24, 2011 at 2:19 PM, Alexander Dalloz <ad+lists at uni-x.org> wrote:
>> Am 24.07.2011 13:03, schrieb Eero Volotinen:
>>> 2011/7/24 yonatan pingle <yonatan.pingle at gmail.com>:
>>>> uploads]# ls | wc -l
>>>> 3123
>>> I assume that you are using ext3 or ext4 filesystems? Both ext3 and
>>> ext4 slows down, if there is too much files in same directory.
>>> XFS-fs is solution to fix this problem.
>>> Eero
>> Seriously, 3123 files in a single directory is not an issue for any of
>> the extX filesystems. Though ext2 probably performs the worst, ext3 and
>> particular ext4 should not have any problem with that small amount of
>> file objects. Given that the filesystem is not already filled nearby 100%.
>>
>> An issue may be, how the code deals with the directory content. Horrible
>> code for sure can impact the speed of the website, but should not affect
>> the system globally.
>>
>> Yonatan, if you really are concerned about the uploads directory, then
>> use vmstat, iostat or sar to check system parameters while the directory
>> is accessed.
>>
>> Your problem is something else, I am pretty sure.
>>
>> Alexander
>>
>> _______________________________________________
>> CentOS mailing list
>> CentOS at centos.org
>> http://lists.centos.org/mailman/listinfo/centos
>>
> 
> 
> Hi, Alexander
> good suggestions, ill monitor I/O and mysql code, sounds like a code
> related issue and not a centos issue after all.
> 
> it runs on ext3  ,i could only guess how to code deals with the dir,
> as it seems to be the site builds the pages using php+mysql for each
> visitor, with about 40K unique visitors a day, that is a lot of I/O.
> 
> This looks like an issue with MySQL after all.
> Queries: 48.0M  qps:   66 Slow:    65.0
> 
> avg-cpu:  %user   %nice %system %iowait  %steal   %idle
>                      0.97    0.00    0.28   97.91    0.00    0.84
> 
>    runq-sz  plist-sz   ldavg-1   ldavg-5  ldavg-15
> 
>          0       102      5.30      3.13      2.06
>          2       120      3.14      2.77      2.22
> 
> we wait and see,
>  tail -f log-slow-queries.log
> /usr/sbin/mysqld, Version: 5.0.67-community-log (MySQL Community
> Edition (GPL)). started with:
> Tcp port: 3306  Unix socket: /var/lib/mysql/mysql.sock
> Time                 Id Command    Argument
> 
> 
> 
> thank you
> 
> 
> 

Do you have cahcing turned on in CMS? That could help.

-- 

Ljubomir Ljubojevic
(Love is in the Air)
PL Computers
Serbia, Europe

Google is the Mother, Google is the Father, and traceroute is your
trusty Spiderman...
StarOS, Mikrotik and CentOS/RHEL/Linux consultant