[CentOS] lots of small files in a folder on Linux centos

[Les Mikesell]

On 7/24/11 4:08 PM, Keith Roberts wrote:
> On Sun, 24 Jul 2011, R P Herrold wrote:
>
>> By using a hash, we remove those constraints, and also gain
>> the virtuous effect for free of self-organizing a relatively
>> level dispersion of files to the destination directories
>
> Not followed the whole thread, but a SQL database index of
> the actual picture files, giving the path into the directory
> structure. Would that work?

You introduce new issues where the name in the database can't be managed 
atomically with the name in the directory that way.  Consider what might happen 
with concurrent operations trying to add different files with the same name - or 
perhaps an add and delete at the same times.

And it still doesn't help with the real problem unless you do something to break 
up the large directory.   Unix-like filesystems guarantee atomic operations in 
filename manipulation, so every time you try to create a file, the system must 
check that the name does not already exist, find an empty slot for the name and 
insert it with the directory locked against other changes until that is 
complete.  Filesystems that index directories can help with the lookup, with the 
tradeoff that additions require an index update.

-- 
    Les Mikesell
     lesmikesell at gmail.com