On 7/24/11 4:08 PM, Keith Roberts wrote:
> On Sun, 24 Jul 2011, R P Herrold wrote:
>
>> By using a hash, we remove those constraints, and also gain
>> the virtuous effect for free of self-organizing a relatively
>> level dispersion of files to the destination directories
>
> Not followed the whole thread, but a SQL database index of
> the actual picture files, giving the path into the directory
> structure. Would that work?
You introduce new issues where the name in the database can't be managed
atomically with the name in the directory that way. Consider what might happen
with concurrent operations trying to add different files with the same name - or
perhaps an add and delete at the same times.
And it still doesn't help with the real problem unless you do something to break
up the large directory. Unix-like filesystems guarantee atomic operations in
filename manipulation, so every time you try to create a file, the system must
check that the name does not already exist, find an empty slot for the name and
insert it with the directory locked against other changes until that is
complete. Filesystems that index directories can help with the lookup, with the
tradeoff that additions require an index update.
--
Les Mikesell
lesmikesell at gmail.com