On Sat, Mar 12, 2011 at 10:14 PM, Nico Kadel-Garcia <nkadel at gmail.com> wrote: > On Sat, Mar 12, 2011 at 4:04 PM, Alain Spineux <aspineux at gmail.com> wrote: >> Thanks to everybody for answering. >> >> I thing >250E6 is a lot and keep decent read and write access speed is unreal >> using mutli-purpose filesystems like ext? and other ?FS. >> I would need a dedicated filesystem for that. >> >> This problem was only a possible solution to another problem. >> I will solve the original problem using another way. >> > > Why, exactly, are you doing this? The normal approach for such dense > repositories is to create a hierarchy of subdirectories. > > File aaa12345 goes in > > $DIR/a/a/a/12345 > > File abc6789 goes in > > $DIR/a/b/c/6789 Try to create this king of tree yourself and when done, remove it. I took hours on my box and it is even faster to keep all files in the same diretory. I looks like working in multiple directories slow down the process also I read other posts and articles and handle more than 100M files become a problem ! 256M is a problem and more than 1G files is a big problem. I was splitting data into files to help me. I will keep them in big files. Regards Anyway it was too slow. > > And whatever is accessing or creating the files is taught the > algorithm used. This requires some programming up front, but helps > prevent precisely the outrageous directory size you describe. > Handling, sorting, and reporting on that many files in one directory > is an old and painful problem. > _______________________________________________ > CentOS mailing list > CentOS at centos.org > http://lists.centos.org/mailman/listinfo/centos > -- Alain Spineux | aspineux gmail com Monitor your iT & Backups | http://www.magikmon.com Free Backup front-end | http://www.magikmon.com/mksbackup Your email 100% available | http://www.emailgency.com