[CentOS] What FileSystems for large stores and very very large stores?

Wed Aug 7 18:42:04 UTC 2013
Matti Aarnio <matti.aarnio at methics.fi>

On 08/07/2013 03:58 AM, Eliezer Croitoru wrote:
> OK so back to the issue in hands.
> The issue is that I have a mail storage for more then 65k users per
> domain and the ext4 doesn't support this size of directory list.
> The reiser FS indeed fits for the purpose but ext4 doesn't even start to
> scratch it.
> Now the real question is that:
> What FS will you use for dovecot backhand to store a domain with more
> then 65k users?
> Eliezer

It was back in 1995 when I had this kind of problem with about 0.05 M accounts, and our solution was used until at least 0.5 M accounts, when I left the company. The filesystem in question back then degraded severely in performance when there were more than about 200 files in a directory.

We ended up cooking our own way using FNV-1a hash, but Dovecot has something similar natively:


The "Directory hashing" is the interesting part, although that explanation does look like needing a complete rewrite.

Having lots of file names in directory will likely mean that a) your directory file is actually grown over time in small extents spanning all over the disk space and  b) thus its reading becomes very inefficient.

Having a hashed subdirectory structure will mean that a 4kB file system block size will likely not overflow , or at most have only a few extend blocks, and their reading will not be _that_ much slower.

Best Regards,  Matti Aarnio