[CentOS] Large scale Postfix/Cyrus email system for 100, 000+ users

Tue Oct 23 18:33:25 UTC 2007
mouss <mlist.only at free.fr>

Ross S. W. Walker wrote:
> mouss wrote:
>> mouss wrote:
>>> Matt Shields wrote:
>>>> I'm trying to set up a large scale email system that 
>> supports 100,000+
>>>> IMAP accounts.  We have an existing frontend web interface 
>> that does a
>>>> lookup on a mysql db to figure out which IMAP server to 
>> connect to for
>>>> each user.  For the email infrastructure we have decided on Postfix
>>>> and Cyrus.  We have configured both to use mysql to get the virtual
>>>> user information.
>>>>
>>>> Because of the way that the infrastructure is (biz 
>> reasons) we are not
>>>> doing shared storage, we have numerous IMAP servers that 
>> we distribute
>>>> accounts across.  As we add more users, we image up a new 
>> IMAP server.
>>>>  For our business's scaling purposes this was the best plan.
>>>>
>>>> What I am having a problem is how do I get postfix to transfer the
>>>> email to the particular IMAP server that the user account is on.  I
>>>> know that I need to use lmtp and transport, but all the examples I
>>>> have seen show forwarding all email to 1 IMAP server.  I would like
>>>> Postfix to do a lookup for each mailbox and determine which IMAP
>>>> server to deliver it to.
>>> There are primarily two ways:
>>>
>>> [virtual aliase]
>>> you can use virtual_alias_maps to redirect foo at example.com to
>>> foo at hostN.example.com, provided the final server accepts 
>> such addresses.
>>> If the final server doesn't accept these, and you use smtp 
>> to relay to,
>>> then you can write the addresses back, using smtp_generic_maps.
>>>
>>> [transport]
>>> an laternative is to use use (per-user) transport_maps. 
>> something like
>>> foo at example.com		relay:[hostN.example.com]
>>>
>>>
>>> In bothe approaches, the mappings can be generated using 
>> sql statements
>>> (mostly CONCAT). something like
>>> ...
>>> query = SELECT concat('relay:[', host, '.example.com]')
>>> 	FROM User
>>> 	where '%u' = user and '%d' = domain
>>>
>>> you get the idea I hope.
> 
> True, it may be better to just have a cron job dump out
> new static maps every 15 minutes or so then to have the
> MTA query on every delivery especially for 100K accounts.
> 

indeed. and if the table has a "status" field so that the script can
download only new or modified entries, then the dump can be made faster.
now, a trigger may even be better so that the dump script doesn't need
to query the full table, but only a small table of new/modified entries
generated by the trigger.