Craig White wrote:
On Tue, 2009-11-24 at 11:00 +0800, Christopher Chan wrote:
Ian Forde wrote:
On Nov 23, 2009, at 5:34 PM, Christopher Chan <christopher.chan@bradbury.edu.hk mailto:christopher.chan@bradbury.edu.hk> wrote:
Les Mikesell wrote:
You probably really want ldap for that sort of thing.
You probably really want to reconsider using ldap for anything that gets loads of changes daily.
In the case of a mail relay, at one point years back I decided to drop (not bounce) all email to bogus recipients at the relay level rather than let it get to (yuck) Exchange, which would bounce it. The trick was having an updated recipient list. My first thought was to query Active Directory for each user, thus getting an up-to-date result.
This turned out to be a *bad* idea for a couple of reasons. 1) if I can't reach AD, mail won't queue up on the relays, which is one of their major functions. 2) I'm making the relays directly dependent on AD latency. 3) any flood of email from outside can cause a large amount of queries against AD, causing a DOS that the relays are supposed to shield the internal network from.
So instead, I found a script to gather the list of users from AD, did some modifications and wrote some wrappers. The result? A script that runs from cron to get the list of valid addresses, convert them into an access file that sendmail (or postfix, in the first case years ago) can use instead. There's a little more latency, but as long as I do some sanity checking (too many changes? Send an alert and don't change the access file) it works just fine. Ldap-based, yes. But loosely coupled. A good compromise in my experience...
Precisely why a buffer like this for sites with a very large user base might want to use cdb. postfix supports cdb and sendmail can get cdb support from sf.net/sendmail-cdb. Both need the tinycdb library though. Even mysql/postgresql could do with a break for legit users.
considering that LDAP is optimized for high amounts of read and minimal writes, the problem with any SMTP daemon querying an LDAP server getting bogged down suggests that other problems are at hand and should be solved. I mean if the primary user/authentication system can't handle the load, you got problems.
I was trumpeting postfix's mysql/postgresql support and then Les says LDAP is the way to go and then I point out that LDAP don't like heavy write environments and you are starting the circle again.
/me tramples LDAP underfoot, gets a horse to trample LDAP, gets a tank to complete the job.
LDAP ain't THE SOLUTION for everything you know.
I admire the workarounds but damn, you have to solve the problems anyway because this surely isn't the only place where this is a problem.
Ian pointed how he needs to 'replicate' a local copy of user 'accounts' from Exchange so that he does not kill Exchange. I just pointed out that this sort of thing can be done also for sites with a very large user base that will want something that is more efficient that Berkeley DB. You can chain lookups in postfix. Check cdb, then check mysql/postgresql. If the account exists in the cdb, then there is no need to check mysql/postgresql. So essentially only non-existent addresses and recently created addresses will result in hits to mysql/postgresql. This is not a work around. This is performance enhancement. Whacking a local cdb will be faster than whacking a mysql/postgresql database. Geez.