Hi all,
As the resident Linux guru, I've just been tasked with costing a webmail setup for about 600 000 users. They each have 10MiB (small, I know) mailboxes. The current setup has about 40 million web page accesses per month.
Has anyone here any experience with this kind of thing? If so, any pointers as to software and hardware used, and any other advice would be appreciated.
TIA
www.atmaill.com www.shupp.org http://www.qmailtoaster.com
I use atmail with multi server mode
Neil Thompson wrote:
Hi all,
As the resident Linux guru, I've just been tasked with costing a webmail setup for about 600 000 users. They each have 10MiB (small, I know) mailboxes. The current setup has about 40 million web page accesses per month.
Has anyone here any experience with this kind of thing? If so, any pointers as to software and hardware used, and any other advice would be appreciated.
TIA
Michael Hill a écrit :
www.atmaill.com www.shupp.org http://www.qmailtoaster.com
I use atmail with multi server mode
Neil Thompson wrote:
Hi all,
As the resident Linux guru, I've just been tasked with costing a webmail setup for about 600 000 users. They each have 10MiB (small, I know) mailboxes. The current setup has about 40 million web page accesses per month.
You can use, of course, squirrelmail ... but you can try too a new one (we use it in Air-Austral for 400 users);
round cube (php, using Ajax, and accessing email accounts using imap): http://www.roundcube.net/
Regards
js.
Has anyone here any experience with this kind of thing? If so, any pointers as to software and hardware used, and any other advice would be appreciated.
TIA
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Michael Hill wrote:
www.atmaill.com www.shupp.org http://www.qmailtoaster.com
I use atmail with multi server mode
What procedures have you taken to make it high availability?
Regards,
Ugo
On Wed, 2007-02-14 at 07:22 +0200, Neil Thompson wrote:
Hi all,
As the resident Linux guru, I've just been tasked with costing a webmail setup for about 600 000 users. They each have 10MiB (small, I know) mailboxes. The current setup has about 40 million web page accesses per month.
Has anyone here any experience with this kind of thing? If so, any pointers as to software and hardware used, and any other advice would be appreciated.
TIA
Personally ... I think horde is good if you are looking to do webmail on an existing mail system. You should be able to tie this to just about any existing IMAP mail server.
If you are looking to install a mailserver and webmail at the same time, I think Scalix is a good bet. It includes the ability to have 25 premium users (ie, 25 people can use the outlook premium mail connector) and everyone else can use IMAP / POP, etc.
I think that the Scalix webmail interface (ajax based) is the best I have ever used.
Johnny Hughes wrote:
On Wed, 2007-02-14 at 07:22 +0200, Neil Thompson wrote:
Hi all,
As the resident Linux guru, I've just been tasked with costing a webmail setup for about 600 000 users. They each have 10MiB (small, I know) mailboxes. The current setup has about 40 million web page accesses per month.
Has anyone here any experience with this kind of thing? If so, any pointers as to software and hardware used, and any other advice would be appreciated.
TIA
Personally ... I think horde is good if you are looking to do webmail on an existing mail system. You should be able to tie this to just about any existing IMAP mail server.
If you are looking to install a mailserver and webmail at the same time, I think Scalix is a good bet. It includes the ability to have 25 premium users (ie, 25 people can use the outlook premium mail connector) and everyone else can use IMAP / POP, etc.
I think that the Scalix webmail interface (ajax based) is the best I have ever used.
I've been using Zimbra (www.zimbra.com). It has some heftier-than-most-solutions server requirements, but the web interface is very nice.
-- jeremy
As the resident Linux guru, I've just been tasked with costing a webmail setup for about 600 000 users. They each have 10MiB (small, I know) mailboxes. The current setup has about 40 million web page accesses per month.
Has anyone here any experience with this kind of thing? If so, any pointers as to software and hardware used, and any other advice would be appreciated.
While not specifically a costing point, if your users are on your LAN, and expect the quickest possible response times, and if your webmail is a collection of pages, and not a single cgi, I recommend placing the pages of your webmail system on a ramdisk.
Barry
On Wed, 14 Feb 2007 07:22:55 +0200 Neil Thompson abraxis@telkomsa.net wrote:
Hi all,
As the resident Linux guru, I've just been tasked with costing a webmail setup for about 600 000 users. They each have 10MiB (small, I know) mailboxes. The current setup has about 40 million web page accesses per month.
Has anyone here any experience with this kind of thing? If so, any pointers as to software and hardware used, and any other advice would be appreciated.
Been there, done that.
If your webmail of choice uses imap to access mailboxes (all php based do), then your primary concern is I/O of your mail storage and your imap server. Cyrus does well on many mail boxes (see fastmail.fm and their blogs for nice example). For storage choose appropriate SAN (i recommend fibrechannel) with many spindles (the more, the better the responsiveness) and nice ammount of write cache. We have about 350k users on 1Tb single cyrus instance, but that turned out to be a bit of a pain when memory error shits down on your filesystem and fsck takes almost a day to fix it. So I'd recommend you to split your users across many smaller cyrus volumes (fastmail.fm did that too) that can be fscked independently in a reasonable amount of time. For filesystem choice, nothing beats reiserfs in maildir like stoarge scenarion.
If you want to know more, just ask.
On Thu, 15 Feb 2007 at 2:35pm, Jure Pečar wrote
For filesystem choice, nothing beats reiserfs in maildir like stoarge scenarion.
*shudder* That may be so in terms of performance, but what about reliability and longevity? Given RH's treatment of non-ext3 FSs (and reiserfs's murky future in general), I'd be *very* hesitant to use reiserfs for anything.
On Thu, 15 Feb 2007 08:40:50 -0500 (EST) Joshua Baker-LePain jlb17@duke.edu wrote:
*shudder* That may be so in terms of performance, but what about reliability and longevity? Given RH's treatment of non-ext3 FSs (and reiserfs's murky future in general), I'd be *very* hesitant to use reiserfs for anything.
As long as your hw is ok, reiserfs works (>=2.4.18, dont know about 2.6). When you get a noncorrected bitflip in memory, it tends to propagate down to fs and make a nonnoticeable or huge disaster, depending on where it lands. That's true for all filesystems, see: http://www.cs.wisc.edu/adsl/Publications/iron-sosp05.pdf
Solaris10 ZFS is immune to that thanks to its cheksumming.
Also, personally I've lost much more data on ext3 than on reiserfs. As people tend to say, YMMV.
It's true that right now there is no filesystem on linux I'd really prefer for such task. Ext3 doesn't cope with the load, ext4 seems to do nothing to improve that, reiser3 is here but limited in today's multicpu machines due to its BKL, reiser4 is "almost there" for at least two years now due to kernel politics, and I still hear mixed things about xfs ... Maybe veritas vxfs, which became free to use on smaller x86_64 machines? http://www.symantec.com/enterprise/sfbasic/index.jsp
On Thu, 15 Feb 2007, Jure [UTF-8] Pe�~Mar wrote:
On Thu, 15 Feb 2007 08:40:50 -0500 (EST) Joshua Baker-LePain jlb17@duke.edu wrote:
*shudder* That may be so in terms of performance, but what about reliability and longevity? Given RH's treatment of non-ext3 FSs (and reiserfs's murky future in general), I'd be *very* hesitant to use reiserfs for anything.
As long as your hw is ok, reiserfs works (>=2.4.18, dont know about 2.6). When you get a noncorrected bitflip in memory, it tends to propagate down to fs and make a nonnoticeable or huge disaster, depending on where it lands. That's true for all filesystems, see: http://www.cs.wisc.edu/adsl/Publications/iron-sosp05.pdf
Solaris10 ZFS is immune to that thanks to its cheksumming.
Also, personally I've lost much more data on ext3 than on reiserfs. As people tend to say, YMMV.
It's true that right now there is no filesystem on linux I'd really prefer for such task. Ext3 doesn't cope with the load, ext4 seems to do nothing to improve that, reiser3 is here but limited in today's multicpu machines due to its BKL, reiser4 is "almost there" for at least two years now due to kernel politics, and I still hear mixed things about xfs ... Maybe veritas vxfs, which became free to use on smaller x86_64 machines? http://www.symantec.com/enterprise/sfbasic/index.jsp
I'm certainly no fs expert, but have you considered GFS in single (nolock) mode?
Barry
On Thu, 15 Feb 2007 08:50:13 -0600 (CST) Barry Brimer lists@brimer.org wrote:
I'm certainly no fs expert, but have you considered GFS in single (nolock) mode?
Yes. Waste of time even becnhmarking it.
Jure Pečar wrote:
*shudder* That may be so in terms of performance, but what about reliability and longevity? Given RH's treatment of non-ext3 FSs (and reiserfs's murky future in general), I'd be *very* hesitant to use reiserfs for anything.
As long as your hw is ok, reiserfs works (>=2.4.18, dont know about 2.6). When you get a noncorrected bitflip in memory, it tends to propagate down to fs and make a nonnoticeable or huge disaster, depending on where it lands. That's true for all filesystems, see: http://www.cs.wisc.edu/adsl/Publications/iron-sosp05.pdf
Solaris10 ZFS is immune to that thanks to its cheksumming.
I think it is a lot to ask from a filesystem to fix memory errors that already happened in the buffer space before writing it out and I don't have a lot of faith in that really working.
Anyway the 'old-school' way of handling a lot of mail users was to use a NetApp filer with whatever number of mail servers you needed for the user load doing NFS mounts and using maildir format to minimize the locking issues. It should still be a sure bet.
Jure Pečar spake the following on 2/15/2007 6:04 AM:
On Thu, 15 Feb 2007 08:40:50 -0500 (EST) Joshua Baker-LePain jlb17@duke.edu wrote:
*shudder* That may be so in terms of performance, but what about reliability and longevity? Given RH's treatment of non-ext3 FSs (and reiserfs's murky future in general), I'd be *very* hesitant to use reiserfs for anything.
As long as your hw is ok, reiserfs works (>=2.4.18, dont know about 2.6). When you get a noncorrected bitflip in memory, it tends to propagate down to fs and make a nonnoticeable or huge disaster, depending on where it lands. That's true for all filesystems, see: http://www.cs.wisc.edu/adsl/Publications/iron-sosp05.pdf
Solaris10 ZFS is immune to that thanks to its cheksumming.
Also, personally I've lost much more data on ext3 than on reiserfs. As people tend to say, YMMV.
It's true that right now there is no filesystem on linux I'd really prefer for such task. Ext3 doesn't cope with the load, ext4 seems to do nothing to improve that, reiser3 is here but limited in today's multicpu machines due to its BKL, reiser4 is "almost there" for at least two years now due to kernel politics, and I still hear mixed things about xfs ... Maybe veritas vxfs, which became free to use on smaller x86_64 machines? http://www.symantec.com/enterprise/sfbasic/index.jsp
I have heard some promising things about JFS, but even though Big Blue is behind it, it is also fairly immature.
Scott Silva wrote:
I have heard some promising things about JFS, but even though Big Blue is behind it, it is also fairly immature.
jfs originated in AIX and it's been there for many years.
IBM ported it to OS/2 back when I was primarily an OS/2 user (prior to RHL 5.0). It's been around for a very long time.
However, use of any filesystem not supported by your vendor (and essentially that's Red Hat here) has its risks, and the maintenance task is all yours.
John Summerfield wrote:
Scott Silva wrote:
I have heard some promising things about JFS, but even though Big Blue is behind it, it is also fairly immature.
jfs originated in AIX and it's been there for many years.
IBM ported it to OS/2 back when I was primarily an OS/2 user (prior to RHL 5.0). It's been around for a very long time.
However, use of any filesystem not supported by your vendor (and essentially that's Red Hat here) has its risks, and the maintenance task is all yours.
JFS appears to receive rather few changes/patches. It would appear to be pretty stable with the second best performance in just about every category in benchmarks. However, the question is whether the lack of bug reports is due to lack of wide spread usage or whether it is really stable code.
Jure Pečar wrote:
O
Also, personally I've lost much more data on ext3 than on reiserfs. As people tend to say, YMMV.
It's true that right now there is no filesystem on linux I'd really prefer for such task. Ext3 doesn't cope with the load, ext4 seems to do nothing to improve that, reiser3 is here but limited in today's multicpu machines due to its BKL, reiser4 is "almost there" for at least two years now due to kernel politics, and I still hear mixed things about xfs ...
I gather Hans has some personal difficulties that may distract from his development of reiserfs4.
I gather Hans has some personal difficulties that may distract from his development of reiserfs4.
Personal difficulties that might not have been had there been no kernel politics in the way. I, personally, look forward to Open Solaris getting more device and filesystem support.
On Sun, 18 Feb 2007 at 6:43pm, Feizhou wrote
I gather Hans has some personal difficulties that may distract from his development of reiserfs4.
Personal difficulties that might not have been had there been no kernel politics in the way. I, personally, look forward to Open Solaris getting more device and filesystem support.
Erm, are you trying to say that kernel politics had something to do with him being arrested and charged with the murder of his wife?
Joshua Baker-LePain wrote:
On Sun, 18 Feb 2007 at 6:43pm, Feizhou wrote
I gather Hans has some personal difficulties that may distract from his development of reiserfs4.
Personal difficulties that might not have been had there been no kernel politics in the way. I, personally, look forward to Open Solaris getting more device and filesystem support.
Erm, are you trying to say that kernel politics had something to do with him being arrested and charged with the murder of his wife?
No, I am trying to say that if kernel politics had not got in the way of reiser4 inclusion into the kernel, Hans Reiser's family situation could have turned out very different from its current sad state of affairs.
For filesystem choice, nothing beats reiserfs in maildir like stoarge scenarion.
If you want to know more, just ask.
--
Jure PeÄar http://jure.pecar.org _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Considering that distros are moving away from resierfs and its future is uncertain I would avoid it if possible. I've also lost a few volumes because of reiserfs so I don't trust it. XFS or JFS perform just as well or even better, ext3 with the right options is also just as fast according to recent benchmarks.
On Thu, 15 Feb 2007 09:21:18 -0500 (EST) wattersm@liquidweb.com wrote:
Considering that distros are moving away from resierfs and its future is uncertain I would avoid it if possible. I've also lost a few volumes because of reiserfs so I don't trust it. XFS or JFS perform just as well or even better, ext3 with the right options is also just as fast according to recent benchmarks.
And what were they benchmarking?
Remember, always benchmark with your application. Recent reports from cyrus mailing lists indicate that ext3 is still not up to the task, dirhash and journal tweaking included.
As others already suggested, using NetApp and NFS with courier or dovecot (cyurs has issues with NFS) is an option to avoid all this linux fs mess. If your budget can afford it :)
-
Barry Brimer -
Feizhou -
Jeremy Rosengren -
John Summerfield -
Johnny Hughes -
Joshua Baker-LePain -
Jure Pečar -
Les Mikesell -
Michael Hill -
Neil Thompson -
Scott Silva -
security -
Ugo Bellavance -
wattersm＠liquidweb.com