----- Message d'origine -----De: "redhat@mckerrs.net" redhat@mckerrs.netDate: Lundi, Novembre 12, 2007 9:25 pmObjet: Re: [CentOS] Need advice on storageÀ: CentOS mailing list centos@centos.org> Guy, > > I don't think raptor hard drives are necessarily value for money > and in your case, I believe that you'd be *much* better served > with some sort of redundancy and not just plain old striping. > > I'd recommend that you stick with software raid and do something > like this; > > > 1 x silicon image PCI 4 port sata controller *not hardware raid* > ($25 from ebay for example) > 3 x 320gb sata hdd (~ $100 each here in Australia) > > for a total of $325 AUD you'd get around 640gb or redundant > storage. > > > 2 x 74gb raptors will cost you $400 here in Australia ..... so > thats $75 *more* for almost 400gb *less* storage that is *not* > redundant ! > > Linux software raid-5 performance is pretty good, my 4 x 400gb > sata's show 155mb/s reads with 'hdparm -t'. > > No brainer ! > > Cheers, > > > Brian. > Hi Brian,  Thanks for your response.  I didn't say it but i can't go more than 2 drives (no more space in the case) and i backup the entire "mail drive" on a LVM backup array of 1.2 TB (3 drives) that serves to backup a windows file server too (Backup Array is 3 drives).  It's not a perfect solution but it cames like it is because the Linux server took more and more job over the years.  The Windoze servers do file serving + AD and Terminal Server.  They have both server grade hardware but this little CentOS 4.5 server running on cheap hardware does almost all the job!  As for RAID 5, with the *poor* CPU i have (Athlon 2500+), i'd be curious how fast it could write with multiples "streams" (server supports 45 WS).  Don't get me wrong, i'm not saying that what you said is not right nor this isn't a good idea!  I'm waiting a budget to change the server but it seems that for now, i have to search a temporary solution that could provide the most speed for the money and taking into account i'm dealing with low end mainboard / cpu.  The next server should be something like the Tyan Transport TA26 (though doesn't seem to have redundant PS) with something like an Adaptec 3085 and SAS 15K Drives.Thanks again for your input!Guy Boisvert

----- Message d'origine -----De: "redhat@mckerrs.net" redhat@mckerrs.netDate: Lundi, Novembre 12, 2007 10:28 pmObjet: Re: Re : Re: [CentOS] Need advice on storageÀ: CentOS mailing list centos@centos.org> > > Guy, > > how many drives can the server take ? and how many are there in > it just now ? > > I'd even go as far as to say install the OS and the data on the > 3 x 320gb sata drives and ditch the 80gb IDE. That is what I do > and it is one of the strengths of software raid, the fact that > you can mix and match raid types on the same physical disks. I'd > go for a triple mirror for /boot and install the rest of the OS > on the RAID-5 set or along with the data (grub can't boot of > raid-5 as far as I'm aware). > > > Alternatively if re-installing or copying your current OS onto > new drives is not practical at this point I'd get 2 x 320g > drives mirror them and also get that extra 1gb of RAM. The extra > RAM could make more of a difference than 10k rpm drives. I dont > think that you'll get anywhere near the performance you'd expect > from a stripe compared to a mirror. As mentioned earlier, > Raptors, IMO, are overrated and they are way hotter and noisier > due to their higher spindle speed. > > How is the box performing ? what are you noticing when it is > slow ? Wait IO ? Can you determine if it is IO bound or CPU > bound ? > > > Cheers, > > Brian. > The server can physically take 5 drives.As for RAM, suggestion taken.  I'll go buy 1 more Gig.  I know that Linux is aggressive when using RAM / Cache so i can just help.When all clients are connected and when they seem to refresh their mail cache and office activity is high, %iowait goes up to 45-55.  I have a script that check and log lsof used by Communigate (every 5 minutes).  Sometimes, i see lsof as high as 130 000.  I had to raise a couple of parameter (in sysctl.conf and limits.conf)because we had Communigate disconnecting clients and i saw messages like:IMAP the 'accept' call failed. Error Code=too many files open in this processI'll probably start another thread for this problem.  The default settings of CentOS for open files were very low.  I had to raise the values and now i'm at 300 000 and i Communigate still display errors from time to time even if my not so precise lsof logging script never shows more than 130 000.  I had even "too many files" errors  with lsof log showing as low as 65 000.  I'm a little confused.  Maybe i'll need more elaborate debugging tools. As you said, i'm in a case where migrating system drive to a RAID 5 array wouldn't be easy.But i have a new possible solution that arosed when i went to see the physical system tonight:  There is a Promise VTRAK 15100 SCSI enclosure in the rack with a free SCSI bus and space to put new SATA drives inside.  I think we have a spare Adaptec 39160 so tomorrow, i'll check that (it's a PCI-X card  but i think it should work PCI too).  If i have the Adaptec, then i'll probably buy 4 x Western-Digital 250 Gigs "YS" (RAID serie) @ about 75$ each, put them inside the VTRAK and configure them in RAID 10 for a total of 500 Gigs.  I briefly checked the VTRAK documentation and they were talking about "up to 200 MBps" throughput and support for NCQ.  It remains to be seems how it could perform with 4 disks in RAID 10.  But it should easily beat a single 200 Gigs IDE drive!For now, i did hdparm -t /dev/hdc (stoopid IDE mail drive!) and it gave me 38.21 MB/sec.Thanks!Guy Boisvert

----- Message d'origine -----De: "redhat@mckerrs.net" redhat@mckerrs.netDate: Mardi, Novembre 13, 2007 2:01 amObjet: Re: Re : Re: Re : Re: [CentOS] Need advice on storageÀ: CentOS mailing list centos@centos.org> > ----- Original Message ----- > As you said, i'm in a case where migrating system drive to a > RAID 5 array wouldn't be easy. > > But i have a new possible solution that arosed when i went to > see the physical system tonight: There is a Promise VTRAK 15100 > SCSI enclosure in the rack with a free SCSI bus and space to put > new SATA drives inside. I think we have a spare Adaptec 39160 so > tomorrow, i'll check that (it's a PCI-X card but i think it > should work PCI too). If i have the Adaptec, then i'll probably > buy 4 x Western-Digital 250 Gigs "YS" (RAID serie) @ about 75$ > each, put them inside the VTRAK and configure them in RAID 10 > for a total of 500 Gigs. I briefly checked the VTRAK > documentation and they were talking about "up to 200 MBps" > throughput and support for NCQ. It remains to be seems how it > could perform with 4 disks in RAID 10. But it should easily beat > a single 200 Gigs IDE drive! > > For now, i did hdparm -t /dev/hdc (stoopid IDE mail drive!) and > it gave me 38.21 MB/sec. > > Thanks! > > Guy Boisvert > > > -- > > Nice kit to have lying around doing nothing ! > > > The PCI-X SCSI card should work in the asus board, but It'll run > as 32bit PCI and therefor your performance will be a maximum of > 128mb/s as that is the theoretical maximum throughput of the > 32bit PCI bus. Given that your NIC and 80gb IDE system hdd are > on the same PCI bus you will be lucky to get 75mb/s I'd say. But > that's almost twice as good as you are currently getting ! > > Good luck. >Thanks for  your input!Actually, the VTrak has 2 independants SCSI Bus and the Winblows fileserver is connected on bus 1.  The bus 2 is free so i can use it.  And there are about 8 free SATA slots free (out of 15).  So this big VTrak unit uses SATA drives and the external interfaces are SCSI.The only thing i dunno for now is the impact on overall performance using 2 servers on the 2 buses.  Is the VTrak controller non blocking?  I'll know soon.As for the PCI bus, you're totally right.  This little crappy mainboard was not engineered to be of server class!  But as i said in a previous post, this little server took more and more job on the go.It does:FTP Server (vsftp)SSH Gateway for road warriors (Transport of Terminal Services Sessions. about 3-4 simul sessions avg)Communigate Server with MAPI connector for about 40-45 users along with WEBMail access (SSL)Sassafras Keyserver for software licences controlBackup server for Winblows file server  and Communigate (rsync snapshots, 900 Gigs used out of 1.2T) All that with an Athlon XP 2500+, 1 Gig ram.  The single 7200 RPM IDE non NCQ is a big part of the bottleneck.So, i'd like to change the server but i won't be able until the new budget and i was asked to try to stretch its life with low budget.  Fortunately, i saw the free VTrak bus and SATA slots!  And i was really lucky that we had an Adaptec 39160 on shelf that was supposed to go elsewhere.I'm very pleased with CentOS 4.  I'm always saying good things about CentOS around me and i often mention that we should all contribute with money to keep this project alive and healthy.  I'd like to contribute with code too but the last line of C code i wrote was on a Sun Sparcstaion 10 about 10-12 years ago!Guy BoisvertP.S.: Sorry for the bad post layout but i'm using my provider's WEBMail (i'm away) and it's really not on par with Thunderbird i usually use...

----- Message d'origine -----De: Kenneth Price kprice@nowyouknow.netDate: Lundi, Novembre 12, 2007 10:58 pmObjet: Re: [CentOS] Need advice on storageÀ: CentOS mailing list centos@centos.org> > You've already received a few responses about reconfiguring your > entireserver, so I'll avoid that route and simply answer your > question straight.> > "No"> > There will not be a "significant" performance boost striping > only 2 drives.> The 10K RPM Raptors alone will definitely increase performance, > but there> is little/no real advantage striping them together. Getting any> significant performance increase from a RAID array really > requires at> *least* 4 drives ... the more the merrier ... although I'll > probably catch> some flack from that statement. :-)> > A previous poster mentioned increasing your RAM from 1Gb to 2Gb. > That was> an excellent idea and should be done in parallel with whatever drive> changes you make.> > Regards,> Ken> >Thanks Ken.That's a good thing to know that software striping won't really boost performance, i then won't waste money on that avenue.  I cannot explain why as i don't know about Linux drivers architecture and general system low level behavior but maybe somebody on this list could give us a clue.As i said on my previous post, i found that the server is inside a rack that has a VTRAK 15100 SCSI to SATA Enclosure with a free SCSI bus (the unit has 2 independant bus) and hard disk slots.  I think we have a spare Adaptec 39160 so if it's the case, i could go with this solution and buy 4 x WD 250 Gigs "YS" (RAID) serie drives @ about 75$ each and put all this in RAID 10.  The unit supports NCQ.  It remains to be seen how well it can perform even if they are talking about "up to 200 MBps".Regards,Guy Boisvert

On Nov 13, 2007 8:03 AM, Ken Price kprice@nowyouknow.net wrote:

I've used both the Promise UltraTrak SX8000 (8-disk RAID50) and UltraTrak SX4000 (4-disk Raid10). Performance was adequate, but I was using Dell PERC2/3 RAID cards. In testing, a plain-jane Adaptec SCSI card coupled with software RAID gave "significantly" better results.

This is beginning to wander a bit off CentOS, but why would you need a PERC card *and* a Promise UltraTrak? The UltraTrak *is* a hardware RAID.

----- Message d'origine -----De: Ken Price kprice@nowyouknow.netDate: Mardi, Novembre 13, 2007 11:04 amObjet: Re: [CentOS] Need advice on storageÀ: centos@centos.org> > Thanks Ken.That's a good thing to know that software striping > won't > > really boost performance, i then won't waste money on that > avenue. > > I cannot explain why as i don't know about Linux drivers > > architecture and general system low level behavior but maybe > > somebody on this list could give us a> > It's not specific to Linux, but RAID in general. You need more > than > two disks to get any significant increase in performance. In > fact, > you may get better performance with Linux software RAID than > hardware > RAID. Linux software RAID is really quite robust.> > > clue.As i said on my previous post, i found that the server is > > > inside a rack that has a VTRAK 15100 SCSI to SATA Enclosure > with a > > free SCSI bus (the unit has 2 independant bus) and hard disk > slots. > > I think we have a spare Adaptec 39160 so if it's the case, i > could > > go with this solution and buy 4 x WD 250 Gigs "YS" (RAID) > serie > > drives @ about 75$ each and put all this in RAID 10. The unit > > > supports NCQ. It remains to be seen how well it can perform > even if > > they are talking about "up to 200 MBps".Regards,Guy Boisvert> > I've used both the Promise UltraTrak SX8000 (8-disk RAID50) and > UltraTrak SX4000> (4-disk Raid10). Performance was adequate, but I was using Dell > > PERC2/3 RAID cards. In testing, a plain-jane Adaptec SCSI card > coupled with software RAID gave "significantly" better results.> > Cheers.> -Ken> Hi Ken,Well, i should be OK with my Adaptec 39160 then.  I'll see if the VTrak 15100 we have is able to push respectable numbers in RAID 10.  But as "redhat@mckerrs.net" stated in another post, it will be plugged in a straight 32 bits PCI bus on a crappy mainboard so it's a temporary solution until we get the budget for a new server.  But it should be relatively easy to beat a single 200 Gig IDE non NCQ drive!Thanks!Guy Boisvert

----- Message d'origine -----De: Shibu C Varughese shibucv@itmission.orgDate: Mardi, Novembre 13, 2007 12:36 pmObjet: Re: [CentOS] Need advice on storageÀ: CentOS mailing list centos@centos.org> if you are planning to do a full upgrade ... in future .. try > out zimbra > from zimbra.com> > -- > ShibuHi Shibu,  I checked that a little while ago along with Scalix.  What i like about Communigate is that it uses standard MBOX files rather than esoteric mailstores and databases.  I'll have to evaluate more deeplay Scalix and Zimbra but my understanding for now is that Communigate is much simpler to deal with as for backup / restore.  I had to restore a couple users' folder and it's a breeze with CGate.  I just copied back the user's directory from our RSync daily snaphot backup and voila!  But i agree, it's only one part of the whole story.Regards,Guy Boisvert