David Finch <david at mytsoftware.com> wrote: > Glad to know it's not just me. It's not. I've seen it too. > Using ext3 on a 3ware raid 5 of four 250gb disks. Doesn't matter what the disks are. The problem is the small cache size on the 3Ware Escalade 7000/8000. They only have 1-4MB of 0 wait state SRAM. 1MB on the original 7200/7210/7400/7410/7800/7810 and 7000-2 and 8000-2, as well the 7006-2 and 8006-2 (xxx6 = 66MHz PCI). 2MB on the 7450/7850 which are, subsequently, the 7500-4/-8 for PATA now, with the 8500-4/-8 and 8506-4/-8 (xxx6 = 66MHz PCI). 4MB on the 7506-12. SRAM is very expensive, both size and cost-wise. It's the logic used in CPU cache and for networking ASICs. But it has little to no wait -- unlike DRAM which is still 40-70ns on reads (many many wait cycles, typically 6-10 for today's 133-266MHz clocks). That's why 3Ware calls the Escalade 7000/8000 series a "storage switch." It's ideal for RAID-0, 1 and 10. This size is a serious issue when it comes to Ext3's journal logic, especially pre-2.4.18 kernels IIRC (maybe it was 2.4.15?). With only 2MB typical (4MB on the 7506-12), the commit of the Ext3 journal exceeds that size -- so the card "stalls" on the write when just committing the journal from the data. > Writes are slow and seem to halt the server until they > complete, but it's not a server where response time or > write speed is critical. You can play with the kernel buffer settings. It's highly recommended for many of the 3Ware Escalade cards, including the 9000 series. But if performance is a consideration, do _not_ use RAID-5 on the 3Ware Escalade 7000/8000. Use RAID-10. You can break over 200MBps _writes_ with RAID-10 on the 7000/8000 series. -- Bryan J. Smith Professional, Technical Annoyance b.j.smith at ieee.org http://thebs413.blogspot.com ---------------------------------------------------- *** Speed doesn't kill, difference in speed does ***