On Mon, 13 Oct 2014 20:15:11 -0500 Joakim Ziegler <joakim at terminalmx.com> wrote: ... > So, if I use iozone -a to test write speeds on the raw device, I get > results in the 500-800MB/sec range, depending on write sizes, which > is about what I'd expect. > > However, when I have an ext4 filesystem on this device, mounted with > noatime and data=writeback, (the filesystem is completely empty) and > I test with dd, the results are less encouraging: ... > Now, I'm sure there can be many reasons for this, but I wonder where > I should start looking to debug this. First I'd suggest comparing apples to apples. That is try doing the dd test on the raw device and compare to dd on ext4. Then you may want to try changing io scheduler from the default cfq to deadline. This typically works better for many raid controllers but ymmv. Also testing with xfs instead of ext4 is probably worth it. xfs usually outperform ext4 in streaming writes (like dd). Of course this raises the question of whether that dd is a useful metric for your actual load... xfs may infact be needed (3T * 7 = 21 TB > ext4 max (if I remember correctly, refer to rh online data for rhel6 to make sure)). Good luck, Peter K