<div><div>Hello,</div><div><br></div><div>Does anyone have experience using a 3ware 9650SE series raid controller on CentOS 6.0?</div><div><br></div><div>I am getting very sporadic throughput with moderately sized files (0.5-2GB) on ext3. I have tried most of the mount time tuning options:</div>
<div><br></div><div>* noatime</div><div>* trying different journal types</div><div>* setting commit=120 - helped a little</div><div><br></div><div>Even after these optimizations it doesn't seem like the raid array is working as it should. After a few 1GB writes (dd from /dev/zero to the raid) kjournald runs for an hour and later writes are really slow. Not only that, using tw_cli is very slow ... whereas tw_cli is superfast if kjournald is not churning away.</div>
<div><br></div><div>So it goes something like this</div><div><br></div><div>* write one 1GB file (486 MB/s)</div><div>* writes another (223 MB/s)</div><div>* any writes before the 120s commit kicks in, is ~200MB/s</div><div>
* commit kicks in and kjournald starts churning</div><div>* writes are all over the map - 6-85MB/s</div><div><br></div><div>Maybe this is just the way it is, but it did not seem to be the case this same hardware was running Fedora (Core) 9 and I have a similar machine where this does not seem to be the case (I can't experiment on it at the moment).</div>
<div><br></div><div>The one thing I do not know, since I did not create the RAID or ext3 filesystem is whether the stride and stripe-width were properly selected to match the 64k chunk size of the raid array. I don't know how to tell from tune2fs ... output below ... its either not there or by another name.</div>
<div><br></div><div>Any help or suggestions are appreciated.</div><div><br></div><div>Austin</div><div><br></div><div>Useful Information</div><div>==============</div><div><br></div><div>============== /proc/mounts =====================</div>
<div><br></div><div>/dev/sda1 /tonga_raid ext3 rw,noatime,errors=continue,nouser_xattr,noacl,commit=120,barrier=1,data=ordered 0 0</div><div><br></div><div><br></div><div>========= tw_cli /c2 show diag =====================</div>
<div><br></div><div>### Time Stamp: 12:34:18 01-Sep-2011</div><div>### Host Name: tonga</div><div>### Host Architecture: x86_64 (64 bit)</div><div>### OS Version: Linux 2.6.32-71.29.1.el6.x86_64</div>
<div>### Model: 9650SE-8LPML</div><div>### Serial #: L326025A8221043</div><div>### Controller ID: 2</div><div>### CLI Version: 2.00.11.016</div><div>### API Version: 2.08.00.017</div>
<div>
### Driver Version: 2.26.02.014RH</div><div>### Firmware Version: FE9X 3.08.00.016</div><div>### BIOS Version: BE9X 3.08.00.004</div><div>### Available Memory: 224MB</div><div><br></div><div>==========================================================================</div>
<div>Diagnostic Information on Controller //.../c2/...</div><div>--------------------------------------------------------------------------</div><div>Event Trigger and Log Information:</div><div>Triggered Event(s) = </div>
<div> ctlreset (controller soft reset) </div><div> fwassert (firmware assert) </div><div> driveerr (drive error) </div><div>Diagnostic log save mode = -</div><div>Parameter table does not exist </div></div><div>
<br>
</div><div><br></div><div><div>========== tune2fs -l /dev/sda1 =================</div><div>tune2fs 1.41.12 (17-May-2010)</div><div>Filesystem volume name: /data</div><div>Last mounted on: <not available></div>
<div>Filesystem UUID: 3c5f6dbb-d5dc-4f85-bc70-9b761c89c86e</div><div>Filesystem magic number: 0xEF53</div><div>Filesystem revision #: 1 (dynamic)</div><div>Filesystem features: has_journal ext_attr resize_inode dir_index filetype needs_recovery sparse_super large_file</div>
<div>Filesystem flags: signed_directory_hash </div><div>Default mount options: user_xattr acl</div><div>Filesystem state: clean</div><div>Errors behavior: Continue</div><div>Filesystem OS type: Linux</div>
<div>Inode count: 427245568</div><div>Block count: 1708965879</div><div>Reserved block count: 85448293</div><div>Free blocks: 809305423</div><div>Free inodes: 426287552</div>
<div>First block: 0</div><div>Block size: 4096</div><div>Fragment size: 4096</div><div>Reserved GDT blocks: 616</div><div>Blocks per group: 32768</div><div>Fragments per group: 32768</div>
<div>Inodes per group: 8192</div><div>Inode blocks per group: 512</div><div>Filesystem created: Tue Sep 9 09:57:44 2008</div><div>Last mount time: Thu Sep 1 12:40:01 2011</div><div>Last write time: Thu Sep 1 12:40:01 2011</div>
<div>Mount count: 11</div><div>Maximum mount count: -1</div><div>Last checked: Tue Aug 30 23:12:49 2011</div><div>Check interval: 0 (<none>)</div><div>Reserved blocks uid: 0 (user root)</div>
<div>Reserved blocks gid: 0 (group root)</div><div>First inode: 11</div><div>Inode size:<span class="Apple-tab-span" style="white-space:pre"> </span> 256</div><div>Journal inode: 8</div>
<div>Default directory hash: tea</div><div>Directory Hash Seed: 0b324311-93a9-4c23-bf15-40965792029b</div><div>Journal backup: inode blocks</div></div><div>===========================================================</div>
<div><br></div><div><br></div><div><div>=================== tw_cli info c2 ============================</div><div><br></div><div>Unit UnitType Status %RCmpl %V/I/M Stripe Size(GB) Cache AVrfy</div><div>------------------------------------------------------------------------------</div>
<div>u0 RAID-5 OK - - 64K 6519.19 OFF OFF </div><div><br></div><div>Port Status Unit Size Blocks Serial</div><div>---------------------------------------------------------------</div>
<div>p0 OK u0 931.51 GB 1953525168 WD-WCASJ1631953 </div><div>p1 OK u0 931.51 GB 1953525168 WD-WCASJ1622428 </div><div>p2 OK u0 931.51 GB 1953525168 WD-WCASJ1639721 </div>
<div>p3 OK u0 931.51 GB 1953525168 WD-WCASJ1636054 </div><div>p4 OK u0 931.51 GB 1953525168 WD-WCASJ1621694 </div><div>p5 OK u0 931.51 GB 1953525168 WD-WCASJ1636292 </div>
<div>p6 OK u0 931.51 GB 1953525168 WD-WCASJ1637586 </div><div>p7 OK u0 931.51 GB 1953525168 WD-WCASJ1637516 </div></div><div><br></div><div><div>===========================================================</div>
</div><div><br></div>