[CentOS] PCI-X/PCIe RAID controller.

Sat Apr 9 20:28:29 UTC 2011
John R Pierce <pierce at hogranch.com>

On 04/09/11 7:08 AM, Christopher Chan wrote:
>> In that case there's no real point upgrading. :-) The drives are U160's.
> The key is the backplane, not the drives. You should be able to get u320
> speeds if you have enough u160 drives

SCSI (parallel) before U320 had a weak spot.  the command and status 
phases of a SCSI command execution were stuck in asynch mode, only the 
data transfer phase could use synchronous ('fast', 'ultra', etc).    
Async SCSI is the old original 3-5Mhz mode, which on a wide bus means 
6-10MB/sec.   This wasn't a big problem with the original Ultra Fast 
Wide 40 and Ultra 80 low voltage differential modes, but when SCSI hit 
U160 it became a potential bottleneck, especially for high rates of 
small block random IOPS on raids with more than a few drives.

IIRC (rusty neurons), a SCSI command takes 16 bytes or so, and a status 
query takes 4-8.   at the typical async 3Mxfer/sec, that means a command 
takes like 5 microseconds.   In 5 microseconds at U160, you could have 
transferred 5*160 or 800 bytes, and a single status poll cycle takes 
another 16-200 bytes, or pretty much 1000 bytes total, eg, for every IO 
operation on the SCSI bus, you lose 1KB worth of potential data 
bandwidth...  So if you're executing 1000 IOPS spread across 4-6 drives, 
you're losing at least 1MB/sec of your channel's bandwidth potential.    
in reality its several times worse than this due to the additional 
overhead of getting on the channel and the typical multiple polling 
cycles required to complete a transfer, I've calculated its as bad as 
10% under heavy IOPS random multiple drive raids

U320 supports synchronous command and status, so its way more than 
double U160 speeds due to the elimination of this overhead.   
Interesting side note, U320 controllers and U320 devices can support 
synchronous command and status even if its running at U160 due to 
backplane and cable loading issues.

thankfully, with FC, SAS, and SATA, these things are all history.