[CentOS] drbd

[John R Pierce]

On 10/12/2014 1:04 PM, Digimer wrote:
>
> If it's only 1 Gbps, then the maximum sustainable write speed is ~120 
> MB/sec (at most; the slower of network or disk determines the max 
> speed). You want the sync rate to be ~30% of maximum speed, or else 
> you will choke out the apps using the DRBD resource and cause them to 
> suffer a huge performance hit. App performance is (max write - sync 
> rate) when a background sync is performed. This is why the default 
> speed is very low.
>
> In your case, you've set the sync rate way above what is possible, and 
> I've seen this dramatically hurt performance and even full-on stall 
> the sync operation. I would set this to '40M' at most. 

verify apparently uses this same speed to limit how fast it runs... 
since its doing checksums on each side, its not actually transferring 
any data across the network, hence why I boosted it to 200M (that left 
the node that initiated the verify with one process at 90% CPU, whihc I 
figure is as fast as it can go)...   as I said, I've set it back to 50M 
now that the verify completed.

btw, originally I had the verify-alg as sha1... my xeon X5650 2.7Ghz 
could only verify about 23MB/sec.  setting it to crc32c sped up to 
180-200MB/sec, which is more reasonable considering this is a 16TB 
volume built from 2 8x3TB drive raid6's striped together, behind a lsi 
9261-8i megaraid sas2 card with 512MB BBU cache.




-- 
john r pierce                                      37N 122W
somewhere on the middle of the left coast