On Fri, 2009-07-24 at 10:21 +0400, Roman Savelyev wrote:
- You are hit by Nagel alghoritm (slow TCP response). You can build DRBD
8.3. In 8.3 "TCP_NODELAY" and "QUICK_RESPONSE" implemented in place. 2. You are hit by DRBD protocol. In most cases, "B" is enought. 3. You are hit by triple barriers. In most cases you are need only one of "barrier, flush, drain" - see documentation, it depens on type of storage hardware.
I have googled the triple barriers thing but cant find that much information.
Would it help if I used IPv6 instead of IPv4?
Ross, here are the results of those tests you suggested: ________________________________________________________________________________________ For completeness here is my current setup:
host1: 10.99.99.2 Xeon Quad-Core 8GB RAM Centos 5.3 64bit 2x 1TB seagate sata disks in software raid level 1 LVM on top of the raid for dom0 root fs and for all domU root FSses
host2: 10.99.99.1 Xeon Dual-Core 8GB RAM Centos 5.3 64bit 2x 1TB seagate sata disks in software raid level 1 LVM on top of the raid for dom0 root fs and for all domU root FSses
common: hosts are connected to local LAN and directly to each other with a CAT6 gigabit crossover.
I have 6 DRBDs running for 5 domUs over the back to back link. DRBD version drbd82-8.2.6-1.el5.centos _______________________________________________________________________ _______________________________________________________________________
Ok, here is what I have done:
_______________________________________________________________________ I have added the following to the drbd config: disk { no-disk-flushes; no-md-flushes; }
That made the resync go up to 50MB/sec after I issued a drbdsetup /dev/drbdX syncer -r 110M
It used to stick around at 11MB/sec
As far as i can tell it has improved the domUs disk access as well.
I do see that there are a lot of warnings to be heeded with disk and metadata flushing...... _______________________________________________________________________
iperf results:
on host 1: # iperf -s ------------------------------------------------------------ Server listening on TCP port 5001 TCP window size: 85.3 KByte (default) ------------------------------------------------------------ [ 5] local 10.99.99.1 port 5001 connected with 10.99.99.2 port 58183 [ ID] Interval Transfer Bandwidth [ 5] 0.0-10.0 sec 1.16 GBytes 990 Mbits/sec
on host 2: # iperf -c 10.99.99.1 ------------------------------------------------------------ Client connecting to 10.99.99.1, TCP port 5001 TCP window size: 73.8 KByte (default) ------------------------------------------------------------ [ 3] local 10.99.99.2 port 58183 connected with 10.99.99.1 port 5001 [ ID] Interval Transfer Bandwidth [ 3] 0.0-10.0 sec 1.16 GBytes 992 Mbits/sec
I am assuming those results are to be expected from a back to back gigabit. _______________________________________________________________________
the dd thing. I think I did this completely wrong, how is this supposed to be done?
this is what i did
host 1: nc -l 8123 | dd of=/mnt/data/1gig.file oflag=direct (/mnt/data is an ext3 FS in LVM mounted on dom0) (Not drbd) i first wanted to try it locally.
host 2: date; dd if=/dev/zero bs=1M count=1000 | nc 10.99.99.2 8123 ; date
I did not wait for it to finish... according to ifstat the average speed I got during this transfer was 1.6MB/sec
_______________________________________________________________________
Any tips would be greatly appreciated.