Hello listmates,
We are working on setting up two private networks linked by a public network which is fast (1 Gbit/s) but potentially insecure. Since the hosts on our two networks need to talk to each other, and do so securely, we have decided to use OpenVPN to connect them, making one gateway a server and the other a client. The connectivity part was easy to establish and worked like a charm. The only problem was, and is, performance.
We have two old PIII-class machines that are being tested for the role of the gateways. We have put new 1 Gbit NIC's in them and they work find for everything (data transmission, DHCP, DNS, routing) except the VPN. When traffic goes through the VPN the OpenVPN process goes to 99% CPU on the server, about 70% CPU on the client and the effective transmission rate goes down to about 6 MB/s whereas in non-VPN mode it can be as high as 50+ MB/s (the top for the 1 Gbit/s is, obviously, 125 MB/s hence with the VPN we are down to about 5% of the capacity).
While this may be usable we would like to hope we can do better. Hence the following questions:
1) Have you used OpenVPN in a similar setup?
2) If so what sort of performance did you see?
3) What kind of equipment did you use?
Personally, I'd like to hope that if we find VPN-enabled gateways with more processing power we'd get drastically better performance. So if you have data to confirm or deny that please share it.
Thanks.
Boris.
Sounds like you need boxes with more cpu.
What's the output when you type: cat /proc/cpuinfo into a shell prompt?
What kind of traffic goes over the link?
Lots of low bandwidth connections? A few high bandwidth connections? Lots of high bandwidth connections? Some combination of the above?
Just testing a bunch of boxes with a variety of specs may give you a better answer the pondering the mysteries of the specifications.
On Thu, Aug 19, 2010 at 10:56 AM, drew einhorn drew.einhorn@gmail.com wrote:
Sounds like you need boxes with more cpu.
What's the output when you type: cat /proc/cpuinfo into a shell prompt?
What kind of traffic goes over the link?
Lots of low bandwidth connections? A few high bandwidth connections? Lots of high bandwidth connections? Some combination of the above?
Just testing a bunch of boxes with a variety of specs may give you a better answer the pondering the mysteries of the specifications. _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Thanks. I actually hoped that other people may have already performed the necessary testing - and that, based on the responses to my email, seems to some extent to have been the case. Here's /proc/cpuinfo on the OpenVPN server:
------------------------------------- [bepstein@dellnikon ~]$ cat /proc/cpuinfo processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 8 model name : Pentium III (Coppermine) stepping : 10 cpu MHz : 996.841 cache size : 256 KB fdiv_bug : no hlt_bug : no f00f_bug : no coma_bug : no fpu : yes fpu_exception : yes cpuid level : 2 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 mtrr pge mca cmov pat pse36 mmx fxsr sse up bogomips : 1993.68
[bepstein@dellnikon ~]$
-------------------------------------
Boris.
On Thu, Aug 19, 2010 at 2:57 PM, Boris Epstein borepstein@gmail.com wrote:
bogomips : 1993.68
If cost is a factor, you should be able to find used hp/Compaq rack-mount servers with Intel Xeon processors with these bogomips:
dmesg | grep -i bogomips Calibrating delay loop (skipped), value calculated using timer frequency.. 6113.34 BogoMIPS (lpj=3056670) Calibrating delay using timer specific routine.. 6112.21 BogoMIPS (lpj=3056109) Calibrating delay using timer specific routine.. 6112.18 BogoMIPS (lpj=3056090) Calibrating delay using timer specific routine.. 6111.93 BogoMIPS (lpj=3055965) Total of 4 processors activated (24449.66 BogoMIPS).
for under $300 each.
Kind regards/ldv
On 8/19/2010 10:29 AM, Boris Epstein wrote:
Hello listmates,
We are working on setting up two private networks linked by a public network which is fast (1 Gbit/s) but potentially insecure. Since the hosts on our two networks need to talk to each other, and do so securely, we have decided to use OpenVPN to connect them, making one gateway a server and the other a client. The connectivity part was easy to establish and worked like a charm. The only problem was, and is, performance.
We have two old PIII-class machines that are being tested for the role of the gateways. We have put new 1 Gbit NIC's in them and they work find for everything (data transmission, DHCP, DNS, routing) except the VPN. When traffic goes through the VPN the OpenVPN process goes to 99% CPU on the server, about 70% CPU on the client and the effective transmission rate goes down to about 6 MB/s whereas in non-VPN mode it can be as high as 50+ MB/s (the top for the 1 Gbit/s is, obviously, 125 MB/s hence with the VPN we are down to about 5% of the capacity).
While this may be usable we would like to hope we can do better. Hence the following questions:
Have you used OpenVPN in a similar setup?
If so what sort of performance did you see?
What kind of equipment did you use?
Personally, I'd like to hope that if we find VPN-enabled gateways with more processing power we'd get drastically better performance. So if you have data to confirm or deny that please share it.
I have an OpenVPN gateway running on an old PII-400 machine with 256M RAM. It works fine for what we need. I have never measured throughput or CPU usage.
I would say that if your CPU is going to 99% when you use the VPN, you would definitely benefit from a faster system. I would suspect that any P4 or higher system would work fine, but maybe someone else that actually uses a high-speed VPN connection could give you a more accurate spec.
We are working on setting up two private networks linked by a public network which is fast (1 Gbit/s) but potentially insecure. Since the hosts on our two networks need to talk to each other, and do so securely, we have decided to use OpenVPN to connect them, making one gateway a server and the other a client. The connectivity part was easy to establish and worked like a charm. The only problem was, and is, performance.
We have two old PIII-class machines that are being tested for the role of the gateways. We have put new 1 Gbit NIC's in them and they work
Why you are using such a antique hardware?
-- Eero
On Thu, Aug 19, 2010 at 12:48 PM, Eero Volotinen eero.volotinen@iki.fi wrote:
We are working on setting up two private networks linked by a public network which is fast (1 Gbit/s) but potentially insecure. Since the hosts on our two networks need to talk to each other, and do so securely, we have decided to use OpenVPN to connect them, making one gateway a server and the other a client. The connectivity part was easy to establish and worked like a charm. The only problem was, and is, performance.
We have two old PIII-class machines that are being tested for the role of the gateways. We have put new 1 Gbit NIC's in them and they work
Why you are using such a antique hardware?
-- Eero _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Dunno. Antique or not, there are still plenty of things it can do for us - so why not use it? :)
Boris.
On Thu, Aug 19, 2010 at 12:48 PM, Eero Volotinen eero.volotinen@iki.fi wrote:
We are working on setting up two private networks linked by a public network which is fast (1 Gbit/s) but potentially insecure. Since the hosts on our two networks need to talk to each other, and do so securely, we have decided to use OpenVPN to connect them, making one gateway a server and the other a client. The connectivity part was easy to establish and worked like a charm. The only problem was, and is, performance.
We have two old PIII-class machines that are being tested for the role of the gateways. We have put new 1 Gbit NIC's in them and they work
Why you are using such a antique hardware?
-- Eero _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Dunno. Antique or not, there are still plenty of things it can do for us - so why not use it? :)
Because they are using a lot of energy compared to i.e. a Draytek Vigor2130. Does gigabit on both sides, 2 VPN tunnels, QoS, all you need and fast. Takes up way less than 15 Watts, earns itself easily back in the first year compared to your old noisy rubbish, and saves thereafter.
Boris. _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Because they are using a lot of energy compared to i.e. a Draytek Vigor2130. Does gigabit on both sides, 2 VPN tunnels, QoS, all you need and fast. Takes up way less than 15 Watts, earns itself easily back in the first year compared to your old noisy rubbish, and saves thereafter.
-- L. de Braal BraHa Systems NL - Terneuzen T +31 115 649333 F +31 115 649444
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
What sort of VPN does Draytek Vigor2130 run? Is it compatible with OpenVPN?
Thanks.
Boris.
On 08/19/10 5:01 PM, Boris Epstein wrote:
What sort of VPN does Draytek Vigor2130 run? Is it compatible with OpenVPN?
googling says, ipsec and pptp. :-/
http://www.draytek.com/user/PdInfoDetail.php?Id=98#PdInfo (gotta look at the user's guide to find that out)
Because they are using a lot of energy compared to i.e. a Draytek Vigor2130. Does gigabit on both sides, 2 VPN tunnels, QoS, all you need and fast. Takes up way less than 15 Watts, earns itself easily back in the first year compared to your old noisy rubbish, and saves thereafter.
-- L. de Braal BraHa Systems NL - Terneuzen T +31 115 649333 F +31 115 649444
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
What sort of VPN does Draytek Vigor2130 run? Is it compatible with OpenVPN?
See here:
http://www.draytek.com/user/PdInfoDetail.php?Id=98
There's also a wireless and VOIP version. Draytek is not the only one selling these appliances, but I have good experiences with these.
Thanks.
Boris. _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
On Thu, 19 Aug 2010, Boris Epstein wrote:
While this may be usable we would like to hope we can do better. Hence the following questions:
- Have you used OpenVPN in a similar setup?
Similar, though it's only a 100Mb/s link.
- If so what sort of performance did you see?
Pretty much maximum throughput for a 100Mb circuit. wget, for instance, reports 11.0MB/s.
- What kind of equipment did you use?
The gateways have Xeon E5405 CPUs and run CentOS 5.
On 08/19/10 7:29 AM, Boris Epstein wrote:
We have two old PIII-class machines that are being tested for the role of the gateways. We have put new 1 Gbit NIC's in them and they work find for everything (data transmission, DHCP, DNS, routing) except the VPN. When traffic goes through the VPN the OpenVPN process goes to 99% CPU on the server, about 70% CPU on the client and the effective transmission rate goes down to about 6 MB/s whereas in non-VPN mode it can be as high as 50+ MB/s (the top for the 1 Gbit/s is, obviously, 125 MB/s hence with the VPN we are down to about 5% of the capacity).
encryption is CPU intensive. you might get a little better throughput if you can force Blowfish cypher over whatever else as it is somewhat lower in CPU overhead, and still considered decently strong, but you still likely won't get wire speed. heck, you can't get wirespeed doing a simple http bulk transfer over gigE with a p3.
note going from P3 to early P4, you need to at least double the clock speed to get about the same performance.. I wouldn't touch a p4 under 2.8Ghz for this. The Core line of CPUs are considerably faster per clock than the P4's. AMD's athlon64/opterons perform much better than P4's at the same clock speeds.
On Thu, Aug 19, 2010, Boris Epstein wrote:
Hello listmates,
We are working on setting up two private networks linked by a public network which is fast (1 Gbit/s) but potentially insecure. Since the hosts on our two networks need to talk to each other, and do so securely, we have decided to use OpenVPN to connect them, making one gateway a server and the other a client. The connectivity part was easy to establish and worked like a charm. The only problem was, and is, performance.
We have two old PIII-class machines that are being tested for the role of the gateways. We have put new 1 Gbit NIC's in them and they work find for everything (data transmission, DHCP, DNS, routing) except the VPN. When traffic goes through the VPN the OpenVPN process goes to 99% CPU on the server, about 70% CPU on the client and the effective transmission rate goes down to about 6 MB/s whereas in non-VPN mode it can be as high as 50+ MB/s (the top for the 1 Gbit/s is, obviously, 125 MB/s hence with the VPN we are down to about 5% of the capacity).
While this may be usable we would like to hope we can do better. Hence the following questions:
- Have you used OpenVPN in a similar setup?
We have a client with offices in 4 cities using a Windows application with remote access (which performs horribly compared to their previous *nix applictioan :-). The main site is in Kansas City, the other three in Texas, and the performance is good enough that people aren't complaining -- much as many prefer the old app.
- If so what sort of performance did you see?
The client is happy, particularly since their software vendor wanted them to get $2,500 Cisco routers for each office, and the Linux boxes cost a lot less including setup and configuration.
Frankly I was amazed that this was adequate for use with Window remote access over relatively slow links with the T1 in KC probably being the potential bottleneck with 3 offices connecting to it.
- What kind of equipment did you use?
Each office has a T1 connection. The KC Linux machine is a general purpose machine doing e-mail, user storage, etc. NAT gateway for the LAN, as well as the OpenVPN with a single Intel(R) Core(TM)2 Duo CPU E4500 @ 2.20GHz and 2GB RAM.
The remote office machines are also NAT gateways for each office's LAN are are running single processor Intel(R) Atom(TM) CPU 330 @ 1.60GHz with 2GB of RAM. These are in small chassis, are very quiet, and seem to work very well. These systems with 80GB SATA drives cost us just under $500 each a couple of years ago, and a bit less today.
All are running CentOS 5.x.
Bill
Bill Campbell wrote:
On Thu, Aug 19, 2010, Boris Epstein wrote:
<snip>
- What kind of equipment did you use?
<snip>>
The remote office machines are also NAT gateways for each office's LAN are are running single processor Intel(R) Atom(TM) CPU 330 @ 1.60GHz with 2GB of RAM. These are in small chassis, are very quiet, and seem to work very well. These systems with 80GB SATA drives cost us just under $500 each a couple of years ago, and a bit less today.
Um, those have more memory, I think, but are pretty much the specs for my HP Mini 110-1100 Netbook.
mark
On 08/19/2010 03:29 PM, Boris Epstein wrote:
- Have you used OpenVPN in a similar setup?
you nver mentioned what the setup is. post your server.conf as a start if you want to compare notes.
- If so what sort of performance did you see?
a very old tyan S3xxx rig, 2x opteron 246HE's, easily sustain about 150 connections and can saturate a 300mbps link ( its a gigabit local port, but only 300mbps makes it out of the rack )
- What kind of equipment did you use?
hardware rng( via padlock ) and careful selection of openssl cypher + interfaces.
- KB
On 08/19/2010 07:29 AM, Boris Epstein wrote:
- Have you used OpenVPN in a similar setup?
For the hell of it, I ran openvpn on two hosts in a 1Gbps LAN.
- If so what sort of performance did you see?
I was able to copy a 707788800 byte ISO in 55 seconds using the default BF-CBC cipher. I used "nc" to transmit the file. The same file over the non-vpn route took only 17 seconds. CPU utilization of the 'nc' process was rather high in the second test, which probably indicates that nc was sending data in fairly small buffers.
I'm at a loss to explain it, but scp was actually much faster, despite encrypting the data twice over (though that was happening on different CPUs, so probably didn't slow things down much). scp copied the same file in 27 seconds over the VPN, and 20 seconds over the LAN.
- What kind of equipment did you use?
Two Dell Inspiron 531s systems. One running CentOS 5, the other Fedora 13. Both have AMD Athlon64 X2 4000+ CPUs. Both have D-Link DGE-530T Gig-E network adapters. The MTU between the systems is 9000.
On Fri, Aug 20, 2010 at 11:54 AM, Gordon Messmer yinyang@eburg.com wrote:
On 08/19/2010 07:29 AM, Boris Epstein wrote:
- Have you used OpenVPN in a similar setup?
For the hell of it, I ran openvpn on two hosts in a 1Gbps LAN.
- If so what sort of performance did you see?
I was able to copy a 707788800 byte ISO in 55 seconds using the default BF-CBC cipher. I used "nc" to transmit the file. The same file over the non-vpn route took only 17 seconds. CPU utilization of the 'nc' process was rather high in the second test, which probably indicates that nc was sending data in fairly small buffers.
I'm at a loss to explain it, but scp was actually much faster, despite encrypting the data twice over (though that was happening on different CPUs, so probably didn't slow things down much). scp copied the same file in 27 seconds over the VPN, and 20 seconds over the LAN.
- What kind of equipment did you use?
Two Dell Inspiron 531s systems. One running CentOS 5, the other Fedora 13. Both have AMD Athlon64 X2 4000+ CPUs. Both have D-Link DGE-530T Gig-E network adapters. The MTU between the systems is 9000. _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Gordon, thanks, very helpful.
I thought ethernet NIC's limit MTU to 1500. Just tried to set mine to 9000 and failed - it is OK with values <= 1500 though.
Boris.
On Fri, Aug 20, 2010, Boris Epstein wrote:
On Fri, Aug 20, 2010 at 11:54 AM, Gordon Messmer yinyang@eburg.com wrote:
On 08/19/2010 07:29 AM, Boris Epstein wrote:
- Have you used OpenVPN in a similar setup?
...
I thought ethernet NIC's limit MTU to 1500. Just tried to set mine to 9000 and failed - it is OK with values <= 1500 though.
Comparisons with gigabit networks seems a pointless given that most VPNs will be used over the Internet which limits bandwidth, how fast is fast enough?
The average small-to-medium business office will probably have a network connection in the neighborhood of T1 speed or perhaps a somewhat faster business cable connection, but these are slower than the slowest ethernet or even WiFi connections.
Bill
On 08/20/2010 10:19 AM, Bill Campbell wrote:
Comparisons with gigabit networks seems a pointless given that most VPNs will be used over the Internet which limits bandwidth, how fast is fast enough?
Boris said in his first email that the link between the two networks was 1 Gps.
On 08/20/2010 09:44 AM, Boris Epstein wrote:
Gordon, thanks, very helpful.
No problem.
I thought ethernet NIC's limit MTU to 1500. Just tried to set mine to 9000 and failed - it is OK with values<= 1500 though.
Depends on the cards. In order to use (or take advantage of) jumbo frames, all of the network gear between two hosts must support it and must be configured to use it. More importantly, all of the equipment on any ethernet segment must have the same MTU.
Gigabit equipment often performs much better when using jumbo frames, but you basically must dedicate an entire LAN to the use of jumbo frames in order to use it at all. The router for that segment normally fragments the packets for use on lines that don't support jumbo frames.
It's unlikely that your two hosts have jumbo frame support between them, but only your ISP would be able to tell you that for certain. I provided benchmarks that were more or less ideal. Faster CPUs or hardware encryption accelerators will get you closer to full gigabit speed.
-
Bill Campbell -
Boris Epstein -
Bowie Bailey -
drew einhorn -
Eero Volotinen -
Gordon Messmer -
John R Pierce -
Karanbir Singh -
Larry Vaden -
Leen de Braal -
m.roth＠5-cent.us -
Paul Heinlein