I think it's baloney mainly because i can't find a mention of it anywhere. Is there REALLY a limit on the number of physical network interfaces in the Kernel?
2010/7/14 William Warren hescominsoon@emmanuelcomputerconsulting.com:
I think it's baloney mainly because i can't find a mention of it anywhere. Is there REALLY a limit on the number of physical network interfaces in the Kernel?
can you really create hardware with huge number or real ethernet controllers?
-- Eero
On Jul 13, 2010, at 8:58 PM, Eero Volotinen eero.volotinen@iki.fi wrote:
2010/7/14 William Warren hescominsoon@emmanuelcomputerconsulting.com:
I think it's baloney mainly because i can't find a mention of it anywhere. Is there REALLY a limit on the number of physical network interfaces in the Kernel?
can you really create hardware with huge number or real ethernet controllers?
With tagged VLANs, bridges and virtual switches, is there a need for that many physical interfaces?
-Ross
On 7/13/2010 9:11 PM, Ross Walker wrote:
On Jul 13, 2010, at 8:58 PM, Eero Volotineneero.volotinen@iki.fi wrote:
2010/7/14 William Warrenhescominsoon@emmanuelcomputerconsulting.com:
I think it's baloney mainly because i can't find a mention of it anywhere. Is there REALLY a limit on the number of physical network interfaces in the Kernel?
can you really create hardware with huge number or real ethernet controllers?
With tagged VLANs, bridges and virtual switches, is there a need for that many physical interfaces?
-Ross
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
ok let me specify. Is there any real limit? I've seen some folks(and been told by a few) that you can't more than 10 physical interfaces in a linux system.
On 7/14/10, William Warren hescominsoon@emmanuelcomputerconsulting.com wrote:
ok let me specify. Is there any real limit? I've seen some folks(and been told by a few) that you can't more than 10 physical interfaces in a linux system.
Googling up a really old 2005 newsgroup thread says some people had 24 physical NIC (6x 4quad) in a system before and one person vaguely remembers a hard 256 limit which would make sense if physical interface count is a byte value.
On 7/14/2010 1:16 AM, Emmanuel Noobadmin wrote:
On 7/14/10, William Warrenhescominsoon@emmanuelcomputerconsulting.com wrote:
ok let me specify. Is there any real limit? I've seen some folks(and been told by a few) that you can't more than 10 physical interfaces in a linux system.
Googling up a really old 2005 newsgroup thread says some people had 24 physical NIC (6x 4quad) in a system before and one person vaguely remembers a hard 256 limit which would make sense if physical interface count is a byte value. _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
can you give me a link to that thread? My googling skills are apparently not up to snuff to find that..:)
On 7/14/10, William Warren hescominsoon@emmanuelcomputerconsulting.com wrote:
On 7/14/2010 1:16 AM, Emmanuel Noobadmin wrote:
Googling up a really old 2005 newsgroup thread says some people had 24 physical NIC (6x 4quad) in a system before and one person vaguely remembers a hard 256 limit which would make sense if physical interface count is a byte value.
can you give me a link to that thread? My googling skills are apparently not up to snuff to find that..:)
2001 poster says kernel 2.4 = 16 at least http://groups.google.com/group/comp.os.linux.hardware/browse_thread/thread/f...
2005 poster says he's seen 6x quad in a Linux server http://linux.derkeiler.com/Newsgroups/comp.os.linux.hardware/2005-01/0431.ht...
-----Original Message----- From: centos-bounces@centos.org [mailto:centos-bounces@centos.org] On Behalf Of Eero Volotinen Sent: Tuesday, July 13, 2010 8:58 PM To: CentOS mailing list Subject: Re: [CentOS] Linux Kernel Physical Interface Limit
2010/7/14 William Warren hescominsoon@emmanuelcomputerconsulting.com:
I think it's baloney mainly because i can't find a mention of it anywhere. Is there REALLY a limit on the number of
physical network
interfaces in the Kernel?
can you really create hardware with huge number or real ethernet controllers?
13x4 = 52 ether chips is possible with COTS hardware. ******************************************************************* This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. This footnote also confirms that this email message has been swept for the presence of computer viruses. www.Hubbell.com - Hubbell Incorporated**
----- "Brian T. Brunner" BBrunner@gai-tronics.com wrote:
-----Original Message----- From: centos-bounces@centos.org [mailto:centos-bounces@centos.org] On Behalf Of Eero Volotinen Sent: Tuesday, July 13, 2010 8:58 PM To: CentOS mailing list Subject: Re: [CentOS] Linux Kernel Physical Interface Limit
2010/7/14 William Warren hescominsoon@emmanuelcomputerconsulting.com:
I think it's baloney mainly because i can't find a mention of it
anywhere. Is there REALLY a limit on the number of
physical network
interfaces in the Kernel?
can you really create hardware with huge number or real ethernet controllers?
13x4 = 52 ether chips is possible with COTS hardware.
Is there some sort of reference for this information? Where does '13x4' come from?
--Tim
Start with a 4 port PCI ethercard http://www.cyberresearch.com/store/industrial-computers-rugged-pcs/industria...
Expand a PCIex1 slot into several PCI slots, claiming 13 PCI slots possible. I haven't tried to assure this is the max possible. http://www.cyberresearch.com/store/industrial-computers-rugged-pcs/industria...
10 PCI slots and a PCIEx16 makes(?) 14 slots http://www.cyberresearch.com/store/industrial-computers-rugged-pcs/industria...
so 13 or 14 X 4 makes 52 or 56 etherchips on one system with COTS.
7 PCIeX16 slots (http://www.evga.com/articles/00501/) @PCIE->4PCI = 28 PCI slots; @x4 chips/slot is 112 (+2 on the mobo) is 1114 interfaces.
Useful? Not my call. Silly enough that somebody will try? Probably.
-----Original Message----- From: centos-bounces@centos.org [mailto:centos-bounces@centos.org] On Behalf Of Tim Nelson Sent: Wednesday, July 14, 2010 10:35 AM To: CentOS mailing list Subject: Re: [CentOS] Linux Kernel Physical Interface Limit
----- "Brian T. Brunner" BBrunner@gai-tronics.com wrote:
-----Original Message----- From: centos-bounces@centos.org [mailto:centos-bounces@centos.org] On Behalf Of Eero Volotinen Sent: Tuesday, July 13, 2010 8:58 PM To: CentOS mailing list Subject: Re: [CentOS] Linux Kernel Physical Interface Limit
2010/7/14 William Warren hescominsoon@emmanuelcomputerconsulting.com:
I think it's baloney mainly because i can't find a
mention of it
anywhere. Is there REALLY a limit on the number of
physical network
interfaces in the Kernel?
can you really create hardware with huge number or real ethernet controllers?
13x4 = 52 ether chips is possible with COTS hardware.
Is there some sort of reference for this information? Where does '13x4' come from?
--Tim _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
******************************************************************* This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. This footnote also confirms that this email message has been swept for the presence of computer viruses. www.Hubbell.com - Hubbell Incorporated**
----- "Brian T. Brunner" BBrunner@gai-tronics.com wrote:
-----Original Message----- From: centos-bounces@centos.org [mailto:centos-bounces@centos.org] On Behalf Of Eero Volotinen Sent: Tuesday, July 13, 2010 8:58 PM To: CentOS mailing list Subject: Re: [CentOS] Linux Kernel Physical Interface Limit
2010/7/14 William Warren hescominsoon@emmanuelcomputerconsulting.com:
I think it's baloney mainly because i can't find a mention of it
anywhere. Is there REALLY a limit on the number of
physical network
interfaces in the Kernel?
can you really create hardware with huge number or real ethernet controllers?
13x4 = 52 ether chips is possible with COTS hardware.
Even if the limit were lower, such as 10 physical interfaces as mentioned before, I have to imagine that the host system would have issues dealing with the number of interrupts needed to *PROPERLY* service all of those interfaces in addition to the other system hardware. They may all work, but your performance may be horrible. In that case, I would think one or two 10Gbe interfaces to a nice switch and the necessary VLANs would perform nicely.
Just my $0.02 USD.
--Tim
On Wed, Jul 14, 2010 at 09:51:51AM -0500, Tim Nelson wrote:
Even if the limit were lower, such as 10 physical interfaces as mentioned before, I have to imagine that the host system would have issues dealing with the number of interrupts needed to *PROPERLY* service all of those interfaces in addition to the other system hardware.
There may (or may not) be another problem. As of a couple of years ago, on some Linux variants (didn't try RHEL/CentOS), I was having trouble even getting 6 NICs (on 3 cards) to work if I had IPv6 turned on. 4 NICs worked fine.
Filed some bug reports, and it was evident from the response that very, very few Linux users ever go > 4 eth's on a system. Thus the lack of properly debugged IPv6 support for that then. Fortunately I don't (yet) need IPv6. When I do, it'll be curious to see if the bug is still there.
Whit
On 07/14/2010 08:17 AM, Whit Blauvelt wrote:
On Wed, Jul 14, 2010 at 09:51:51AM -0500, Tim Nelson wrote:
Even if the limit were lower, such as 10 physical interfaces as mentioned before, I have to imagine that the host system would have issues dealing with the number of interrupts needed to *PROPERLY* service all of those interfaces in addition to the other system hardware.
There may (or may not) be another problem. As of a couple of years ago, on some Linux variants (didn't try RHEL/CentOS), I was having trouble even getting 6 NICs (on 3 cards) to work if I had IPv6 turned on. 4 NICs worked fine.
Filed some bug reports, and it was evident from the response that very, very few Linux users ever go> 4 eth's on a system. Thus the lack of properly debugged IPv6 support for that then. Fortunately I don't (yet) need IPv6. When I do, it'll be curious to see if the bug is still there.
I've got six machines with 6 Gb interfaces (two on motherboard, 4 on a card) right now (the design called for 3 bonded pairs on separate nets for redundancy). I haven't tried IPV6 on them. I had 'issues' with bonding and VMs though.
On Wed, Jul 14, 2010 at 08:52:02AM -0700, Benjamin Franz wrote:
Filed some bug reports, and it was evident from the response that very, very few Linux users ever go> 4 eth's on a system. Thus the lack of properly debugged IPv6 support for that then. Fortunately I don't (yet) need IPv6. When I do, it'll be curious to see if the bug is still there.
I've got six machines with 6 Gb interfaces (two on motherboard, 4 on a card) right now (the design called for 3 bonded pairs on separate nets for redundancy). I haven't tried IPV6 on them. I had 'issues' with bonding and VMs though.
My trouble didn't involve bonding or VMs, just using 5 interfaces at once (dual-homed WAN, DMZ, LAN, direct crossover failover backup). Having IPv6 turned on was screwing up using more than 4 for IPv4. Suspect it was a kernel limitation that in likelihood is since fixed.
Whit
On Wed, 2010-07-14 at 08:52 -0700, Benjamin Franz wrote:
I've got six machines with 6 Gb interfaces (two on motherboard, 4 on a card) right now (the design called for 3 bonded pairs on separate nets for redundancy). I haven't tried IPV6 on them. I had 'issues' with bonding and VMs though.
--- Can you give me "sar -I SUM" the last timed entry intr/s? How many CPUs? I'm not questioning you but on the curious side.
John
On 07/14/2010 10:30 AM, JohnS wrote:
On Wed, 2010-07-14 at 08:52 -0700, Benjamin Franz wrote:
I've got six machines with 6 Gb interfaces (two on motherboard, 4 on a card) right now (the design called for 3 bonded pairs on separate nets for redundancy). I haven't tried IPV6 on them. I had 'issues' with bonding and VMs though.
Can you give me "sar -I SUM" the last timed entry intr/s? How many CPUs? I'm not questioning you but on the curious side.
On the heaviest loaded machine:
10:10:01 AM sum 1637.48 10:20:01 AM sum 1640.73 10:30:01 AM sum 1653.58 10:40:01 AM sum 1617.78 10:50:01 AM sum 1727.97 11:00:01 AM sum 1767.88 11:10:01 AM sum 1798.93 11:20:01 AM sum 1782.14
Average: INTR intr/s Average: sum 1365.55
This is on a dual processor machine with a total of 8 cores.
The highest I see on any of the machines for the last 24 hours is a brief (one ten minute interval) peak of 5300 intr/second during system backups and nothing over 3000 otherwise.
On 07/13/2010 05:51 PM, William Warren wrote:
I think it's baloney mainly because i can't find a mention of it anywhere. Is there REALLY a limit on the number of physical network interfaces in the Kernel?
Most of the limits on the number of any device come from the major/minor numbers on the device nodes. Since ethernet adapters don't have device nodes, I'd be surprised if there were any practical limit.
I'm able to create at least 1025 VLANs on my ethernet adapter, and VLANs are "real" interfaces in the kernel.
-
Benjamin Franz -
Brunner, Brian T. -
Eero Volotinen -
Emmanuel Noobadmin -
Gordon Messmer -
JohnS -
Ross Walker -
Tim Nelson -
Whit Blauvelt -
William Warren