This is that system with the missing management port, and I'm still fighting it. Everything *looks* right:
3: enp6s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 00:25:90:0a:42:87 brd ff:ff:ff:ff:ff:ff inet 192.168.0.100/24 brd 192.168.0.255 scope global enp6s0
ip route 192.168.0.0/24 dev enp6s0 proto kernel scope link src 192.168.0.100
and ipmitool lan print IP Address Source : Static Address IP Address : 192.168.0.132 Subnet Mask : 255.255.255.0 MAC Address : 00:25:90:0a:42:92 <...> Default Gateway IP : 192.168.0.100 Default Gateway MAC : 00:25:90:0a:42:87
I can ping 192.168.0.1... yet I'm getting martians on the real network.
There's no firewall on enp6s0. Any clues as to what I'm missing?
mark "No illudium Q-36 space modulators, please"
Once upon a time, mark m.roth@5-cent.us said:
This is that system with the missing management port, and I'm still fighting it. Everything *looks* right:
3: enp6s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 00:25:90:0a:42:87 brd ff:ff:ff:ff:ff:ff inet 192.168.0.100/24 brd 192.168.0.255 scope global enp6s0
ip route 192.168.0.0/24 dev enp6s0 proto kernel scope link src 192.168.0.100
and ipmitool lan print IP Address Source : Static Address IP Address : 192.168.0.132 Subnet Mask : 255.255.255.0 MAC Address : 00:25:90:0a:42:92 <...> Default Gateway IP : 192.168.0.100 Default Gateway MAC : 00:25:90:0a:42:87
No, that does not look right. You have configured the gateway of the IPMI to be the host OS side of the NIC. You can't do that... in a lot of systems I've seen, the IPMI side of the NIC can't even talk to the host OS on the network.
The IPMI LAN is an independent controller, separate from the host OS. It does not use any routing/firewall/etc. from the host OS. It is just another device on the network that happens to share the same physical port as the host. It should be configured to talk to the same network gateway and such as the host OS.
Think of it as if you have two independent systems inside one box; a PC and an IPMI device. It is similar to them being two devices with an ethernet switch between them (and another port to the outside world). It doesn't actually work that way (because they are sharing the physical port), but it is close.
On Thu, Jul 12, 2018 at 07:27:58PM -0500, Chris Adams wrote:
Default Gateway IP : 192.168.0.100 Default Gateway MAC : 00:25:90:0a:42:87
No, that does not look right. You have configured the gateway of the IPMI to be the host OS side of the NIC. You can't do that... in a lot of systems I've seen, the IPMI side of the NIC can't even talk to the host OS on the network.
From previous emails, I gather that mark can't find the way to set
which interface the IPMI BMC uses, so he's setting the BMC's IP settings to use one of the NICs as a gateway. This is not how you make that setting (it won't work) but I can see where he's coming from.
In my experience, it's either hard-wired to a particular interface. This should be documented, otherwise you need another computer on the same network or connected with a crossover cable to figure it out.
Sometimes you can set the interface that IPMI uses in the BIOS or through 'ipmitool'.
For Dell hardware, the ipmitool command that ships with CentOS7 has an 'ipmitool delloem lan set <mode>' which lets you choose which interface to use.
============================================================================== # ipmitool delloem lan set
lan set <Mode> sets the NIC Selection Mode : on iDRAC12g OR iDRAC13g : dedicated, shared with lom1, shared with lom2,shared with lom3,shared with lom4,shared with failover lom1,shared with failover lom2,shared with failover lom3,shared with failover lom4,shared with Failover all loms, shared with Failover None). on other systems : dedicated, shared, shared with failover lom2, shared with Failover all loms. ==============================================================================
If I'm using a system with a shared setup, I don't set up networking on that interface at all. At least with Dells, even if you set up an IP on the interface, it can't talk to the BMC from the OS using the shared interface. Ping doesn't work, 'ipmitool -I lanplus' doesn't work, http doesn't work. You need to connect from another host. I have a private management network that we use for IPMI/iLO systems, because those BMC interfaces are known to be an attack vector.
Hopefully, this is enough information to explain that you need to find out which interface your IPMI device is using, and to use appropriate IP settings, and to *NOT* use the IP/MAC from any OS interfaces as your IPMI device's gateway.
On Fri, Jul 13, 2018 at 08:34:26AM -0400, Jonathan Billings wrote:
In my experience, it's either hard-wired to a particular interface. This should be documented, otherwise you need another computer on the same network or connected with a crossover cable to figure it out.
Sometimes you can set the interface that IPMI uses in the BIOS or through 'ipmitool'.
Sorry, the above was a run-on sentence I edited to break up into two paragraphs, but I meant "either it's hard-wired to a particular interface or you can change it in the BIOS or with ipmitool".
Please excuse my confusing email.
Jonathan Billings wrote:
On Thu, Jul 12, 2018 at 07:27:58PM -0500, Chris Adams wrote:
Default Gateway IP : 192.168.0.100 Default Gateway MAC : 00:25:90:0a:42:87
No, that does not look right. You have configured the gateway of the IPMI to be the host OS side of the NIC. You can't do that... in a lot of systems I've seen, the IPMI side of the NIC can't even talk to the host OS on the network.
From previous emails, I gather that mark can't find the way to set which interface the IPMI BMC uses, so he's setting the BMC's IP settings to use one of the NICs as a gateway. This is not how you make that setting (it won't work) but I can see where he's coming from.
In my experience, it's either hard-wired to a particular interface. This should be documented, otherwise you need another computer on the same network or connected with a crossover cable to figure it out.
Sometimes you can set the interface that IPMI uses in the BIOS or through 'ipmitool'.
<snip> Thanks for the info; the thing I never understood in the documentation was the business of "shared with lomx" - is that for a second management port?
Aos, my manager tells me to think of the BMC as a completely separate computer, which it is, but that has *no* contact with the o/s.
On the other hand... *sigh* - I've solved the original issue: y'see, the server's just below the middle of my chest in the rack, and the fans stick out about a cm or two. I went back into the room last night, with a flashlight... and bent down, and lo and behold, there *was* a perfectly good management port. Connected that, did a warm reboot of the bmc, and all is well.
My manager, the other admin I work with, and I were all too tall to see the port, so I guess there is some use for short people*....
mark
* Of course I'm hearing Randy Newman's Short People in my head.
-
Chris Adams -
Jonathan Billings -
mark