Andrew and Dennis are spot on. Their conclusions about your server being connected to an access port and not a trunk port would be my conclusion as well.
On Sat, Jan 24, 2015 at 9:11 AM, Dennis Jacobfeuerborn < dennisml@conversis.de> wrote:
Hi Boris, what I'd like to know is the actual VLAN configuration of the switch port (link-type and tagged and untagged VLANs). When I look at the switchport coniguration here I get (among other things):
... Port link-type: trunk Tagged VLAN ID : 8, 1624 Untagged VLAN ID : 10 ...
Here is my suspicion: Your ports have an access link-type with an untagged VLAN ID of 48. That would explain why the moment you configure an IP from that VLAN on eth0 you get connectivity because then the packets the Linux box sends are untagged as the switch would expect them to be. If you only put an address on eth0.48 then the packets get tagged by Linux but if the switch port is not configured to receive the packets for VLAN 48 as tagged then it will simply drop these packets and you will not get connectivity.
Additionally, the switch should gripe about 802.1q BPDUs. Check the in-memory system log (or syslog server if you have configured that).
show logging | i 1Q
Example: 1w1d: %SPANTREE-2-RECV_1Q_NON_TRUNK: Received 802.1Q BPDU on non trunk FastEthernet0/2 on vlan 100.
So getting the actual VLAN config of the switch port would help to determine if the switch actually expects to receive the packets the way you send them from the Linux box.
+1 Let's see the config for the switch port your server is connected to.