I just got 50 Mbps symmetric fiber from AT&T and it includes a /56 of IPv6 addresses, replacing a much slower ADSL line. I never tried to get IPv6 working on the old connection. I'm using CentOS 7 as a gateway and it's worked great for several versions for IPv4.
I'm not seeing any IPv6 default route on the WAN interface. I suspect I'm not getting route announcements. I think I have all the IPv6 variables in ifcfg-em2 set right. But I do notice that the accept_ra file in proc for that interface has value 1, not 2. Changing it to 2 doesn't change anything, though. No route appears.
While I wait for an answer to my trouble ticket, is there some way to verify that I'm not receiving any RA packets? Is there a way to force a solicitation for one? Is there a tcpdump invocation I can use to watch for them? Are there log messages that will tell me when an RA has been seen and added to the routing table or ignored?
I struggled with this under CentOS 7. I think there is a bug.
You can run /usr/sbin/radvdump to print out RAs. Leave it running for some minutes.
I had this in my /etc/sysctl.d/50-net6.conf (on C7):
# # IPv6 Forwarding # net.ipv6.conf.all.forwarding = 1 net.ipv6.conf.default.forwarding = 1 net.ipv6.conf.all.accept_ra = 1 net.ipv6.conf.default.accept_ra = 1 # # Fix bug to received RAs from Router # Disable forwarding on enp4s0f1 interface so we still get RAs # net.ipv6.conf.enp4s0f1.forwarding = 0 #net.ipv6.conf.enp4s0f1.accept_ra = 1
where enp4s0f1 is the WAN interface. Note that ipv6 forwarding still works. I used my C7 as a firewall/gateway.
I am running 8.1 now.
Alan
Once upon a time, Kenneth Porter shiva@sewingwitch.com said:
I just got 50 Mbps symmetric fiber from AT&T and it includes a /56 of IPv6 addresses, replacing a much slower ADSL line. I never tried to get IPv6 working on the old connection. I'm using CentOS 7 as a gateway and it's worked great for several versions for IPv4.
I'm not seeing any IPv6 default route on the WAN interface. I suspect I'm not getting route announcements. I think I have all the IPv6 variables in ifcfg-em2 set right. But I do notice that the accept_ra file in proc for that interface has value 1, not 2. Changing it to 2 doesn't change anything, though. No route appears.
While I wait for an answer to my trouble ticket, is there some way to verify that I'm not receiving any RA packets? Is there a way to force a solicitation for one? Is there a tcpdump invocation I can use to watch for them? Are there log messages that will tell me when an RA has been seen and added to the routing table or ignored?
I haven't touched AT&T's IPv6, but the typical way WAN IPv6 works is:
- gateway sends a router solicitation and gets a router advertisement with "stateful config" set, which tells gateway to do DHCPv6 (but default route comes from RA) - gateway does DHCPv6 to get a WAN IP - after that completes, gateway does DHCPv6 for prefix delegation
If you are running NetworkManager, then IIRC the accept_ra flag doesn't matter, because NM manages everything rather than have the kernel handle autoconfiguration (because NM needs to know what's going on with IPs).
As for watching, "tcpdump -p -i <interface> -v ip6" should show everything (and since you don't have any routing yet, you don't really need to filter out anything else!). You could filter "ip6 and multicast", because RAs and DHCPv6 (and ND, neighbor discovery, the counterpart to ARP) are all multicast.
On 4/28/2020 3:17 PM, Chris Adams wrote:
- gateway sends a router solicitation and gets a router advertisement with "stateful config" set, which tells gateway to do DHCPv6 (but default route comes from RA)
I'm not seeing any outbound IPv6 traffic from my CentOS 7 box on the WAN interface. I do see RA's emitting from the LAN interface, from radvd. Is there some setting in NM tells it to send solicitations? Is there some way to push one manually?
Once upon a time, Kenneth Porter shiva@sewingwitch.com said:
On 4/28/2020 3:17 PM, Chris Adams wrote:
- gateway sends a router solicitation and gets a router advertisement with "stateful config" set, which tells gateway to do DHCPv6 (but default route comes from RA)
I'm not seeing any outbound IPv6 traffic from my CentOS 7 box on the WAN interface. I do see RA's emitting from the LAN interface, from radvd. Is there some setting in NM tells it to send solicitations? Is there some way to push one manually?
What's in /etc/sysconfig/network-scripts/ifcfg-<name>? I wonder if you have IPv6 disabled.
I'm not using a "regular" (CentOS, Fedora, etc.) Linux as a gateway; I have OpenWRT on a dedicated box. I couldn't find a way to handle the prefix delegation with the typical desktop/server tools (but it has been a while since I looked). OpenWRT has their own daemon for that.
However, my local systems are all sending RA solicitations and getting DHCPv6-assigned addresses with NetworkManager (which matches the first steps of what you need on the WAN, just not the prefix delegation).
On 4/28/2020 4:22 PM, Chris Adams wrote:
What's in /etc/sysconfig/network-scripts/ifcfg-<name>? I wonder if you have IPv6 disabled.
Pasted below. V6 definitely works. I have a second server and gave it a WAN address and I can connect between them using their WAN addresses. That's what told me that my ip6tables weren't screwed up and that the problem lay elsewhere. That's when I hit on looking at the route table and seeing no default IPv6 route.
I'm not using a "regular" (CentOS, Fedora, etc.) Linux as a gateway; I have OpenWRT on a dedicated box. I couldn't find a way to handle the prefix delegation with the typical desktop/server tools (but it has been a while since I looked). OpenWRT has their own daemon for that.
However, my local systems are all sending RA solicitations and getting DHCPv6-assigned addresses with NetworkManager (which matches the first steps of what you need on the WAN, just not the prefix delegation).
I'm using OpenWrt at home and it's working mostly fine there. Except with my Android phone. I'm not getting a DNS setting for V6, but I do have the setting in the router's config file. The Win10 clients work fine, though. Apparently Android has issues with DHCPv6, and I'm betting it's interfering with my SLAAC config.
LAN config, ifcfg-em1:
TYPE=Ethernet PROXY_METHOD=none BROWSER_ONLY=no BOOTPROTO=none DEFROUTE=yes IPV4_FAILURE_FATAL=no IPV6INIT=yes IPV6_AUTOCONF=no IPV6_DEFROUTE=no IPV6_FAILURE_FATAL=no IPV6_ADDR_GEN_MODE=stable-privacy NAME=em1 UUID=fbf59f72-2d60-4dd3-8870-c14a1acb4337 DEVICE=em1 ONBOOT=yes DHCP_CLIENT_ID=saruman IPV6_PRIVACY=no ZONE=internal IPADDR=10.169.1.128 PREFIX=9 IPV6ADDR=2001:1890:1837:5B11::100/60
WAN config, ifcfg-em2:
TYPE=Ethernet PROXY_METHOD=none BROWSER_ONLY=no BOOTPROTO=none DEFROUTE=yes IPV4_FAILURE_FATAL=no IPV6INIT=yes IPV6_AUTOCONF=yes IPV6_DEFROUTE=yes IPV6_FAILURE_FATAL=no IPV6_ADDR_GEN_MODE=stable-privacy NAME=em2 UUID=fe08c9df-9913-47f5-be8b-47dad168bcc4 DEVICE=em2 ONBOOT=yes IPADDR=12.3.140.156 PREFIX=29 GATEWAY=12.3.140.153 DNS1=127.0.0.1 DOMAIN="mpa.lan dhcp.mpa.lan microprecisionautomation.com" ZONE=external IPV6FORWARDING=yes IPV6_PEERDNS=no # AT&T assigns us 2001:1890:1837:5B00::/56 # we use the 2nd /64 for our DMZ IPV6ADDR_SECONDARIES=2001:1890:1837:5B01::100/64
Once upon a time, Kenneth Porter shiva@sewingwitch.com said:
I'm using OpenWrt at home and it's working mostly fine there. Except with my Android phone. I'm not getting a DNS setting for V6, but I do have the setting in the router's config file. The Win10 clients work fine, though. Apparently Android has issues with DHCPv6, and I'm betting it's interfering with my SLAAC config.
Yeah, Android refuses to support DHCPv6, so you either have to have IPv4 DNS or SLAAC. I have IPv4 DNS on my home network, so don't have an issue. I did just look, and OpenWRT is putting the DNS option for SLAAC in the RA, so that should work too (but I think that's something relatively recent for OpenWRT).
I didn't get that you have a static assignment (presumably a business connection) - they may not do RAs on that (I don't at my ISP job). Business connections (or at least, connections with static assignments) tend to operate differently. For that, they should have given you a static v6 address and gateway, just like they did for v4.
So... there's one thing you could try (but probably won't work to a regular router interface) - see if there's a MAC-derived fe80::/64 link-local address on their end. Get the MAC of the gateway from the v4 ARP entry and expand it to a LL v6 address as fe80::xxxx:xxff:fexx:xxxx (split the MAC, put ff:fe in the middle). Try ping6 that address with %em2 appended (have to append the interface when using link-local addresses). I doubt it'll work, since I know Juniper (which IIRC AT&T likes) doesn't assign those (I can't remember for sure about Cisco and don't have a handy test target).
And frankly, giving you a /56 is pretty crappy, since ARIN rules say to give every site a /48. I'd only do a /56 for a home connection prefix delegation. But, that's AT&T! :)
--On Tuesday, April 28, 2020 10:16 PM -0500 Chris Adams linux@cmadams.net wrote:
I didn't get that you have a static assignment (presumably a business connection) - they may not do RAs on that (I don't at my ISP job). Business connections (or at least, connections with static assignments) tend to operate differently. For that, they should have given you a static v6 address and gateway, just like they did for v4.
I didn't think to ask when we were turning up the v4 and phones. I'm betting there's a setting in their gateway box but I'm waiting for them to give me the credentials to log into it.
So... there's one thing you could try (but probably won't work to a regular router interface) - see if there's a MAC-derived fe80::/64 link-local address on their end. Get the MAC of the gateway from the v4 ARP entry and expand it to a LL v6 address as fe80::xxxx:xxff:fexx:xxxx (split the MAC, put ff:fe in the middle). Try ping6 that address with %em2 appended (have to append the interface when using link-local addresses). I doubt it'll work, since I know Juniper (which IIRC AT&T likes) doesn't assign those (I can't remember for sure about Cisco and don't have a handy test target).
Good idea but alas it's not routing. I can ping6 their gateway but it won't route after I add it to the route table for the WAN interface. While pinging a remote server, in a second shell I can tcpdump icmp6 packets and I see the packets going out but no replies coming back.
And frankly, giving you a /56 is pretty crappy, since ARIN rules say to give every site a /48. I'd only do a /56 for a home connection prefix delegation. But, that's AT&T! :)
I'd just read about that when researching this. Maybe they decided that since we only have about a dozen people at our site, we won't have a lot of subnets. What do small offices DO with 256 public subnets, anyway? I suppose eventually we'll have an IoT subnet on every person.
Once upon a time, Kenneth Porter shiva@sewingwitch.com said:
--On Tuesday, April 28, 2020 10:16 PM -0500 Chris Adams linux@cmadams.net wrote:
And frankly, giving you a /56 is pretty crappy, since ARIN rules say to give every site a /48. I'd only do a /56 for a home connection prefix delegation. But, that's AT&T! :)
I'd just read about that when researching this. Maybe they decided that since we only have about a dozen people at our site, we won't have a lot of subnets. What do small offices DO with 256 public subnets, anyway? I suppose eventually we'll have an IoT subnet on every person.
The idea with IPv6 is not to even necessarily think about it in terms of direct numbers, but in layers. It is not uncommon to have several layers of routers, firewalls, guest wifi networks, etc., and each layer should request a prefix delegation from its parent. So rather than 256 subnets, think about it as 8 layers (at most... but if a layer has more than 2 children, you have fewer layers available).
So for example, if your Internet gateway has a desktop firewall, a guest wifi, a public DMZ, and a development lab gateway connected, and you want to allow for more things at that layer, there's 3 of your 8 bits in a /56. If the dev lab needs to fan out more, and maybe your public DMZ needs to break up for production and QA-testing networks, and you add a VPN concentrator to the desktop network... you can go through those bits fast.
In IPv4, people would just NAT the crap out of everything, having to tunnel from one NATted network to another, making life really difficult. The plan is no NAT in IPv6, so allow for all potential allocations up front.
Also, allocations should be larger than necessary and sparse, so that you never need another allocation (even if you grow to 1000 employees and multiple buildings on a campus). This is to hopefully prevent routing tables from exploding like IPv4 did (and also to avoid you having to renumber everything just to stay in a single block).
I discovered that IPv6 is sort of working when I got an email rejection from Comcast for not having an IPv6 PTR record. I discovered I could telnet to port 25 on their MX server over IPv6! I then found I could tracroute6 to them, but I couldn't to my Linode VPS in Fremont. It gets to the data center and stops. Going the other way, my Linode can traceroute6 almost to my AT&T-hosted server. Neither can reach the open port 25 on the other, but both can reach mx1.comcast.net via IPv6.
I dropped the firewall ("systemctl stop firewalld", confirmed with "ip6tables -L -v -n") on my Fremont server and still can't traceroute6 or ping6 it. Nor can I do so from my home system on Comcast.
What else could be blocking ping6, traceroute6, and telnet to port 25 on IPv6?
From AT&T host to Linode host:
[root@saruman ~]# traceroute6 2600:3c01::f03c:91ff:febc:e22d traceroute to 2600:3c01::f03c:91ff:febc:e22d (2600:3c01::f03c:91ff:febc:e22d), 30 hops max, 80 byte packets 1 2001:1890:1837:5b00::1 (2001:1890:1837:5b00::1) 3.230 ms 3.181 ms 3.137 ms 2 2001:1890:c05:2f3a::ee6e:61cd (2001:1890:c05:2f3a::ee6e:61cd) 6.906 ms 6.901 ms 6.870 ms 3 ok3ca81crs.ipv6.att.net (2001:1890:ff:ffff:12:123:6:98) 8.305 ms 8.254 ms 8.258 ms 4 sffca21crs.ipv6.att.net (2001:1890:ff:ffff:12:122:110:65) 14.199 ms 14.164 ms 14.153 ms 5 sffca401igs.ipv6.att.net (2001:1890:ff:ffff:12:122:114:5) 9.861 ms 10.163 ms * 6 2001:1890:1fff:41e:192:205:32:222 (2001:1890:1fff:41e:192:205:32:222) 9.789 ms 8.078 ms 7.365 ms 7 linode-ic-342731-sjo-b21.c.telia.net (2001:2000:3080:1ac4::2) 6.788 ms 6.327 ms 6.262 ms 8 2600:3c01:3333:4::2 (2600:3c01:3333:4::2) 7.294 ms 7.728 ms 7.705 ms 9 * * *
From Linode to AT&T:
[root@linode2 ~]# traceroute6 2001:1890:1837:5b01::100 traceroute to 2001:1890:1837:5b01::100 (2001:1890:1837:5b01::100), 30 hops max, 80 byte packets 1 2600:3c01::8678:acff:fe0d:79c1 (2600:3c01::8678:acff:fe0d:79c1) 0.783 ms 2600:3c01::8678:acff:fe0d:a641 (2600:3c01::8678:acff:fe0d:a641) 0.834 ms 0.972 ms 2 2600:3c01:3333:4::1 (2600:3c01:3333:4::1) 0.645 ms 0.646 ms 2600:3c01:3333:2::1 (2600:3c01:3333:2::1) 0.508 ms 3 ix-ae-67-0.tcore1.sqn-san-jose.ipv6.as6453.net (2001:5a0:1000:500::a1) 1.059 ms 2001:550:2:3::d9:1 (2001:550:2:3::d9:1) 1.967 ms ix-ae-67-0.tcore1.sqn-san-jose.ipv6.as6453.net (2001:5a0:100 4 be2063.ccr21.sjc01.atlas.cogentco.com (2001:550:0:1000::9a36:1a1) 2.340 ms 2001:1890:1fff:415:192:205:37:69 (2001:1890:1fff:415:192:205:37:69) 3.209 ms 4.412 ms 5 sffca21crs.ipv6.att.net (2001:1890:ff:ffff:12:122:114:6) 7.353 ms 6.769 ms 5.379 ms 6 ok3ca81crs.ipv6.att.net (2001:1890:ff:ffff:12:122:110:66) 4.785 ms 6.158 ms 6.073 ms 7 2001:1890:ff:ffff:12:122:119:125 (2001:1890:ff:ffff:12:122:119:125) 3.401 ms 3.406 ms 3.469 ms 8 ok3ca81crs.ipv6.att.net (2001:1890:ff:ffff:12:122:110:66) 8.444 ms 8.415 ms 2001:1890:c05:2f3a::116e:61cd (2001:1890:c05:2f3a::116e:61cd) 5.816 ms 9 2001:1890:ff:ffff:12:122:119:125 (2001:1890:ff:ffff:12:122:119:125) 4.688 ms 4.607 ms 4.565 ms 10 * * 2001:1890:c05:2f3a::116e:61cd (2001:1890:c05:2f3a::116e:61cd) 6.732 ms 11 * * *
Once upon a time, Kenneth Porter shiva@sewingwitch.com said:
I discovered that IPv6 is sort of working when I got an email rejection from Comcast for not having an IPv6 PTR record. I discovered I could telnet to port 25 on their MX server over IPv6! I then found I could tracroute6 to them, but I couldn't to my Linode VPS in Fremont. It gets to the data center and stops. Going the other way, my Linode can traceroute6 almost to my AT&T-hosted server. Neither can reach the open port 25 on the other, but both can reach mx1.comcast.net via IPv6.
Yeah, unfortunately things like that can happen, v4 or v6 (like I couldn't get to a local TV station's website a little while ago from my home connection, but could from elsewhere).
From your traceroutes, it kind of looks like it's possible that it's
something on your gateway (but I'm not really sure). Do you have any IPv6 firewall running there?
One other note about mail on v6 - not only do you need to have a valid reverse (with matching forward) DNS record, you probably need to do TLS with a valid cert (Let's Encrypt is free and easy).
--On Thursday, April 30, 2020 6:08 PM -0500 Chris Adams linux@cmadams.net wrote:
From your traceroutes, it kind of looks like it's possible that it's something on your gateway (but I'm not really sure). Do you have any IPv6 firewall running there?
I dropped the firewall just to make sure it wasn't the issue. My connections to Linode eventually started working. Comparison to the traceroute I posted earlier shows a different router just before my server, so I suspect they had a bad router. I still can't connect the other way, towards AT&T. But routing must be working because replies from outbound connections are getting back. I'd file a ticket with AT&T but they still haven't added my account to their ticket system after more than a week of service. So I'm stuck playing tag with the sales consultants.
One other note about mail on v6 - not only do you need to have a valid reverse (with matching forward) DNS record, you probably need to do TLS with a valid cert (Let's Encrypt is free and easy).
Thanks for reminding me. I added an IPv6-only name for testing and for my reverse and need to add that to my certificate.
-
Alan McRae -
Chris Adams -
Kenneth Porter