Quick Intro to IPv6

This post strives to provide an overview where (and why) IPv6 is different from IPv4. The intended audience are folks with a solid understanding of IPv4 but not too much exposure to IPv6 so far (I hear such an audience still exists ;-), and the post is intentionally kept short (regular readers of this blog may imagine that I’d love to extensively rant on several of the below items. some of them would deserve full posts on their own). Also I won’t go into technical detail too much.
In a nutshell the post tries to summarize why, under the hood, IPv6 is quite different from IPv4, and what those differences are.

Design Objectives

In order to understand certain elements of IPv6, it’s helpful to keep in mind that it was mainly developed in the mid-90s. It hence tried to solve some of the issues & challenges found in networking at the time, besides introducing general new ideas.
For the purpose of this post the following objectives are of interest:

• autonomy: hosts should be able to come up with the configuration of basic IP parameters on their own, without the need for human intervention/administration or the need for additional services (like DHCP).
• (restoration of the) end-to-end principle: hosts should be able to communicate with each other without ‘the network’ providing functions besides simple packet forwarding.
• optimization: come up with some changes that ‘make network communications more efficient and hence faster’ such as replacing broadcast by multicast, or simplifying the IP header.

And, yes, the one main reason why IPv6 gets deployed in many environments today, that is larger address space, played a role, too.

The Unpleasant Reality

In case you’ve already been working a bit with IPv6, and you have scratched your head while reading the above section, thinking sth along the lines: “wait a second, those ideas haven’t really worked out, they’ve only been implemented half-baked, or they have even added a lot of complexity and operational pain”, you’re fully right.
Several factors have contributed to the mess we have today, like:

• Dynamics within standard bodies based on (seemingly) voluntary work like the IETF, including the composition of working groups, their politics plus the associated way of finding compromises etc.
• Over-engineering in general, further fueled by certain incentives within those working groups
• Lack of ‘feedback from the field’: during the first 15 years or so after the initial specification not much deployment happened, so nobody told those well-intentioned and smart – seriously, no irony here: they are, but the ecosystem is complex in itself – engineers that what many networks needed was just more address space, and that all the other shiny enhancements and features primarily introduced complexity and operational efforts. And things got worse with every year that passed, with regard to protocol complexity, and with regard to the inability to make fundamental changes of the design.

I’m aware that some of these things and developments are hard to understand from a technical perspective or from a 2021 point-of-view, but protocol development doesn’t happen in a vacuum, and of course in hindsight it’s always 20/20.
Point is: from a deployment perspective, accept IPv6 as it is (the boat for significant changes has long sailed), and drive the right conclusions for your operational decisions.

Technical Elements & Changes

In this section I’ll list some of the main technical elements of IPv6 which are new for those coming from IPv4, and which play a huge role in both the way how an IPv6 stack works (by default) and how ‘an IPv6-enabled network’ behaves in general:

• Router advertisements (‘RAs’): these are packets sent out by routers to their adjacent networks which carry information that enables hosts to perform autoconfiguration (remember the above autonomy objective). Understanding these packets, and their operational implications is crucial for smooth operations of the vast majority of IPv6 networks. I might add here that, based on some of the factors of the 2nd section, RAs are super-complex packets themselves, so they are somewhat metaphorical for the state of IPv6 ;-).
• The link-local address (‘LLA’): in contrast to IPv4 where one and the same address is usually used both for communication within a subnet and with remote hosts, IPv6 strictly differentiates between local communication and non-local communication (the latter happening through a router/’the default gateway’). This differentiation includes a special address only used for local purposes. It uses the prefix fe80::/10.
• Multicast: the approach of ‘general broadcasting’ when communication with multiple or ‘unknown’ hosts in the local subnet is needed, was replaced by using multicast groups (their addresses start with ‘ff’) for these types of communication. Combined with new/additional interactions (like RAs), at least in the local network (the ‘local link’ in IPv6 terms) one will usually see a lot of multicast traffic with different addresses, and for different purposes. Evidently this has a number of operational implications (which, again, are outside the scope of this post).

To some lesser extent one could add IPv6 extension headers (EHs) to this list, but – luckily – there’s a fair chance that many of you joining the IPv6 world in 2021 won’t ever see them in operational practice (besides security filters dropping them), so no need to discuss them further here.

As one can see, quite a few architectural changes have happened between IPv4 and IPv6. Understanding them can help to make well-informed decisions during the deployment of IPv6.

IPv6 Duplicate Address Detection

In this post I’ll take a closer look at IPv6 Duplicate Address Detection (aka ‘DAD’, which evidently bears all of types of jokes and wordplays). While the general mechanism should be roughly familiar to everybody working with IPv6 there are some interesting intricacies under the hood, some of which might even have operational implications.

DAD was already part of the initial specification of SLAAC in RFC 1971 (dating from 1996), which was then obsoleted by RFC 2462. RFC 4429 describes a modification called ‘Optimistic Duplicate Address Detection’. Neighbor discovery and SLAAC, incl. DAD, were later updated/specified in the RFCs 4861 and 4862 which are considered the main standards as of today. Finally DAD was enhanced in RFC 7527 but that’s of minor relevance here.

Its goal is to avoid address conflicts (within the scope of a respective address). To do so it is supposed to perform a specific verification procedure (‘ask a certain question’) and subsequently to act on the result of that procedure. However, as we will see, namely the latter can depend on a number of circumstances, in particular on the type of the address/IID.

How to ask the question?

Generally speaking a host is expected to perform the following (for a given unicast address):

• send a Neighbor Solicitation (ICMPv6 type 135) message.
• use the unspecified address (::, see RFC 4291, section 2.5.2) as source address, the requested unicast address’s Solicited-Node multicast address (SNMA, see RFC 4291, section 2.7.1) as target address and put the to-be-used unicast address as target address into the ICMPv6 payload.

This can look like this (ref. RFC 2464 for the ’33:33′ in the Ethernet multicast address):

It should be noted that RFC 4862 states that “Duplicate Address Detection MUST be performed on all unicast addresses prior to assigning them to an interface, regardless of whether they are obtained through stateless autoconfiguration, DHCPv6, or manual configuration”, but in practice this can be turned off on the OS level (and there might even exist situations where this could be desirable, see below). Still, the general verification procedure is mostly identical on the vast majority of operating systems.

Shall we wait for a response?

This is where the differences between scenarios start. As stated above RFC 4429 describes a thing called ‘Optimistic DAD’. The idea here is put an address into an ‘optimistic’ state right after sending out the NS and thereby make the address operational pretty much immediately (with some minor restrictions like not to send certain packets with said address as the Source Link-Layer Address Option [SSLAO]). This optimization is supposed to be used when – as of RFC 4429 section 3.1 – “the address is based on a most likely unique interface identifier” such as an EUI-64 generated one, a randomly generated one (Privacy Extensions, RFC 4941, more info here), a Cryptographically Generated Address (as for example used by Apple devices, see here) or a DHCPv6 address (note that the concept of ‘stable’ addresses as of RFC 7217 did not exist at the time). Optimistic DAD explicitly “SHOULD NOT be used for manually entered addresses”.
As of today it’s a fair assumption that all ‘client operating systems’ use Optimistic DAD, as can be observed in the above example, but this does not apply to servers using static addresses. This is how it looks like on macOS Big Sur (note that the router solicitation is sent already two milliseconds after the DAD neighbor solicitation)

What if the response indicates a conflict?

This is where things (differences) become really interesting. While RFC 4429 has a dedicated section on the ‘Collision Case’ (sect. 4.2), it remains relatively vague, includes terms like ‘hopefully’ 😉, and states that an address collision “may incur some penalty to the ON [optimistic node], in the form of broken connections, and some penalty to the rightful owner of the address” (which doesn’t sound right to me…).
RFC 4862 mandates (in “5.4.5.  When Duplicate Address Detection Fails”) that in case of a collision of an EUI-64 generated address the IPv6 operation of the respective interface “SHOULD be disabled”, but “MAY be continued” in other (address generation) scenarios. Furthermore “the node SHOULD log a system management error”.
An interface with a static address where DAD failed could look like this:

inet6 2001:db8:320:104::9/64 scope global tentative dadfailed 
valid_lft forever preferred_lft forever

So, overall no guidance is provided here how to proceed in case of a detected conflict for addresses based on RFC 3972 (CGAs), RFC 4941 (Privacy Extensions) or RFC 7217 (‘Stable IIDs’), but this may be specified in other places (see below), and/or might be left to the implementors of individual OS stacks. Many years ago Christopher Werny and myself performed some testing for Windows and Linux, creating various scenarios with address collisions, and from the top of my head I recall that their behavior was both quite different and not necessarily intuitive (sorry I don’t remember details).

CGAs have a dedicated Collision Count parameter which can be “incremented during CGA generation to recover from an address collision detected by duplicate address detection” (RFC 3972, section 3).

RFC 4941 includes this (with the TEMP_IDGEN_RETRIES defaulting to the value 3):

RFC 8415 on DHCPv6 specifies as follows (with a DEC_MAX_RC parameter indicating the number of client-side retries of getting a new address. it defaults to the value 4):

Furthermore the DHCPv6 server “SHOULD mark the addresses declined by the client so that those addresses are not assigned to other clients”.
I’m not sure about the exact sequence of things when the client uses optimistic DAD (which in turn should be the default for DHCPv6 addresses).

tl:dr of this section: the exact behavior of reacting to an address collision might not always be the same, and it might depend on several circumstances.

Operational Implications (1): Service Bindings

As laid out above optimistic DAD is not supposed to be performed when static IPv6 addresses are used. This can create issues when during system boot a service is to be bound to an address which is still in ‘tentative’ state (during DAD), as discussed in this thread (also interesting comment there at the bottom, on the differences re: DAD between FreeBSD and NetBSD).
This could look like this:

020/09/26 10:08:22 [emerg] 11298#11298: bind() to [2001:db8:104:1700::12]:80 failed (99: Cannot assign requested address)

Apparently this may be fixed by touching the following sysctl but I don’t fully understand its mechanism, so this might only work in certain scenarios:

sysctl net.ipv6.ip_nonlocal_bind=1

In any case the delay induced by DAD (with static addresses) should be considered for service bindings during startup.

Operational Implications (2): cni0 interface stuck in DAD

I once heard of a case where the cni0 bridge interface on Kubernetes clusters was stuck in DAD when initialized by standard CentOS initscripts (which in turn was difficult to troubleshoot as it only had veth members and wasn’t bound to any physical interface). This could presumably only be solved by disabling DAD as a whole. That might be a debatable approach (I for one think this is perfectly doable even in other settings once one has sufficient control over the [static] address assignment mechanisms), but for completeness sake here’s the relevant sysctl (from the current Linux kernel documentation):

Suffice to say that DAD might kick in various ways and in the context of different dependencies, so one has to be aware of its inner workings and of its role during interface initialization.
To contribute to such an understanding was the exact point of this post ;-). Thank you for reading so far, and as always I’m happy to receive feedback on any channel incl. Twitter.

A Holistic Look on SLAAC and DHCPv6

At first a very happy new year to all readers, and all the best for 2021!

While I wrote a few posts about IPv6-related topics in the past – for many years here and later on the present blog – it seems I never contributed to the ‘classic SLAAC vs. DHCPv6’ debate, besides documenting the behavior of different OSs in a long-expired Internet Draft + in this talk at RIPE70 and besides some ranting about DHCPv6.
Let’s change this today ;-). Here’s a post on those two. It’s a bit inspired by Fernando‘s recent talk on CVE-2020-16898 at the UK IPv6 Council Annual Meeting 2020 (some notes from the event here) in which he briefly covered the differences between both provisioning approaches incl. this slide:

I’d like to add some further perspective on the two. For that purpose I think it’s helpful to differentiate between the (network- or ‘environment level’-) operator point of view and the host perspective, as different parties (potentially pursuing very different objectives) can be responsible for those. Also it might make sense to differentiate between deployment scenarios like data centers or campus networks (some initial discussion of those here).
Let’s first look at the host/the ‘client’ which receives parameters or ‘instructions’ from one of the mechanisms (or from both, in certain settings). From its perspective the following objectives come to mind:

• Completeness: for the IP-level operation of a host certain parameters (some discussion here) and information bits are needed, and those have to be provided by ‘the environment’, that is from network components, additional systems offering certain services and the like.
• Simplicity: whatever happens keep it as simple as possible, from a general configuration perspective but also when it comes to components (like pieces of software, e.g. daemons, which have to be started, configured, maintained, patched etc.).
• Security: the security attributes (or lack thereof) of the two mechanisms can play a role for the provisioning strategy in a given environment. At this point it should be noted that, in general, I think that inherent security issues of RAs and DHCPv6 packets should be addressed on the infrastructure level and not by hosts themselves (hence rules 5 and 6 in this post).
• Support: this is seemingly evident, but still worth a mention/consideration. The configuration approach has to be supported at all (at least in an operationally feasible way, e.g. without installing additional 3rd party components) by the client platform(s) in question for a specific network or use case. Here it should be noted that as of RFC 8504 IPv6 Node Requirements SLAAC is a mandatory element of an IPv6 stack (‘MUST’ be supported, in RFC 2119 terminology) whereas DHCPv6 ‘SHOULD’ be supported (while this capital-letter statement in an RFC is a much stronger claim than a ‘common language’ interpretation of ‘should’, it’s still weaker than a ‘MUST’). For example it’s well known that Android does not support DHCPv6, which might have implications on the overall configuration approach of an environment.

From the perspective of the ‘operator’ (of the network infrastructure and/or of the servers being part of an overall provisioning picture, like DHCPv6 or TFTP servers for UEFI netboot) the following objectives may play a role:

• Functionality: it has to work ;-).
• Simplicity: whatever happens keep it as simple as possible.
• Security: the fewer components involved, the less vulnerability exposure.
• Support: different use cases and client platforms.
• Traceability: in many environments the capability to identify systems involved in security violations (be them security incidents, be them violations of intellectual property regulations, e.g. in university networks) is an important consideration. Traditional thinking often stated DHCPv6 would strictly be needed for this, but this might not be the case.
• Control: for system management and inventory purposes in data centers there’s often the desire to assign IP(v6) addresses to systems in a controlled way. Usually this precludes the use of SLAAC.

Now let’s have a look at both mechanisms in different scenarios.

Client perspective, campus networks

In the vast majority of networks I’ve seen in the last years going with SLAAC (with RDNSS) was/is considered fully sufficient. Recent examples of v6-only Wi-Fi networks like this one or this one) took this route, and I’ve also seen this in large conference networks (e.g. Cisco Live Europe 2019). I’m aware that back in 2019 Chris Werny and I recommended (in our talk on IPv6-only in Wi-Fi networks at Troopers) going with an approach of both SLAAC+RDNSS and stateless DHCPv6 in parallel, but I don’t think this is still needed in 2021, except for networks

• where support for clients without RDNSS abilities is needed (namely Windows before Win10)
• where IP parameters other than address(es), default gateway and DNS resolvers have to be provisioned by the network infrastructure (as opposed to be provisioned via other centralized mechanisms influencing the behavior of hosts like MS Active Directory/Group Policies or the like). VoIP telephones are an often-cited example of systems possibly needing DHCPv6, but I’ve yet to encounter such a deployment irl.

In campus networks going with SLAAC clearly wins with regard to the objectives ‘simplicity’ and ‘support’.

Client perspective, data center networks

Let’s first assume that we usually see a very different mix of operating systems in such networks (than in campus networks). Let’s further assume that from the operator side DHCPv6 might be a preferred option (pursuing the ‘simplicity’ objective and assuming that at least *some* systems in the DC might require DHCPv6 for netboot purposes). From my perspective there’s two aspects/objectives that deserve special attention here:

• Security: while probably most dual-stacked clients in campus networks run a DHCP client, this is not necessarily the case in DC environments. Adding a DHCP(v6) client on systems can increase their vulnerability exposure, e.g. see CVE 2018-1111 in RHEL discovered a while ago by Felix Wilhelm with whom I had the pleasure to work for a few years.
• Support: I have seen a few special-purpose appliances (both virtual and physical ones) commonly found in DC environments which only supported SLAAC, but not DHCPv6. Obviously ‘just installing dhclient’ might not be an option for such systems.

For the above reasons SLAAC might be a viable, or even required, option for hosts in data center networks, too, but this might collide with an organization’s system management & control approach. On the other hand SLAAC and particularly container networks have a lot in common (think existence of ephemeral entities and reliance on DNS).

Operator perspective, campus networks

In case the operator is responsible only for the network infrastructure, but not for the client environments (which is a common case in ISP scenarios), there’s usually a strong focus on ‘support’ (of as many different client platforms as possible) but not on any other of the above objectives. With regard to this one SLAAC is the better option.

A network operator can choose to provide both mechanisms at the same time, in order to support as many heterogenous clients as possible (which can lead to interesting results on the client side, e.g. see the Comcast approach described here. I still don’t understand the reasoning for doing SLAAC [with A-flag] and managed DHCPv6 in parallel).

In case the operator is also responsible for the hosts (e.g. enterprise networks), the picture is usually more nuanced. Here often traceability plays a major role. DHCPv6 is often considered helpful in this context, but it might not be the only option.

I’d hence say that going with SLAAC (only) provides advantages from the (enterprise) operator perspective as well, provided the traceability angle can be solved without DHCPv6. It might also facilitate things in the security context (focus on rogue RAs only instead of addressing DHCPv6 related threats as well).

Operator perspective, data center

As of 2021, UEFI netboot over IPv6 practically always requires DHCPv6 (option 59 et al.) so it’s a fair assumption that DHCPv6 can be found in many data centers.

As I discussed in this post on ‘IPv6 configuration approaches for servers’ there might furthermore be the approach to (ab)use DHCPv6 reservations as a mechanism to assign specific addresses to specific systems, based on a centralized database (or several ;-). In this case the infrastructure (DHCPv6 relay agents running on ToR switches, and the DHCPv6 servers themselves) often needs to support RFC 6939, which, as far as I can see, might still not be easy as of today, given the lack of support of RFC 6939 in network gear. Another potential obstacle: the (from the DHCPv6 process perspective) ‘client’ – which actually is a ‘server’ 😉 – could show up with different MAC addresses during different stages of the netboot process which can create additional challenges… I may discuss intricacies of this in another post ;-). Overall it seems that IPv6-based netboot is a real PITA in many environments, e.g. see this presentation on v6-only at the Imperial College London.

Still, for the purpose of our discussion we have to note that DHCPv6 may very well be a mandatory requirement in many DC environments. The operator then has to decide between pursuing the ‘simplicity’ objective (by going with DHCPv6 as the only possible approach) and prioritizing the ‘support’ objective (by allowing both mechanisms for hosts in the DC, which ofc increases the overall complexity of the environment).

Personally I think that supporting SLAAC in DC environments might be inevitable in the mid-term due to the considerations laid out above in the ‘Client perspective in DCs’ section. As an additional note I have to admit I’ve never been a big fan of DHCPv6 at all as I’ve seen too many interoperability and instability issues during my years performing projects in various large organizations (pre mid-2019), but things might have developed to the better in the interim. Happy (seriously!) to hear your success story of a stable large-scale deployment of DHCPv6, either in a campus network or in a data center.

Thanks for reading so far, and happy IPv6 networking in 2021 to you all!

Notes from the UK IPv6 Council Annual Meeting 2020

Today the UK IPv6 Council held their annual meeting. These have been great events for many years (e.g. see 2019, 2018, 2017). Many thanks to Veronika McKillop and Tim Chown for organizing it! In the following I’ll discuss some of the talks (full agenda here).

Colin Donohue & Ian Hallissy: The AIT Experience with IPv6 Only Wifi

Colin and Ian work at the Athlone Institute of Technology (AIT), a technical university in the Irish Midlands with 6000+ students. They started their IPv6 journey back in 2012, and decided to use a 2020 refresh of the Wifi infrastructure to switch from dual stack to IPv6 only incl. the management plane and the control plane. This would allow to easier identify IPv6 issues, to manage one protocol instead of two, and to provide a live IPv6 environment for research, testing and development. Here’s some relevant technical details of their approach:

They implemented the infrastructure with Aruba components (related vendor material with some technical details on the case study here ;-), and the NAT64 function is handled by Fortigate firewalls, with seemingly sufficient performance. With some small exceptions everything could be done in a v6-only way:

Ondřej Caletka from RIPE NCC asked how they (plan to) achieve their stated goal of traceability (e.g. in case of security incidents) given they only use SLAAC (yes, DHCPv6 might not be needed to achieve that objective). They responded that they expect (Aruba) ClearPass to handle this.
Another question was if they had a fallback plan in case some important application didn’t work properly without IPv4. Such a plan does not really exist – their stance was: “if urgently needed, we could add IPv4 on a VLAN basis, but we strive to avoid this”.

Overall a super-interesting presentation which clearly showed, like Jen Linkova’s recent talk at RIPE81, that v6-only Wifi (even for a heterogeneous user base) is doable in 2020.

Slides can be found here.

Pavel Odintsov: DDoS Challenges in IPv6 environment

Pavel is the author of FastNetMon. He started his talk laying out protocol-level DoS techniques on layer 3 and layer 4. He then discussed IPv6-related challenges from a DDoS protection perspective

and how he dealt with those when adding full IPv6-support to the latest FastNetMon community editions.

Slides can be found here.

Sam Defriez: Community Fibre IPv6 Update

This was one of the provider case study talks I really love at the event. Community Fibre is a London-focused (commercial) broadband ISP. The main objective of their IPv6 deployment was simply to avoid spending a fortune on IPv4, but instead use the money on “putting fibre in the ground”, as Sam phrased it. Seems reasonable and much more sustainable to me as well ;-). He discussed a few challenges during their IPv6 implementation, nicely illustrated by this chart:

They do DHCPv6 based on ISC’s Kea which had a few teething problems. Furthermore about 20% of their BNGs didn’t support DHCPv6 relay which was later fixed by Huawei, supposedly due to pressure from another, bigger ISP. Finally they suffered from the Cogent-Google (de-) peering issue (for those not aware of this see this discussion on the NANOG mailing list), which they solved by peering directly with Google.

Slides can be found here.

Erik Nygren (Akamai): CDN Provider’s view of IPv6 in 2020

Erik explained Akamai’s IPv6 efforts and developments since they first enabled it for HTTP traffic back in 2011. He then discussed in which spaces they see below-average IPv6 deployment: enterprise organizations (presumably due to backend systems & supporting processes not being IPv6-ready) and in the gaming industry (probably no incentives to add IPv6 once initial launch was v4-only).
He also mentioned a recent peak of 28 Tbps IPv6 traffic due to a specific event. (I wonder if that one happened in July? 🤔😉). Here are some interesting numbers he showed:

Asked if they see DDoS attacks happening over IPv6 he mentioned that they see a few, but they don’t consider this a relevant issue so far (not least as Prolexic, their DDoS protection service, supports IPv6 now).

Fernando Gont: Deeper dive into the recent Windows ICMPv6 vulnerability 

Fernando spoke about CVE-2020-16898, an RCE vulnerability in the Windows 10 IPv6 stack due to (mis-) handling of router advertisements (for the initial Microsoft advisory and some related discussion see here). The details of this vulnerability are quite interesting, and they were laid out in two independent posts, one from Francisco Falcon and the other one from Adam Zabrocki. In a nutshell the underlying problem is incorrect parsing of the (length of the) RDNSS option which in turns leads to improper buffer management. This can only be exploited once the malicious RA is sent heavily fragmented. Such a (fragmented) local ICMPv6 datagram shouldn’t be processed at all if the Windows IPv6 stack correctly implemented RFC 6980, which Fernando authored in 2013 (a related post about the latest Windows IPv6 stack here).
A very similar vulnerability was just recently identified by Francisco Falcon in FreeBSD (see here).

The slides can be found here.

---

Another great event from the UK IPv6 Council. Apparently 48 people joined, and there were some real good discussions among IPv6 practitioners. I expect the videos to be published soon, and I will update this post once that has happened.

Happy holidays to you all. Stay safe and healthy!

IPv6 Security Best Practices

That’s an ambitious title, from many regards.
Still, late 2020 might finally be time that we, as the IPv6 community, try to come up with a set of simple IPv6 security best practices to be used both as guidance and in a checklist manner.
One of the earliest of such efforts goes back to my friend Eric Vyncke, yet that one dates from 2007 (btw, Eric is also the maintainer of the useful ‘IPv6 Deployment Aggregated Status’ site). I had the pleasure to co-author an Internet-Draft on “Operational Security Considerations for IPv6 Networks”, but after years of discussions on the mailing list it has been stuck in ‘IETF review hell’ for a while (I paraphrase this term from Geoff Huston’s essay on “The Making of an RFC in today’s IETF“). Currently it doesn’t feel like that any of the authors incl. myself has any motivation or energy left to drive this further.
I’ll hence use this blog to formulate some ideas. As I have done IPv6 projects predominantly in large enterprise environments (before taking over my current role in mid-2019) there’ll be a certain focus on that space, though many of the thoughts should be applicable in various types of organizations. Also I’ll try to keep it simple, which isn’t always easy for me 😂.

Harden your servers

I recently wrote a piece about hardening of IPv6 stacks. For most systems the security benefits gained from it might not be worth the associated operational effort, but for systems in data centers hardening should be on your list of IPv6 security measures, especially when those systems are configured with static (global) addresses. Simply not offering certain interactions usually strengthens the security posture of systems or network devices.

Review packet filtering rules

In pretty much all environments packet filtering, either on the network level (via firewalls or ACLs on routers) or on the host level contributes to the overall security stance. In the course of your IPv6 deployment you might want to check those rule sets, as IPv6 brings technical changes which in turn require adapted rules, like

• Different functionalities of ICMPv6 (some additional info here, here, and here)
• IPv6 extension headers (see also next section)
• Different network behavior within subnets, that is – in IPv6 lingo – ‘on the
  local link’
• An altered addressing architecture (more on this here, here, or here)

An organization’s IPv6 transition might also provide a good opportunity to review rule sets from a broader perspective (“do we still need all those rules sitting in there since 15 years?”), but maybe you shouldn’t overload an IPv6 deployment effort with the ambition to cure deficiencies in your company’s security governance space at the same time ;-). Usually the former is complex enough.

Drop packets with extension header types 0, 43, 60 at network borders

There’s an active, and worth a read, Internet-Draft on “Operational Implications of IPv6 Packets with Extension Headers” (authored by Gert Doering, Fernando Gont and others). It includes this section:

This nicely summarizes the potential security issues of those headers, more information here or here. The main point, however, being that there’s no compelling (or other) use case for any production use of EH types 0,43, and 60 at all. If there are no practical benefits of a certain functionality, but many associated risks – why would one ever allow such stuff to enter one’s network?

Strongly consider dropping all inbound IPv6 fragments

Security problems related to IPv6 extension headers are often amplified when fragmentation comes into play (see here or here). There’s hope that the amount of fragmented IPv6 packets in the Internet right now significantly decreases due to the recent DNS flag day 2020. In recent years the majority of IPv6 fragments has been DNS traffic; nowadays one should only see very few TCP fragments, if at all, given TCP has the MSS to deal with situations requiring to split a payload, and at the same time the black hole detection of most OSs has become quite mature.
You might face some resistance when coming up with this measure. It could be an idea to perform an extra amount of testing incl. documentation, to closely observe counters of related ACLs for a while, and/or to come up with some training to identify MTU related connection issues during troubleshooting or via network telemetry. I also encourage all involved parties to read RFC 8900 “IP Fragmentation Considered Fragile”.

In shared L2 subnets perform risk analysis for RAs, NDP, MLD, and DHCPv6

All these packet types can cause security issues on the local link. Over the years there’s been some debate if the operational effort related to deploying associated controls on the switch port level (often called “First Hop Security” features) is worth the resulting risk reduction, in particular as from my experience several vendor approaches have shown serious teething problems. One may also keep in mind that at least in the past some of these features could be easily evaded.
There are RFCs on addressing rogue RAs (RFC 6105) and on protecting against unsolicited DHCPv6 packets (RFC 7610); see also next section. Eric Vyncke, Antonios Atlasis and myself once came up with the idea of an ‘MLD-guard’ feature but this didn’t take off, presumably because bad things doable with MLD are considered less severe than those related to the other packet types. In the projects mentioned above I primarily saw RA Guard and DHCPv6 Guard getting implemented (hence this post from 2015).

Understanding the risks associated with these functionalities & packet types, and if or how to protect against those, is part of your security-related homework of the IPv6 journey. I’ll discuss some of this in the following (yes, some overlap of the rules here ;-).

In shared L2 subnets with Ethernet drop RAs and server-side DHCPv6 messages on all access ports by default. Same for Wi-Fi on the controller or AP level

As stated at other occasions this is simply basic network hygiene. Many network operators integrate this into their ‘access port security templates’ (for physical switches, for virtual ones it’s a different story), and barring very specific conditions it’s not only safe but highly advisable to drop these packets. Looking at RFC 8415 section 7.3 ‘server-side DHCPv6 messages’ should encompass Advertise, Reply & Reconfigure messages. Common DHCPv6-Shield (that’s the ‘official’ name of the feature as of RFC 7610) implementations like Cisco’s DHCPv6 Guard probably filter exactly those.

In Wi-Fi networks such filtering can often be done on the controller or on the AP level, and evidently even fewer legit scenarios with such packets exist there than, say, in campus Ethernet networks. The related configuration approaches may be quite different though, e.g. see this talk by Christopher Werny from the Troopers 2016 IPv6 Security Summit, or this guide for Cisco 9800s. Last year I did some cursory testing on its effectiveness on a specific platform (results here).
Given the differences of commercial enterprise Wi-Fi solutions re: the exact configuration steps and their default settings, I recommend going with a combined approach of auditing (which, indeed, you periodically do anyway, right? 😉 ) and configuration steps (to happen in an automated manner during device deployment, maybe).

Think hard about a source of IPv6 truth for processes like asset management or vulnerability management

The importance of asset management and of a proper source of truth for it is, of course, also true for IPv4 environments. There’s a reason why the NIST Cybersecurity Framework starts with the Identify function…
On the other hand most of us know that the simple question “what do we have?” might not be simple to answer, even if executives or regulators may think so. However, in IPv4 networks one could always try to cover blind spots by simply scanning address ranges in a sequential manner, in order to identify ‘alive’ systems (which are then subjected to further enumeration steps). Surprisingly many organizations still do this today. Point is that such an approach is just not feasible in IPv6 subnets. Consequently one has to come up with different methods (such as ingesting data from neighbor tables, incorporating DNS AAAA records etc. – I’ll discuss some of these in another post), and I recommend to start thinking about this better sooner.

Take care that the vulnerability management framework is IPv6-ready

As an IPv6 practitioner one definitely wants to avoid being responsible for increasing the risk exposure of an environment, so taking care of the IPv6-readiness of an organizations’s vulnerability management process makes sense. This usually includes three main fields of activities:

• A source of truth which provides ‘alive’ IPv6 systems/addresses, see above
• IPv6 capabilities of the tools & techniques used for the vulnerability scanning itself
• IPv6 reachability of all to-be-scanned endpoints from the scanner networks

Reflect on network security telemetry and on threat detection in the age of IPv6

I’ve discussed ‘Organizational security implications for IPv6’ on the APNIC blog a few months ago. Furthermore I wrote down some initial thoughts on threat detection in IPv6 networks back in 2016 here, but overall I feel this needs more critical reflection at another occasion (read: stay tuned for an upcoming post ;-).

Suffice to say that, whatever type of smart machinery and/or human intelligence you may be using to look at large sets of security-related data, you’ll have to re-think and adapt some of that for IPv6.

Evaluate the implications of dual-stack and of v6-only + NAT64 on security processes

Some overlap here with the previous point, still this deserves a dedicated mention in the list. Both deployment strategies bring some changes/challenges when it comes to analyzing & namely correlating connection data and traffic flows. In dual-stack environments one endpoint might show up with different addresses, from different families and with differing ‘lifetimes’. In networks using translation techniques one and the same connection might have an IPv4 part and an IPv6 part (initially IPv6 and subsequently IPv4 in case of translation ‘close to the client’ = with NAT64, or initially IPv6 and then IPv4 in case of translation ‘close to the server’, e.g. on load balancers or reverse proxies). Either way all of these scenarios require to re-evaluate the idea of a ‘connection identity’.

---

Tl;dr: (not only) from a security perspective the advent of IPv6 requires technical implementation steps and process changes. Here’s a summary of the rules I discussed:

Some Notes on Hardening IPv6 Stacks

Initially I wanted to start a long-planned post on “IPv6 Security Best Practices”, with a particular focus on enterprise environments, but then I realized I might need some ramp-up clarifications beforehand.

First it might be helpful to understand that actual ‘IPv6 security measures’ in a given environment might heavily depend on the ‘deployment mode’. As mentioned in a recent Twitter thread I increasingly think that the specific, let’s call it: ‘use case’ of IPv6 in a given (sub-) network, plays a huge role for the way it’s configured and operated, incl. the security side of things. Look at these three examples:

IPv6 in the data center: this is often done with static addressing (even though other approaches might exist) which includes a high level of control (on the one hand the desire for this is the reason for the former and on the other hand it’s an operational prerequisite), security is frequently implemented by/contributed to by at least one layer of network-based traffic filtering etc.

IPv6 in a mostly Wi-Fi based campus network: here commonly SLAAC comes into play, RA Guard and filtering of server-side DHCPv6 packets are considered basic network hygiene (implemented either by means of active configuration steps or by default on the platform of choice, see this thread), in a given enterprise environment one can expect a certain homogeneity of the involved operating systems, and one can also expect some level of administrative control with regard to the (IPv6-related) settings on the hosts.

IPv6 in networks populated mostly by ‘IoT systems’ which in turn usually implies a low level of control over the individual systems and their settings as well as a high degree of heterogeneity when it comes to their IPv6 stacks, together with – maybe – different protection needs (re: traffic confidentiality or system availability) when compared to data center networks.

While the above are just examples, I think that distinguishing between different deployment models, and namely understanding that those exist, can play a significant role for the discussion of security controls applicable in a specific environment. This becomes clear when one looks at a potential security control which has been the subject of some debates for a while, that is the ‘hardening’ of IPv6 stacks. There are people who think that ‘IPv6 is primarily a network thing, and hence the network folks should take care of securing it’, while others think that (host-level) hardening is an essential part of a holistic infrastructure security framework. Also it’s a topic of common interest in IPv6 security circles – the ‘Hardening IPv6’ guides for different OSs which my friend Antonios Atlasis wrote in a joint project back in 2014 (you can find those documents here) have been downloaded a lot over the years.

Let’s have a quick look at typical ‘IP hardening’ measures, and where/how to implement those. Based on IPv6 projects I’ve done in very large enterprise networks between 2013 and mid-2019, I for one would say that host-level hardening of IPv6 by means of operational configuration measures (not to be confused with shipping hardened IPv6 stacks from a vendor, see this discussion on the APNIC blog) should only be done in data center networks, if at all. Reason mainly being that, from my experience, in other deployment scenarios the ratio of operational feasibility vs. the achieved security benefit/risk reduction usually isn’t very good. Put differently: one might invest a lot of energy, and expectations, into hardening measures without gaining much real-life security. Please note that I’m not stating this lightly as I’ve always been a big fan of hardening in my security career, and I still am. It’s just that in the IPv6 world hardening might be overrated (as IPv6 by design assumes a high level of autonomy for individual hosts, and deviating from this usually involves significant operational effort). Furthermore some measures which would actually be useful (such at just not processing IPv6 extension headers of the types 0,43,60 and/or dropping all IPv6 fragments by default on a given stack) are commonly not available, again due to IPv6 design decisions. As I laid out in my Troopers 16 IPv6 Security Summit talk on “Protecting Hosts in IPv6 Networks” and in the recent IPv6 stack level security post, potential measures can be broken down to the following categories:

• Removal of unused IPv6 features or helper protocols
• Static configuration of IPv6 parameters incl. disabling ‘dynamic functionalities’
• Tweaking of IPv6-parameters

In the following I’ll discuss some details on those, with sample parameters from the Linux world (documentation of those I mention for the current kernel here. also note that some parameters can be configured on various levels like ‘all’, ‘default’ or interface-specific, so multiple configuration steps might be needed).

Removal of unused IPv6 features or helper protocols

In case of a static (address) configuration approach, SLAAC can (and should 😉 be disabled, via configuring
net.ipv6.conf.default.autoconf

Getting rid of MLD might be an idea, too, but you should thoroughly understand its role and the implications (see this series of posts on MLD that we wrote some years ago). I’m not aware how to disable it by a sysctl on Linux or FreeBSD anyway.

Static configuration of IPv6 parameters incl. disabling ‘dynamic functionalities’

Common candidates in this space encompass disabling the processing of router advertisements and disabling the processing of ICMPv6 redirects, via touching
net.ipv6.conf.default.accept_ra
net.ipv6.conf.default.accept_redirects

I’d say that the 1st one is a common thing in data center environments, whereas the 2nd isn’t configured very often (see also results of this Twitter poll I did a while ago). Also keep in mind that the RA parameter might/should also be configured on ‘all’ plus on the interface level. Furthermore it’s noteworthy that Linux nowadays allows some granularity wrt the pieces of an RA that will be accepted (e.g. default route, PIO, MTU).

Tweaking of IPv6-parameters

There are two types of measures of interest here: those that control security-related specifications (e.g. RFC 6980) and those which enforce upper limits of IPv6-related data structures. From the 1st category on Linux there is only
net.ipv6.conf.all.suppress_frag_ndisc
which has a reasonable default value (‘1’ = ‘enabled’), and I’m not aware of any configuration knob on other OSs.

From the second category one might have a look at max thresholds for the number of (default) routes or addresses (which shouldn’t be needed anyway when SLAAC is disabled, see above), or at limits in the context of fragmentation/reassembly (provided fragments are allowed in[to] the environment in the first place, which might already be debatable from a security & network stability perspective …). Looking at the Linux kernel, the following parameters come to mind in this space:
net.ipv6.conf.all.max_addresses
net.ipv6.ip6frag_high_thresh & net.ipv6.ip6frag_time

Finally, from a security point of view, one might have a look at this one:
net.ipv6.icmp.echo_ignore_multicast

However, frankly, I don’t remember many environments configuring the settings in this last section at all. Still those might be worth a closer look, together with proper testing in case you consider touching them ;-). Most operating systems don’t even offer ways to do that.

tl:dr: the chosen IPv6 security strategy will depend on the deployment scenario. When hardening is part of this strategy, there’s usually only a limited number of parameters which are worth looking at, from an OpEx-to-security gains perspective.

Thanks for reading so far. As always I’m happy to receive feedback.

A Quick Security Evaluation of IPv6

In my last post on “IPv6 Security on the Stack Level” I mentioned that the journey to a secure IPv6 environment might start with a properly secured stack which then would have to be followed by operational processes & tools.
Accidentally I just re-read an old post on “Protocol Properties & Attack Vectors“, and I’d hence like to take the opportunity to discuss IPv6 from the perspective of the approach laid out in that post. While this will be an admittedly somewhat abstract exercise – regular readers know I have a soft spot for such reflections 😉 – it can help to develop an understanding of the types of operational controls one might need in a complex environment, in order to properly secure an IPv6 deployment effort.
The factors listed in that post were the following:

• Scope of interaction
• Integrated security capabilities
• Impact on packet forwarding
• Execution of an election process
• Amount of state
• Ratio of (numbers & sizes) of requests to responses

Let’s have a look at IPv6 from these angles. As stated in another slightly aged post on “IPv6 & Complexity“, talking about “the IPv6 protocol” here encompasses helper protocols to be found in most IPv6 networks like ICMPv6.

Scope of interaction

This boils down to “from where can a threat vector usually interact with $PROTOCOL” (with ‘threat vector’ including attackers or misconfigured other systems), which obviously can have a huge impact on the risk exposure of a system running it. For example security vulnerabilities of an LLDP implementation (like those discussed here) can only be exploited from an L2-adjacent node, whereas vulnerabilities in routing protocols often require the ability to send multicast packets (which in turn means an attackers has to be on a subnet shared with a router interface that has the protocol in question enabled).
Discussing this property from an IPv6 point of view is particularly interesting for (at least) three reasons:

• The by-default presence of the link-local address (LLA), together with joining several multicast groups, might allow for interesting interaction possibilities (once an attacker is in the same subnet).
• The latter two technical properties (LLA & multicast stuff) didn’t exist in IPv4 which once again makes clear that IPv6 is very different from IPv4 from a security perspective. Let me just emphasize: thinking that “IPv6 security is like IPv4 security, just with longer addresses” might lead to the implementation of inappropriate controls.
• Some IPv6 helper functionalities are supposed to interact mostly by means of multicast which in turn is supposed to be limited by L3 boundaries, and some implement “Generalized TTL Security Mechanism“-style checks in order to make sure only link-local sourced packets are processed (e.g. see the sections on “Message Validation” in RFC 4861). One may note I used “supposed” several times here 😉

Let’s assume one operates an IPv6 router and one wants to allow SSH access for a certain network segment (only), to a specific interface (represented by a specific address). A related, as it is/was called in Cisco space, ‘Infrastructure ACL‘ could roughly look like this:

permit tcp 2001:db8:10::/64 host 2001:db8::1 eq 22    # allow SSH access from management network 2001:db8:10::/64 
deny tcp any host 2001:db8::1 eq 22         # deny SSH from anywhere else
permit ipv6 any any # allow all transit traffic

Such an ACL would still leave SSH access open via the LLA of the specific interface, so for other systems in the same subnet. This might pose a relevant risk, or not. Point is that the mere existence of multiple addresses with different reachability properties might require adapted packet filtering rules for a number of systems. Also, generally speaking, moving packet filtering closer to the nodes (instead of doing it on the network level) might make sense in IPv6 times, but I’ll discuss the practical implications of the intellectual exercise performed in this post in yet another, upcoming, one 😉

In the context of recently published CVE-2020-16898 (for some background see these two threads and Francisco Falcon‘s great “Beware the Bad Neighbor” write-up) there was some discussion if one could attack a system from a non-local origin. Fernando Gont mentioned on the ipv6hackers mailing list (btw, here some more IPv6-related mailing lists) that he had performed some testing and it seems “*BSD, Linux, MS Windows” do properly perform implement the relevant checks (“GTSM”, see above. essentially checking if the received packet [still] has a hop-limit of 255). If that’s true let’s hope that Windows does that check before it parses (the length of) the RDNSS option…
In any case it should be noted that a while ago major router vendors were affected by a vulnerability that allowed sending unicast NDP/RA packets over the Internet to their devices, which in turn led to DoS conditions given the high priority queues/pipelines those packets are/were processed in. Chris Werny demoed this for Cisco devices in this post, and here’s the related Juniper advisory.

Integrated security capabilities

This area of evaluation is about the security capabilities which the protocol brings with it. More importantly it’s about their operational feasibility, read: (provided there are any such capabilities) can those be used in a way that provides a reasonable operational-effort-to-risk-reduction ratio?

Again, looking at IPv6 provides an interesting picture here. Following some naïve 90s “crypto will solve all network sec problems” optimism, there were hopes that ‘integrated IPsec’ would solve the looming security issues of the Internet. As we know, this hasn’t really happened yet (but backported-to-IPv4 IPsec became a success story in itself).

Furthermore there’s ‘SEcure Neighbor Discovery (SEND)’ as of RFC 3971, but let me be blunt here: the idea of enhancing the security of a by-itself highly complex network infrastructure protocol by certificate-based mechanisms is somewhat ridiculous from an operations perspective. For that good reason SeND has never gained any ground, and I don’t expect that to change anytime soon. Actually I expect SeND to be forgotten at some point ;-).

Alas those hopes (on IPsec) meant that existing authentication capabilities were removed (or replaced by practically unusable IPsec-based variants) from higher layer infrastructure protocols such as VRRPv3 or OSPFv3. For the latter this was later ‘corrected’ by re-introducing an authentication header in RFC 7166 (RFC 6506), which in turn reflected operational reality and needs from pre-IPv6 times.

Overall the in-practice absence of available built-in security mechanisms has produced two things of interest for an IPv6 security practitioner:

• the inability to establish (crypto-based) trust relationships for core helper protocols led to the development of compensating controls on the network level. The most prominent example is RA Guard which replaces protocol-inherent authentication by a ‘topology-based’ approach. Think: “if you can’t use a watchword to authenticate a messenger of a sensitive piece of information, then at least restrict the door through which the messenger can enter the room”.
• several later RFCs (such as RFC 6980 or RFC 7112) try to address IPv6 security issues (stemming from overly liberal design decisions) by inducing additional checks. While on the 1st glance this may seem a good idea, it should be kept in mind that additional checks usually increase the code base of an IPv6 stack. Not everybody would agree that more lines of code necessarily lead to more security…

Impact on packet forwarding

Let’s paraphrase this one as: “by means of an attack against $PROTOCOL, can we alter the future path taken by IP packets?” (which could either lead to Denial-of-Service scenarios or eavesdropping/Man-in-the-Middle attacks against unencrypted higher layer protocols). There’s a reason for the popularity of dsniff and Ettercap in the good old IPv4 days. IPv4’s helper protocol ARP was an ideal candidate for attacks: no integrated security capabilities while at the same time having an impact on the (local) destination of IP packets. Also the majority of existing attacks against DNS has the eventual objective to alter the path that IP packets take, based on falsified name-to-address mappings.

So what about IPv6 in this realm? Unfortunately (from the perspective in focus here) IPv6 introduced a very powerful function to provision default routes (and DNS servers) to hosts within a subnet, that is router advertisements. Keeping the absence of feasible security capabilities for NDP in mind (see previous section) we end up with a unfortunate combination of:

• a mighty control mechanism.
• which does not bring any authentication or integrity control properties.
• is expected to run by default on all end notes and which might even be needed for their full IPv6 life.

Furthermore pretty much every other main IPv6 helper protocol (NDP, DHCPv6, MLD) can be attacked in a way that allows to modify the future path of (some) IPv6 packets on the local link, which means all of those have to be carefully considered in a shared L2-environment, such as campus networks. A silver line being here that quite some enterprise Wi-Fi offerings nowadays come with built-in capacities to filter this stuff.

Execution of an election process

This again is a quite interesting one. Some of you might recall those days when Yersinia was the main tool for the network part of security assessments, and one of main reasons for this was the simple fact that quite a few infrastructure protocols (e.g. STP or HSRP) performed elections (of active forwarding devices) without many security precautions. Similarly in this talk from Black Hat Europe 2010 we discussed how to win a specific election in Cisco-based Wi-Fi networks (using the WLCCP protocol).

Looking at IPv6 it becomes clear there are many election procedures. Hosts choose between routers to act as their default gateway based on different router preferences as of RFC 4191 (which is the reason why all attack tools ofc send their RAs with router pref=high). The MLD querier role can change when a “query [is] received from a router with a lower IP address” (literal quote from RFC 2710 ;-), and some helpers select information on a ‘last received packet’ basis when facing multiple alternatives.

Amount of state

Here let’s look at ‘state’ as the amount of memory needed on hosts or network devices  in order to perform the expected functions of $PROTOCOL, or to perform related security functions (e.g. packet filtering or NIPS). The more state, the higher the susceptibility to resource starvation, either from an attack (DoS) or just from a misconfiguration (human error).

IPv6 has well-known issues in this regard (such as neighbor cache exhaustion), and certain helper packets (e.g. router advertisements or some related to MLDv2) are super-complex by themselves (leading to respective parsing efforts + resources for storing associated data structures).

Evidently the biggest issue in this space is related to IPv6 extension headers and fragmentation. Besides other security problems they may cause (see this paper) both stacks of local systems and network or security devices can have a hard time when it comes to packet reassembly and subsequent security decisions. RFC 8200 ‘allows’ up to one minute (“within 60 seconds”) between the 1st fragment of a packet and others needed for reassembly – how would any security device trying to look at a full (reassembled) packet in 2020 networks handle this?? Limiting the resources spent on this, and with this the depth of inspection, seems the only reasonable strategy here. From an operations perspective – and regular readers won’t be surprised at this point 😉 – just dropping all packets with EHs like HbH, RH, DestOptions or fragmentation might also be a good idea for some networks (not least as fragmentation can be expected to decrease over time, given latest developments like the DNS flag day. I have another post on that subject in the works).

Ratio of (numbers & sizes) of requests to responses

This is the aspect that is relevant for amplification attacks. If an attacker, either from a spoofed IP address or in the context of multicast, can trigger a lot of response packets (‘ideally’ of a large size), then DoS situations can be created. Protocols being unauthenticated and/or mostly stateless (running over UDP might play a role here) can influence these abilities, too.
In the IPv6 world several such scenarios come to mind, like:

• a single router solicitation (RS) can trigger an RA where “the usual case is to multicast the response to the all-nodes group” (RFC 4861, section 6.2.6), which in turn can be particularly annoying in Wi-Fi networks (which is why it’s a good idea to send RAs via unicast in such networks, see this talk from Troopers #TR16).
• triggering MLD report messages (as shown in this talk) from many hosts can create significant packet load on the local subnet.

---

In this post I discussed properties of IPv6 (incl. helper protocols) which can have an impact on the overall security and stability of environments using it. While this was meant to be a somewhat intellectual exercise, I will use this as a basis for a discussion of common IPv6 security measures in an upcoming post. Stay tuned & thank you for reading so far ;-).

IPv6 Security on the Stack Level

A few weeks ago Scott Hogg, one of the smartest persons around in the field of IPv6 security, published a post titled “7 points your security team needs to know about IPv6 (but probably doesn’t)“. He listed ‘IPv6 needs to be secured from the onset, not retroactively’ as one of those. Given my own security education started in the 90s where one (still) considered ‘hardening’ as an integral part of an overall security landscape ;-), this made me reflect on the potential properties or attributes of an IPv6 stack itself which one would label as ‘a secure one’. I mean it all starts with the stack, right? – so “from the onset” could mean exactly that: begin an IPv6 journey with sufficiently secure implementations, then take care of (security-wise) proper operational handling.
At that point the question ‘just’ is: what does a ‘secure’ IPv6 stack look like/incorporate? This is what I want to discuss in a informal manner in this post.

Code base & code quality

As the recent Ripple20 vulnerabilities have shown, the vulnerability surface of a specific TCP/IP stack can play a huge role for the security of devices using the respective stack. Subsequently it can make sense to look at the origins and the code base of an implementation, e.g. on the one hand the FreeBSD IPv6 stack (stemming from the KAME project) is usually considered a stable and secure one, but on the other hand this excellent technical report from 2018 (supervised by my friend Sergey Bratus) reviewed a number of vulnerabilities it had in earlier years.

Privacy properties of address generation approach

The privacy implications of the initial EUI-64 based SLAAC addresses have been discussed extensively in the past (btw there are some tidbits on the political dimensions of the choice of EUI-64 in Brian Carpenter’s book “Network Geeks. How They Built the Internet“). There was an article “Where’s All The Outrage About The IPv6 Privacy Threat?” already in 1999 (I could only find a version of it buried in this page, in the SixXS archives which are a nice resource anyway if you have an interest in the history of IPv6) and the first RFC on Privacy Extensions was published in 2001. Also this is a talk which Chris Werny and myself gave on the topic at the Heise IPv6 Kongress 2012.
In any case from today’s perspective it’s usually considered to be beneficial when a SLAAC generated address fulfills mainly two requirements:

• sufficiently privacy-preserving in the sense that an IID can not be tracked across different networks.
• some amount of ‘stability’ (read: the address does not change [very often]) as long as the respective interface stays in/with the same subnet/prefix.

Several address generation mechanisms which serve these objectives exist, namely the one described by Fernando Gont in RFC 7217 (but also Cryptographically Generated Addresses [CGAs] as of RFC 3972 can fulfill the above objectives).

Interactions

By definition any implementation of the Internet Protocol can be expected to process input (read: packets) from many untrusted/untrustworthy communication partners 😉.
Still (at least) two questions come to mind here:

• how does an implementation handle input/packets which it has to process as part of its core functionality (like parsing the IP header, or processing NS/NA packets sent from other hosts on the local link)?
• which additional interactions (=> vulnerability surfaces) does an implementation offer?

The first question relates to the (secure) parsing capabilities of a stack (see the above section on code quality) and potentially also to limits of resources which can be consumed by an untrusted party sending legitimate packets (see the below section on limits on data structures). Let’s hence focus on the second question, and let me paraphrase it a bit: which additional IPv6 ‘subprotocols’ does an implementation bring and, more importantly, which of those are enabled/started by default? There’s three of those which deserve a bit of attention: MLD, DHCPv6 and mDNS.

I for one still think that the current practice of IPv6 stacks coming with MLD enabled by default goes back to a misunderstanding of section 7.2.1 of RFC 4861 where ambiguous language is used (‘is done’ can be prescriptive [‘normative’ as of IETF lingo] or descriptive), and that the vast majority of IPv6 stacks wouldn’t need MLD to function properly. From a security perspective MLD can be abused in many ways (see the ‘MLD Considered Harmful‘ presentation from the Troopers IPv6 Security Summit 2015), but one has to accept that pretty much all implementations come with MLD enabled (Antonios Atlasis mentioned OpenBSD as an exception in this post from 2014), not least as the MLD-related of the IPv6 Node Requirements RFC seems to imply that any use of multicast addresses (including link-local uses such as ff02::1 or the Solicited-Node Address as of RFC 4291 which is employed in the context of neighbor discovery) requires MLD. While I’d like to point out that generally MLD doesn’t make much sense for multicast communication on the local link (which is btw why quite a few implementations of MLD snooping explicitly do not process LL multicast groups), but the ‘let’s consider the usefulness of MLD’ ship has long sailed.

DHCPv6 could be another candidate for the ‘do we really need this at all/enabled by default?’ category as several DHCP (client) implementations have been vulnerable in the past (e.g. see CVE-2019-0547 or the very nice CVE 2018-1111 discovered by my former colleague Felix Wilhelm), but then again pretty much all current IPv6 implementations come with an enabled-by-default client-side DHCPv6 process.

The same applies to mDNS: even Microsoft IPv6 implementations have it since a while, (see this post) and RFC 8504 encourages its use in section 9 of the document. Wearing a security hat one might still be tempted to ask: “do we need this by default?”, especially when one has read the “An Attack-in-Depth Analysis of multicast DNS and DNS Service Discovery“ paper by Antonios).

Security properties of the stack

There’s a number of RFCs describing – I’m hesitant to write ‘mandating’ 😉 – security-oriented ways of handling IPv6 packets with specific properties, incl.

• RFC 6980 Security Implications of IPv6 Fragmentation with IPv6 Neighbor Discovery
• RFC 7112 Implications of Oversized IPv6 Header Chains (which subsequently found its way into RFC 8200)
• RFC 8021 Generation of IPv6 Atomic Fragments Considered Harmful (also incorporated in RFC 8200)

With regard to our initial question one should clearly take those into account when it comes to evaluating an IPv6 stack’s overall security, but it can be quite difficult to find out for an individual stack (extensive testing as described here might be a good road…).

Limits on data structures

As laid out above, an IPv6 stack has to digest all types of input from untrusted sources. This can include quite complex packets (like router advertisements) which in turn can lead to memory- and/or computation-intensive processes on the local system. A properly secure(d) IPv6 stack should hence impose limits on certain data structures. Some of you might recall the “IPv6 readiness update” initially released by Microsoft in 2012, presumably as a response to RA flooding performed at the time with Marc Heuse‘s THC-IPV6 attack toolkit. In general data structures subject to similar restrictions can include:

• Neighbor cache entries, namely ‘incomplete’ ones (for some discussion see this post)
• IPv6 default routes
• SLAAC generated addresses based on the prefix information option (PIO) in RAs (also called ‘honored PIOs’).

(Additional) Security features

Following a wide-spread industry approach of bolting on security (features) to things-to-be-secured one might be tempted to cure inherent security weaknesses of IPv6 (such as the lack of security properties of router advertisements) by bringing in additional elements. Bonus points in this space are usually awarded for complex cryptography based stuff, like for example in the IPv6 realm SEcure Neighbor Discovery (SEND).

Still regular readers of my posts probably know that I do not necessarily share the opinion that additional lines of code always increase the security posture of a component 😉 (here: an IPv6 stack) , so I’m not sure if ‘supplementary security features’ make it to my list of a ‘secure IPv6 stack’. Happy to take suggestions from you, dear readers, here…

Assurance

Finally the resources (time, skills, diversity etc.) spent on security assessments of the respective stack might obviously constitute a (main?) contributing factor to the overall security of an IPv6 stack. Evaluating the security of an individual implementation (c|sh)ould hence include finding out about those. For example from that angle one could look at the aforementioned IPv6 stack of BSD-derived systems and draw some conclusions (e.g. CVE-2019-5597 initially found in OpenBSD also affected FreeBSD and apparently even Solaris whereas CVE-2019-5608 doesn’t seem to affect other stacks from the BSD family, but ‘only’ components directly using FreeBSD, see for example this advisory. Also there has been some speculation if CVE-2020-1603 in Junos OS is related to the latter, or to CVE-2019-5611). In any case some research on security assessments of an individual IPv6 stack should be part of a ‘secure stack’ evaluation.

I hope the above provided some food for thought as for IPv6 security from the point of view of IPv6 stacks – I’m happy to receive any type of feedback, on any channel.
Thank you for reading so far! 😉

IPv6 Security & Capability Testing, Part 2

In this post I want to give an overview of IPv6 security testing efforts performed by various researchers in the past. As laid out in the first part of this series such testing can be considered one of the pillars of an overall IPv6 security strategy in complex environments. I will also try to develop a bit of a classification approach as for the ‘questions’ that the individual testing efforts were supposed to answer.

One of the earliest efforts was undoubtedly Marc Heuse‘s PacSec 2005 talk on “Attacking the IPv6 Protocol Suite” in which he discussed the now famous THC IPv6 toolkit for the 1st time in public. That talk was mainly focused on general IPv6 properties and how to exploit them. Another milestone of IPv6 security research (albeit quite a few years later than Marc’s talk) was Antonios Atlasis‘ Black Hat EU 2012 talk on “Attacking IPv6 Implementation Using Fragmentation“. I saw that one live as I presented at the same event myself, and I remember being impressed by the huge security impact that seemingly small implementation nuances can have (e.g. CVE-2012-4444 in the Linux kernel was one outcome of that research). I also realized that IPv6 fragmentation can be a really bad thing from a security perspective. That work was heavily focused on the ‘how do different implementations behave [when confronted with specific packets], and why does it matter’ angle. Highlighting that differences between IPv6 stacks exist, and that these might be relevant for an implementation’s security posture, was an important contribution/message in itself.
Antonios then expanded his research into looking at the way how various firewalls handled IPv6 fragmentation and IPv6 extension headers (2013, together with Christopher “Chris” Werny), and how IDPS systems could be evaded by means of extension headers (2014. see also the “Hiding in Complexity” talk by Marc Heuse in 2015). Similar testing was done by Marc & Fernando Gont (“Security Assessments of IPv6 Networks and Firewalls“, 2013) and by Johannes Weber in his master thesis (“IPv6 Security Test Laboratory”, 2013). All these research efforts explicitly looked at both IPv6 implementation differences on security components and at their resilience when being exposed to individual (and potentially ‘peculiar’) IPv6 packets. At the time another group of researchers performed various related activities at the University of Potsdam (see also their publications); a few results of their work were presented at the Troopers IPv6 Security Summit 2014.

Some of the above papers included (more or less detailed) descriptions of test cases, but the most comprehensive taxonomy of tests to be performed with regard to the secure implementation of IPv6 on operating systems or network devices could be found in the “Testing the security of IPv6 implementations” paper published in March 2014 by a group of (mostly) Dutch researchers (incl. Cristofaro Mune and Albert Spruyt):

Since then not much has happened in the space of IPv6 security research centered around implementation differences of OSs or network devices (I myself contributed to lab testing of various OSs in the context of SLAAC/DHCPv6, IPv6 source address selection or RFC 6980 support, although besides the latter none of these had a specific security perspective). A notable exception is a Ph.D. thesis on “Measuring IPv6 Resilience and Security” published in 2019, but they mostly looked at the real-life distribution of specific patterns and IPv6 security issues on an Internet scale (as opposed to performing tests in one individual environment and/or in a lab). While this is super-interesting research it is supposed to answer a different (set of) research question(s). Also there is an interesting paper/thesis on “IPv6 Security Issues in Linux and FreeBSD Kernels: A 20-Year Retrospective” from 2018 (authored by Jack Cardwell, with my old friend Sergey Bratus as advisor).
On the other hand I can confirm from IPv6-focused security assessments in customer environments which Chris and I led between 2016 and early 2019 that significant implementation differences still exist(ed) in more recent times. Maybe it’s hence time to revive such testing efforts, now that IPv6 deployment gains ever more traction in enterprise environments? In another post I will discuss how a framework for testing IPv6 implementations from different (not only security-centric) perspectives could look like.

Stay tuned, and thank you for reading so far ;-).

IPv6 Security & Capability Testing, Part 1

Let’s be realistic about this: the advent of any new technology in a complex IT environment can lead to additional vulnerability exposure and hence to additional risk. Given the fundamental role of IP for practically all communication acts and all digital services in today’s organizations this is particularly true for IPv6.

On the other hand as an IPv6 practitioner one definitely wants to avoid being responsible for significantly increasing the risk exposure of an environment, not least as it’s already hard in itself to drive the deployment of a protocol which shines by a lot of added – and, maybe, from today’s 20/20 perspective: unnecessary – complexity while at the same time, rightfully or not, being perceived as bringing just one single advantage, that is larger address space.

This is why I want to discuss in a small series of posts some thoughts how to deal with this situation. What are the things to keep in mind or to focus on, when IPv6 is gaining traction in any sufficiently large, complex and heterogeneous environment?

Said additional exposure can stem from several factors:

• Lack of operational experience (we as the IPv6 community can try to contribute to curing this by training & education, but of course there will be gaps for some time/years). These two papers (both from 2016) lay out some differences between IPv4 and IPv6 from an Internet scanning perspective, and quite some of those observed differences could be attributed to disparities in operational practices.
• Lack of available security mechanisms & processes. I’m not the biggest fan of the ‘let’s just transfer our security architecture & technologies from the IPv4 world to IPv6’ approach, but it’s a commonly found and – considering the often significant efforts of deploying IPv6 – fully legitimate approach (I mean we as IPv6 practitioners shouldn’t expect our valued peers from the security groups to heavily re-engineer their stuff just because of our new thing, and I’m fully serious here). On the other hand it might just be that such a ‘simply transfer’ strategy can’t be pursued due to hardware limitations (TCAM memory comes to mind) or due to performance penalties arising from IPv6. While from 2013, this diagram (from a presentation at the ‘IPv6 Hackers’ meeting at IETF 87) might give an idea of the latter:
  
• Specifics of the new technology in question. In case of IPv6 this might encompass factors like new protocol features with possible security impact (router advertisements or MLD come to mind), the generally huge variety when it comes to IPv6 stacks & their properties, and finally the potential immaturity of those stacks. I mean in 2020 we shouldn’t see things like this or this anymore, right?
  

This last aspect is the reason why I think that the following three elements should be part of the security side of each IPv6 deployment in a complex environment:

• Make sure that any IPv6-enabled system is properly protected by (at least) some packet filtering mechanism (be it an ACL on the network layer or a host-based component) from the very beginning. This is particularly important as you should always assume global reachability due to IPv6 GUAs and due to the potential lack of control your security groups might have over route announcements and route filtering. How to audit this (existence of packet filter on some layer for all systems) can be an interesting challenge, and I might get back to this at a later point/post.
• Make sure that all IPv6-enabled systems are covered by your organization’s vulnerability scanning process in case such a thing exists. Given one can not reasonably scan IPv6 subnets sequentially this might actually require some changes to said process.
• Create transparency with regard to the capabilities & properties of individual IPv6 stacks to be found in the organization, e.g. when it comes to RFC 6980 support or to their susceptibility as for malformed IPv6 packets.

The actual vulnerability exposure and the associated risk level of your IPv6 deployment will highly depend on these three factors. In the next post I will particularly focus on the third of those, that is on ideas and approaches how to create such transparency.

Stay tuned, and thanks for reading so far.

Does One Need DHCP(v6)?

I’ve covered DHCPv6 quite a bit in the past, e.g. in these two ‘A Deep Dive into DHCPv6’ blogposts, this one on RFC 6939, a few other contributions discussing it in some way (1, 2, 3) and a talk on ‘Secure & Reliable DHCPv6’ at the Troopers #TR15 IPv6 Security Summit (slides & video).

I also had a number of Twitter exchanges on DHCPv6, and one of the communication partners recently reached out to me with an interesting question. That person asked: “we consider avoiding DHCPv6 in our campus networks, but the security team insists on using it to be able to properly identify systems which act in a malicious way. what’s your opinion?” – In this post I’ll try to give a response.

Before I start with a technical discussion myself let me further add that several IPv6 deployment case studies mention all types of issues with DHCPv6, like this one from CERN, this one from Microsoft and this one from Deutsche Telekom. You might think about this for a second… (if this wasn’t clear, let me rephrase: in general DHCPv6 might be a piece of crap whose use should be carefully considered from a production [-readiness] perspective).

In ancient IPv4 times, in enterprise networks usually DHCP served one or several of the following functions:

• 1a) provisioning of dynamic addresses (in a more or less centralized manner).

• 1b) provisioning of other IP-related parameters/properties needed by clients (default gateway, DNS resolvers, NTP servers, others e.g. place to find firmware/boot image etc.)

• 2) simple access control (like ‘only MAC addresses registered in CMDB get an IP address from DHCP’, which is supposed to prevent unauthorized network access. I’ve once seen this in a large automotive ICS environment).

• 3) contribute to DFIR activities by providing mapping of MAC addresses to IP addresses found in log files, PCAPs etc.

Now let’s look at those from an IPv6 perspective.

• 1) For address provisioning DHCPv6 is no longer needed (hosts can do that on their own, via SLAAC), for the default gateway it can’t be used, and the vast majority of systems can get their DNS resolvers via RAs (option 25/RDNSS as of RFC 8106; see also Chris‘ post when Windows 10 eventually had it). That leaves distributing NTP servers which in turn some operating systems take into their own hands anyway (e.g. Windows).

• 2) That ‘simple access control’ approach could be considered a curiosity for specific environments, and some of you might be reminded of those times when a printer’s test page (containing a MAC address) was sufficient to get access to enterprise networks running complex & expensive ‘NAC’ solutions…

That leaves us with the ‘we need DHCP(v6) for DFIR contributions’ thing, which was at the core of the initial question. To evaluate (the need for) this we should keep in mind that during a host’s lifecycle on a network a lot of properties can potentially be observed at various points. These include

• the MAC address(es)
• IPv6 address(es)
• IPv4 address(es)
• FQDN
• mDNS-related information or similar (NetBIOS names)
• username, e.g. in authentication context
• X.509 certs (machine-/user-based) and namely the wealth of data they contain
• RADIUS-related information

The correlation of the first two ones (MAC address to IPv6 addresses) or, maybe even better, to a username might hence be easily inferable from other information which is observable/can-be-logged at some point of a host’s lifecycle, e.g. X.509 cert based info in the context of Wireless EAP protocol (EAP-TLS or PEAP in most enterprise networks).
Read: you might not need DHCPv6 logs for that but just have a look at other sources available anyway (Bryan ‘The Packet Master’ Fite once coined the term ‘latent capabilities’ for this approach). In the following I’ll provide some examples.

Probably the best known of such capabilities is the ‘ipv6 neighbor binding logging’ command from Cisco’s IP Snooping framework which is available on most of their (physical) enterprise level switches (please note that I’ve never been a big fan of most features of that tool set, due to it’s complexity and immaturity in [test] deployments I’ve seen, but that logging capability is definitely helpful).

Another example would be the ‘NDP Monitoring‘ feature on Palo Alto devices which I learned about from Johannes Weber:

Finally here’s a very nice example which my friend Eric Vyncke showed in his session on IPv6 security at last year’s Cisco Live Europe:

tl;dr: instead of using DHCPv6 (bringing its own complexities and troubles) to solve that security requirement it might be a much better idea to analyze which data bits one might have available in an environment anyway and how to correlate them.
Assuming that some of you readers are still in the early stages of the IPv6 deployment in your organization you might even be able to file respective feature requests (e.g. ‘logging of IPv6 address(es) and MAC address of a client directly after its association in a wireless network’) to the vendor of your network gear.