Sep 10 2018

Are BGPs security features working yet?

This post is a textual version of a talk I gave at NLNOG 2018, You can watch the talk below if that’s your preferred medium:

BGP has had a problem for quite a while, most of the time when we hear about this in the news outside of the networking word it is referred to as a “BGP Hijack”. Which can be better phrased as “someone routed someone else’s addresses to them”.

The reason that this is a thing is because BGP wasn’t exactly built with the idea of verifying ownership of IP address blocks in mind, so the BGP software doesn’t actually know if a route to some IP addresses is legitimate or not.

There are currently two suggestions to fix this, BGPSec and RPKI, both of them involve cryptography in one way or another. The current state of adoption means that BGPSec while defined, is science fiction in the real world. The alternative RPKI spec though is seeing deployment!

RPKI works somewhat in the same way that TLS does. There are a set of root certificate authorities and downstream people start signing what their IP ranges should be announced as, and who they should be announced by.

Currently speaking, the deployment of RPKI signatures on IP addresses is getting pretty good:

Or if we look at the RIPE region alone:

We see a significantly higher adoption.

However this is not the whole story, It’s great if you are signing your IP ranges (also known as prefixes), but the process of RPKI involves networks also validating the routes that they import from their peers and vendors.

RPKI Validation is a little messy, to start with you likely need to run another device on your network that your router talks to check if prefixes should be accepted or not. This is not helped by the fact that software to do this is not fully mature and documented yet and routers that have not been maintained for a while will very likely need upgrading (and for some vendors that upgrade carries significant risk of chaos)

But let’s recap on what RPKI is supposed to be preventing

This is a “standard” setup, here we have a ISP with a single upstream transit peer, and a Internet eXchange. Here there is a single path to a prefix (illustrated with a book)

The problem comes when someone evil on the IX announces the prefix, Because either the routing path is shorter, or that the router has been configured to be bias to the IX (because it’s cheaper). This is bad because the router has just followed a hijacked prefix because it knew no better, It had no way to know it was hijacked!

This is where RPKI comes in, because the prefix is now locked down to the netmask + AS number, the hijack over the IX will not validate and the router will prefer the correct route over the bad one.

Observant readers may suddenly realise a hole in this design, since bypassing this would only require writing the path on the attackers side to make the hijack prefix keep the original AS number, and just have the hijackers ASN as a upstream.

This does work, however if a router is still operating on “shortest BGP path = best path” this reduces the chance that the path will be accepted as the best one.

Let’s look at another situation:

Here we have another very common setup, in this case the ISP does not have a internet exchange connection, it just has two upstream ISPs.

Let’s say that the attacker has managed to have their hijack reach transit providers, this is quite common as there are still many providers who either have bad filters, or non existent filters on what prefixes their customers should be able to announce.

With RPKI filtering, both incoming BGP feeds should be considered invalid for that prefix, and the router should have nowhere to route the packets back. While packets can still come in from that hijack, almost all systems need duplex communication to function correctly (TCP being the critical one)

However here comes the next issue in practise, In a lot of cases providers will send a default route to their customers, this provides traffic a route even if the provider has not declared it could route traffic there. It can sometimes be useful and valid in cases.

Because it provides a “last ditch” method of getting packets to a place. It means that these hijacks still work, because even though the hijack route is rejected it will still take a default route to get back there.

Fixing this should be easy right? Just remove the default route or blackhole traffic in the case of there being no valid route to get there. Well here comes the next roadblock to overcome.

The business case for rejecting what could be 0.87% of client traffic is a hard one to argue. In a ideal world this 0.87% of IP address prefixes should be doomed anyway. However giving that RPKI deployment is only just starting to happen, these prefixes for the most part can still access most of the internet.

So. With all of this in mind, how does the internet currently do on deployment of RPKI other than just signing validation, giving that for RPKI to work you need to both sign and validate.

For this, we are going need to send out a lot of packets…

Armed with prefixes that are purposefully signed to be invalid, and a single valid prefix. I sent out many ICMP Pings to all IP addresses using the “green” control prefix.

Should a response come back in, pings are sent out on each of the other invalid “red” prefixes to test if a response makes it back

This leaves us with buckets of hosts! Matching hosts inside these buckets we can tell if their ISPs are doing RPKI validation.

If they appear only in the green bucket, then they are doing RPKI validation correctly! If they appear in the green bucket, and the red ARIN bucket, then they are employing some level of RPKI validation, however have suffered from some default configurations in software (more on that in a moment). If they are in all buckets, then they are not doing any RPKI validation at all.

However let’s look at the final counts:

The lower the number, the more enforcement there is. As you can see there is a 0.1 million difference between ARIN and RIPE. This is likely because of the defaults of all RPKI validation software. See, the ARIN TAL requires you to download it separately and with that it carries a Relaying Party Agreement that is 3 pages of legal speak that has some alarming statements in there. This fact means that by default, the ARIN TAL (a critical part in verifying a ARIN RPKI validity) is not included by default in software.

With that in mind, I can present to you the currently leading AS numbers in RPKI deployment:

Or presented nicely:

57598   | MD | ripencc  | SHA-AS, MD
15426   | NL | ripencc  | XENOSITE Amsterdam, NL
34968   | NL | ripencc  | IUNXI, NL
35470   | NL | ripencc  | XL-AS, NL
34762   | BE | ripencc  | COMBELL-AS, BE
28878   | NL | ripencc  | SIGNET-AS, NL
39647   | NL | ripencc  | REDHOSTING-AS, NL
8455    | NL | ripencc  | ATOM86-AS ATOM86, NL
21155   | NL | ripencc  | ASN-PROSERVE Amsterdam, NL
197902  | NL | ripencc  | HOSTNET, NL
24679   | DE | ripencc  | SSERV-AS, DE
20559   | NL | ripencc  | FUNDAMENTS-AS, NL
8608    | NL | ripencc  | QINIP Esprit Telecom B.V., NL
200831  | NL | ripencc  | MIHOSNET, NL
30870   | NL | ripencc  | TRANS-IX-AS Trans-iX, NL
29028   | NL | ripencc  | COMPUKOS-AS, NL
24586   | NL | ripencc  | NL-INTERMAX B.V., NL
34756   | NL | ripencc  | ASN-GVRH, NL
8312    | NL | ripencc  | ZYLON-AS, NL
202955  | NL | ripencc  | IAHOSTER, NL
201975  | NL | ripencc  | UNISCAPEB IT-Services & Hosting, NL
41480   | NL | ripencc  | SYSTEMEC-AS, NL
201290  | NL | ripencc  | BLACKGATE, NL
39637   | NL | ripencc  | NETLOGICS-AS, NL
8587    | NL | ripencc  | INFRACOM-AS, NL
50554   | NL | ripencc  | NCBV-BACKBONE, NL
61349   | NL | ripencc  | MAXITEL, NL
58075   | NL | ripencc  | X2COM, NL
59980   | NL | ripencc  | MIJNDOMEIN, NL
24730   | NL | ripencc  | ASN-NETHOLDING, NL
60820   | NL | ripencc  | WIFI4ALL-AS, NL
202916  | NL | ripencc  | IPS, NL
28747   | BE | ripencc  | EASYHOST-COLO-AS, BE
34215   | NL | ripencc  | ATINET, NL
42812   | NL | ripencc  | DT-IT, NL
48729   | NL | ripencc  | O4S-AS, NL
199456  | GB | ripencc  | VLDTECH-ASN, GB
60950   | NL | ripencc  | CLOUDNL-AS, NL
202016  | NL | ripencc  | DOMINOICT, NL
61429   | NL | ripencc  | AS-CASTOR, NL
35027   | NL | ripencc  | ASN-SEVENP, NL
21073   | NL | ripencc  | ZORANET-AS Amsterdam, NL
41153   | NL | ripencc  | GNTEL-AS, NL
49627   | NL | ripencc  | SPEAKUP, NL
61147   | NL | ripencc  | CALLHOSTED-AS Callhosted NL
42585   | NL | ripencc  | NETWORKING4ALL, NL
15703   | NL | ripencc  | TRUESERVER-AS TrueServer BV, NL
15879   | NL | ripencc  | KPN-INTERNEDSERVICES, NL
35260   | NL | ripencc  | IU-NET, NL
62353   | NL | ripencc  | ASN-DATAPLACE, NL
202947  | NL | ripencc  | Multi ICT B.V., Almere, NL
34141   | NL | ripencc  | IN2IP-AS, NL
41960   | NL | ripencc  | NEXTPERTISE Nextpertise, NL
20495   | NL | ripencc  | WEDARE wd6.NET B.V, NL
52144   | NL | ripencc  | NOTUBIZ, NL
42755   | NL | ripencc  | DATAFIBER, NL

A keen eye will notice that this list is 91% Dutch! With some Belgium ISP’s leading next at 3%

All of the alive and protected hosts could fit in roughly a /15 of address space (128k IP addresses)

However if you sort this by total hosts active, rather than RPKI compliance, the image looks a lot more bleak:

The first 100% RPKI Invalid dropper only appears at rank #271 on this list.

Using the data, we can also see that the providers that have not downloaded the ARIN TAL. Either because they were not aware that they needed to, or could not agree to the agreement they have with it.

This is frustrating to watch. Since it means that ROA signing ARIN prefixes will be less secure than ROA signing others, until these restrictions are abolished. Even after that it will have a long term knock on effect as software updates take a long time to propagate to end networks.

One good place for RPKI validation to start happening would be DNS resolvers, Since this year someone hijacked a decent chunk of Amazons Route 53 to steal some internet (cryptographic) money. This event promoted Amazon to start signing some of their prefixes (they don’t however block invalid RPKI traffic on their network yet, see a pattern?)

To test adoption of RPKI in DNS resolvers I setup DNS servers on the IP prefixes that were used to scan for ISP’s validating, and found that no major resolver sites behind a RPKI validated network, that includes: 1.1.1.1, 4.2.2.1, 8.8.8.8, 9.9.9.9, and 80.80.80.80.

I even tested using RIPE Atlas and found only one probe had a DNS resolver that blocked invalid RPKI prefixes, and that probe was on a network listed above as a 100% compliant network.

The future to a cryptographic signed prefix world is a long way from reality, but that should hopefully not stop us from trying.

---

You can find the data generated by this post+talk here:

Formatted spreadsheet

Original CSVs

I would like to thank some groups for helping out on this,

• Job Snijders
• Nepal Research and Education Network (NREN)
• Japan Network Information Center / PPP-EXP
• NTT Communications
• LARUS Cloud Service Ltd

If you liked this post, you may want to check out the rest of the blog, to stay up to date on my new writings you can either follow me on Twitter or use RSS