In the fall I did a blog post and talk on RPKI about how the current methods of measuring RPKI deployment are broken because they do not take into account network operators actually verifying their imported routes.
If you have not read that post I recommend you do so now to understand the next results: https://blog.benjojo.co.uk/post/are-bgps-security-features-working-yet-rpki
The post singles out two major points:
A) Users who try to verify their route imports can find themselves unable to verify ARIN routes, due to a legal agreement issue.
B) People are signing their routes with ROAs but not validating their imports, making it a vanity exercise at best
In this update, I will be focusing on the two above points:
For the legal point of view, the findings of this talk got posted on NANOG by Job Snijders from NTT Communications:
Dear NANOG,
I'd like to draw attention to a very concerning development: it appears
that the ARIN TAL is not as widely deployed as other RPKI TALs (such as
RIPE or APNIC's TAL). This means that ARIN members are needlessly put at
higher risk.
Ben Cartwright-Cox performed RPKI research a few weeks ago where he
compared the (un)reachability of RPKI Invalid announcements using ARIN,
RPKI, APNIC and JPNIC space. The full write-up is available here:
https://blog.benjojo.co.uk/post/are-bgps-security-features-working-yet-rpki
I quote Ben's assessment:
"""Using the data, we can also see that the providers that have not
downloaded the ARIN TAL. Either because they were not aware that
they needed to, or could not agree to the agreement they have with
it.
This is frustrating to watch. Since it means that ROA signing ARIN
prefixes will be less secure than ROA signing others, until these
restrictions are abolished. Even after that it will have a long term
knock on effect as software updates take a long time to propagate to
end networks."""
In other words, when creating RPKI ROAs for your resources, ARIN members
are getting *LESS* in return compared to say RIPE members.
As ARIN member I'd hope and expect the ARIN organisation to go above and
beyond to distribute their RPKI TAL as widely as possible. (Think:
proactively submitting the ARIN TAL to relevant open source projects,
making the TAL available for download without restrictions or
limitations).
At the NANOG 74 meeting David Whisnick will talk about legal barriers to
RPKI adoption and he'll offer suggestions for reform. I think Ben's
observation that the ARIN TAL is less widely deployed is a direct result
from the legal barriers that David identified.
https://pc.nanog.org/static/published/meetings//NANOG74/daily/day_2.html#talk_1767
I fear that if no action is taken (e.g. when none of the RPKI Cache
Validators can include the ARIN TAL in their software distribution [1]),
we'll continue to see the gap between ARIN and the other RIRs widen.
Software developers have indicated that the RPA is problematic and
prevents BGP implementations from shipping "secure by default" software:
https://lists.arin.net/pipermail/arin-consult/2018-April/001087.html
When ARIN members create ROAs, I assume those members would also like
networks *outside* the ARIN region to honor those ROAs and help prevent
propagation of incorrect BGP announcements. The ARIN member and the
operator of the foreign network may not even have any languages in
common! I fear that limited deployment of the ARIN TAL is detrimental to
the business interests of resource holders in the ARIN region.
Kind regards,
Job
This provoked a response from John Curran, the CEO of ARIN:
>> I quote Ben's assessment:
>>
>> """Using the data, we can also see that the providers that have not
>> downloaded the ARIN TAL. Either because they were not aware that
>> they needed to, or could not agree to the agreement they have with
>> it.”"
I believe that you correctly characterize the situation; i.e:
1) Either they were not aware they needed it (note this is trivial
to fix with outreach/education and requires skills well within the range
of anyone who is going to be doing routing validation based on RPKI data), or
2) They could not agree to ARIN RPA agreement (for which the most cited
reason is the indemnification clause, but perplexing given agreement to other
indemnification clauses such as RIPE’s Certification services.)
Further analysis and characterization of those not using the ARIN TAL would help
enormously in understanding which of the scenarios above is dominant, and
prioritize efforts for addressing the situation.
Based on this response, the status of the legal issues on the ARIN TAL will not change any time soon. John Curran appears to not see a problem with what is basically a CA certificate needing a legal agreement clause.
This basically caused a mini pile on John, with many people on the list disagreeing with him.
Based on this, the gap between people validating RIPE signed prefixes vs. ARIN prefixes is going to widen.
A unique loss for networks with ARIN space (Almost all networks in the US and Canada), since no other RIR has this kind of legal block on a TAL/CA.
On the issue of networks not validating their prefix imports with regards to RPKI, we can re run the same scans as last time to check if the RPKI protected internet is growing or not:
This time we have the following RIR’s to test with
We now have a full list of ASN’s that are fully RPKI compliant:
15426 | NL | ripencc | XENOSITE Amsterdam, NL
35470 | NL | ripencc | XL-AS, NL
34968 | NL | ripencc | IUNXI, NL
12859 | NL | ripencc | NL-BIT BIT BV, NL
28878 | NL | ripencc | SIGNET-AS, NL
39647 | NL | ripencc | REDHOSTING-AS, NL
8455 | NL | ripencc | ATOM86-AS ATOM86, NL
29290 | NL | ripencc | ALPHAMEGA, NL
21155 | NL | ripencc | ASN-PROSERVE Amsterdam, NL
197902 | NL | ripencc | HOSTNET, NL
200831 | NL | ripencc | MIHOSNET, NL
29028 | NL | ripencc | COMPUKOS-AS, NL
24586 | NL | ripencc | NL-INTERMAX, NL
24679 | DE | ripencc | SSERV-AS, DE
34756 | NL | ripencc | ASN-GVRH, NL
12907 | DE | ripencc | IPANDMORE, DE
51050 | NL | ripencc | H4HOSTING-AS, NL
30870 | NL | ripencc | TRANS-IX-AS, NL
51483 | DE | ripencc | SASG Cecinastr. 70, DE
8312 | NL | ripencc | ZYLON-AS, NL
201975 | NL | ripencc | UNISCAPEB IT-Services & Hosting, NL
8608 | NL | ripencc | QINIP Esprit Telecom B.V., NL
8587 | NL | ripencc | INFRACOM-AS, NL
42755 | NL | ripencc | DATAFIBER, NL
20495 | NL | ripencc | WEDARE wd6.NET B.V, NL
50554 | NL | ripencc | NCBV-BACKBONE, NL
58075 | NL | ripencc | X2COM, NL
8283 | NL | ripencc | COLOCLUE-AS, NL
15922 | NL | ripencc | QWEB-AS, NL
47409 | DE | ripencc | TPI-AS, DE
59980 | NL | ripencc | MIJNDOMEIN, NL
61429 | NL | ripencc | AS-CASTOR, NL
28747 | BE | ripencc | EASYHOST-COLO-AS, BE
47543 | NL | ripencc | ATOM86-AS, NL
42812 | NL | ripencc | DT-IT, NL
34215 | NL | ripencc | ATINET, NL
199456 | GB | ripencc | VLDTECH-ASN, GB
24730 | NL | ripencc | ASN-NETHOLDING, NL
60820 | NL | ripencc | WIFI4ALL-AS, NL
202916 | NL | ripencc | IPS, NL
48729 | NL | ripencc | O4S-AS, NL
60950 | NL | ripencc | CLOUDNL-AS, NL
3333 | EU | ripencc | RIPE-NCC-AS, NL
61575 | BR | lacnic | UNIVERSIDADE FEDERAL DE MINAS GERAIS, BR
202016 | NL | ripencc | DOMINOICT, NL
35027 | NL | ripencc | ASN-SEVENP, NL
41153 | NL | ripencc | GNTEL-AS, NL
42585 | NL | ripencc | NETWORKING4ALL, NL
61147 | NL | ripencc | CALLHOSTED-AS, NL
49627 | NL | ripencc | SPEAKUP, NL
39022 | NL | ripencc | DEEPMEDIA-AS, NL
47863 | NL | ripencc | SDSK, NL
198694 | NL | ripencc | DOTXS-, NL
59752 | DE | ripencc | JPRU-AS, DE
44187 | NL | ripencc | DEANONE-AS, NL
204030 | NL | ripencc | SERAC-, NL
198260 | IE | ripencc | AFILIAS-AMS2, IE
27970 | CW | lacnic | OnePacket Networks Inc., CW
197000 | EU | ripencc | RIPE-NCC-AUTHDNS-AS, NL
203822 | NL | ripencc | MKB-WEBHOSTER, NL
202022 | NL | ripencc | FLEXYZ, NL
41458 | RU | ripencc | MVSGROUP-AS, RU
202591 | NL | ripencc | DIGIWARE, NL
206389 | NL | ripencc | LIBERNET, NL
41008 | BE | ripencc | CEGEKA-LEUVEN, BE
205915 | NL | ripencc | PINKROCCADE, NL
49820 | NL | ripencc | PICTURA-NET, NL
39146 | NL | ripencc | MACHCLOUD-AS, NL
199408 | NL | ripencc | BOL-COM, NL
35316 | NL | ripencc | PRODACOM-AS, NL
44780 | DE | ripencc | EVERSCALE-AS, DE
1140 | NL | ripencc | SIDN, NL
58138 | NL | ripencc | KORTON_INTERNET, NL
40490 | US | arin | AS-AFILIAS - Afilias, Inc., US
199743 | NL | ripencc | HPS, NL
31433 | NL | ripencc | BAKKERGROEP, NL
59929 | NL | ripencc | FLEXIBELT, NL
48112 | RO | ripencc | XINDI-AS, RO
49685 | NL | ripencc | ITIS-AS Signet B.V., NL
58186 | NL | ripencc | GAMEHOUSEEUROPE, NL
59427 | NL | ripencc | MASSXESS, NL
201311 | NL | ripencc | ITCREATION, NL
203318 | NL | ripencc | ASBIZWAY, NL
203179 | NL | ripencc | PROACTMCS, NL
2121 | EU | ripencc | RIPE-MEETING-AS, NL
30132 | US | arin | ISC-F-AS - Internet Systems Consortium, Inc., US
198730 | NL | ripencc | SBP, NL
34106 | NL | ripencc | TELEGRAAF-AS, NL
39002 | RO | ripencc | OPTI-AS, RO
57400 | NL | ripencc | INTERVIA-AS, NL
50906 | BE | ripencc | LUCIAD, BE
31172 | NL | ripencc | QFAST-AS, NL
61963 | AE | ripencc | MARKAVIP, AE
61200 | NL | ripencc | MEDIAMONKS-AS, NL
47632 | NL | ripencc | CONTINUITY_SOLUTIONS, NL
205839 | NL | ripencc | SIOC-AS, NL
204754 | NL | ripencc | SYSACTIVE, NL
42836 | NL | ripencc | SCHUBERGPHILIS, NL
12337 | DE | ripencc | NORIS-NETWORK, DE
15703 | NL | ripencc | TRUESERVER-AS TrueServer BV AS number, NL
35260 | NL | ripencc | IU-NET, NL
8315 | NL | ripencc | AMSIO, NL
62353 | NL | ripencc | ASN-DATAPLACE, NL
200023 | NL | ripencc | QONNECTED-AS Qonnected B.V., NL
34762 | BE | ripencc | COMBELL-AS, BE
202947 | NL | ripencc | Multi ICT B.V, NL
61349 | NL | ripencc | MAXITEL, NL
199522 | NL | ripencc | TEDAS-AS, NL
202955 | NL | ripencc | IAHOSTER, NL
21385 | DE | ripencc | TNIB Trusted Network GmbH, DE
41480 | NL | ripencc | SYSTEMEC-AS, NL
34141 | NL | ripencc | IN2IP-AS, NL
41960 | NL | ripencc | NEXTPERTISE Nextpertise, NL
20559 | NL | ripencc | FUNDAMENTS-AS, NL
57866 | NL | ripencc | FUSIX-AS, NL
22567 | US | arin | CWGS - CWGS ENTERPRISES, LLC, US
Just like last time, The Netherlands are leading the way and leaving the rest of the networking world behind:
93 NL
8 DE
4 BE
3 US
2 RO
1 AE
1 BR
1 CW
1 GB
1 IE
1 RU
This is mostly due to a handful of tier 2 providers who are RPKI filtering on behalf of their customers, in total this brings us to 116 ASNs that are fully RPKI protected, for reference the previous post had this number at 85.
Sadly, the gap between ASNs not importing the ARIN TAL is also going up.
New | No change
AS12306
AS12822
AS14348
AS16188
AS17833
AS197729
AS20098
AS202034
AS2037
AS204723
AS20755
AS22821
AS26967
AS30766
AS33824
AS394053
AS44152
AS46491
AS46787
AS49871
AS54255
AS54300
AS54912
AS62520
AS6733
Giving that ARIN Leadership does not seem to want to budge on the legal issue for now, the gap between networks that do import all TALs, and the networks that import all but ARIN is going to widen. This is happening at a critical time as well since as the number of networks doing some form of ROV (reject routes based on invalid RPKI status) double since September.
Out of all networks that do ROV, around 20% of them do not have the ARIN TAL installed, meaning their policy of rejecting invalid prefixes are undermined by this. This issue seems to mainly impact ARIN networks, since there are more ARIN networks that do not have the ARIN TAL installed, than ones that do:
12306 | DE | ripencc | PLUSLINE, DE
12822 | DE | ripencc | LYNET-AS Hamburg, Luebeck, DE
14348 | US | arin | URI-AS - University of Rhode Island, US
16188 | DE | ripencc | PUNKT, DE
17833 | KR | apnic | NCOM-AS-KR NCOM ltd., KR
197729 | DE | ripencc | TWENTYONE-CLOUD, DE
20098 | US | arin | BCBS-AL - Blue Cross and Blue Shield of Alabama, US
202034 | DE | ripencc | REDREPLY-AS, DE
2037 | US | arin | CSUFRESNO - California State University at Fresno, US
204723 | DE | ripencc | HACON, DE
20755 | DE | ripencc | NET-LAB Frankfurter Str. 99, DE
22821 | US | arin | AIPSO-COM - AIPSO, US
26967 | US | arin | DCSL-6-26967 - Digium Cloud Services, LLC, US
30766 | DE | ripencc | GGEWNET-AS Dammstrasse 68, DE
33824 | DE | ripencc | CONSOL-AS, DE
394053 | US | arin | NAICWEB - National Association of Insurance Commissioners, US
44152 | DE | ripencc | SMARTHOUSE-AS, DE
46491 | US | arin | MEC-ASN - Metropolitan Educational Council, US
46787 | US | arin | RXLINC-INC - RXLINC LLC, US
49871 | DE | ripencc | DDG-AS, DE
54255 | US | arin | ONDEMAND - On Demand Networks Inc, US
54300 | US | arin | LIGHTSPEED-VOICE - Lightspeed Voice, US
54912 | US | arin | BAYAL-IP1 - Bay Alarm Company, US
62520 | US | arin | EMASON-ASN - eMASON, Inc, US
6733 | DE | ripencc | DIMDI Waisenhausgasse 36-38a, DE
In addition, RPKI got a nice press boost from Cloudflare, a large CDN.
In their post they also call out the ARIN legal agreement as a possible trouble maker, and then mentioned they are making ways to implement filtering themselves.
And based on my scans, they do not appear to be doing this in a way that is globally viewable. After some chats with some of the Cloudflare engineers on the project, it became clear that for the moment, they are only applying RPKI filtering on their peered routes, so my scanning setup (since those packets will be coming in though transit, and even if not, they will have other routes though transit) will not see their efforts.
Though I do question the point of doing RPKI this way, I look forward to seeing them fully implement RPKI on all imports.
That all being said, Vasco Asturiano (an engineer at Cloudflare) has done some very cool visuals on RPKI ROA’s and Certs as well:
Current view of #RPKI ~18k certs (blue) and ~11k ROAs (red) on a #Hilbert curve, for whole IPv4 space and 2001::/16 IPv6. Credit to @lpoinsig for scraping the data. pic.twitter.com/qphV9Grwuj
— Vasco Asturiano (@vastur) November 16, 2018
Closing off I would like to thank the following orgs and people for IP space and/or other resources:
If you want to look at the raw data I collected for this round of scanning, you can find the data here in gzipped CSV form: link
I will try and do these round ups at least every half year, but post xmas I will work on automating these scans and publish data at least every month.
To keep up to date with that, you can either use the blog’s RSS, or follow me on twitter.
Until next time!
Related Posts:
Just how long do DNS resolvers cache last? (2017)
Are BGPs security features working yet? (2018)
Random Post: