Basically what's happened here is that Telecom Malaysia told one of Level3's networks (AS3549 - Global Crossing) that it was capable of delivering traffic to... anywhere on the Internet. Global Crossing apparently didn't have the usual sanity checks in place.
Because of how BGP works, once GBLX decided the route seemed reasonable, it immediately proceeded to dump huge amounts of traffic on the tiny Telecom Malaysia. This isn't incorrect behavior on GLBX's part (accepting the routes was incorrect, but sending the traffic was correct once they were accepted). This is way more traffic than Telecom Malaysia was prepared to handle.
Every tier 1 network gets rid of traffic as soon as possible (because it reduces their costs), completely ignoring performance or whether a route seems sensible. Telecom Malaysia therefore claimed to the most attractive route from anywhere near Malaysia to most of the world.
Level3 is one of the biggest telecoms in the world -- and in fact the biggest by far in that region. This means that most of the Internet probably stopped working for anyone anywhere near Malaysia.
Is it actually normal to have some checks for this now? When this happened a few years ago in Australia the apnic writeup (someone has linked below) essentially said there was no scalable solution yet.
when the number of originated or announced routes are "reasonable", prefix lists, as they are called, are generated and applied to BGP sessions. when the originated or announced routes is substantial and/or fluid, there are little to no prefix lists; allowing anything to be announced and accepted.
in an ideal world that has not existed for over a decade, IRR data could be used to automagically generate accurate prefix lists but for all the usual reasons (inaccurate, stale data; obtuse interface; etc) that just isn't done on a wide scale.
You can put a prefix limit on a session (that's the "little" in little to no) but that doesn't stop someone from announcing 10,000 more specific routes to popular (YouTube, Facebook, Twitter, Akamai) destinations.
>Every tier 1 network gets rid of traffic as soon as possible
Really? That's not been my experience with L3 and a few others. They seem to prefer customer routes always, even when prepended. But I don't really understand this too in-depth.
Do we know that this happened by accident? Because if all it takes is a few routing packages that is a very attractive option for any state that wants to attack another.
However, it'd be a good idea for CAs to look at any "domain validated" certs issued during this time and re-confirm. Really, any service that does insecure account resets or takeovers should be a bit more wary than usual.
Basically what's happened here is that Telecom Malaysia told one of Level3's networks (AS3549 - Global Crossing) that it was capable of delivering traffic to... anywhere on the Internet. Global Crossing apparently didn't have the usual sanity checks in place.
Because of how BGP works, once GBLX decided the route seemed reasonable, it immediately proceeded to dump huge amounts of traffic on the tiny Telecom Malaysia. This isn't incorrect behavior on GLBX's part (accepting the routes was incorrect, but sending the traffic was correct once they were accepted). This is way more traffic than Telecom Malaysia was prepared to handle.
Every tier 1 network gets rid of traffic as soon as possible (because it reduces their costs), completely ignoring performance or whether a route seems sensible. Telecom Malaysia therefore claimed to the most attractive route from anywhere near Malaysia to most of the world.
Level3 is one of the biggest telecoms in the world -- and in fact the biggest by far in that region. This means that most of the Internet probably stopped working for anyone anywhere near Malaysia.