GitHub was hit yesterday by what is reported to be the biggest distributed denial of service (DDoS) attack ever. According to GitHub Engineering, the site was shut down by the attack from 17:21 to 17:26 UTC on February 28. Afterwards, the website maintained intermittent functionality between 17:26 and 17:30 before fully recovering.
The nine-minute attack was by no means the longest disruption GitHub has faced. Back in 2015, we reported on a DDoS attack against the code repository that appeared to be originating from the Chinese government that lasted for the better part of a weekend. However, what makes the recent attack more significant and severe is not the length of time the site was affected, but the sheer volume of data that was sent that caused the shutdown.
At the peak of the attack, GitHub was flooded with data coming in at 1.35Tbps. The previous largest DDoS attack ever recorded was closer to 1.1Tbps. The second phase of the attack, which was causing intermittent interruptions, was only spiking at around 400Gbps.
GitHub assured its clients that no data was compromised during the event.
“To note, at no point was the confidentiality or integrity of your data at risk. We are sorry for the impact of this incident and would like to describe the event, the efforts we’ve taken to drive availability, and how we aim to improve response and mitigation moving forward.”
The attack used an obscure attack vector to amplify its power. The technique, called a "memcached reflection" attack, exploits the memcached protocol over UDP port 11211. Cloudflare is reporting that it has seen an increase in this vector over the last few days.
This is how it works. An attacker spoofs a victim’s IP address and sends a small request to several memcached servers. The servers then send out “legitimate” responses to the target as if it was the one requesting the data.
In practice, this method can magnify an attack by a factor of more than 51,000. In other words, for every byte the attacker sends out, the victim is receiving up to 51KB.
“15 bytes of request can trigger a 134KB of response sent to the unfortunate target. This is amplification factor of 10,000x! In practice we've seen a 15 byte request result in a 750KB response (that's a 51,200x amplification),” Cloudflare said.
In response to the incident, GitHub said it has moved the affected facility's inbound traffic to Akamai. Engineers say this will “help provide additional edge network capacity.” It will also be working to improve its automated intervention protocols and monitoring its infrastructure to reduce its mean time to recovery (MTTR) when mitigating future attacks.