LiteSpeed Addresses HTTP/2 DoS Advisories

Summary

On Tuesday, August 13th, 2019 a family of eight HTTP/2 security advisories was disclosed publicly in Vulnerability Note VU#605641.  We examined our software and discovered that the flagship LiteSpeed Web Server and LiteSpeed ADC products are not vulnerable to seven of the eight potential attacks.  In fact, even the one troublesome attack cannot cause any service outage: all it can do is waste computer resources.  Nevertheless, today, August 15th, 2019, LiteSpeed Technologies releases updated server software that addresses all of these security vulnerabilities:

• LSWS 5.4.1 (CVE-2019-9516)
• ADC 2.5.1 (CVE-2019-9516)
• OpenLiteSpeed 1.5.5 and 1.4.50  (CVE-2019-9512, CVE-2019-9515, CVE-2019-9516, CVE-2019-9518)

Note: We have also added the fixes to LSWS 5.3.8 build 7.

Timeline

• Spring 2019: Netflix finds vulnerabilities in several HTTP/2 implementations.
• 1pm EDT, Tue Aug 13: HTTP/2 security advisories are disclosed.
• 3 pm EDT, Tue Aug 13: LiteSpeed begins tests to see whether its software is vulnerable.
• 10 pm EDT, Tue Aug 13: Tests are completed and plan of action is agreed upon.
• 1 am EDT, Wed Aug 14: Software fixes are complete.
• Wed Aug 14: Fixes are ported to all our products and verification tests are performed.
• 5pm EDT, Thu Aug 15: Security updates for LiteSpeed software are available.

Background

As early as May of this year, security researchers at Netflix examined several HTTP/2 implementations.  What they discovered were several avenues for potential DoS attacks.  Netflix teamed up with Google to inform HTTP/2 software vendors of potential vulnerabilities and to coordinate a synchronized software update rollout.  No one alerted us, however, and we learned about these vulnerabilities two days ago from the public announcement. Meanwhile, other vendors have had the luxury to spend months to fix their software.

LiteSpeed HTTP/2 Implementation Holds Up Well

The good news is that LiteSpeed software does well in the face of these attacks.  In fact, only one of the potential vulnerabilities, CVE-2019-9516 (a.k.a. Zero-Length Headers Leak) is present in LiteSpeed Web Server and ADC.  A malicious client that sends a stream of zero-length header names and values causes the server to waste CPU cycles and to keep allocating memory for the duration of the connection.  In our testing, even when under such an attack, the server continued to process other connections with no perceptible performance impact. LiteSpeed closed the offending connections after 60 seconds, releasing excess memory.  Nevertheless, even though this attack cannot cause a service outage, the wasted CPU cycles is something we won’t accept.

Advisories in Detail

Below, we list the individual security advisories and their potential impact on our software — LSWS, ADC, and OpenLiteSpeed — before today’s new releases.  However, the new releases mitigate all of these attack scenarios.

CVE-2019-9511 “Data Dribble”

This attack does not affect LiteSpeed software

CVE-2019-9512 “Ping Flood”

Under this attack, LSWS and ADC use slightly more memory than usual and use 100% CPU, but continue to serve other traffic.  LiteSpeed closes the attacking connection in 20 seconds.

OpenLiteSpeed enters a busy loop and cannot serve other traffic.

CVE-2019-9513 “Resource Loop”

LiteSpeed’s HTTP/2 implementation uses an efficient priority mechanism (not a tree) and thus pays no price when priorities change.

CVE-2019-9514 “Reset Flood”

This attack does not affect LiteSpeed software

CVE-2019-9515 “Settings Flood”

Same as Ping Flood above: LSWS and ADC cope well. OpenLiteSpeed is effectively DoSed.

CVE-2019-9516 “0-Length Headers Leak”

LSWS, ADC, and OpenLiteSpeed keep on allocating memory and using CPU in this attack scenario, but are still able to serve other traffic.  LiteSpeed closes the attacking connection in 60 seconds.

The damage is limited to wasting computer resources: CPU and memory.  The amount of memory the server allocates is proportional to the volume of incoming traffic.  Because the server closes the malicious connections after a period of time and releases memory, this attack cannot take down LSWS or ADC.

CVE-2019-9517 “Internal Data Buffering”

This attack does not affect LiteSpeed software

CVE-2019-9518 “Empty Frames Flood”

This attack does not affect LSWS and ADC: no memory increase and no impact to service.  More CPU cycles are used to process frames.

OpenLiteSpeed gets stuck in a loop and cannot serve other traffic.

Mitigation

The updates introduce several heuristics to detect an abnormally high frequency of control or unusual (e.g. empty) frames.  LiteSpeed closes offending connections forthwith.

When the outgoing buffer is under pressure, the processing of incoming frames is suspended, preventing allocating excessive amounts of memory to keep up with incoming data.  This feedback mechanism is a natural and effective defense against potential future DoS attacks.

Looking Ahead

It surprises us that we didn’t receive notification of the impending security advisories.  Not only were we the HTTP/2 pioneer — at one point powering 96.5% of all websites that used HTTP/2 — but we also offer the best HTTP/2 performance in the world, beating the next fastest implementation by a factor of 8.  (Head over to https://http2benchmark.org/ to see for yourself.)

We have reached out to relevant parties to ensure that LiteSpeed Technologies is kept in the loop the next time around.

In all, it was two busy days.  We are proud of the limited impact these security advisories had on our software and of delivering security updates in such a short time.