Why LiteSpeed and WAF is Pure Love

Do you use WAF (Web Application Firewalls) to protect your sites? Our guest blogger, Lucas Rolff does. Lucas is, as he says, “the everything man – sales, support, accounting, coffee drinker, and operations guy” for Hosting4Real. Today he’ll be sharing his experience implementing WAF. See how its impact on his systems changed dramatically when he switched from Apache to LiteSpeed!

So, without further ado, here’s Lucas’ story in his own words:
—

At Hosting4Real, we run WAF across our fleet. This is done mainly to protect against known vulnerabilities in the open source software our customers use. It also protects against generic attacks such as SQL injection, cross-site scripting, backdoors etc.

We use the Comodo CWAF ruleset. CWAF contains hundreds of rules for SQL injection, cross-site scripting and, more importantly, rules targeting specific software vulnerabilities in WordPress, Joomla, Magento and other known CMS’s.

WAF and Apache

The issue with having an extensive list of WAF rules is that it tends to hurt your performance, both in your rules latency, and in the amount of resources your system requires to handle a request.

On certain shared hosting servers with a particular traffic patterns, we noticed Apache would sometimes take up 100-200% (1-2 cores) alone for processing requests within mod_security. Eventually, we got to the point where we tried turning off mod_security rules, and we instantly saw a drop in CPU usage of about 100% (1 core).

WAF and LiteSpeed

In early-mid 2018 we decided to switch to LiteSpeed for various reasons, one of them being the CPU load experienced while using our Comodo WAF ruleset. After the switch to LiteSpeed, which had already decreased our CPU usage compared to Apache without mod_security, we decided to enable our ruleset again. We went in with the expectation that we’d see an increase in CPU load similar to the drop we saw when turning it off in Apache. To our surprise, we saw almost zero difference in the CPU usage on the server, despite running with the full set enabled.

We didn’t believe this, so we talked to LiteSpeed core developers to figure it out!

LiteSpeed uses it’s very own implementation to parse rules, instead of relying on a generic mod_security library. This has allowed LiteSpeed to optimize the way regular expressions are matched, as well as utilize an internal cache.

What is even more impressive is that LiteSpeed has managed to only run the rules for dynamic requests. This is a big difference compared to how Apache and other web servers handle this.

Apache, for example, will execute your rules for every single request, even the pictures of cats and hamsters. This means we’re maybe saving 40-90% processing power alone by this optimization.

Effectively, we went from a server using up 180-200% of our CPU the majority of the time due to mod_security, down to roughly 100% when using LiteSpeed. What is even more amazing is that the decrease of 80-100% CPU is despite an increase in traffic of 10.8% (from 42.06 million to 46.6 million requests).

LiteSpeed vs. Apache WAF

In the screenshot above you’ll see how our CPU usage changed.

In the first section we see Apache with CWAF modsec rules. At the first red line we decided to disable the rules for the given server. At the second red line in mid-June, we decided to switch to LiteSpeed and then enable the CWAF modsec rules again, decreasing the CPU by a further 10-25%.

In the above screenshot, we see that the traffic is rather stable with an increase in traffic in the last couple of weeks.

And if we take a look at the Unix load, we also see a massive drop there as well.

As you can see over time at the beginning of the Unix load graph, we’re seeing a slow increase in load – the load increase is consistent with the rate of growing traffic over the months, and also as new customers would get on board the server.

How to Enable CWAF in cPanel for LiteSpeed

Since LiteSpeed requires another set of rules compared to the Apache ones from CWAF, we have to use a new vendor. You can use “ModSecurity™ Vendors” from cPanel within WHM.

In the ModSecurity™ Vendors page, click Add Vendor. Supply a Vendor Configuration URL, i.e. https://waf.comodo.com/doc/meta_comodo_litespeed.yaml. Click Load, which loads the vendor information in the fields below, and then click Save.

This will automatically enable the ruleset, and ensure that updates to the ruleset are downloaded every night. It will also trigger a restart request to LiteSpeed.  So, the rules will be active seconds after you’ve clicked the Save button.

The Server Hardware Specs

CPU: Intel Xeon E5-1630 v4 @ 3.70GHz (4C/8T)
RAM: 64GB DDR4 2400 Mhz ECC Memory
Disks: 2x480GB Intel DC S3520 SSDs (for MySQL) and 2x2TB HGST Ultrastar 7K4000 Spinning disks for OS and files

Who is Hosting4Real?

We are a small Dutch-based company focusing on high-performance shared web hosting. We try to kill the bad reputation of the shared hosting market by offering extremely fast (and scalable) web hosting. We’re addicted to performance, and we use best-in-class software such as LiteSpeed to help us handle traffic peaks, as well as our own in-house-developed Content Delivery Network

—
Our thanks go to Lucas for sharing his story. Readers, if you have a LiteSpeed success story that you would like to share, drop by our Slack community, and let us know what you have in mind!