ModSecurity Performance Comparison: Apache, nginx, LiteSpeed

Measuring ModSecurity Performance

Each of the primary web server vendors includes the ability to run a Web Application Firewall (WAF), often described as ModSecurity (the terms are used here interchangeably). We wanted to answer the following questions about ModSecurity performance on Apache, nginx, and LiteSpeed:

• What is ModSecurity’s impact on server performance?
• Which rulesets performed better: the OWASP ModSecurity Core Rule Set or Comodo?
• Which implementation performed better: ModSecurity 2, ModSecurity 3, or LiteSpeed’s proprietary implementation?
• Whose ModSecurity configuration performed better: Apache, nginx, OpenLiteSpeed, or LiteSpeed Enterprise?

ModSecurity’s Impact on Server Performance

Web servers are often benchmarked, but here we document the specific impact of ModSecurity on overall server performance. The servers tested are:

• Apache v2.4.41 using ModSecurity engine v2.9.2
• nginx v1.17.6 using the nginx ModSecurity connector v1.0.1 and the ModSecurity engine v3.0.3
• OpenLiteSpeed v1.6.4 using ModSecurity engine v3.0.4
• LiteSpeed Enterprise v5.4.2 using its internal ModSecurity engine

To be thorough, the tests include a concurrent volume of:

• Static data
• Dynamic data using a simple PHP script
• A cached WordPress front page

By showing these separately, differences between servers can be highlighted.

The environment for both the web server machine and the client machine is:

• Vultr high frequency VM
• Network Connected 10Gb.
• 1 CPU, 3.8Ghz, gen 6 VM
• 1 Gb memory
• CentOS 7

OWASP vs. Comodo Rulesets

The tests are performed using OWASP ModSecurity Core Rule Set (v3.3) and the Open Source Comodo rules (v1.225). Comodo differentiates between v2.x rules which are used by Apache and LiteSpeed Enterprise and v3.x rules which are used by nginx and OpenLiteSpeed.

ModSecurity 2 vs. ModSecurity 3 vs. LiteSpeed Proprietary

The fact that there are differences in rules to be used between servers highlights the fact that each server has a different engine for processing ModSecurity rules.

• Apache uses the older ModSecurity v2.x engine. That engine was initially written specifically for Apache and is optimized for it.
• nginx and OpenLiteSpeed use the newer ModSecurity v3.x engine. This engine was re-written from scratch specifically to remove the dependencies in Apache, and was written in a higher level language (C++ rather than C).
• LiteSpeed Enterprise has its own proprietary engine, highly optimized and well integrated into the LiteSpeed server.

ModSecurity Configuration in Apache vs. nginx vs. LiteSpeed

Each server must be configured to use ModSecurity, both for the engine and for accessing the rules.

• The easiest to configure is LiteSpeed Enterprise as the WAF configuration is incorporated into the WebAdmin interface. The ModSecurity engine is fully integrated into the server so there is no specific ModSecurity installation.
• OpenLiteSpeed includes the ModSecurity Plugin as part of the distribution and requires only configuration of the Plugin in order to be recognized and use the rules. This is part of the WebAdmin interface, but the documentation must be followed as the procedures are not obvious.
• For Apache you must obtain the mod_security Plugin independently from the server software. Once obtained and installed, follow the instructions, as configuring the rules is not an obvious process.
• To obtain the rules engine for nginx you must install the source code and mod_security Plugin separately, compile the Plugin and install it. Once it is installed, you can configure it to use the rules. This is a cumbersome, error-prone, time-consuming process.

Luckily, you can avoid the headache of configuring ModSecurity manually by using the scripts provided in the http2benchmark project.

Benchmark Tools Used

To provide a stable test bed to demonstrate ModSecurity performance differences, http2benchmark (available here and documented here) was used as a base. It’s advantages are many, but in particular it installs and configures web servers of your choosing on your server and runs tests from a client of your choosing. This lets you validate the performance in your own environment.

The main server configurations can be found here, if you’d like to look them over.

Since http2benchmark is managed by scripts, scripts were created to setup a WAF for your environment using OWASP rules for servers installed by http2benchmark for Apache, nginx, Litespeed Enterprise and OpenLiteSpeed.

Note that there is documentation on how to use the scripts. For example, if you wish to configure an Apache, nginx and Litespeed Enterprise web server installed by http2benchmark you would run the following scripts on the test server as root:

./config_apache_modsec.sh
./config_nginx_modsec.sh
./config_lsws_modsec.sh

These scripts allow you to control which servers are installed with the WAF, and to even use a different set of rules (the documentation describes the use of Comodo rules rather than OWASP rules).

After configuring the server, you need to move over to the client system and run the /opt/benchmark.sh script to start the benchmarks.

Benchmark Test Results

Each test shows a particular run of h2load with three variations: ModSecurity disabled, ModSecurity enabled with the OWASP rules, and ModSecurity enabled with the Comodo rules. The actual command for the first test is:

h2load -n 1000 -c 10 -t 1 -T 5 -m 10 -H 'Accept-Encoding: gzip,deflate' https://benchmark.com/hello.php

The purpose is to specifically show the differences in performance between each of the servers processing a particular type of data and to show the effect of the ModSecurity processing on the server’s ability to process data. With http2benchmarks, higher numbers represent better performance.

You can find the raw data from these tests here.

Dynamic PHP File Benchmarks

The first test uses a dynamic PHP file: a file where the output is generated by the script and PHP engine. The server must process the input request and do some initial validation. The ModSecurity engine is involved in processing the request and validating that it is acceptable.

The server software will then pass the request to the PHP engine (under server control), which generates the output. The output is passed back to the server for additional ModSecurity processing before the output can be passed back to the client.

This test is particularly important as it uses generated (dynamic) data. Other than the request source, the server can not cache the request output either to avoid additional file I/O or to avoid ModSecurity engine use.

The percentage of non-ModSecurity speed once ModSecurity is enabled:

The performance of each server relative to Apache:

The results show:

• LiteSpeed Enterprise started as the best performer with ModSecurity disabled and continued as the best performer regardless of the rules used.
• Apache ran the slowest with ModSecurity disabled but then ran better than OpenLitespeed or nginx with ModSecurity enabled due to its use of the highly optimized ModSecurity v2.x engine.
• OpenLiteSpeed ran slower with ModSecurity disabled than nginx, but it ran faster or with comparable speeds with ModSecurity enabled.
• nginx’s best times were without ModSecurity. With ModSecurity enabled it ran the slowest of the group.
• Comodo rules are slower in all cases than the OWASP rules.

Static HTML File Benchmarks

The next test uses a 1K static file: a simple GET request which is served up quickly by the server. The server does limited processing on the input request to determine access to the file to read, obtains the file (generally out of cache), compresses it in this case and then sends the response to the client.

The ModSecurity engine is involved in processing the request and validating that it is acceptable.

This test is particularly important as it shows the impact of caching both of the file and of the ModSecurity response to this request. Since the request and the output do not change, the server can avoid both reading the data many times (which all servers do) and processing it through ModSecurity (which the LiteSpeed servers do).

The percentage of non-ModSecurity speed once ModSecurity is enabled:

The performance of each server relative to Apache:

The results show:

• LiteSpeed Enterprise started as the best performer with ModSecurity disabled and continued as the best performer regardless of the rules used.
• OpenLiteSpeed ran faster than any server but LiteSpeed Enterprise in all cases.
• Apache ran the slowest with ModSecurity disabled but then ran better than nginx with ModSecurity enabled due to its use of the highly optimized ModSecurity v2.x engine.
• nginx’s best times were without ModSecurity. With ModSecurity enabled it ran the slowest of the group.

Cached WordPress Front Page Benchmarks

The final test is a cached WordPress front page environment where a number of pages are served simultaneously. This is designed to represent a real-world server environment as closely as possible.

It combines elements of the prior two tests:

• Multiple, simultaneous operations
• Requests of static data
• Requests of dynamic data.

This test is particularly important as it comes closest to representing how data is served in a real server environment. For cache solutions, Apache was paired with W3Total Cache, nginx with FastCGI cache , and both LiteSpeed servers with LSCache.

The percentage of non-ModSecurity speed once ModSecurity is enabled:

The performance of each server relative to Apache:

The results show:

• LiteSpeed Enterprise started as the best performer with ModSecurity disabled and continued as the best performer regardless of the rules used.
• OpenLiteSpeed ran faster than any server but LiteSpeed Enterprise in all cases.
• Apache ran the slowest with ModSecurity disabled but then ran better than nginx with ModSecurity enabled due to its use of the highly optimized ModSecurity v2.x engine.
• nginx’s best times were without ModSecurity. With ModSecurity enabled it ran the slowest of the group.

LiteSpeed Enterprise ModSecurity Performance is Best

• In all of the tests run, LiteSpeed Enterprise was the clear winner.
• Both LiteSpeed Enterprise and OpenLiteSpeed ran better in serving up pages in the real-world environment.
• Apache’s use of the ModSecurity v2 engine provided better performance than the v3 engine used by nginx.
• nginx ran well without ModSecurity enabled, but degraded the most once it was enabled.
• Comodo rules are a bit slower than the OWASP rules in almost all environments.