Performance Testing Namespaces

Namespaces in Linux are often used to implement containers and are great at administering sandboxes where files, users, processes and just about all system resources are separated from all other processes. Many commonly used applications, including Docker and Kubernates, use namespaces to implement their sandboxes. Kernel support for namespaces makes them virtually bulletproof and as such, namespaces are considered the gold standard for creating safe sandbox environments.

OpenLiteSpeed supports both the bubblewrap tool and Linux native namespaces to meet the needs of a safe sandbox environment. But while they both provide a safe environment (bubblewrap actually uses namespaces in its implementation), there are differences between them in execution with OpenLiteSpeed. In this article, we’ll be performance testing namespaces and bubblewrap in OpenLiteSpeed.

Differences between Bubblewrap and Namespaces

OpenLiteSpeed’s support of bubblewrap is quite simple: set up a command line, and then execute it and allow bubblewrap to implement the sandbox. There are special command line options for generating the bubblewrap command line. These include custom /etc/passwd and /etc/group files to meet your needs, and file copies to generate files with the correct permissions. Bubblewrap itself does the bulk of the work.

The namespaces feature in OpenLiteSpeed performs virtually all of the same tasks, but does so in a different way. Instead of a command line, you specify the name of a configuration file, which contains commands in it. Also, when a set of namespaces for a given user’s environment is created, it is persisted so that later executions do not have to recreate it. They can simply join it using facilities similar to nsenter. This should result in significantly better performance, but does it? That’s what we will aim to find out in this article.

Goals

• Determine the conditions where namespaces and bubblewrap perform the same.
• Determine the conditions where namespaces perform better than bubblewrap and how much improvement there is.

In an earlier blog which described OpenLiteSpeed’s use of bubblewrap, PHP processes were used to describe the feature. And while PHP is good for demonstrating how it works, and it remains the most common use of bubblewrap in OpenLiteSpeed, it is not the best tool to show performance differences between bubblewrap and namespaces.

LiteSpeed highly optimizes PHP, and starts a small number of lsphp processes to serve any number of PHP requests. Thus the setup is one time for each execution of lsphp. It’s still worth using namespaces as doing so removes the overhead of executing an intermediary process. The lsphp program is run directly rather than through bubblewrap, so the actual performance gain is slight.

Plain CGI executables provide the largest gain in performance and are the best tool to demonstrate the value of LiteSpeed’s native namespaces over bubblewrap’s implementation of namespaces. These are programs which are directly run by the web server and do their communications with the server though standard I/O and environment variables (see Common Gateway Interface).

Performance Benchmarks

In https://http2benchmark.org/ the test tool of choice is h2load, thus we used it for testing performance here as well.

Test environment

• Hardware: Intel i7, with 16GB of memory.
• Software: VM with Ubuntu 22.04, configured to use 8GB of memory and 3 processors. VirtualBox running on Windows.
• Test version of OpenLiteSpeed with a pre-release of namespace support. Running 3 workers to utilize the 3 processors.
• After each test, we ran unmount_ns to assure that a subsequent test would not benefit from a persisted namespace. Also, we stopped and restarted OpenLiteSpeed after each test as well.
• We tested using the sample CGI program included with OpenLiteSpeed: cgi-bin/helloworld. This program is simple enough to incur minimal overhead and thus demonstrate the value of any performance changes between the tests.

Test results

Small Load Test

Command line:

h2load -n 10000 -c 10 https://127.0.0.1:8089/cgi-bin/helloworld

Where:

• -n 10000: 10,000 total requests
• -c 10: 10 concurrent requests

Bubblewrap results:

• 69 seconds
• 145 requests per second
• 7 K/sec

Namespace results:

• 16 seconds
• 613 requests per second
• 29 K/sec

Without namespace or bubblewrap:

• 13 seconds
• 761 requests per second
• 35 K/sec

This shows in a small number of requests a 431% improvement in clock time and 423% improvement in throughput. The penalty for namespaces is 23% in clock time and 24% in throughput.

Large Load Test

Command line:

h2load -n 100000 -c 100 -m 10 https://127.0.0.1:8089/cgi-bin/helloworld

Where:

• -n 100000: 100,000 total requests
• -c 100: 100 concurrent requests
• -m 10: 10 streams, or since this is http, the number of requests in flight at a time.

Bubblewrap results:

• 1123 seconds
• 89 requests per second
• 8 K/sec

Namespace results:

• 159 seconds
• 628 requests per second
• 36 K/sec

Without namespace or bubblewrap:

• 125 seconds
• 799 requests per second
• 43 K/sec

This shows in a larger number of requests a 706% improvement in clock time and 706% improvement in throughput. The penalty for namespaces is only 27% in clock time and 27% in throughput.

Conclusion

In conclusion, it is clear that for CGI requests, you can accomplish substantial performance gains using namespaces over bubblewrap. These performance gains merely hint at the other resource gains in CPU, memory, processes, etc. And the value of a sandbox comes at a reasonable cost in performance and overhead.