Integrated Bubblewrap Support in LiteSpeed Web Servers
The LiteSpeed family of high performance web servers offers a number of ways to isolate requests, thereby providing protection against attack. In upcoming releases of LiteSpeed Enterprise (6.0) and OpenLiteSpeed (1.6.15), we have added integrated support for bubblewrap by Flatpak, providing even greater isolation and protection against attack.
What’s so great about bubblewrap?
Bubblewrap is a lightweight sandbox application written by Flatpak, and is described in their wiki here. What it does is implement Linux namespaces, which basically gives the application (LiteSpeed in this case) a full sandbox, which includes operating-system-supported isolated mounts, user/group IDs, interprocess communications, users, cgroups, host names and more.
For example, with isolated mount namespaces: each request can only see the file system designated for that request and can even be denied write access to any files within any directories you specify.
With isolated users and groups, the user will only be able to see their own user ID, group ID and capabilities. The default even creates independent
/etc/group files with the contents you specify.
With isolated interprocess communications each process can only see the named pipes, or Unix Domain Sockets you specify, which limits the applications that a process can contact. It’s impossible to communicate with applications without specified file system access.
Similar to, but perhaps less comprehensive than, CloudLinux CageFS, a bubblewrap’d process is utterly independent. Thus significantly reducing the opportunities for an attack.
Why integrate it?
Use of bubblewrap may be configured into previous versions of LiteSpeed (or other web servers), however, it requires complex reconfiguration of your server in order to call the
bwrap program directly, and call your program from it. By integrating it into LiteSpeed, bubblewrap works for a large number of application types and requires a very simple configuration change.
Integration allows it to operate on CGI applications as well as FCGI applications, which would not be possible without integration.
How to bubblewrap in LiteSpeed Web Server
In most cases you can enable bubblewrap in your existing LiteSpeed configuration by setting Bubblewrap Container to On in your security configuration. Details for OpenLiteSpeed are here.
If you do not customize a Bubblewrap Command (and most users will not) it will use the default of:
/bin/bwrap --ro-bind /usr /usr --ro-bind /lib /lib --ro-bind-try /lib64 /lib64 --ro-bind /bin /bin --ro-bind /sbin /sbin --dir /var --dir /tmp --proc /proc --symlink../tmp var/tmp --dev /dev --ro-bind-try /etc/localtime /etc/localtime --ro-bind-try /etc/ld.so.cache /etc/ld.so.cache --ro-bind-try /etc/resolv.conf /etc/resolv.conf --ro-bind-try /etc/ssl /etc/ssl --ro-bind-try /etc/pki /etc/pki --ro-bind-try /etc/man_db.conf /etc/man_db.conf --ro-bind-try /home/$USER /home/$USER --bind-try /var/lib/mysql/mysql.sock /var/lib/mysql/mysql.sock --bind-try /home/mysql/mysql.sock /home/mysql/mysql.sock --bind-try /tmp/mysql.sock /tmp/mysql.sock --unshare-all --share-net --die-with-parent --dir /run/user/$UID '$PASSWD 65534' '$GROUP 65534'
The key to the isolation is the bubblewrap parameters
--unshare-allspecifies all of the isolation mentioned above.
--share-netis required to allow your application access to networking.
Let’s see the isolation this gives you. The first parameter is simply the location of the bubblewrap program. The rest of the parameters will:
- Mount the following directories (and their subdirectories) as read-only:
/lib64(if it exists)
/etc/ssl(if it exists)
/etc/pki(if it exists)
/home/(your user’s directory) (if it exists)
- Mount the following files as read-only:
/etc/localtime(if it exists)
/etc/ld.so.cache(if it exists)
/etc/resolv.conf(if it exists)
/etc/man_db.conf(if it exists)
- Mount the following files as read-write:
/var/lib/mysql/mysql.sock(if it exists)
/home/mysql/mysql.sock(if it exists)
/tmp/mysql.sock(if it exists).
- Create the following empty directories:
/run/user/(your user’s ID)
- Create symbolic links:
- Provide minimal access to:
- Terminate when LiteSpeed terminates
- Create custom:
/etc/passwdfile with only your user’s entry and
/etc/groupfile with only your group’s entry and
Most programs will be able to run with these defaults and the result is great protection with minimum configuration.
If your program is not in one of the mounted directories, uses a file not in the list of files or directories, or needs write access to a file not in the writable list, your program won’t be able to access what it needs and will fail. In that case you will need to create a customized command line.
How to customize it
The customization is all in the Bubblewrap Command command line. Basically it uses the bubblewrap parameters from your version of bubblewrap with some additional tokens which LiteSpeed adds to provide better customization:
$USERis replaced with the actual user name of your user.
$UIDis replaced with the user ID of your user.
$GIDis replaced with the group ID of your group.
$PASSWDis replaced with the creation of a custom
/etc/passwdfile with a single user line in it, for the user you’re running as. If you enclose this option in single quotes you can add space-separated additional users. The default is
’$PASSWD 65535’which creates a 2 line
/etc/hostsfile, with your user and the nobody user.
$GROUPis just like
$PASSWDbut creates a custom
Note that most users will be able to use the default Bubblewrap Command by leaving it blank.
A simple example
The simplest example is the worst because it gives your application no additional protections. Set Bubblewrap Container to
On and Bubblewrap Command to:
/bin/bwrap --dev-bind / /
A better example
This is a much better example of a Bubblewrap Command as it mounts even fewer files and directories than the default. It includes no
/etc/man_db.conf, no database sockets, and only your user in
/etc/passwd and your group in
bin/bwrap --ro-bind /usr /usr --ro-bind /lib /lib --ro-bind-try /lib64 /lib64 --ro-bind /bin /bin --ro-bind /sbin /sbin --dir /tmp --proc /proc --dev /dev --ro-bind-try /etc/localtime /etc/localtime --ro-bind-try /etc/resolv.conf /etc/resolv.conf --ro-bind-try /etc/ssl /etc/ssl --ro-bind-try /etc/pki /etc/pki --ro-bind-try /home/$USER /home/$USER --unshare-all --share-net --die-with-parent --dir /run/user/$UID $PASSWD $GROUP
Give it a try!
To try it, upgrade to a version of LiteSpeed Enterprise or OpenLiteSpeed that supports bubblewrap, and turn it on for an application. This will show you the power of an integrated sandbox like bubblewrap inside of a powerful web server like LiteSpeed.