Benchmark Comparison on Serving Small Static Files: LiteSpeed vs Apache vs Nginx

To perform a fair benchmark requiring a lot of preparation, as you need to know the detail of each product and set appropriate configuration. Recently, we got chance to do some internal testing. It’s experimental, however the result is good enough to make some points and may be useful to our users, so we would like to share it in a series of blog posts first. Feel free to provide any feedback, so we can improve on our formal benchmark tests later on.

This time we will compare web server performance on serving small static files. Comparing small files is standard practice for benchmark, as this can reveal web server performance instead of network being saturated by large files first. We will compare LiteSpeed, Apache and Nginx with different configurations on serving small static files.

Software version: LiteSpeed 4.1RC2, Apache 2.2.14, Nginx 0.7.61.

Test Result

Round Apache
(worker)
Apache
(event)
Apache
(prefork)
Nginx
(1 worker)
LiteSpeed
(1 CPU)
Non-Keep-Alive
1 2570.63 3192.56 3342.65 6616.52 9217.25
2 2555.7 3150.34 3275.96 7098.36 9474.72
3 2562.29 3159.37 3309.69 9609.46 9526.48
4 2534.73 3181.72 3392.23 6525.97 9550.62
Average 2555.84 3171.00 3330.13 7462.58 9442.27
Rating 100 124 130 292 369
Keep-Alive
1 4957.55 6571.35 6668.33 13862.73 27487.78
2 4609.48 6482.42 6595.71 13729.05 28333.26
3 4321.93 6709.53 6655.88 12319.22 27872.33
4 4384.76 6323.04 6633.4 13123.33 28788.51
Average 4568.43 6521.58 6638.33 13258.58 28120.47
Rating 100 143 145 290 616

Conclusion

LiteSpeed is apparently a big winner in this test. Apache worker and event model does not perform better than prefork model in serving small files. We are also surprised with LiteSpeed performance over Nginx, especially on keep-alive test.

Details

Server Software apache/2.2.14 (mpm prefork, worker, event), default setting + disable access log
nginx/0.7.61 (1 worker) default setting + disable access log
litespeed/4.1RC2 (1cpu license), default setting + disable access log
Server Hardware Dual Athlon MP 2000+/266FSB/256KB L2 Cache
Memory: 1GB PC2100
Motherboard: MSI K7D
Hard Drive: 36GB 1000RPM SCSI drive
Server OS Linux CentOS 5.4
Client Software ApacheBench (ab) 2.3
Client Host Hardware 2 XEON E5410 2.33GHz with 8GB memory
Motherboard: ASUS DSEB-DG with 4Gpbs LAN ports
Hard Drive: 4 SCSI 10Krpm RAID0
Network Switch Dell PowerConnect 2624 24-Port Gigabit Ethernet Switch
Client VM Linux 2.6.24-ovz-1.9-default #1 SMP
openvz VM with 1GB memory, bridged Network connection over 1Gbps NIC

Build apache with different model:

$ ./configure –prefix=/usr/local/apache-event –enable-mods-shared=all –with-mpm=event
$ ./configure –prefix=/usr/local/apache-event –enable-mods-shared=all –with-mpm=worker
$ ./configure –prefix=/usr/local/apache-event –enable-mods-shared=all –with-mpm=prefork

Test small html page

#cat test.html
<html>
<head>
<title>Test page</title>
</head>
<body>
<h1>This is a test page</h1>
</body>
</html>

Document Path: /test.html
Document Length: 101 bytes
Estimation target: Requests per second:[#/sec] (mean) from ab

Test Parameters

non-keep-alive:
nginx: ab -n 200000 -c 200 192.168.0.40:48130/test.html
litespeed: ab -n 200000 -c 200 192.168.0.40:48088/test.html
mpm prefork: ab -n 200000 -c 200 192.168.0.40:48124/test.html
mpm worker: ab -n 200000 -c 200 192.168.0.40:48120/test.html
mpm event: ab -n 200000 -c 200 192.168.0.40:48122/test.html

keep-alive:
nginx: ab -k -n 200000 -c 200 192.168.0.40:48130/test.html
litespeed: ab -k -n 200000 -c 200 192.168.0.40:48088/test.html
mpm prefork: ab -k -n 200000 -c 200 192.168.0.40:48124/test.html
mpm worker: ab -k -n 200000 -c 200 192.168.0.40:48120/test.html
mpm event: ab -k -n 200000 -c 200 192.168.0.40:48122/test.html

Note: aio is not used in this test. For a small benchmark file like this, aio will actually introduce extra function call / cpu cycle which does not help, as the file is already in memory (kernel i/o buffer).

Tags: , , , , ,

Comments are closed.