LiteSpeed + LSAPI vs. Apache + Passenger vs. Nginx + Passenger

To perform a fair benchmark requiring a lot of preparation, as you need to know the detail of each product and set appropriate configuration. Recently, we got chance to do some internal testing. It’s experimental, as all tests were run within 1 box. We bind LSWS to 1 CPU, and leave another CPU to ab. The result here is good enough to make some points and may be useful to our users, so we would like to share in a series of blog posts first. Feel free to provide any feedback, so we can improve in our formal benchmark test later on.

In our last test, we studied the memory consumption of Ruby vs Ruby Enterprise Edition using Typo as a test case. Since Typo itself is a bottleneck when cache is disabled, performance among different web servers cannot be revealed.

This naturally leads to our next interested topic “Comparing the performance of the interface between web server and Ruby engine”.

This time, we focus on performance of the interface between web server and Ruby engine by using the simplest “Hello World” without database. We tested 3 configurations, all based on Ruby Enterprise 1.8.7: LiteSpeed/4.1RC2 + REE, Apache/2.2.14 (worker MPM) + REE + Phusion Passenger and Nginx/0.7.61  + REE + Phusion Passenger.

Test method is still based on http://www.rubyenterpriseedition.com/comparisons.html

Software Version: LSWS 4.1RC2, Nginx/0.7.61, Apache/2.2.14(Worker MPM), Ruby Enterprise Edition 1.8.7, Rails 2.2, Passenger 2.2.5

Test Result

Configuration Req/Sec Memory Usage
LiteSpeed + REE + LSAPI 280 146M ~ 158M
Nginx + REE + Passenger 212 153M
Apache(worker MPM) + REE + Passenger 200 178M

Conclusion

Based on above simple tests, LiteSpeed + REE + LSAPI are the preferred configuration.


Details

Client Software ApacheBench (ab) 2.3
Server Hardware Dual Athlon MP 2000+/266FSB/256KB L2 Cache
Memory: 1GB PC2100
Motherboard: MSI K7D
Hard Drive: 36GB 1000RPM SCSI drive
Server OS Linux CentOS 5.4
This test focuses on performance, so the simplest “Hello World” without database is used.
LSWS 4.1RC2 – Bind listener to 1 processor, disable cache. (equivalent to 1cpu license)

LiteSpeed + REE + LSAPI

LSWS 4.1RC2 + Ruby Enterprise 1.8.7 + Rails 2.2 + Hello
(Server–>Ruby Rails–>Ruby Path:/usr/local/ruby-enterprise-1.8.7-2009.10/bin/ruby)
Rails Context max connections: 8
~>(date;free -m;ab -n 20000 -c 100 http://127.0.0.1:48089/hello;date;free -m) 2>&1|tee ttt;echo;egrep “(buffers/cache)|(Requests per second)” ttt

Round Free Memory (M)
Before
Free Memory (M)
After
Memory Usage
( Before – After)
Request/Second
[#/sec](mean)
1 876 730 146M 281.40
2 877 734 143M 281.59
3 877 720 157M 267.05
4 877 719 158M 279.84

Test again: restart LiteSpeed after each round of test

Round Free Memory (M)
Before
Free Memory (M)
After
Memory Usage
( Before – After)
Request/Second
[#/sec](mean)
1 873 772 101M 281.63
2 872 747 125M 227.86
3 872 714 158M 280.69
4 872 714 158M 281.00

Nginx + REE + Passenger


Nginx/0.7.61 + Ruby Enterprise 1.8.7 + Passenger 2.2.5 + Rails 2.2 + Hello
test script:
~>(date;free -m;ab -n 20000 -c 100 127.0.0.1:48130/hello;date;free -m)2>&1|tee ttt;echo;egrep “(buffers/cache)|(Requests per second)” ttt
To stop Nginx: /usr/local/nginx-passenger/sbin>./nginx -s stop
To start Nginx: /usr/local/nginx-passenger/sbin>./nginx

    Notes:

  1. There are totally 8 ruby processes running during the test by checking “top” and “pstree”. That is why we set LiteSpeed rails context max connections to 8 to compare.
  2. Nginx+REE+Passenger won’t free memory immediately after “ab” testing, unlike LiteSpeed which frees memory immediately after “ab” finishes. Therefore Nginx has to be restarted after each round of test.
Round Free Memory (M)
Before
Free Memory (M)
After
Memory Usage
( Before – After)
Request/Second
[#/sec](mean)
1 869 715 154M 211.47
2 868 715 153M 211.84
3 868 715 153M 213.39

Apache + REE + Passenger


Apache/2.2.14(Worker MPM) + Ruby Enterprise 1.8.7 + Passenger 2.2.5 + Rails 2.2 + Hello
(date;free -m;ab -n 20000 -c 100 127.0.0.1:48120/hello;date;free -m)2>&1|tee ttt;echo;egrep “(buffers/cache)|(Requests per second)” ttt
To restart Apache: /usr/local/apache-worher/bin>./apachectl restart

    Notes:

  • Same as Nginx, Apache needs to be restarted after each round of test.
Round Free Memory (M)
Before
Free Memory (M)
After
Memory Usage
( Before – After)
Request/Second
[#/sec](mean)
1 861 681 180M 201.12
2 860 682 153M 199.88
3 861 683 178M 200.73

Tags: , , , , , , , , , ,

Comments are closed.