LiteSpeed + LSAPI vs. Apache + Passenger vs. Nginx + Passenger

December 16th, 2009 by Benchmarks , LiteSpeed Web Server , Performance Comments Off on LiteSpeed + LSAPI vs. Apache + Passenger vs. Nginx + Passenger

To perform a fair benchmark requiring a lot of preparation, as you need to know the detail of each product and set appropriate configuration. Recently, we got chance to do some internal testing. It’s experimental, as all tests were run within 1 box. We bind LSWS to 1 CPU, and leave another CPU to ab. The result here is good enough to make some points and may be useful to our users, so we would like to share in a series of blog posts first. Feel free to provide any feedback, so we can improve in our formal benchmark test later on.

In our last test, we studied the memory consumption of Ruby vs Ruby Enterprise Edition using Typo as a test case. Since Typo itself is a bottleneck when cache is disabled, performance among different web servers cannot be revealed.

This naturally leads to our next interested topic “Comparing the performance of the interface between web server and Ruby engine”.

This time, we focus on performance of the interface between web server and Ruby engine by using the simplest “Hello World” without database. We tested 3 configurations, all based on Ruby Enterprise 1.8.7: LiteSpeed/4.1RC2 + REE, Apache/2.2.14 (worker MPM) + REE + Phusion Passenger and Nginx/0.7.61  + REE + Phusion Passenger.

Test method is still based on http://www.rubyenterpriseedition.com/comparisons.html

Software Version: LSWS 4.1RC2, Nginx/0.7.61, Apache/2.2.14(Worker MPM), Ruby Enterprise Edition 1.8.7, Rails 2.2, Passenger 2.2.5

Test Result

ConfigurationReq/SecMemory Usage
LiteSpeed + REE + LSAPI280146M ~ 158M
Nginx + REE + Passenger212153M
Apache(worker MPM) + REE + Passenger200178M

Conclusion

Based on above simple tests, LiteSpeed + REE + LSAPI are the preferred configuration.


Details

Client SoftwareApacheBench (ab) 2.3
Server HardwareDual Athlon MP 2000+/266FSB/256KB L2 Cache
Memory: 1GB PC2100
Motherboard: MSI K7D
Hard Drive: 36GB 1000RPM SCSI drive
Server OSLinux CentOS 5.4
This test focuses on performance, so the simplest “Hello World” without database is used.
LSWS 4.1RC2 – Bind listener to 1 processor, disable cache. (equivalent to 1cpu license)

LiteSpeed + REE + LSAPI

LSWS 4.1RC2 + Ruby Enterprise 1.8.7 + Rails 2.2 + Hello
(Server–>Ruby Rails–>Ruby Path:/usr/local/ruby-enterprise-1.8.7-2009.10/bin/ruby)
Rails Context max connections: 8
~>(date;free -m;ab -n 20000 -c 100 http://127.0.0.1:48089/hello;date;free -m) 2>&1|tee ttt;echo;egrep “(buffers/cache)|(Requests per second)” ttt

RoundFree Memory (M)
Before
Free Memory (M)
After
Memory Usage
( Before – After)
Request/Second
[#/sec](mean)
1876730146M281.40
2877734143M281.59
3877720157M267.05
4877719158M279.84

Test again: restart LiteSpeed after each round of test

RoundFree Memory (M)
Before
Free Memory (M)
After
Memory Usage
( Before – After)
Request/Second
[#/sec](mean)
1873772101M281.63
2872747125M227.86
3872714158M280.69
4872714158M281.00

Nginx + REE + Passenger


Nginx/0.7.61 + Ruby Enterprise 1.8.7 + Passenger 2.2.5 + Rails 2.2 + Hello
test script:
~>(date;free -m;ab -n 20000 -c 100 127.0.0.1:48130/hello;date;free -m)2>&1|tee ttt;echo;egrep “(buffers/cache)|(Requests per second)” ttt
To stop Nginx: /usr/local/nginx-passenger/sbin>./nginx -s stop
To start Nginx: /usr/local/nginx-passenger/sbin>./nginx

    Notes:

  1. There are totally 8 ruby processes running during the test by checking “top” and “pstree”. That is why we set LiteSpeed rails context max connections to 8 to compare.
  2. Nginx+REE+Passenger won’t free memory immediately after “ab” testing, unlike LiteSpeed which frees memory immediately after “ab” finishes. Therefore Nginx has to be restarted after each round of test.
RoundFree Memory (M)
Before
Free Memory (M)
After
Memory Usage
( Before – After)
Request/Second
[#/sec](mean)
1869715154M211.47
2868715153M211.84
3868715153M213.39

Apache + REE + Passenger


Apache/2.2.14(Worker MPM) + Ruby Enterprise 1.8.7 + Passenger 2.2.5 + Rails 2.2 + Hello
(date;free -m;ab -n 20000 -c 100 127.0.0.1:48120/hello;date;free -m)2>&1|tee ttt;echo;egrep “(buffers/cache)|(Requests per second)” ttt
To restart Apache: /usr/local/apache-worher/bin>./apachectl restart

    Notes:

  • Same as Nginx, Apache needs to be restarted after each round of test.
RoundFree Memory (M)
Before
Free Memory (M)
After
Memory Usage
( Before – After)
Request/Second
[#/sec](mean)
1861681180M201.12
2860682153M199.88
3861683178M200.73


Related Posts


Comments