LiteSpeed + LSAPI vs. Apache + Passenger vs. Nginx + Passenger

To perform a fair benchmark requiring a lot of preparation, as you need to know the detail of each product and set appropriate configuration. Recently, we got chance to do some internal testing. It’s experimental, as all tests were run within 1 box. We bind LSWS to 1 CPU, and leave another CPU to ab. The result here is good enough to make some points and may be useful to our users, so we would like to share in a series of blog posts first. Feel free to provide any feedback, so we can improve in our formal benchmark test later on.

In our last test, we studied the memory consumption of Ruby vs Ruby Enterprise Edition using Typo as a test case. Since Typo itself is a bottleneck when cache is disabled, performance among different web servers cannot be revealed.

This naturally leads to our next interested topic “Comparing the performance of the interface between web server and Ruby engine”.

This time, we focus on performance of the interface between web server and Ruby engine by using the simplest “Hello World” without database. We tested 3 configurations, all based on Ruby Enterprise 1.8.7: LiteSpeed/4.1RC2 + REE, Apache/2.2.14 (worker MPM) + REE + Phusion Passenger and Nginx/0.7.61  + REE + Phusion Passenger.

Test method is still based on http://www.rubyenterpriseedition.com/comparisons.html

Software Version: LSWS 4.1RC2, Nginx/0.7.61, Apache/2.2.14(Worker MPM), Ruby Enterprise Edition 1.8.7, Rails 2.2, Passenger 2.2.5

Test Result

Conclusion

Based on above simple tests, LiteSpeed + REE + LSAPI are the preferred configuration.

Details

LiteSpeed + REE + LSAPI

LSWS 4.1RC2 + Ruby Enterprise 1.8.7 + Rails 2.2 + Hello
(Server–>Ruby Rails–>Ruby Path:/usr/local/ruby-enterprise-1.8.7-2009.10/bin/ruby)
Rails Context max connections: 8
~>(date;free -m;ab -n 20000 -c 100 http://127.0.0.1:48089/hello;date;free -m) 2>&1|tee ttt;echo;egrep “(buffers/cache)|(Requests per second)” ttt

Test again: restart LiteSpeed after each round of test

Nginx + REE + Passenger

Nginx/0.7.61 + Ruby Enterprise 1.8.7 + Passenger 2.2.5 + Rails 2.2 + Hello
test script:
~>(date;free -m;ab -n 20000 -c 100 127.0.0.1:48130/hello;date;free -m)2>&1|tee ttt;echo;egrep “(buffers/cache)|(Requests per second)” ttt
To stop Nginx: /usr/local/nginx-passenger/sbin>./nginx -s stop
To start Nginx: /usr/local/nginx-passenger/sbin>./nginx

Notes:
1. There are totally 8 ruby processes running during the test by checking “top” and “pstree”. That is why we set LiteSpeed rails context max connections to 8 to compare.
2. Nginx+REE+Passenger won’t free memory immediately after “ab” testing, unlike LiteSpeed which frees memory immediately after “ab” finishes. Therefore Nginx has to be restarted after each round of test.

Apache + REE + Passenger

Apache/2.2.14(Worker MPM) + Ruby Enterprise 1.8.7 + Passenger 2.2.5 + Rails 2.2 + Hello
(date;free -m;ab -n 20000 -c 100 127.0.0.1:48120/hello;date;free -m)2>&1|tee ttt;echo;egrep “(buffers/cache)|(Requests per second)” ttt
To restart Apache: /usr/local/apache-worher/bin>./apachectl restart

Notes:
• Same as Nginx, Apache needs to be restarted after each round of test.