We haven’t done benchmarks since LSWS 2.0 due to various reasons… People may discount what we say because this is a vendor version… The main reason is that we are really busy with other high-priority things, like new feature enhancement, bug fixes and support… We treat every user’s feedback seriously. So to answer those people who request new benchmark, YES, we know it is OVERDUE, and it is on our TODO list after 4.1 stable released.
To perform a fair benchmark requiring a lot of preparation, as you need to know the detail of each product and set appropriate configuration. Recently, we got chance to do some internal testing. It’s experimental, as all tests were run within 1 box. We bind LSWS to 1 CPU, and leave another CPU to ab. The result here is good enough to make some points and may be useful to our users, so we would like to share in a series of blog posts first. Feel free to provide any feedback, so we can improve on our formal benchmark test later on.
In official site of Ruby Enterprise Edition, it compares REE with LiteSpeed 3.3.11 standard version (4-25-2008 released). According to the comparison, LiteSpeed is highest on memory usage. That test was flawed because each Ruby process occupies a lot of memory, and the default setting in LiteSpeed spawns 10 Ruby processes which are higher than that in other test cases. So the memory usage graph is basically illustrated how much memory Ruby processes used. Also in order to reduce the memory in real-time, the default setting also tries to destroy Ruby process whenever it is idle for a short time. This is not good for benchmark testing, as the server has to spend extra CPU in creating/destroying Ruby processes as this is not real-world scenario.
However, what we are interested here is “How about LiteSpeed + Ruby Enterprise Edition?”
Test method is based on http://www.rubyenterpriseedition.com/comparisons.html
1. REE Saves Memory
| LiteSpeed + Ruby Enterprise |
165M |
| LiteSpeed + regular Ruby |
264M |
| Memory Saving with REE |
(264 – 165) / 264 = 37.5% |
This is close to “33% less memory on average” claimed by REE web site.
2. REE’s Copy-on-write Works with LiteSpeed
In order to test this, we manually enable/disable copy-on-write function by modifying lsws/fcgi-bin/RailsRunner.rb:
GC.copy_on_write_friendly = true|false
| LiteSpeed + REE with Copy-on-Write |
165M |
| LiteSpeed + REE without Copy-on-Write |
234M |
| Memory Saving with Copy-on-Write |
(234 – 165) / 234 = 29~30% |
Conclusion
Based on above simple tests, we recommend using REE with LiteSpeed.
Next, we’ll touch on another interesting topic “LiteSpeed vs Apache + Passenger vs Nginx + Passenger”.
Read the rest of this entry »
