After reading Scoundrel's blog "Looking for Optimal Solution: Benchmark Results Summary and Findings", I am pretty surprised that LSWS + LSAPI falls behind. The best performer in his tests is nginx + FCGI. "All right, let me spend some time benchmarking them." I said to myself.

Here is our test environment:

Server: Dell PowerEdge SC1425

    CPU: Dual Intel Xeon 3.0GHz/800MHz FSB/2MB L2 Cache
    Memory: 1024MB DDR2 400MHz(2X512MB) ECC DIMM
    Hard Drive: 80GB 7200RPM SATA drive
    NIC: on board Intel PRO/1000 Gigabit Adapter
    Cent OS: CentOS 4.2

Client: Custom Built

    Motherboard: MSI K7D
    CPU: Dual Athlon MP 2000+/266FSB/256KB L2 Cache
    Memory: 1GB PC2100
    Hard Drive: 36GB 1000RPM SCSI drive
    NIC: Intel PRO/1000 Gigabit
    OS: RedHat 8.0/Kernel 2.4.22smp

Benchmark Tool: ApacheBench (ab) 1.3.33

We found that ab in Apache 2.x is not as fast as that in Apache 1.3.x release.

On the server we installed the latest software packages, they are: Ruby: 1.8.5 Rails: 1.1.6 Ruby-LSAPI: 1.6 Ruby FCGI: 0.8.7 nginx: 0.4.0 LSWS Enterprise: 2.2

We created a simple "Hello world" application like Scoundrel did, the URL is like http://192.168.0.60:/hello We use our easy configuration approach to configure the Rails application with 4 LSAPI worker processes, took me about 1 minute, fire up ab -n -c http://192.168.0.60:8080/hello

Got:

    Server Software:        LiteSpeed
    Server Hostname:        192.168.0.60
    Server Port:            8080

    Document Path:          /hello
    Document Length:        47 bytes

    Concurrency Level:      100
    Time taken for tests:   26.667 seconds
    Complete requests:      10000
    Failed requests:        0
    Broken pipe errors:     0
    Total transferred:      2650360 bytes
    HTML transferred:       470000 bytes
    Requests per second:    375.00 [#/sec] (mean)
    Time per request:       266.67 [ms] (mean)
    Time per request:       2.67 [ms] (mean, across all concurrent requests)
    Transfer rate:          99.39 [Kbytes/sec] received

    Connnection Times (ms)
          min  mean[+/-sd] median   max
    Connect:        0     0    0.1      0     3
    Processing:    35   265   90.9    246  1730
    Waiting:       31   265   90.9    246  1730
    Total:         35   265   90.9    246  1730

    Percentage of the requests served within a certain time (ms)
      50%    246
      66%    257
      75%    268
      80%    275
      90%    300
      95%    392
      98%    658
      99%    696
     100%   1730 (last request)

Tried a couple times, the result is pretty consistent, all are around 370 rps.

OK, it is nginx's turn. compile, install and start it. Good, the index page is accessible, it works. Next is to figure out how to configure the Rails FCGI. After digging on the internet for a quite while, finally made it work.

Fist, I need start Rails dispatch.fcgi manually with a FCGI spawning tool, I found that only "cgi-fcgi" coming with FCGI SDK can spawn multiple FCGI instances sharing the same listening socket, which is required for nginx, for this test. I use command

   cgi-fcgi -start -connect 127.0.0.1:4001 ./dispatch.fcgi 4

to start 4 instances of dispatch.fcgi, all listen on the same server socket. OK, dispatch.fcgi started. Before start dispatch.fcgi, I remember to do "export RAILS_ENV=production" to make sure dispatch.fcgi will run in production mode.

What about nginx configured. After some digging, I end up with adding following configuration to the default configuration to make it work.

    location /hello {
        fastcgi_pass   127.0.0.1:4001;
        include        conf/fastcgi_params;
    }

Clear the session cache directory then fire up: ab -n -c http://192.168.0.60:8080/hello

Got

    Server Software:        nginx/0.4.0
    Server Hostname:        192.168.0.60
    Server Port:            80

    Document Path:          /hello
    Document Length:        47 bytes

    Concurrency Level:      100
    Time taken for tests:   35.416 seconds
    Complete requests:      10000
    Failed requests:        0
    Broken pipe errors:     0
    Total transferred:      2590000 bytes
    HTML transferred:       470000 bytes
    Requests per second:    282.36 [#/sec] (mean)
    Time per request:       354.16 [ms] (mean)
    Time per request:       3.54 [ms] (mean, across all concurrent requests)
    Transfer rate:          73.13 [Kbytes/sec] received

    Connnection Times (ms)
                  min  mean[+/-sd] median   max
    Connect:        0     0    0.1      0     3
    Processing:     8   203 1677.3     20 32021
    Waiting:        8   203 1677.2     20 32021
    Total:          8   203 1677.4     20 32023

    Percentage of the requests served within a certain time (ms)
      50%     20
      66%     23
      75%     26
      80%     27
      90%     31
      95%     43
      98%   1170
      99%   5857
     100%  32023 (last request)

Tried a couple times, the results is not very consistent, falls in the range of 250-330 rps.

OK, let's try UNIX domain socket hoping for better performance, fire up another group of dispatch.fcgi listening at /tmp/rails.sock, change nginx configuration to

    location /hello {
        fastcgi_pass   unix:/tmp/rails.sock;
        include        conf/fastcgi_params;
    }

Restart nginx, and run 'ab'

    Server Software:        nginx/0.4.0
    Server Hostname:        192.168.0.60
    Server Port:            80

    Document Path:          /hello
    Document Length:        383 bytes

    Concurrency Level:      100
    Time taken for tests:   17.883 seconds
    Complete requests:      10000
    Failed requests:        5835
       (Connect: 0, Length: 5835, Exceptions: 0)
    Broken pipe errors:     0
    Non-2xx responses:      4165
    Total transferred:      3735375 bytes
    HTML transferred:       1869440 bytes
    Requests per second:    559.19 [#/sec] (mean)
    Time per request:       178.83 [ms] (mean)
    Time per request:       1.79 [ms] (mean, across all concurrent requests)
    Transfer rate:          208.88 [Kbytes/sec] received

    Connnection Times (ms)
                  min  mean[+/-sd] median   max
    Connect:        0     0    0.2      0    10
    Processing:     5   177  135.9    257  1632
    Waiting:        1   177  136.0    257  1631
    Total:          5   177  135.8    258  1632

    Percentage of the requests served within a certain time (ms)
      50%    258
      66%    280
      75%    295
      80%    302
      90%    327
      95%    345
      98%    372
      99%    382
     100%   1632 (last request)

It looks good at first glance, but when you take a closer look, you will find that there are many failed requests. Tried a couple times, results are similar.

My conclusion is that using UNIX domain socket with nginx is not reliable at all. I think it is because nginx does not pipeline requests like what LSWS does, but attempts to make 100 concurrent connections to upstream, while upstream is only capable of handling 4 concurrent connections when UNIX domain socket is used. Using TCP socket is more reliable as TCP server socket backlog can buffer a few pending connections, but I bet problem will occur when concurrent level is higher than the backlog buffer.

You may think I am biased, may be, because I knew our product inside out and LSWS has been properly tuned for sure, but for nginx, I can only use the default setting without much tuning.

All right, so much for the benchmarking. It is not possible to have a exactly same test environment and configuration for everyone who does benchmark tests, the result may vary dramatically. So, it is better to do your own benchmark before making decisions, I only trust my own results.

Trac SCM/Project Management software is one of the most popular Python-based application available today. Many open-source projects are hosting their developement source, bug tracker, and wiki all using a single Trac install. It goes to show how much feature has been crammed into Trac and how well integrated it is with its sub-features.

Frankly, it's pretty amazing to see how many Trac users there are currently. A year ago, I saw a few Trac installations. Today, it would be strange not to see a Trac install for every 5 software project I come accross.

I just posted a new Trac SCM + FastCGI + LiteSpeed how-to for all the Trac lovers out there and to also illustrate how easy it is to integrate any FastCGI application with LiteSpeed. Trac SCM has been added to our live demos.

Lastly, I have also updated many of the configuration for our LiteSpeed Live Demos. The configurations have been simplified/streamlined, when applicable, so new users get a sense of the shortest-way possible of integrating their their apps with LiteSpeed web server.

We here at LiteSpeed feels like it's about time for a company blog where we can muse about internal/external affairs and add give us another way to reach out to our growing community.

The blog is hosted on Typo, one of the most rounded Ruby on Rails application. Though php-based WordPress is a favorite of ours and we have a great relationship with them, we really wanted a Rails platform for this blog.

Just a few days ago we released a very important 2.2 release of our core product, the LiteSpeed Web Server. Quite a bit of work was put into the 2.2 release by everyone on the team and especially George, the company founder and lead developer.

With LiteSpeed Web Server 2.2, Ruby on Rails has never been easier. It's just a matter of a few clicks within our web admin gui to setup your first full-production ready Rails app.

Easy Rails Wiki How-To