PHP 7 vs HHVM Benchmark Series 2: WordPress

Based on the results of the Hello World Benchmark Test, we now know that LiteSpeed and OpenLiteSpeed have the smallest IPC overhead and provides the best performance for PHP 7 and HHVM of the web servers tested. We will be using OpenLiteSpeed to do the 2nd benchmark test where we will be targeting a sample WordPress site to see the performance difference. We know from our first test that PHP 7 is 140% faster than HHVM on hello world. How about when using WordPress?

This benchmark was conducted using the same server as in the first one.

Server configuration:

Typical VPS Server in an OpenVZ container
Running CentOS 7
2 cores of Intel(R) Xeon(R) CPU E5-1620 @ 3.60GHz
4G RAM
OpenLiteSpeed 1.4.8

HHVM – 3.8.0
PHP 7.0.0 – Git latest version dated 07/24/2015
PHP 5.6 – 5.6.11

Updates:

As indicated from the HHVM blog, HHVM 3.8.0 was finally made available in mid-July and ran up to 13.9% faster than HHVM 3.7.0 ( http://hhvm.com/blog/9803/hhvm-3-8-0 ). To be fair, we definitely did not want to miss this performance improvement and have accordingly updated the existing HHVM 3.7.1-Dev to 3.8.0-Dev. We are also happy to see, after this upgrade, a more detailed phpinfo page for HHVM which previously only showed the text “HipHop”.

We also upgraded lsphp7 to the latest git version dated 07/24/2015 and lsphp56 to 5.6.11.

PHP 7 and PHP 56 have Zend Opcode cache enabled.

The HHVM OSS performance benchmark suite (https://github.com/hhvm/oss-performance) was used as a guideline for this testing. Of Course we have replaced Nginx with OpenLiteSpeed in this scenario. PHP 7 and PHP 5.6 communicate with OpenLiteSpeed through LSAPI while HHVM uses FastCGI over a Unix socket instead of a TCP socket.

WordPress setup:

WordPress files were unzipped from https://github.com/hhvm/oss-performance/tree/master/targets/wordpress/wordpress-4.2.0.tar.gz without installation. wp-config.php was copied to the WordPress root folder. dbdump.sql was imported to the database. Listen port 8092 must be configured on OpenLiteSpeed to point to the WordPress Virtual Host location. You may also need to change the open file limit to clear a siege error. Something like the following:

ulimit -n 100000

Benchmark tool:
Siege was used this time instead of ab, to closer align with the OSS Performance project. Siege was run from a separate CentOS 6.6 server. Since there were reports that Siege 3.x was buggy, Siege 2.78 was built for this test.

We use siege to simulate 10 concurrent users with 100 reps. The WordPress url list from the OSS performance project has also been used with modest changes to remove bad URLs, with 140 lines of URLs still in the list. Benchmark command is as follows:

siege278 -b -q -c 10 -r 100 -f wordpress.urls

Result:
Without further ado, the results of this round of competition:

HHVM > PHP 7 >> PHP 56

HHVM 3.8.0-dev + OpenLiteSpeed 1.4.8

Transactions: 1000 hits
Availability: 100.00 %
Elapsed time: 10.76 secs
Data transferred: 5.07 MB
Response time: 0.11 secs
Transaction rate: 92.94 trans/sec
Throughput: 0.47 MB/sec
Concurrency: 9.92
Successful transactions: 1000
Failed transactions: 0
Longest transaction: 0.15
Shortest transaction: 0.03

PHP 7- Git latest version dated 07/24/2015 + ZendOpcache Enabled + OpenLiteSpeed 1.4.8

Transactions: 1000 hits
Availability: 100.00 %
Elapsed time: 11.54 secs
Data transferred: 11.42 MB
Response time: 0.11 secs
Transaction rate: 86.66 trans/sec
Throughput: 0.99 MB/sec
Concurrency: 9.88
Successful transactions: 1000
Failed transactions: 0
Longest transaction: 0.20
Shortest transaction: 0.02

PHP 56 – 5.6.11 + ZendOpcache Enabled + OpenLiteSpeed 1.4.8

Transactions: 1000 hits
Availability: 100.00 %
Elapsed time: 27.83 secs
Data transferred: 11.42 MB
Response time: 0.28 secs
Transaction rate: 35.93 trans/sec
Throughput: 0.41 MB/sec
Concurrency: 9.94
Successful transactions: 1000
Failed transactions: 0
Longest transaction: 0.51
Shortest transaction: 0.07

Highlights and Conclusions

• For the WordPress test, HHVM beats PHP 7, running up to 7% faster.
• If we recall HHVM’s reported lockdown results, HHVM claimed to be 18.7% faster on a WordPress workload. We can endorse HHVM’s faster performance; however, we did not experience as much as 18%.
• PHP 7 claims to be up to two times faster than PHP 5.6, which is true according to our test. In this benchmark, PHP 7 runs about 2.4 times faster than PHP 5.6, while HHVM runs about 2.6 times faster than PHP 5.6.

For our second test in the PHP 7 vs HHVM Benchmarking series, we see HHVM beating PHP 7 on WordPress, but what about other applications? Will HHVM still come out on top? Stay with us as the next round of benchmarking is coming soon.

What do you think about this test? What would you like to see next? If you have any feedback/comments/recommendations, we are keen to hear from you. Shoot us an email at info@litespeedtech.com.