Wickedly Fast WordPress on SmartOS with Varnish and nginx

Sometimes using WordPress CMS isn’t exactly a speedy experience, every page that is rendered hogs the CPU as it goes though the thousands of lines of code and performs multiple SQL database queries.  99% of the time, the content is unchanged so it makes a lot more sense to serve that content statically – directly out of RAM.

Enter our “hero” Varnish Cache, a web application accelerator also known as a caching HTTP reverse proxy. You simply install it in front of any web services that talk HTTP and configure it to cache the contents. This activity really makes WordPress fly.

In fact, Varnish makes WordPress response times outrageously fast. So fast, that speed increases in the order of 300–1000X can typically be expected. Since your content is served statically out of RAM, your CPU remains mainly idle. This setup will let you serve hundreds of thousands of users per day effortlessly and turn your existing rig into a lean, mean web-serving machine.

5000 Pages served per second with almost no cpu load

Almost 2 Million pages served in 6 minutes

It is undeniable fact that millions of high traffic blogs and company websites are built using WordPress. This is because WordPress is easy to use and maintain, the codebase is very mature and if you use a recommended framework such as the superb “Genesis Framework“, you can effortlessly tick all the necessary boxes when it comes to SEO best practice including page optimisation and social outreach.

Nowadays, we are all overloaded with web content that is driven at us from all angles and its literally “screaming for our attention”. As a consequence, the common web user has become so impatient, that if the desired web page has not loaded by the time we blink, we move on.

Lets face it, we spend a huge amount of time, painstakingly writing great content for our audiences.  But, if you get your WordPress setup wrong, you’re in for some real pain. You will get penalised by both users and search engines caused by slow page loads, crappy plugins, congested databases, security breaches and ultimately the inability to serve high traffic loads when you really need to.

But, get it right and…. “shazam!”  Your snappy, secure and responsive website should survive the dreaded “slashdot effect” without a hiccup.

In the following guide, I will demonstrate how to set up a blazingly fast WordPress site using a SmartOS zone with minimal resources to effortlessly handle extremely high traffic loads and without breaking a sweat.

Although this guide is specifically tuned for WordPress on SmartOS, most of the optimisation points and concepts discussed here are applicable to other operating systems and web environments.

We will be using:

• SmartOS base64 13.3.1 dataset
• 1024 MB Ram for our VPS.
• Percona MySQL for the database.
• nginx for the web server.
• PHP 5.5 with its built in Zend OPcache for PHP script cache.
• Varnish Cache as a in memory web application accelerator.
• WordPress 3.8.1 + Demo content for testing

No additional WordPress cache plugins are required and won’t be used .  The best rule of thumb for WordPress configuration is to reduce bloat and keep your plugins to a bare minimum.

If we are effectively handling our caching at the server level where it belongs, lets not install any additional caching plugins in WordPress itself as it’s simply not necessary and just adds to bloat.

Varnish-Cache is an amazing beast! as you will soon see. In our tutorial for simplicity sake, we are running everything in a single zone/vps. In production environments it would make more sense to have a single varnish zone instance in front of all your WordPress sites.

The advantage of the setup below,  is that there are no external dependencies and everything is maintained within the zone itself.

Ok, you are probably thinking, hey dude enough yapping its time to get crackin! So, lets get going with the actual setup steps.

Package Installation

pkgin -fy up
pkgin in percona-server-5.6 nginx php-5.5 php55-mysqli php55-mcrypt php55-mbstring unzip php55-zlib php55-gd php55-zip php55-gd php55-curl php55-imap php55-json php55-mysql php55-pdo_mysql php55-bcmath php55-fpm nano php55-dom php55-xmlrpc php55-opcache varnish

Notice that GCC is installed as a varnish dependency, this is because Varnish compiles its VCL configuration file on start-up. Once all packages have been installed we move on to the nginx configuration.

nginx Configuration

Here we will be setting up nginx to run our WordPress site as a virtual host. We setup nginx to listen on port 8080 and later we will be configuring varnish to listen on port 80 and pass traffic to nginx on 8080.

mkdir -p /opt/local/www/mysite.net
mkdir /opt/local/etc/nginx/sites-available
mkdir /opt/local/etc/nginx/sites-enabled
vi /opt/local/etc/nginx/nginx.conf
vi /opt/local/etc/nginx/sites-available/mysite.net.conf
cd /opt/local/etc/nginx/sites-enabled/
ln -s /opt/local/etc/nginx/sites-available/mysite.net.conf
nginx -t
vi /opt/local/etc/php-fpm.conf
mkdir /opt/local/www/mysite.net
svcadm enable nginx
svcadm enable php-fpm

nginx.conf

user www www;
worker_processes  1;

events {
    worker_connections 768;
    multi_accept on;
}


http {

# Let NGINX get the real client IP for its access logs
    set_real_ip_from 127.0.0.1;
    real_ip_header X-Forwarded-For;

    # Basic Settings
    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;
    keepalive_timeout 20;
    client_max_body_size 15m;
    client_body_timeout 60;
    client_header_timeout 60;
    client_body_buffer_size  1K;
    client_header_buffer_size 1k;
    large_client_header_buffers 4 8k;
    send_timeout 60;
    reset_timedout_connection on;
    types_hash_max_size 2048;
    server_tokens off;

    include /opt/local/etc/nginx/mime.types;
    default_type application/octet-stream;

# Logging Settings
    # access_log /var/log/nginx/access.log;
    error_log /var/log/nginx/error.log;


# Log Format
    log_format main '$remote_addr - $remote_user [$time_local] '
    '"$request" $status $body_bytes_sent "$http_referer" '
    '"$http_user_agent" "$http_x_forwarded_for"';


# Gzip Settings
    gzip on;
    gzip_disable "msie6";
    gzip_vary on;
    gzip_proxied any;
    gzip_comp_level 6;
    gzip_min_length 512;
    gzip_buffers 16 8k;
    gzip_http_version 1.1;
    gzip_types text/css text/javascript text/xml text/plain text/x-component
    application/javascript application/x-javascript application/json
    application/xml  application/rss+xml font/truetype application/x-font-ttf
    font/opentype application/vnd.ms-fontobject image/svg+xml;


#   include /usr/local/etc/nginx/vhosts/*;
  include /opt/local/etc/nginx/sites-enabled/*;
}

mysite.net.conf

server {
    # Default server block blacklisting all unconfigured access
    listen 8080 default_server;
    server_name _;
    return 444;
}

server {
    # Configure the domain that will run WordPress
    server_name  mysite.net;
    listen 8080 ;
    port_in_redirect off;
    access_log  /var/log/nginx/mysite.net-access.log;
    error_log  /var/log/nginx/mysite.net-error.log;
    server_tokens off;
    autoindex off;

    client_max_body_size 15m;
    client_body_buffer_size 128k;

    # WordPress needs to be in the webroot of /var/www/ in this case
    root   /opt/local/www/mysite.net;
    index index.html index.htm index.php;
    try_files $uri $uri/ /index.php?q=$uri&$args;

    # Define default caching of 24h
    expires 86400s;
    add_header Pragma public;
    add_header Cache-Control "max-age=86400, public, must-revalidate, proxy-revalidate";

    # deliver a static 404
    error_page 404 /404.html;
    location  /404.html {
        internal;
    }

    # Deliver 404 instead of 403 "Forbidden"
    error_page 403 = 404;

    # Do not allow access to files giving away your WordPress version
    location ~ /(\.|wp-config.php|readme.html|license.txt) {
        return 404;
    }

    # Add trailing slash to */wp-admin requests.
    rewrite /wp-admin$ $scheme://$host$uri/ permanent;

    # Don't log robots.txt requests
    location = /robots.txt {
        allow all;
        log_not_found off;
        access_log off;
    }

    # Rewrite for versioned CSS+JS via filemtime
    location ~* ^.+\.(css|js)$ {
        rewrite ^(.+)\.(\d+)\.(css|js)$ $1.$3 last;
        expires 31536000s;
        access_log off;
        log_not_found off;
        add_header Pragma public;
        add_header Cache-Control "max-age=31536000, public";
    }

    # Aggressive caching for static files
    # If you alter static files often, please use
    # add_header Cache-Control "max-age=31536000, public, must-revalidate, proxy-revalidate";
    location ~* \.(asf|asx|wax|wmv|wmx|avi|bmp|class|divx|doc|docx|eot|exe|gif|gz|gzip|ico|jpg|jpeg|jpe|mdb|mid|midi|mov|qt|mp3|m4a|mp4|m4v|mpeg|mpg|mpe|mpp|odb|odc|odf|odg|odp|ods|odt|ogg|ogv|otf|pdf|png|pot|pps|ppt|pptx|ra|ram|svg|svgz|swf|tar|t?gz|tif|tiff|ttf|wav|webm|wma|woff|wri|xla|xls|xlsx|xlt|xlw|zip)$ {
        expires 31536000s;
        access_log off;
        log_not_found off;
        add_header Pragma public;
        add_header Cache-Control "max-age=31536000, public";
    }

    # pass PHP scripts to Fastcgi listening on Unix socket
    # Do not process them if inside WP uploads directory
    # If using Multisite or a custom uploads directory,
    # please set the */uploads/* directory in the regex below
    location ~* (^(?!(?:(?!(php|inc)).)*/uploads/).*?(php)) {
        try_files $uri = 404;
        fastcgi_split_path_info ^(.+.php)(.*)$;
        fastcgi_pass   127.0.0.1:9000;
                                #fastcgi_pass unix:/var/run/php-fpm.socket;
        fastcgi_index index.php;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        include fastcgi_params;
        fastcgi_intercept_errors on;
        fastcgi_ignore_client_abort off;
        fastcgi_connect_timeout 60;
        fastcgi_send_timeout 180;
        fastcgi_read_timeout 180;
        fastcgi_buffer_size 128k;
        fastcgi_buffers 4 256k;
        fastcgi_busy_buffers_size 256k;
        fastcgi_temp_file_write_size 256k;
    }

    # Deny access to hidden files
    location ~ /\. {
        deny all;
        access_log off;
        log_not_found off;
    }

}

# Redirect all www. queries to non-www
# Change in case your site is to be available at "www.yourdomain.tld"
server {
    listen 8080;
    server_name www.mysite.net;
    rewrite ^ $scheme://mysite.net$request_uri? permanent;
}

The below fpm config file is basically the default, you should however tune yours more aggressively to match your expected traffic loads.

php-fpm.conf

[global]
pid = run/php-fpm.pid
[www]
user = www
group = www
listen = 127.0.0.1:9000
pm = dynamic
pm.max_children = 5
pm.start_servers = 2
pm.min_spare_servers = 1
pm.max_spare_servers = 3

In php.ini we add some best practice php settings to lock down security and prevent some php exploits. In addition we enable Zend Opcache and add some tuning parameters. Opcache will compile and cache our PHP scripts without the need to execute them.

php.ini

[PHP]
engine = On
short_open_tag = Off
ignore_user_abort = Off
asp_tags = Off
precision = 14
y2k_compliance = On
output_buffering = 4096
zlib.output_compression = Off
implicit_flush = Off
unserialize_callback_func =
serialize_precision = 17
allow_call_time_pass_reference = Off
safe_mode = Off
safe_mode_gid = Off
safe_mode_include_dir =
safe_mode_exec_dir =
safe_mode_allowed_env_vars = PHP_
safe_mode_protected_env_vars = LD_LIBRARY_PATH
disable_functions = "apache_child_terminate, apache_setenv, define_syslog_variables, escapeshellarg, escapeshellcmd, eval, exec, fp, fput, ftp_connect, ftp_exec, ftp_get, ftp_login, ftp_nb_fput, ftp_put, ftp_raw, ftp_rawlist, highlight_file, ini_alter, ini_get_all, ini_restore, inject_code, mysql_pconnect, openlog, passthru, php_uname, phpAds_remoteInfo, phpAds_XmlRpc, phpAds_xmlrpcDecode, phpAds_xmlrpcEncode, popen, posix_getpwuid, posix_kill, posix_mkfifo, posix_setpgid, posix_setsid, posix_setuid, posix_setuid, posix_uname, proc_close, proc_get_status, proc_nice, proc_open, proc_terminate, shell_exec, syslog, system, xmlrpc_entity_decode"
disable_classes =
zend.enable_gc = On
expose_php = On
max_execution_time = 30
max_input_time = 60
memory_limit = 128M
error_reporting = E_ALL & ~E_DEPRECATED
display_errors = Off
display_startup_errors = Off
log_errors = On
log_errors_max_len = 1024
ignore_repeated_errors = Off
ignore_repeated_source = Off
report_memleaks = On
track_errors = Off
html_errors = Off
variables_order = "GPCS"
request_order = "GP"
register_globals = Off
register_long_arrays = Off
register_argc_argv = Off
auto_globals_jit = On
post_max_size = 20M
magic_quotes_gpc = Off
magic_quotes_runtime = Off
magic_quotes_sybase = Off
auto_prepend_file =
auto_append_file =
default_mimetype = "text/html"
include_path = ".:/opt/local/lib/php"
doc_root =
user_dir =
enable_dl = Off
file_uploads = On
upload_tmp_dir = /tmp
upload_max_filesize = 15M
default_charset = "UTF-8"
max_file_uploads = 20
allow_url_fopen = Off
allow_url_include = Off
default_socket_timeout = 30
extension=pdo.so
extension=mysqli.so
extension=mcrypt.so
extension=mbstring.so
extension=zlib.so
extension=gd.so
extension=zip.so
extension=curl.so
extension=imap.so
extension=json.so
extension=mysql.so
extension=pdo_mysql.so
extension=bcmath.so
extension=xmlrpc.so
extension=dom.so
zend_extension=opcache.so

[Date]
date.timezone = Australia/Melbourne
[filter]
[iconv]
[intl]
[sqlite]
[sqlite3]
[Pcre]
[Pdo]
[Pdo_mysql]
pdo_mysql.cache_size = 2000
pdo_mysql.default_socket=
[Phar]
[Syslog]
define_syslog_variables  = Off
[mail function]
SMTP = localhost
smtp_port = 25
mail.add_x_header = On
[SQL]
sql.safe_mode = Off
[ODBC]
odbc.allow_persistent = On
odbc.check_persistent = On
odbc.max_persistent = -1
odbc.max_links = -1
odbc.defaultlrl = 4096
odbc.defaultbinmode = 1
[Interbase]
ibase.allow_persistent = 1
ibase.max_persistent = -1
ibase.max_links = -1
ibase.timestampformat = "%Y-%m-%d %H:%M:%S"
ibase.dateformat = "%Y-%m-%d"
ibase.timeformat = "%H:%M:%S"
[MySQL]
mysql.allow_local_infile = On
mysql.allow_persistent = Off
mysql.cache_size = 2000
mysql.max_persistent = -1
mysql.max_links = -1
mysql.default_port =
mysql.default_socket =
mysql.default_host =
mysql.default_user =
mysql.default_password =
mysql.connect_timeout = 60
mysql.trace_mode = Off
[MySQLi]
mysqli.max_persistent = -1
mysqli.allow_persistent = On
mysqli.max_links = -1
mysqli.cache_size = 2000
mysqli.default_port = 3306
mysqli.default_socket =
mysqli.default_host =
mysqli.default_user =
mysqli.default_pw =
mysqli.reconnect = Off
[mysqlnd]
mysqlnd.collect_statistics = On
mysqlnd.collect_memory_statistics = Off
[OCI8]
[PostgreSQL]
pgsql.allow_persistent = On
pgsql.auto_reset_persistent = Off
pgsql.max_persistent = -1
pgsql.max_links = -1
pgsql.ignore_notice = 0
pgsql.log_notice = 0
[Sybase-CT]
sybct.allow_persistent = On
sybct.max_persistent = -1
sybct.max_links = -1
sybct.min_server_severity = 10
sybct.min_client_severity = 10
[bcmath]
bcmath.scale = 0
[browscap]
[Session]
session.save_handler = files
session.use_cookies = 1
session.use_only_cookies = 1
session.name = PHPSESSID
session.auto_start = 0
session.cookie_lifetime = 0
session.cookie_path = /
session.cookie_domain =
session.cookie_httponly =
session.serialize_handler = php
session.gc_probability = 1
session.gc_divisor = 1000
session.gc_maxlifetime = 1440
session.bug_compat_42 = Off
session.bug_compat_warn = Off
session.referer_check =
session.entropy_length = 0
session.cache_limiter = nocache
session.cache_expire = 180
session.use_trans_sid = 0
session.hash_function = 0
session.hash_bits_per_character = 5
url_rewriter.tags = "a=href,area=href,frame=src,input=src,form=fakeentry"
[MSSQL]
mssql.allow_persistent = On
mssql.max_persistent = -1
mssql.max_links = -1
mssql.min_error_severity = 10
mssql.min_message_severity = 10
mssql.compatability_mode = Off
mssql.secure_connection = Off
[Assertion]
[COM]
[mbstring]
[gd]
[exif]
[Tidy]
tidy.clean_output = Off
[soap]
soap.wsdl_cache_enabled=1
soap.wsdl_cache_dir="/tmp"
soap.wsdl_cache_ttl=86400
soap.wsdl_cache_limit = 5
[sysvshm]
[ldap]
ldap.max_links = -1
[mcrypt]
[dba]
[xsl]
opcache.memory_consumption=128
opcache.interned_strings_buffer=8
opcache.max_accelerated_files=4000
opcache.revalidate_freq=60
opcache.fast_shutdown=1
opcache.enable_cli=1

MySQL / Percona Database Configuration

I have included the full my.cnf file that I used, it is pretty much standard but with a number of small tweaks to reduce the memory overhead and optimise query cache for WordPress.

vi /opt/local/etc/my.cnf
svcadm enable percona:default

Lets create the database.

mysql -u root
SET PASSWORD FOR ''@'localhost' = PASSWORD('password');
SET PASSWORD FOR 'root'@'localhost' = PASSWORD('password');
CREATE DATABASE wordpressdb;
FLUSH PRIVILEGES;
grant all privileges  on wordpressdb.* to wordpressdba@localhost identified by 'wordpresspass';
quit;

my.cnf

[client]
port            = 3306
socket          = /tmp/mysql.sock
default-character-set = utf8

[mysqld]
user = mysql
port = 3306
basedir = /opt/local
datadir = /var/mysql
socket = /tmp/mysql.sock
bind-address = 127.0.0.1
default-storage-engine = innodb
character-set-server = utf8
skip-external-locking
log_warnings
skip_name_resolve
server-id = 1
key_buffer_size = 16M
sort_buffer_size = 1M
read_buffer_size = 1M
read_rnd_buffer_size = 4M
myisam_sort_buffer_size = 64M

innodb_data_home_dir = /var/mysql
innodb_log_group_home_dir = /var/mysql
innodb_data_file_path = ibdata1:100M:autoextend
innodb_buffer_pool_size = 16M
innodb_log_file_size = 400M
innodb_log_buffer_size = 8M
innodb_flush_log_at_trx_commit = 2
innodb_lock_wait_timeout = 50
innodb_file_per_table
innodb_doublewrite = 0
innodb_io_capacity = 1500
innodb_read_io_threads = 8
innodb_write_io_threads = 8

slow_query_log_file = /var/log/mysql/slowquery.log
slow_query_log = 1
log_slow_filter = "full_scan,tmp_table_on_disk,filesort_on_disk"
log_slow_verbosity = "full"

table_open_cache = 512
thread_cache_size = 1000
query_cache_size = 64M
query_cache_limit = 2M
query_cache_strip_comments = 1
query_cache_type = 1
back_log = 64
thread_concurrency = 32
tmpdir = /tmp
max_connections = 50
max_allowed_packet = 24M
max_join_size = 4294967295
net_buffer_length = 2K
thread_stack = 128K
tmp_table_size = 64M
max_heap_table_size = 64M

binlog_format=mixed
log-bin = /var/log/mysql/bin.log
log-error = /var/log/mysql/error.log

expire_logs_days = 7

[mysqldump]
quick
max_allowed_packet = 16M

[mysql]
no-auto-rehash

[myisamchk]
key_buffer_size = 128M
sort_buffer_size = 128M
read_buffer = 2M
write_buffer = 2M

[mysqlhotcopy]
interactive-timeout

Varnish cache configuration

The settings within the varnish config files control things including:

• Do not serve cached content if we are logged into WordPress admin.
• Allow cache purging only from specific hosts.
• How to correctly handle compression
• Pass actual clients requesting ip address to the logs
• Remove cookies in certain situations
• Do not cache certain wordpress folders e.g. wp-admin
• How to purge content out of the cache

vi /opt/local/etc/default.vcl
mkdir /opt/local/etc/varnish
vi /opt/local/etc/varnish/purge.vcl
chown -R varnish:varnish /opt/local/etc/varnish/
vi /opt/local/lib/svc/manifest/varnish.xml  # change port to 80
svccfg delete varnish
svccfg validate /opt/local/lib/svc/manifest/varnish.xml
svccfg import /opt/local/lib/svc/manifest/varnish.xml
svcadm enable svc:/pkgsrc/varnish:default

default.vcl

backend default {
    .host = "127.0.0.1";
    .port = "8080";
}

acl purge {
        "127.0.0.1";
        "10.1.1.17";
        "localhost";
        "mysite.net";
}

include "/opt/local/etc/varnish/purge.vcl";

sub vcl_recv {

if (req.http.Accept-Encoding) {
        if (req.http.Accept-Encoding ~ "gzip") {
            # If the browser supports it, we'll use gzip.
            set req.http.Accept-Encoding = "gzip";
        }
        else if (req.http.Accept-Encoding ~ "deflate") {
            # Next, try deflate if it is supported.
            set req.http.Accept-Encoding = "deflate";
        }
        else {
            # Unknown algorithm. Remove it and send unencoded.
            unset req.http.Accept-Encoding;
        }
    }

    if (req.restarts == 0) {
        if (req.http.x-forwarded-for) {
            set req.http.X-Forwarded-For =
            req.http.X-Forwarded-For + ", " + client.ip;
        } else {
            set req.http.X-Forwarded-For = client.ip;
        }
    }

# Don't serve cached pages to logged in users
        if ( req.http.cookie ~ "wordpress_logged_in" || req.url ~ "vaultpress=true" ) {
                return( pass );
        }


    if (req.request != "GET" &&
        req.request != "HEAD" &&
        req.request != "PUT" &&
        req.request != "POST" &&
        req.request != "TRACE" &&
        req.request != "OPTIONS" &&
        req.request != "DELETE") {
            return (pipe);
    }

    # Do not cache these paths
    if (req.url ~ "^/wp-cron\.php$" ||
        req.url ~ "^/xmlrpc\.php$" ||
        req.url ~ "^/wp-admin/.*$" ||
        req.url ~ "^/wp-includes/.*$" ||
        req.url ~ "\?s=") {
            return (pass);
    }

    if (req.request != "GET" && req.request != "HEAD") {
        return (pass);
    }
    if (!(req.url ~ "wp-(login|admin)") &&
        !(req.url ~ "&preview=true" ) ) {
        unset req.http.cookie;
    }

    if (req.http.Authorization || req.http.Cookie) {
        return (pass);
    }
    return (lookup);
}


sub vcl_fetch {
  # remove some headers we never want to see
    unset beresp.http.Server;
    unset beresp.http.X-Powered-By;

    if (!(req.url ~ "wp-(login|admin)")) {
        unset beresp.http.set-cookie;
        set beresp.ttl = 96h;
    }

    # don't cache response to posted requests or those with basic auth
    if ( req.request == "POST" || req.http.Authorization ) {
        return (hit_for_pass);
    }

    # don't cache search results
    if( req.url ~ "\?s=" ){
        return (hit_for_pass);
    }

    # only cache status ok
    if ( beresp.status != 200 ) {
        return (hit_for_pass);
    }

    if (beresp.ttl <= 0s ||
        beresp.http.Set-Cookie ||
        beresp.http.Vary == "*") {
            set beresp.ttl = 120 s;
            return (hit_for_pass);
    }
    return (deliver);
}

purge.vcl

# There are 3 possible behaviors of purging.

# Regex purging
# Treat the request URL as a regular expression.
sub purge_regex {
        ban("obj.http.X-Req-URL ~ " + req.url + " && obj.http.X-Req-Host == " + req.http.host);
}

# Exact purging
# Use the exact request URL (including any query params)
sub purge_exact {
        ban("obj.http.X-Req-URL == " + req.url + " && obj.http.X-Req-Host == " + req.http.host);
}

# Page purging (default)
# Use the exact request URL, but ignore any query params
sub purge_page {
        set req.url = regsub(req.url, "\?.*$", "");
        ban("obj.http.X-Req-URL-Base == " + req.url + " && obj.http.X-Req-Host == " + req.http.host);
}


# The purge behavior can be controlled with the X-Purge-Method header.
#
# Setting the X-Purge-Method header to contain "regex" or "exact" will use
# those respective behaviors.  Any other value for the X-Purge header will
# use the default ("page") behavior.
#
# The X-Purge-Method header is not case-sensitive.
#
# If no X-Purge-Method header is set, the request url is inspected to attempt
# a best guess as to what purge behavior is expected.  This should work for
# most cases, although if you want to guarantee some behavior you should
# always set the X-Purge-Method header.

sub vcl_recv {
        if (req.request == "PURGE") {
                if (client.ip !~ purge) {
                        error 405 "Not allowed.";
                }

                if (req.http.X-Purge-Method) {
                        if (req.http.X-Purge-Method ~ "(?i)regex") {
                                call purge_regex;
                        } elsif (req.http.X-Purge-Method ~ "(?i)exact") {
                                call purge_exact;
                        } else {
                                call purge_page;
                        }
                } else {
                        # No X-Purge-Method header was specified.
                        # Do our best to figure out which one they want.
                        if (req.url ~ "\.\*" || req.url ~ "^\^" || req.url ~ "\$$" || req.url ~ "\\[.?*+^$|()]") {
                                call purge_regex;
                        } elsif (req.url ~ "\?") {
                                call purge_exact;
                        } else {
                                call purge_page;
                        }
                }

                error 200 "Purged.";
        }
}

sub vcl_fetch {
        set beresp.http.X-Req-Host = req.http.host;
        set beresp.http.X-Req-URL = req.url;
        set beresp.http.X-Req-URL-Base = regsub(req.url, "\?.*$", "");
}

sub vcl_deliver {
        unset resp.http.X-Req-Host;
        unset resp.http.X-Req-URL;
        unset resp.http.X-Req-URL-Base;
}

varnish.xml

<?xml version="1.0"?>
<!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1">
<service_bundle type="manifest" name="varnish">
  <service name="pkgsrc/varnish" type="service" version="1">
    <create_default_instance enabled="false" />
    <single_instance />
    <dependency name="network" grouping="require_all" restart_on="error" type="service">
      <service_fmri value="svc:/milestone/network:default" />
    </dependency>
    <dependency name="filesystem" grouping="require_all" restart_on="error" type="service">
      <service_fmri value="svc:/system/filesystem/local" />
    </dependency>
    <method_context>
      <method_environment>
        <envvar name='PATH' value='/opt/local/sbin:/opt/local/bin:/sbin:/usr/sbin:/usr/bin' />
      </method_environment>
    </method_context>
    <exec_method type="method" name="start" exec="/opt/local/sbin/varnishd -a %{listen} -l %{size} -f %{config_file} -u varnish -g varnish" timeout_seconds="60" />
    <exec_method type="method" name="stop" exec=":kill" timeout_seconds="60" />
    <property_group name="startd" type="framework">
      <propval name="duration" type="astring" value="contract" />
      <propval name="ignore_error" type="astring" value="core,signal" />
    </property_group>
    <property_group name="application" type="application">
      <propval name="config_file" type="astring" value="/opt/local/etc/default.vcl" />
      <propval name="listen" type="astring" value="0.0.0.0:80" />
      <propval name="size" type="astring" value="64M" />
    </property_group>
    <stability value="Evolving" />
    <template>
      <common_name>
        <loctext xml:lang="C">Varnish daemon</loctext>
      </common_name>
    </template>
  </service>
</service_bundle>

WordPress install

Download the latest version of WordPress and extract it into the directory that our nginx virtual host is referencing. We have already created the database so all we need to do now, is just complete the install steps via the web interface wizard.

cd /opt/local/www/mysite.net/
wget http://wordpress.org/latest.tar.gz
tar -xvzf latest.tar.gz
mv wordpress/* .
rm -rf wordpress/
chown -R www:www /opt/local/www/mysite.net

Open a web browser and confirm you can get to the site on both the nginx and the varnish ports.

nginx port:

http://mysite.net:8080

varnish port:

http://mysite.net

If you have configured everything correctly as above, then you should see your brand new WordPress site being served on both port 80 and 8080.

Testing and Results

Now that it's up and running lets see what this baby can do!

For the initial trial I have created a single test post on the site called "test". It includes a 5000 word article and multiple images.

I am using apache bench running on another computer on the network to do the testing. I will be watching the load on the vps using prstat while the benchmark is being run.

Lets simulate 100 surfers being served the page 10,000 times.

Without Varnish

Requests per second: 73.88
Total: 28 processes, 170 lwps, load averages: 3.77, 1.82, 0.72

                                 ZONE  RSS(MB)  CAP(MB)    NOVER  POUT(MB)
 384c53e7-dc41-4e2f-845b-ab6effc8b7df      319     1024        0         0

➜  ~  ab -kc 100 -n 10000 http://mysite.net:8080/test/
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking mysite.net (be patient)
Completed 1000 requests
Completed 2000 requests
Completed 3000 requests
Completed 4000 requests
Completed 5000 requests
Completed 6000 requests
Completed 7000 requests
Completed 8000 requests
Completed 9000 requests
Completed 10000 requests
Finished 10000 requests

Server Software:        nginx
Server Hostname:        mysite.net
Server Port:            8080

Document Path:          /test/
Document Length:        0 bytes

Concurrency Level:      100
Time taken for tests:   135.351 seconds
Complete requests:      10000
Failed requests:        0
Write errors:           0
Non-2xx responses:      10000
Keep-Alive requests:    0
Total transferred:      4060000 bytes
HTML transferred:       0 bytes
Requests per second:    73.88 [#/sec] (mean)
Time per request:       1353.514 [ms] (mean)
Time per request:       13.535 [ms] (mean, across all concurrent requests)
Transfer rate:          29.29 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.3      0       5
Processing:    15 1347 126.4   1359    1536
Waiting:       15 1347 126.3   1359    1536
Total:         20 1347 126.1   1359    1536

Percentage of the requests served within a certain time (ms)
  50%   1359
  66%   1370
  75%   1380
  80%   1389
  90%   1407
  95%   1424
  98%   1458
  99%   1471
 100%   1536 (longest request)

With Varnish - It's 113X Faster with zero load

Requests per second: 8357.32
Total: 28 processes, 169 lwps, load averages: 0.02, 0.00, 0.00

                                 ZONE  RSS(MB)  CAP(MB)    NOVER  POUT(MB)
 384c53e7-dc41-4e2f-845b-ab6effc8b7df      319     1024        0         0

➜  ~  ab -kc 100 -n 10000 http://mysite.net/test/
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking mysite.net (be patient)
Completed 1000 requests
Completed 2000 requests
Completed 3000 requests
Completed 4000 requests
Completed 5000 requests
Completed 6000 requests
Completed 7000 requests
Completed 8000 requests
Completed 9000 requests
Completed 10000 requests
Finished 10000 requests

Server Software:
Server Hostname:        mysite.net
Server Port:            80

Document Path:          /test/
Document Length:        12922 bytes

Concurrency Level:      100
Time taken for tests:   1.197 seconds
Complete requests:      10000
Failed requests:        0
Write errors:           0
Keep-Alive requests:    0
Total transferred:      133804300 bytes
HTML transferred:       129463908 bytes
Requests per second:    8357.32 [#/sec] (mean)
Time per request:       11.966 [ms] (mean)
Time per request:       0.120 [ms] (mean, across all concurrent requests)
Transfer rate:          109203.63 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        2    3   0.9      3      19
Processing:     3    9   1.8      9      25
Waiting:        2    3   1.0      3      19
Total:          7   12   1.8     12      28

Percentage of the requests served within a certain time (ms)
  50%     12
  66%     12
  75%     12
  80%     12
  90%     13
  95%     14
  98%     17
  99%     19
 100%     28 (longest request)

A dramatic difference no doubt?

Making WordPress play nice with Varnish cache

Now that we have Varnish up and running there are a couple of simple plugins that I recommend you should install. This is to make WordPress clear the varnish cache whenever you change content so that users get served a fresh version of the page. The second is to instruct WordPress that we are using nginx as our webserver.

• nginx Compatibility
• Varnish HTTP Purge

Conclusion & Takeaways

Varnish configuration is complex yet very powerful. I am by no means a varnish expert, but the settings used here, work well for me. I am sure there is additional room for further optimisation. The benchmarking above is not comprehensive, and a proper test would involve multiple pages being queried in random workloads by clients coming from different ip addresses. That being said, I am sure you would agree this is amazingly fast and can handle a ton of traffic.