Sometimes using WordPress CMS isn’t exactly a speedy experience, every page that is rendered hogs the CPU as it goes though the thousands of lines of code and performs multiple SQL database queries. 99% of the time, the content is unchanged so it makes a lot more sense to serve that content statically – directly out of RAM.
Enter our “hero” Varnish Cache, a web application accelerator also known as a caching HTTP reverse proxy. You simply install it in front of any web services that talk HTTP and configure it to cache the contents. This activity really makes WordPress fly.
In fact, Varnish makes WordPress response times outrageously fast. So fast, that speed increases in the order of 300–1000X can typically be expected. Since your content is served statically out of RAM, your CPU remains mainly idle. This setup will let you serve hundreds of thousands of users per day effortlessly and turn your existing rig into a lean, mean web-serving machine.
5000 Pages served per second with almost no cpu load
Almost 2 Million pages served in 6 minutes
It is undeniable fact that millions of high traffic blogs and company websites are built using WordPress. This is because WordPress is easy to use and maintain, the codebase is very mature and if you use a recommended framework such as the superb “Genesis Framework“, you can effortlessly tick all the necessary boxes when it comes to SEO best practice including page optimisation and social outreach.
Nowadays, we are all overloaded with web content that is driven at us from all angles and its literally “screaming for our attention”. As a consequence, the common web user has become so impatient, that if the desired web page has not loaded by the time we blink, we move on.
Lets face it, we spend a huge amount of time, painstakingly writing great content for our audiences. But, if you get your WordPress setup wrong, you’re in for some real pain. You will get penalised by both users and search engines caused by slow page loads, crappy plugins, congested databases, security breaches and ultimately the inability to serve high traffic loads when you really need to.
But, get it right and…. “shazam!” Your snappy, secure and responsive website should survive the dreaded “slashdot effect” without a hiccup.
In the following guide, I will demonstrate how to set up a blazingly fast WordPress site using a SmartOS zone with minimal resources to effortlessly handle extremely high traffic loads and without breaking a sweat.
Although this guide is specifically tuned for WordPress on SmartOS, most of the optimisation points and concepts discussed here are applicable to other operating systems and web environments.
We will be using:
- SmartOS base64 13.3.1 dataset
- 1024 MB Ram for our VPS.
- Percona MySQL for the database.
- nginx for the web server.
- PHP 5.5 with its built in Zend OPcache for PHP script cache.
- Varnish Cache as a in memory web application accelerator.
- WordPress 3.8.1 + Demo content for testing
No additional WordPress cache plugins are required and won’t be used . The best rule of thumb for WordPress configuration is to reduce bloat and keep your plugins to a bare minimum.
If we are effectively handling our caching at the server level where it belongs, lets not install any additional caching plugins in WordPress itself as it’s simply not necessary and just adds to bloat.
Varnish-Cache is an amazing beast! as you will soon see. In our tutorial for simplicity sake, we are running everything in a single zone/vps. In production environments it would make more sense to have a single varnish zone instance in front of all your WordPress sites.
The advantage of the setup below, is that there are no external dependencies and everything is maintained within the zone itself.
Ok, you are probably thinking, hey dude enough yapping its time to get crackin! So, lets get going with the actual setup steps.
Package Installation
pkgin -fy up pkgin in percona-server-5.6 nginx php-5.5 php55-mysqli php55-mcrypt php55-mbstring unzip php55-zlib php55-gd php55-zip php55-gd php55-curl php55-imap php55-json php55-mysql php55-pdo_mysql php55-bcmath php55-fpm nano php55-dom php55-xmlrpc php55-opcache varnish
Notice that GCC is installed as a varnish dependency, this is because Varnish compiles its VCL configuration file on start-up. Once all packages have been installed we move on to the nginx configuration.
nginx Configuration
Here we will be setting up nginx to run our WordPress site as a virtual host. We setup nginx to listen on port 8080 and later we will be configuring varnish to listen on port 80 and pass traffic to nginx on 8080.
mkdir -p /opt/local/www/mysite.net mkdir /opt/local/etc/nginx/sites-available mkdir /opt/local/etc/nginx/sites-enabled vi /opt/local/etc/nginx/nginx.conf vi /opt/local/etc/nginx/sites-available/mysite.net.conf cd /opt/local/etc/nginx/sites-enabled/ ln -s /opt/local/etc/nginx/sites-available/mysite.net.conf nginx -t vi /opt/local/etc/php-fpm.conf mkdir /opt/local/www/mysite.net svcadm enable nginx svcadm enable php-fpm
nginx.conf
user www www; worker_processes 1; events { worker_connections 768; multi_accept on; } http { # Let NGINX get the real client IP for its access logs set_real_ip_from 127.0.0.1; real_ip_header X-Forwarded-For; # Basic Settings sendfile on; tcp_nopush on; tcp_nodelay on; keepalive_timeout 20; client_max_body_size 15m; client_body_timeout 60; client_header_timeout 60; client_body_buffer_size 1K; client_header_buffer_size 1k; large_client_header_buffers 4 8k; send_timeout 60; reset_timedout_connection on; types_hash_max_size 2048; server_tokens off; include /opt/local/etc/nginx/mime.types; default_type application/octet-stream; # Logging Settings # access_log /var/log/nginx/access.log; error_log /var/log/nginx/error.log; # Log Format log_format main '$remote_addr - $remote_user [$time_local] ' '"$request" $status $body_bytes_sent "$http_referer" ' '"$http_user_agent" "$http_x_forwarded_for"'; # Gzip Settings gzip on; gzip_disable "msie6"; gzip_vary on; gzip_proxied any; gzip_comp_level 6; gzip_min_length 512; gzip_buffers 16 8k; gzip_http_version 1.1; gzip_types text/css text/javascript text/xml text/plain text/x-component application/javascript application/x-javascript application/json application/xml application/rss+xml font/truetype application/x-font-ttf font/opentype application/vnd.ms-fontobject image/svg+xml; # include /usr/local/etc/nginx/vhosts/*; include /opt/local/etc/nginx/sites-enabled/*; }
mysite.net.conf
server { # Default server block blacklisting all unconfigured access listen 8080 default_server; server_name _; return 444; } server { # Configure the domain that will run WordPress server_name mysite.net; listen 8080 ; port_in_redirect off; access_log /var/log/nginx/mysite.net-access.log; error_log /var/log/nginx/mysite.net-error.log; server_tokens off; autoindex off; client_max_body_size 15m; client_body_buffer_size 128k; # WordPress needs to be in the webroot of /var/www/ in this case root /opt/local/www/mysite.net; index index.html index.htm index.php; try_files $uri $uri/ /index.php?q=$uri&$args; # Define default caching of 24h expires 86400s; add_header Pragma public; add_header Cache-Control "max-age=86400, public, must-revalidate, proxy-revalidate"; # deliver a static 404 error_page 404 /404.html; location /404.html { internal; } # Deliver 404 instead of 403 "Forbidden" error_page 403 = 404; # Do not allow access to files giving away your WordPress version location ~ /(\.|wp-config.php|readme.html|license.txt) { return 404; } # Add trailing slash to */wp-admin requests. rewrite /wp-admin$ $scheme://$host$uri/ permanent; # Don't log robots.txt requests location = /robots.txt { allow all; log_not_found off; access_log off; } # Rewrite for versioned CSS+JS via filemtime location ~* ^.+\.(css|js)$ { rewrite ^(.+)\.(\d+)\.(css|js)$ $1.$3 last; expires 31536000s; access_log off; log_not_found off; add_header Pragma public; add_header Cache-Control "max-age=31536000, public"; } # Aggressive caching for static files # If you alter static files often, please use # add_header Cache-Control "max-age=31536000, public, must-revalidate, proxy-revalidate"; location ~* \.(asf|asx|wax|wmv|wmx|avi|bmp|class|divx|doc|docx|eot|exe|gif|gz|gzip|ico|jpg|jpeg|jpe|mdb|mid|midi|mov|qt|mp3|m4a|mp4|m4v|mpeg|mpg|mpe|mpp|odb|odc|odf|odg|odp|ods|odt|ogg|ogv|otf|pdf|png|pot|pps|ppt|pptx|ra|ram|svg|svgz|swf|tar|t?gz|tif|tiff|ttf|wav|webm|wma|woff|wri|xla|xls|xlsx|xlt|xlw|zip)$ { expires 31536000s; access_log off; log_not_found off; add_header Pragma public; add_header Cache-Control "max-age=31536000, public"; } # pass PHP scripts to Fastcgi listening on Unix socket # Do not process them if inside WP uploads directory # If using Multisite or a custom uploads directory, # please set the */uploads/* directory in the regex below location ~* (^(?!(?:(?!(php|inc)).)*/uploads/).*?(php)) { try_files $uri = 404; fastcgi_split_path_info ^(.+.php)(.*)$; fastcgi_pass 127.0.0.1:9000; #fastcgi_pass unix:/var/run/php-fpm.socket; fastcgi_index index.php; fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name; include fastcgi_params; fastcgi_intercept_errors on; fastcgi_ignore_client_abort off; fastcgi_connect_timeout 60; fastcgi_send_timeout 180; fastcgi_read_timeout 180; fastcgi_buffer_size 128k; fastcgi_buffers 4 256k; fastcgi_busy_buffers_size 256k; fastcgi_temp_file_write_size 256k; } # Deny access to hidden files location ~ /\. { deny all; access_log off; log_not_found off; } } # Redirect all www. queries to non-www # Change in case your site is to be available at "www.yourdomain.tld" server { listen 8080; server_name www.mysite.net; rewrite ^ $scheme://mysite.net$request_uri? permanent; }
The below fpm config file is basically the default, you should however tune yours more aggressively to match your expected traffic loads.
php-fpm.conf
[global] pid = run/php-fpm.pid [www] user = www group = www listen = 127.0.0.1:9000 pm = dynamic pm.max_children = 5 pm.start_servers = 2 pm.min_spare_servers = 1 pm.max_spare_servers = 3
In php.ini we add some best practice php settings to lock down security and prevent some php exploits. In addition we enable Zend Opcache and add some tuning parameters. Opcache will compile and cache our PHP scripts without the need to execute them.
php.ini
[PHP] engine = On short_open_tag = Off ignore_user_abort = Off asp_tags = Off precision = 14 y2k_compliance = On output_buffering = 4096 zlib.output_compression = Off implicit_flush = Off unserialize_callback_func = serialize_precision = 17 allow_call_time_pass_reference = Off safe_mode = Off safe_mode_gid = Off safe_mode_include_dir = safe_mode_exec_dir = safe_mode_allowed_env_vars = PHP_ safe_mode_protected_env_vars = LD_LIBRARY_PATH disable_functions = "apache_child_terminate, apache_setenv, define_syslog_variables, escapeshellarg, escapeshellcmd, eval, exec, fp, fput, ftp_connect, ftp_exec, ftp_get, ftp_login, ftp_nb_fput, ftp_put, ftp_raw, ftp_rawlist, highlight_file, ini_alter, ini_get_all, ini_restore, inject_code, mysql_pconnect, openlog, passthru, php_uname, phpAds_remoteInfo, phpAds_XmlRpc, phpAds_xmlrpcDecode, phpAds_xmlrpcEncode, popen, posix_getpwuid, posix_kill, posix_mkfifo, posix_setpgid, posix_setsid, posix_setuid, posix_setuid, posix_uname, proc_close, proc_get_status, proc_nice, proc_open, proc_terminate, shell_exec, syslog, system, xmlrpc_entity_decode" disable_classes = zend.enable_gc = On expose_php = On max_execution_time = 30 max_input_time = 60 memory_limit = 128M error_reporting = E_ALL & ~E_DEPRECATED display_errors = Off display_startup_errors = Off log_errors = On log_errors_max_len = 1024 ignore_repeated_errors = Off ignore_repeated_source = Off report_memleaks = On track_errors = Off html_errors = Off variables_order = "GPCS" request_order = "GP" register_globals = Off register_long_arrays = Off register_argc_argv = Off auto_globals_jit = On post_max_size = 20M magic_quotes_gpc = Off magic_quotes_runtime = Off magic_quotes_sybase = Off auto_prepend_file = auto_append_file = default_mimetype = "text/html" include_path = ".:/opt/local/lib/php" doc_root = user_dir = enable_dl = Off file_uploads = On upload_tmp_dir = /tmp upload_max_filesize = 15M default_charset = "UTF-8" max_file_uploads = 20 allow_url_fopen = Off allow_url_include = Off default_socket_timeout = 30 extension=pdo.so extension=mysqli.so extension=mcrypt.so extension=mbstring.so extension=zlib.so extension=gd.so extension=zip.so extension=curl.so extension=imap.so extension=json.so extension=mysql.so extension=pdo_mysql.so extension=bcmath.so extension=xmlrpc.so extension=dom.so zend_extension=opcache.so [Date] date.timezone = Australia/Melbourne [filter] [iconv] [intl] [sqlite] [sqlite3] [Pcre] [Pdo] [Pdo_mysql] pdo_mysql.cache_size = 2000 pdo_mysql.default_socket= [Phar] [Syslog] define_syslog_variables = Off [mail function] SMTP = localhost smtp_port = 25 mail.add_x_header = On [SQL] sql.safe_mode = Off [ODBC] odbc.allow_persistent = On odbc.check_persistent = On odbc.max_persistent = -1 odbc.max_links = -1 odbc.defaultlrl = 4096 odbc.defaultbinmode = 1 [Interbase] ibase.allow_persistent = 1 ibase.max_persistent = -1 ibase.max_links = -1 ibase.timestampformat = "%Y-%m-%d %H:%M:%S" ibase.dateformat = "%Y-%m-%d" ibase.timeformat = "%H:%M:%S" [MySQL] mysql.allow_local_infile = On mysql.allow_persistent = Off mysql.cache_size = 2000 mysql.max_persistent = -1 mysql.max_links = -1 mysql.default_port = mysql.default_socket = mysql.default_host = mysql.default_user = mysql.default_password = mysql.connect_timeout = 60 mysql.trace_mode = Off [MySQLi] mysqli.max_persistent = -1 mysqli.allow_persistent = On mysqli.max_links = -1 mysqli.cache_size = 2000 mysqli.default_port = 3306 mysqli.default_socket = mysqli.default_host = mysqli.default_user = mysqli.default_pw = mysqli.reconnect = Off [mysqlnd] mysqlnd.collect_statistics = On mysqlnd.collect_memory_statistics = Off [OCI8] [PostgreSQL] pgsql.allow_persistent = On pgsql.auto_reset_persistent = Off pgsql.max_persistent = -1 pgsql.max_links = -1 pgsql.ignore_notice = 0 pgsql.log_notice = 0 [Sybase-CT] sybct.allow_persistent = On sybct.max_persistent = -1 sybct.max_links = -1 sybct.min_server_severity = 10 sybct.min_client_severity = 10 [bcmath] bcmath.scale = 0 [browscap] [Session] session.save_handler = files session.use_cookies = 1 session.use_only_cookies = 1 session.name = PHPSESSID session.auto_start = 0 session.cookie_lifetime = 0 session.cookie_path = / session.cookie_domain = session.cookie_httponly = session.serialize_handler = php session.gc_probability = 1 session.gc_divisor = 1000 session.gc_maxlifetime = 1440 session.bug_compat_42 = Off session.bug_compat_warn = Off session.referer_check = session.entropy_length = 0 session.cache_limiter = nocache session.cache_expire = 180 session.use_trans_sid = 0 session.hash_function = 0 session.hash_bits_per_character = 5 url_rewriter.tags = "a=href,area=href,frame=src,input=src,form=fakeentry" [MSSQL] mssql.allow_persistent = On mssql.max_persistent = -1 mssql.max_links = -1 mssql.min_error_severity = 10 mssql.min_message_severity = 10 mssql.compatability_mode = Off mssql.secure_connection = Off [Assertion] [COM] [mbstring] [gd] [exif] [Tidy] tidy.clean_output = Off [soap] soap.wsdl_cache_enabled=1 soap.wsdl_cache_dir="/tmp" soap.wsdl_cache_ttl=86400 soap.wsdl_cache_limit = 5 [sysvshm] [ldap] ldap.max_links = -1 [mcrypt] [dba] [xsl] opcache.memory_consumption=128 opcache.interned_strings_buffer=8 opcache.max_accelerated_files=4000 opcache.revalidate_freq=60 opcache.fast_shutdown=1 opcache.enable_cli=1
MySQL / Percona Database Configuration
I have included the full my.cnf file that I used, it is pretty much standard but with a number of small tweaks to reduce the memory overhead and optimise query cache for WordPress.
vi /opt/local/etc/my.cnf svcadm enable percona:default
Lets create the database.
mysql -u root SET PASSWORD FOR ''@'localhost' = PASSWORD('password'); SET PASSWORD FOR 'root'@'localhost' = PASSWORD('password'); CREATE DATABASE wordpressdb; FLUSH PRIVILEGES; grant all privileges on wordpressdb.* to wordpressdba@localhost identified by 'wordpresspass'; quit;
my.cnf
[client] port = 3306 socket = /tmp/mysql.sock default-character-set = utf8 [mysqld] user = mysql port = 3306 basedir = /opt/local datadir = /var/mysql socket = /tmp/mysql.sock bind-address = 127.0.0.1 default-storage-engine = innodb character-set-server = utf8 skip-external-locking log_warnings skip_name_resolve server-id = 1 key_buffer_size = 16M sort_buffer_size = 1M read_buffer_size = 1M read_rnd_buffer_size = 4M myisam_sort_buffer_size = 64M innodb_data_home_dir = /var/mysql innodb_log_group_home_dir = /var/mysql innodb_data_file_path = ibdata1:100M:autoextend innodb_buffer_pool_size = 16M innodb_log_file_size = 400M innodb_log_buffer_size = 8M innodb_flush_log_at_trx_commit = 2 innodb_lock_wait_timeout = 50 innodb_file_per_table innodb_doublewrite = 0 innodb_io_capacity = 1500 innodb_read_io_threads = 8 innodb_write_io_threads = 8 slow_query_log_file = /var/log/mysql/slowquery.log slow_query_log = 1 log_slow_filter = "full_scan,tmp_table_on_disk,filesort_on_disk" log_slow_verbosity = "full" table_open_cache = 512 thread_cache_size = 1000 query_cache_size = 64M query_cache_limit = 2M query_cache_strip_comments = 1 query_cache_type = 1 back_log = 64 thread_concurrency = 32 tmpdir = /tmp max_connections = 50 max_allowed_packet = 24M max_join_size = 4294967295 net_buffer_length = 2K thread_stack = 128K tmp_table_size = 64M max_heap_table_size = 64M binlog_format=mixed log-bin = /var/log/mysql/bin.log log-error = /var/log/mysql/error.log expire_logs_days = 7 [mysqldump] quick max_allowed_packet = 16M [mysql] no-auto-rehash [myisamchk] key_buffer_size = 128M sort_buffer_size = 128M read_buffer = 2M write_buffer = 2M [mysqlhotcopy] interactive-timeout
Varnish cache configuration
The settings within the varnish config files control things including:
- Do not serve cached content if we are logged into WordPress admin.
- Allow cache purging only from specific hosts.
- How to correctly handle compression
- Pass actual clients requesting ip address to the logs
- Remove cookies in certain situations
- Do not cache certain wordpress folders e.g. wp-admin
- How to purge content out of the cache
vi /opt/local/etc/default.vcl mkdir /opt/local/etc/varnish vi /opt/local/etc/varnish/purge.vcl chown -R varnish:varnish /opt/local/etc/varnish/ vi /opt/local/lib/svc/manifest/varnish.xml # change port to 80 svccfg delete varnish svccfg validate /opt/local/lib/svc/manifest/varnish.xml svccfg import /opt/local/lib/svc/manifest/varnish.xml svcadm enable svc:/pkgsrc/varnish:default
default.vcl
backend default { .host = "127.0.0.1"; .port = "8080"; } acl purge { "127.0.0.1"; "10.1.1.17"; "localhost"; "mysite.net"; } include "/opt/local/etc/varnish/purge.vcl"; sub vcl_recv { if (req.http.Accept-Encoding) { if (req.http.Accept-Encoding ~ "gzip") { # If the browser supports it, we'll use gzip. set req.http.Accept-Encoding = "gzip"; } else if (req.http.Accept-Encoding ~ "deflate") { # Next, try deflate if it is supported. set req.http.Accept-Encoding = "deflate"; } else { # Unknown algorithm. Remove it and send unencoded. unset req.http.Accept-Encoding; } } if (req.restarts == 0) { if (req.http.x-forwarded-for) { set req.http.X-Forwarded-For = req.http.X-Forwarded-For + ", " + client.ip; } else { set req.http.X-Forwarded-For = client.ip; } } # Don't serve cached pages to logged in users if ( req.http.cookie ~ "wordpress_logged_in" || req.url ~ "vaultpress=true" ) { return( pass ); } if (req.request != "GET" && req.request != "HEAD" && req.request != "PUT" && req.request != "POST" && req.request != "TRACE" && req.request != "OPTIONS" && req.request != "DELETE") { return (pipe); } # Do not cache these paths if (req.url ~ "^/wp-cron\.php$" || req.url ~ "^/xmlrpc\.php$" || req.url ~ "^/wp-admin/.*$" || req.url ~ "^/wp-includes/.*$" || req.url ~ "\?s=") { return (pass); } if (req.request != "GET" && req.request != "HEAD") { return (pass); } if (!(req.url ~ "wp-(login|admin)") && !(req.url ~ "&preview=true" ) ) { unset req.http.cookie; } if (req.http.Authorization || req.http.Cookie) { return (pass); } return (lookup); } sub vcl_fetch { # remove some headers we never want to see unset beresp.http.Server; unset beresp.http.X-Powered-By; if (!(req.url ~ "wp-(login|admin)")) { unset beresp.http.set-cookie; set beresp.ttl = 96h; } # don't cache response to posted requests or those with basic auth if ( req.request == "POST" || req.http.Authorization ) { return (hit_for_pass); } # don't cache search results if( req.url ~ "\?s=" ){ return (hit_for_pass); } # only cache status ok if ( beresp.status != 200 ) { return (hit_for_pass); } if (beresp.ttl <= 0s || beresp.http.Set-Cookie || beresp.http.Vary == "*") { set beresp.ttl = 120 s; return (hit_for_pass); } return (deliver); }
purge.vcl
# There are 3 possible behaviors of purging. # Regex purging # Treat the request URL as a regular expression. sub purge_regex { ban("obj.http.X-Req-URL ~ " + req.url + " && obj.http.X-Req-Host == " + req.http.host); } # Exact purging # Use the exact request URL (including any query params) sub purge_exact { ban("obj.http.X-Req-URL == " + req.url + " && obj.http.X-Req-Host == " + req.http.host); } # Page purging (default) # Use the exact request URL, but ignore any query params sub purge_page { set req.url = regsub(req.url, "\?.*$", ""); ban("obj.http.X-Req-URL-Base == " + req.url + " && obj.http.X-Req-Host == " + req.http.host); } # The purge behavior can be controlled with the X-Purge-Method header. # # Setting the X-Purge-Method header to contain "regex" or "exact" will use # those respective behaviors. Any other value for the X-Purge header will # use the default ("page") behavior. # # The X-Purge-Method header is not case-sensitive. # # If no X-Purge-Method header is set, the request url is inspected to attempt # a best guess as to what purge behavior is expected. This should work for # most cases, although if you want to guarantee some behavior you should # always set the X-Purge-Method header. sub vcl_recv { if (req.request == "PURGE") { if (client.ip !~ purge) { error 405 "Not allowed."; } if (req.http.X-Purge-Method) { if (req.http.X-Purge-Method ~ "(?i)regex") { call purge_regex; } elsif (req.http.X-Purge-Method ~ "(?i)exact") { call purge_exact; } else { call purge_page; } } else { # No X-Purge-Method header was specified. # Do our best to figure out which one they want. if (req.url ~ "\.\*" || req.url ~ "^\^" || req.url ~ "\$$" || req.url ~ "\\[.?*+^$|()]") { call purge_regex; } elsif (req.url ~ "\?") { call purge_exact; } else { call purge_page; } } error 200 "Purged."; } } sub vcl_fetch { set beresp.http.X-Req-Host = req.http.host; set beresp.http.X-Req-URL = req.url; set beresp.http.X-Req-URL-Base = regsub(req.url, "\?.*$", ""); } sub vcl_deliver { unset resp.http.X-Req-Host; unset resp.http.X-Req-URL; unset resp.http.X-Req-URL-Base; }
varnish.xml
<?xml version="1.0"?> <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="manifest" name="varnish"> <service name="pkgsrc/varnish" type="service" version="1"> <create_default_instance enabled="false" /> <single_instance /> <dependency name="network" grouping="require_all" restart_on="error" type="service"> <service_fmri value="svc:/milestone/network:default" /> </dependency> <dependency name="filesystem" grouping="require_all" restart_on="error" type="service"> <service_fmri value="svc:/system/filesystem/local" /> </dependency> <method_context> <method_environment> <envvar name='PATH' value='/opt/local/sbin:/opt/local/bin:/sbin:/usr/sbin:/usr/bin' /> </method_environment> </method_context> <exec_method type="method" name="start" exec="/opt/local/sbin/varnishd -a %{listen} -l %{size} -f %{config_file} -u varnish -g varnish" timeout_seconds="60" /> <exec_method type="method" name="stop" exec=":kill" timeout_seconds="60" /> <property_group name="startd" type="framework"> <propval name="duration" type="astring" value="contract" /> <propval name="ignore_error" type="astring" value="core,signal" /> </property_group> <property_group name="application" type="application"> <propval name="config_file" type="astring" value="/opt/local/etc/default.vcl" /> <propval name="listen" type="astring" value="0.0.0.0:80" /> <propval name="size" type="astring" value="64M" /> </property_group> <stability value="Evolving" /> <template> <common_name> <loctext xml:lang="C">Varnish daemon</loctext> </common_name> </template> </service> </service_bundle>
WordPress install
Download the latest version of WordPress and extract it into the directory that our nginx virtual host is referencing. We have already created the database so all we need to do now, is just complete the install steps via the web interface wizard.
cd /opt/local/www/mysite.net/ wget http://wordpress.org/latest.tar.gz tar -xvzf latest.tar.gz mv wordpress/* . rm -rf wordpress/ chown -R www:www /opt/local/www/mysite.net
Open a web browser and confirm you can get to the site on both the nginx and the varnish ports.
nginx port:
http://mysite.net:8080
varnish port:
http://mysite.net
If you have configured everything correctly as above, then you should see your brand new WordPress site being served on both port 80 and 8080.
Testing and Results
Now that it's up and running lets see what this baby can do!
For the initial trial I have created a single test post on the site called "test". It includes a 5000 word article and multiple images.
I am using apache bench running on another computer on the network to do the testing. I will be watching the load on the vps using prstat while the benchmark is being run.
Lets simulate 100 surfers being served the page 10,000 times.
Without Varnish
Requests per second: 73.88
Total: 28 processes, 170 lwps, load averages: 3.77, 1.82, 0.72
ZONE RSS(MB) CAP(MB) NOVER POUT(MB) 384c53e7-dc41-4e2f-845b-ab6effc8b7df 319 1024 0 0
➜ ~ ab -kc 100 -n 10000 http://mysite.net:8080/test/ This is ApacheBench, Version 2.3 <$Revision: 655654 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ Benchmarking mysite.net (be patient) Completed 1000 requests Completed 2000 requests Completed 3000 requests Completed 4000 requests Completed 5000 requests Completed 6000 requests Completed 7000 requests Completed 8000 requests Completed 9000 requests Completed 10000 requests Finished 10000 requests Server Software: nginx Server Hostname: mysite.net Server Port: 8080 Document Path: /test/ Document Length: 0 bytes Concurrency Level: 100 Time taken for tests: 135.351 seconds Complete requests: 10000 Failed requests: 0 Write errors: 0 Non-2xx responses: 10000 Keep-Alive requests: 0 Total transferred: 4060000 bytes HTML transferred: 0 bytes Requests per second: 73.88 [#/sec] (mean) Time per request: 1353.514 [ms] (mean) Time per request: 13.535 [ms] (mean, across all concurrent requests) Transfer rate: 29.29 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 0.3 0 5 Processing: 15 1347 126.4 1359 1536 Waiting: 15 1347 126.3 1359 1536 Total: 20 1347 126.1 1359 1536 Percentage of the requests served within a certain time (ms) 50% 1359 66% 1370 75% 1380 80% 1389 90% 1407 95% 1424 98% 1458 99% 1471 100% 1536 (longest request)
With Varnish - It's 113X Faster with zero load
Requests per second: 8357.32
Total: 28 processes, 169 lwps, load averages: 0.02, 0.00, 0.00
ZONE RSS(MB) CAP(MB) NOVER POUT(MB) 384c53e7-dc41-4e2f-845b-ab6effc8b7df 319 1024 0 0
➜ ~ ab -kc 100 -n 10000 http://mysite.net/test/ This is ApacheBench, Version 2.3 <$Revision: 655654 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ Benchmarking mysite.net (be patient) Completed 1000 requests Completed 2000 requests Completed 3000 requests Completed 4000 requests Completed 5000 requests Completed 6000 requests Completed 7000 requests Completed 8000 requests Completed 9000 requests Completed 10000 requests Finished 10000 requests Server Software: Server Hostname: mysite.net Server Port: 80 Document Path: /test/ Document Length: 12922 bytes Concurrency Level: 100 Time taken for tests: 1.197 seconds Complete requests: 10000 Failed requests: 0 Write errors: 0 Keep-Alive requests: 0 Total transferred: 133804300 bytes HTML transferred: 129463908 bytes Requests per second: 8357.32 [#/sec] (mean) Time per request: 11.966 [ms] (mean) Time per request: 0.120 [ms] (mean, across all concurrent requests) Transfer rate: 109203.63 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 2 3 0.9 3 19 Processing: 3 9 1.8 9 25 Waiting: 2 3 1.0 3 19 Total: 7 12 1.8 12 28 Percentage of the requests served within a certain time (ms) 50% 12 66% 12 75% 12 80% 12 90% 13 95% 14 98% 17 99% 19 100% 28 (longest request)
A dramatic difference no doubt?
Making WordPress play nice with Varnish cache
Now that we have Varnish up and running there are a couple of simple plugins that I recommend you should install. This is to make WordPress clear the varnish cache whenever you change content so that users get served a fresh version of the page. The second is to instruct WordPress that we are using nginx as our webserver.
Conclusion & Takeaways
Varnish configuration is complex yet very powerful. I am by no means a varnish expert, but the settings used here, work well for me. I am sure there is additional room for further optimisation. The benchmarking above is not comprehensive, and a proper test would involve multiple pages being queried in random workloads by clients coming from different ip addresses. That being said, I am sure you would agree this is amazingly fast and can handle a ton of traffic.
Jun 01, 2014 @ 11:26:56
Awesome post…
Little issue I hit, varnish can interfere with password protected pages/posts in wordpress.
Add this to vcl_recv
# password protected pages miss cache
if( req.http.Cookie ~ “wp-postpass_” ){
return (pass);
}
Thanks to buffcleb @ wordpress support forums for the tip.
Jul 07, 2014 @ 17:37:54
Thanks for the tip. I have added it to my notes 😉
Oct 23, 2014 @ 11:12:55
Have you ever tried intergrate them with REDIS?
Tried it on my test server at http://ca1.5ribu.net using nginx, php-fpm, mariadb and redis.
Nov 01, 2014 @ 01:30:12
No I have never tried to do that. Sounds awesome!
Whats the performance like, and do you have a How-To guide on your setup?
Nov 06, 2014 @ 14:58:20
I used redis (Full Page Caching) over a year ago doing a lot of internal tests on various different custom nginx configurations. But honestly, Redis didn’t offer any more speed or do anything that all of the other caches do.. It’s crazy, that once you get a rock solid nginx.conf and virtual host configuration, you can test all most every combination of cache systems, but in the end, I find enhanced disk based caching for pages, database and objects to be the fastest overall on Hex/Xeon processors with ultra fast SSD drives.. We have a high traffic site, resource usage is low, page speeds are extremely fast..
The above set up in the article, is very similar to the one I read about Zillows configuration, http://engineering.zillow.com/on-wordpress-scaling-and-performance-the-zillow-way/. They say that adding Memcached and using https://wordpress.org/plugins/memcached/ for object caching along with varnish is crazy fast as well.
I am curious about testing this set up with 3 of my other wordpress sites, each running a different configuration mainly for benchmarks.. I’m curious how much faster, if any, the above set up will be from the post.
There is a lot of good information here! One of the best posts I’ve seen yet out of hundreds probably, for optimizing WordPress with specific focus on Nginx!
Thanks!
Chris
Nov 06, 2014 @ 15:48:41
Hi there Chris – thanks for the detailed comment.
I recently been using plain Opcache + nginx compiled with fast-cgi cache purge support. Its wickedly fast and does not have the complexity of Varnish setups. Then using a tiny purge plugin in wordpress for selective purge on edit etc. I made a dataset with it its hosted on datasets.at If you want to check it out. More info here as well.
http://blog.smartcore.net.au/community-smartos-datasets/
Nov 06, 2014 @ 20:28:40
Wicked Stuff, Thanks Mark!
I honestly know nothing about the Smart OS, but this sounds similar to an Idea I posted recently. It’s crazy that no one has taken a Linux distro, and customized it specifically for speed, optimized and scalable, while incorporating a cache system and also adding direct caching support to the wordpress core.. Something similar to a true optimized virtual appliance. Or in this case, Smart OS.. This is the stuff I do everyday! Like your build above, I spend hours testing configurations! Again, thats an awesome little setup!
Thanks for sharing!
Chris
Nov 06, 2014 @ 20:59:12
Chris – thanks for the kind words.
Let me know how it goes if you manage to test it and get a SmartOS setup going. Warning once you go SmartOS, it will be hard to go back 😉
Dec 29, 2014 @ 10:58:45
Thanks for information – very good ! Please write more about SmartOS !