Awstats and Nginx setup on SmartOS

These days, I am doing a lot of dedicated WordPress VPS sites for clients. I always use SmartOS zones and Nginx wherever possible. Best practice is always to setup Google analytics and Google Webmaster tools to analyse traffic metrics and what is happening from a SEO perspective.

I also don’t like to have all my eggs in one basket and think its important to have our own stats generated from the local nginx server logs. This is great for understanding semi real-time traffic patterns as well as seeing some information that may not be included from other analytics.

I also like to use some of the filtering mechanisms available with awstats to remove certain types of traffic from the generated stats e.g. I do not want to see stats for people logged into WordPress as admin users. I also like to protect the stats with nginx password authentication to protect the information from prying eyes.

This setup assumes you

already have Nginx installed and working. This is how I do my set-up’s.

Lets install the required packages and create the awstats conf file. I like to integrate Geoip information into awstats, its always great to see Country and City information.

pkgin in awstats GeoIP-1.4.8 GeoLiteCity-201402 p5-Geo-IP
cp /opt/local/share/examples/awstats/awstats.conf /opt/local/awstats/cgi-bin/awstats.vitahealth.conf
cd /opt/local/awstats/cgi-bin/
vi /opt/local/awstats/cgi-bin/awstats.vitahealth.conf
vi /opt/local/awstats/cgi-bin/awstats-fcgi.php
chmod 711 /opt/local/awstats/cgi-bin/awstats-fcgi.php
chown www:www /opt/local/awstats/cgi-bin/awstats-fcgi.php

Below is the edited awstats configuration file with comments stripped out. I have added Geoip support as well as using REGEX to ignore log lines which have “wp-admin” in them.

Filename : awstats.vitahealth.conf

LogFile="/var/log/nginx/vitahealth.net.au-access.log"
LogType=W
LogFormat=1
LogSeparator=" "
SiteDomain="vitahealth.net.au"
HostAliases="vitahealth.net.au"
DNSLookup=0
DirData="."
DirCgi="/cgi-bin"
DirIcons="/icon"
AllowToUpdateStatsFromBrowser=0
AllowFullYearView=2
EnableLockForUpdate=0
DNSStaticCacheFile="dnscache.txt"
DNSLastUpdateCacheFile="dnscachelastupdate.txt"
SkipDNSLookupFor=""
AllowAccessFromWebToAuthenticatedUsersOnly=0
AllowAccessFromWebToFollowingAuthenticatedUsers=""
AllowAccessFromWebToFollowingIPAddresses=""
CreateDirDataIfNotExists=0
BuildHistoryFormat=text
BuildReportFormat=html
SaveDatabaseFilesWithPermissionsForEveryone=0
PurgeLogFile=0
ArchiveLogRecords=0
KeepBackupOfHistoricFiles=0
DefaultFile="index.php index.html"
SkipHosts=""
SkipUserAgents=""
SkipFiles="REGEX[^/wp-admin/]"
SkipReferrersBlackList=""
OnlyHosts=""
OnlyUserAgents=""
OnlyUsers=""
OnlyFiles=""
NotPageList="css js class gif jpg jpeg png bmp ico rss xml swf"
ValidHTTPCodes="200 304"
ValidSMTPCodes="1 250"
AuthenticatedUsersNotCaseSensitive=0
URLNotCaseSensitive=0
URLWithAnchor=0
URLQuerySeparators="?;"
URLWithQuery=0
URLWithQueryWithOnlyFollowingParameters=""
URLWithQueryWithoutFollowingParameters=""
URLReferrerWithQuery=0
WarningMessages=1
ErrorMessages=""
DebugMessages=0
NbOfLinesForCorruptedLog=50
WrapperScript=""
DecodeUA=0
MiscTrackerUrl="/js/awstats_misc_tracker.js"
LevelForBrowsersDetection=2
LevelForOSDetection=2
LevelForRefererAnalyze=2
LevelForRobotsDetection=2
LevelForSearchEnginesDetection=2
LevelForKeywordsDetection=2
LevelForFileTypesDetection=2
LevelForWormsDetection=0
UseFramesWhenCGI=1
DetailedReportsOnNewWindows=1
Expires=0
MaxRowsInHTMLOutput=1000
Lang="auto"
DirLang="./lang"
ShowMenu=1
ShowSummary=UVPHB
ShowMonthStats=UVPHB
ShowDaysOfMonthStats=VPHB
ShowDaysOfWeekStats=PHB
ShowHoursStats=PHB
ShowDomainsStats=PHB
ShowHostsStats=PHBL
ShowAuthenticatedUsers=0
ShowRobotsStats=HBL
ShowWormsStats=0
ShowEMailSenders=0
ShowEMailReceivers=0
ShowSessionsStats=1
ShowPagesStats=PBEX
ShowFileTypesStats=HB
ShowFileSizesStats=0
ShowDownloadsStats=HB
ShowOSStats=1
ShowBrowsersStats=1
ShowScreenSizeStats=0
ShowOriginStats=PH
ShowKeyphrasesStats=1
ShowKeywordsStats=1
ShowMiscStats=a
ShowHTTPErrorsStats=1
ShowSMTPErrorsStats=0
ShowClusterStats=0
AddDataArrayMonthStats=1
AddDataArrayShowDaysOfMonthStats=1
AddDataArrayShowDaysOfWeekStats=1
AddDataArrayShowHoursStats=1
IncludeInternalLinksInOriginSection=0
MaxNbOfDomain = 10
MinHitDomain  = 1
MaxNbOfHostsShown = 10
MinHitHost    = 1
MaxNbOfLoginShown = 10
MinHitLogin   = 1
MaxNbOfRobotShown = 10
MinHitRobot   = 1
MaxNbOfDownloadsShown = 10
MinHitDownloads = 1
MaxNbOfPageShown = 10
MinHitFile    = 1
MaxNbOfOsShown = 10
MinHitOs      = 1
MaxNbOfBrowsersShown = 10
MinHitBrowser = 1
MaxNbOfScreenSizesShown = 5
MinHitScreenSize = 1
MaxNbOfWindowSizesShown = 5
MinHitWindowSize = 1
MaxNbOfRefererShown = 10
MinHitRefer   = 1
MaxNbOfKeyphrasesShown = 10
MinHitKeyphrase = 1
MaxNbOfKeywordsShown = 10
MinHitKeyword = 1
MaxNbOfEMailsShown = 20
MinHitEMail   = 1
FirstDayOfWeek=1
ShowFlagLinks=""
ShowLinksOnUrl=1
UseHTTPSLinkForUrl=""
MaxLengthOfShownURL=64
HTMLHeadSection=""
HTMLEndSection=""
MetaRobot=0
Logo="awstats_logo6.png"
LogoLink="http://www.awstats.org"
BarWidth   = 260
BarHeight  = 90
StyleSheet=""
color_Background="FFFFFF"
color_TableBGTitle="CCCCDD"
color_TableTitle="000000"
color_TableBG="CCCCDD"
color_TableRowTitle="FFFFFF"
color_TableBGRowTitle="ECECEC"
color_TableBorder="ECECEC"
color_text="000000"
color_textpercent="606060"
color_titletext="000000"
color_weekend="EAEAEA"
color_link="0011BB"
color_hover="605040"
color_u="FFAA66"
color_v="F4F090"
color_p="4477DD"
color_h="66DDEE"
color_k="2EA495"
color_s="8888DD"
color_e="CEC2E8"
color_x="C1B2E2"
 LoadPlugin="geoip GEOIP_STANDARD /opt/local/share/GeoIP/GeoIP.dat"
 LoadPlugin="geoip_city_maxmind GEOIP_STANDARD /opt/local/share/GeoIP/GeoLiteCity.dat"
ExtraTrackedRowsLimit=500

Then we need to create the awstats fcgi file that will work with nginx to generate and display the awstats information without needing to pre-generate static html files. We also change the permissions to make sure it is executable.

Filename : awstats-fcgi.php

<?php
$descriptorspec = array(
        0 => array('pipe', 'r'),  // stdin is a pipe that the child will read from
        1 => array('pipe', 'w'),  // stdout is a pipe that the child will write to
        2 => array('pipe', 'w')   // stderr is a file to write to
);

$newenv = $_SERVER;
$newenv['SCRIPT_FILENAME'] = $_SERVER['X_SCRIPT_FILENAME'];
$newenv['SCRIPT_NAME'] = $_SERVER['X_SCRIPT_NAME'];

if (is_executable($_SERVER['X_SCRIPT_FILENAME'])) {
        $process = proc_open($_SERVER['X_SCRIPT_FILENAME'], $descriptorspec, $pipes, NULL, $newenv);

        if (is_resource($process)) {
                fclose($pipes[0]);
                $head = fgets($pipes[1]);

                while (strcmp($head, "\n")) {
                        header($head);
                        $head = fgets($pipes[1]);
                }

                fpassthru($pipes[1]);
                fclose($pipes[1]);
                fclose($pipes[2]);

                $return_value = proc_close($process);
        } else {
                header('Status: 500 Internal Server Error');
                echo('Internal Server Error');
        }
} else {
        header('Status: 404 Page Not Found');
        echo('Page Not Found');
}
?>

Virtual Host Setup

I like to setup a separate virtual host for my stats stuff and then secure the access with a basic username and password. I also setup a rewrite /awstats that automatically redirects to “vhost/cgi-bin/awstats.pl?config=vitahealth” so that we do not have to input the whole string each time we want to view the stats. The username and password can be generated with htpasswd or an online password generator if you do not have htpasswd installed.

vi /opt/local/etc/nginx/sites-available/stats.vitahealth.net.au.conf
cd /opt/local/etc/nginx/sites-enabled/
ln -s /opt/local/etc/nginx/sites-available/stats.vitahealth.net.au.conf
vi /opt/local/awstats/pass.txt
chmod 640 /opt/local/awstats/pass.txt
chown -R www:www /opt/local/awstats

Filename : stats.vitahealth.net.au.conf

server {
        listen 0.0.0.0:80;
        server_name stats.vitahealth.net.au;
        auth_basic            "Restricted";
        auth_basic_user_file  /opt/local/awstats/pass.txt;
        root /opt/local/awstats;
        index index.html;

        location ~ ^/awstats {
                rewrite ^ http://stats.vitahealth.net.au/cgi-bin/awstats.pl?config=vitahealth;
        }

        location /awstats/classes/ {
                alias /opt/local/awstats/classes/;
        }

        location /awstats/css/ {
                alias /opt/local/awstats/css/;
        }

        location /awstats/icon/ {
                alias /opt/local/awstats/icon/;
        }

        location /awstats-icon/ {
        alias /opt/local/awstats/icon/;
    }

        location /awstats/js/ {
                alias /opt/local/awstats/js/;
        }


        # Dynamic stats.
        location ~ ^/cgi-bin/(awredir|awstats)\.pl {
                gzip off;
                fastcgi_pass 127.0.0.1:9000;
                fastcgi_param SCRIPT_FILENAME /opt/local/awstats/cgi-bin/awstats-fcgi.php;
                fastcgi_param X_SCRIPT_FILENAME /opt/local/awstats$fastcgi_script_name;
                fastcgi_param X_SCRIPT_NAME $fastcgi_script_name;
                include fastcgi_params;
        }
}

Generate stats and setup Cron

We are almost done, last step is to configure cron to crunch our log files every 15 minutes so the stats are always semi up to date.

crontab -e

and then add the following entry to your cron.

00,15,30,45 * * * * /opt/local/awstats/cgi-bin/awstats.pl -update -config=vitahealth &gt;/dev/null 2&gt;&amp;1

Thats it we are done. To access your stats open a web-browser and go to stats.yourdomain/awstats

Mark Slatem ⋅ SmartOS and friends ⋅ nginx, smartos, tutorial, wordpress ⋅ No comments

Awstats and Nginx setup on SmartOS

Virtual Host Setup

Generate stats and setup Cron

Mark Slatem

Leave a Reply Cancel