Sunday, March 23, 2008

AB/Multivariate/Segmentation Testing– Choosing the best Page/Combination/Segment

Web Analytics is not just pulling numbers from the Web Analytics tools and reporting numbers, it is much more than that. As an Analyst, your task is to report the numbers in a presentable and descriptive manner with analysis but also give suggestions on how to continuously improve the website in terms of appeal and ease of use. This article will touch base on the 3 major website conversion testing methods: AB Testing, Multivariate Testing and Segmentation Testing.

AB Testing or Bucket Test as you might call it is a way to test 2 variations of a page in order to determine which page is more successful in terms of pulling people towards a website goal. We split traffic on each page based on a defined proportion (50/50, 90/10). Usually the first step where there are most chances of convincing a user to buy something is the homepage as this is the page that defines a website in terms of visual appeal, product listing, ease of navigation, color scheme and even text. These things go a long way in generating a sense of trust in the users. For e.g. we have a website whose business model is selling books and there are 2 AB Test pages with the first page being the Control (original) having a discount offer banner on the top and the second page doesn’t have any discount offer banner but a slideshow listing the top selling books. We will then put tracking codes on both these pages and identify each page with a unique identifier in order to measure each page’s success. This success is usually measured in terms of the Conversion ratio which can be measured as: Visitors on the Conversion page/Visitors on the Test/Home Page. Some tools that can help you achieve AB Testing are Google Website Optimizer, Verster, SiteSpect and SplitAnalyzer. Below is a screenshot depicting a typical AB test report.

The second and the most efficient method to optimize your page is known as a Multivariate Testing. In this method, we don’t use multiple pages to determine a higher conversion rate but we test the same page by changing the elements around. For e.g. we can test headlines, colors, buttons and images on the same page by correlating each element with each other to find the best possible combination/element. The method used to achieve this is known as the Taguchi Method. This method takes into consideration the sample size and how long a test should run in order to have a clear winner. Google Website Optimizer uses this method to measure a multivariate test. This tool lets you view the various combinations and elements separately and then based on the Taguchi method decides the winner. All you need to do is add a tracking code on to your page and Google Website Optimizer will then split the traffic equally on each element separately. Please look at the below screenshot to understand how Google Website Optimizer measures a multivariate test. Some other tools that can help you achieve multivariate Testing are Google Website Optimizer, Optimost, Memetrics, Verster, Maxymiser and Omniture Test&Target.

Another form of testing which can be used to target specific segments like country, keywords, referring domains and direct traffic is known as Segmentation. In this we split the traffic based on a defined segment and then track the success factor/behavior pattern of each segment. This is the first step to achieve Behavioral Targeting which is in itself a very powerful way to understand user patterns. Please look at the below mockup segmentation test report screenshot.
Again the best KPI to measure the success is the Conversion rate. Some tools that can help you achieve Segmentation are Google Analytics (Free), Kefta and Maxymiser.

Sunday, March 9, 2008

Content Management System Walkthrough – Joomla and Drupal

A Content Management System is used to handle website content which can include text, images, audio, video etc. A CMS allows you to create a website without having any prior knowledge of programming. For the past couple of weeks I tried my hands on open source content management systems. The 2 premier open source content management systems in the market are Joomla and Drupal. The names sound funny but these 2 tools are one of the most robust tools available which can give any other CMS tools a run for their money. It was a tough couple of weeks to figure out the installation process of setting up the Content Management systems. After going through tons of articles on the Web, I managed to successfully install these on my machine.

Through this article I’ll try to share my experience in setting up the CMS tools in a local environment. Following are a few tools you would need on your machine in order to properly configure these CMS tools:

1) IIS (Internet Information Services/Apache Web Server: You can install IIS on your Windows XP machine by going to Control Panel -> Add or Remove Programs -> Windows Components Wizard and select IIS from the list as shown in the figure to install it. IIS is a Web Server which enables you to publish your webpages in a local/global environment. I didn’t use Apache but you can use it instead of IIS.

2) PHP (Hypertext Preprocessor): PHP is a Server Side programming language used to produce dynamic Web Pages. We don’t need PHP to do any sort of programming in this walkthrough but it is pivotal as the CMS tools we are using contain pages created in PHP. PHP can be downloaded at Once you have installed the PHP executable and setup your IIS Server, you can test whether your PHP is working correctly or not. You would need to create a document test.php and enter the following text in it: ?php phpinfo();? included in <> tags. Save the test.php and place it in this folder: C:\Inetpub\wwwroot\test.php which is your IIS Server’s root directory. Once all the above steps are complete, test your PHP installation by going to this URL http://localhost/test.php. If you see the page as shown in the image, it means that PHP is successfully installed on your machine.

3) MySQL Server: MySQL is a free open source database that has all the features that an expensive server can have (For e.g. SQL Server, TeraData costs can go up to thousands of dollars). Joomla and Drupal make use of MySQL as the back-end which provide storage for all text content and configuration settings. You can download MySQL Server from but don’t forget to download the MySQL administrator (Image below) as well which will provide a front end GUI (Graphical User Interface) for your databases. Configuring and Installing MySQL is very simple and easy but if you do face any problems, you can always search the Web. It is however mandatory to create a username and password which you will have to use in order to install Joomla and Drupal. (Note: One of the biggest problems I faced after the installation of MySQL and PHP was getting MySQL compatible/talk to PHP. Basically there are a lot of extensions or Dlls that you might have to play with or change in order to create a successful connection between MySQL and PHP. I still can’t figure out which exact extension was responsible for making these 2 tools compatible but it is either php_mysql.dll or php_mysqli.dll. So if you are unlucky, then you might have to wait a little longer than expected to make Joomla or Drupal work. However, I finally managed to make these 2 tools talk by installing EasyPHP which is explained next.)

4) EasyPHP: This tool allows quick compatibility of PHP with the Web Servers and databases. It was only after installing this tool was I able to access Joomla and Drupal on localhost. This tool will install all the extensions that PHP needs in order to create a connection with MySQL. You can install EasyPHP from Once you have installed and executed EasyPHP, you can see an E icon on the taskbar (bottom right of your screen). Right click on that and go to Configuration -> PHP Extension. Once you see the box as shown in the below image, you should select all the extensions listed in the box as it will install the necessary Dlls.

Once you have completed the above steps, then you need to install Joomla which you can download from and Drupal from Choose the latest installation versions (I chose these files and drupal-6.1.tar.gz) and extract these files in the root directory of your Web Server (C:\Inetpub\wwwroot\joomla and C:\Inetpub\wwwroot\drupal). Once you have extracted these files, you would need to create respective databases for these 2 tools in MySQL Administrator as shown in the below image.

Finally you would need to go to http://localhost/joomla/index.php and http://localhost/drupal/index.php in order to install these 2 tools on your machine. While installing these tools, it would be required to create a connection with MySQL Server in order for these CMS tools to publish their files on the database by entering your MySQL credentials as defined earlier. Once your installation is complete, you should see the following pages as shown in the last 2 images.

All in all it was a nice experience learning the CMS installation procedure as it has added a new dimension to my career and also a skill-set. Hopefully you like this article and let me know if you have any questions regarding the rigorous installation process as I’ll try my best to help you.