Automated web shop testing with WebdriverIO

We’ve recently migrated our website to a new environment. While we were at it, we thought we could complement our simple uptime-Testing with some automated web shop testing, so that we can sleep at night knowing our site is OK. We’ve implemented some tests with WebdriverIO and thought we’d share the procedure, in case you’re looking for a quick guide. The entire test script is available for download here.

Our test scenario

When somebody buys Ghostlab from our website, they have to do several things: add Ghostlab to the cart, go to the cart page, click the checkout button, provide all required information in the checkout form, pay for the order, and finally they will expect to receive their license information via E-mail. And that’s exactly the procedure we want to pack into an automated test run.

  • Automated test steps
  • Go to Ghostlab landing page
  • Click “Buy” link to get to WooCommerce product page
  • Add Ghostlab to the cart
  • Click on “View cart” to see the cart contents
  • Proceed to checkout
  • Complete checkout form
  • Pay – we’ll do that with a trick
  • On order completion, receive E-mail with license key

 

Set up WebdriverIO project

Before you can get started, you have to set up a WebdriverIO project. It’s really easy, just follow the WebdriverIO guide. What you’re doing is basically:

  • set up your project directory
  • download selenium server and driver – be sure to start the server after download.
  • install webbdriverio via npm

Once you’re set up, let’s directly create a test runner for our project. Just follow the instructions for how to complete the testrunner setup dialog (WebdriverIO guide, “Let’s get serious”). Note that as of today (January 13 2017), there is one additional setup step in the dialog that is not described in the guide – I just selected the default value.

I’m assuming that you now have your test/specs directory. Now create a JavaScript file there that lets you write your test. You should end up with a setup comparable to the one in the image below.

# To run your test(s), simply execute the 
# following from the command line in the 
# project directory. 
./node_modules/.bin/wdio wdio.conf.js
Why WebdriverIO?
There are many tools to create automated tests. When we set up our tests, we were looking for something that was powerful, well-documented, easy to setup, and locally available (i.e. no SaaS). We gave several tools a quick spin, and WebdriverIO convinced us because the time to your first hello world is really short, and it is very well documented.

Now on to the actual test steps…we will only be looking at a small set of features of the Webdriver API – for the full reference, see WebdriverIO API Docs.

1: Navigate to landing page, follow to WooCommerce product page, add Ghostlab to cart

We want to start from our product landing page, so we need to point the browser to that location.

browser.url('https://www.vanamco.com/ghostlab');

On this page, there is a Buy button. We’d expect visitors who want to buy Ghostlab to click it and get to the WooCommerce product page. After clicking the button, we need to wait for the WooCommerce product page to load before we can perform our further actions. Therefore, we’re waiting for an element to be present that was not on the landing page, but is on the WooCommerce product page, namely the “Add to cart” button.

browser.click('=Buy');
browser.waitForExist('.single_add_to_cart_button.button', 5000);

Both the click and waitForExist function expect a Selector as first argument. A selector is used to specify the element in question – the one that we want to click on, or the one we’re waiting for to be present.

We’re specifying a timeout of 5000 milliseconds for waitForExist – in case the page does not load within this time, the test will fail. So make sure you specify a timeout long enough for your page to load.

Selectors
There’s a wide range of possible selectors, we’ve found the link text (‘=Buy’, on anchor elements) and CSS selectors to be easy to handle. Of course, which selectors best suit you depends on the content of your page. As soon as you start using selectors, be aware that changes in your page might break your test. For example, if we were to change the text of the buy button to “Get one now”, the test would no longer work. See the Developer Guide for a full reference of available selectors.

As soon as the WooCommerce product page has loaded, there will be an element to add the product to the cart, and our test run will continue. Next we actually want to add 1x Ghostlab to the cart by pressing the button. When this is done, the page will show a confirmation message that we’ve added it to the cart – and there will be a link that takes us to our shopping cart. So we wait for the confirmation to appear, then follow the cart link.

browser.click('.single_add_to_cart_button.button');
browser.waitForExist('.woocommerce-message > a', 5000);
browser.click('.woocommerce-message > a');

2: Enter coupon code so we don’t have to provide payment information, continue to checkout

As soon as the cart is displayed, we enter a coupon code we’ve prepared for the test that gives us 100% off. This means that our order totals 0$ and we can perform the checkout without entering payment information.  We detect whether the cart has loaded by checking for the presence of the coupon code field. We then wait for the confirmation message WooCommerce writes to the page to show that the code was applied, and verify that it was successfully applied by examining the content of the message.

browser.waitForExist('input[name="coupon_code"]');
browser.setValue('input[name="coupon_code"]', config.couponCode);
browser.click('input[name="apply_coupon"]');
browser.waitForExist('.woocommerce-message', 5000);
var msg = browser.getText('.woocommerce-message').trim();
assert(msg === 'Coupon code applied successfully.', 'Did not see coupon code applied successfully message');

Once the coupon code is applied, we go to the checkout form by clicking the proceed to checkout link. We detect the load of the checkout form by checking for the presence of the place order button – which we will use in the end to actually place our order.

browser.click('.wc-proceed-to-checkout > a');
browser.waitForExist('input[name="woocommerce_checkout_place_order"]');

3: Complete checkout form

Now we can fill out the checkout form. We’ve stored the field selectors and the associated values in a separate array over which we can iterate – of course, you could also solve this differently. One field we have to treat specially is the country selector – this is not a simple select, but a JavaScript-enhanced select component where you can search for values, then select them by clicking them. We are unable to simply set the value as with a regular input field, so we need to perform the click-search-select pattern that a real user would use.

 

config.checkoutForm = [
 {s: 'input[name="billing_first_name"]', v: 'Automatic'} /*, ... */
];

for (var i = 0; i < config.checkoutForm.length; i++) {
    var item = config.checkoutForm[i];
    browser.setValue(item.s, item.v);
 }
browser.click('.country_select');
myWait(1000);
browser.setValue('#s2id_autogen1_search', 'Switzerland');
myWait(500);
browser.click('.select2-result-label');

 

We are using the myWait function. This will simply make the test wait for a specified number of milliseconds by using the waitUntil function in combination with some time math. We use it on the JavaScript select box to make sure that the page has had the time to first render the search box, then filter down the results. We’ll later use the myWait function before trying to fetch the license e-mail.

/*
 * Simple function to wait for a specified number of milliseconds (roughly)
 */
function myWait(ms) {
    console.log('Waiting for ' + (ms/1000) + ' seconds');
    var now = new Date().getTime();
    browser.waitUntil(function(){
        return (new Date().getTime() - now) > ms;
    }, ms + 1000)
}

4: Place order, verify order confirmation, extract order number

Now that the form is completely filled out, we can place the order. For that, we simply click the place order button. After the order has been processed, we will see a confirmation page. We verify that we have reached it by waiting for the thank you note, and by examining the content of the message. Also, we extract the order number assigned to our order by the system. We need this to check if we have received the license e-mail.

browser.click('input[name="woocommerce_checkout_place_order"]');
browser.waitForExist('.woocommerce-thankyou-order-received');
msg = browser.getText('.woocommerce-thankyou-order-received').trim();
assert(msg === 'Thank you. Your order has been received.', 'Did not see order has been received message.');

console.log('Extracting order number...');
var orderNumber = browser.getText('.woocommerce-thankyou-order-details.order_details > li:first-child > strong').trim();

 

5: Verify that license e-mail has been sent

On order placement, our backend should have generated a license key for Ghostlab and sent it to the e-mail address we have provided in the checkout form. In order to complete the test, we need to see if that e-mail has actually been delivered. For that, we use a node imap package to connect to the e-mail account we provided on purchase. Before we try and fetch the e-mail, we wait for 30 seconds, otherwise, it might not have been delivered yet.

Mocha timeouts
By default, mocha (which we’re using in our example) has a timeout of 10s – so when we would wait for 30s, that would fail, as the global timeout would apply after 10s. You can increase the mocha timeout in the mocha options (mochaOpts.timeout), this can be found in the file wdio.conf.js, which is automatically created for your project.

We know that the Subject text of that license e-mail is unique – it is some static text combined with the order number (that’s the unique part). So we can simply search for a message with the expected subject, and will be satisfied if the number of messages returned by the search is exactly 1. Of course, we could go on to examine the content of the license email, maybe even try to activate the software product with the license – but we’ll leave that for some other time.

// Let's wait a bit before we try to fetch the E-mail
myWait(10 * 1000);

var subject = 'Ghostlab 2 license for your order ' + orderNumber + ' (1 x Ghostlab)'
var foundEmail = false;
imap.once('ready', function(){
  imap.openBox('INBOX', true, function(err, box){
    imap.search([ 'UNSEEN', ['SUBJECT', subject] ], function(err, results) {
      if (err) 
        throw err;
       var resultsLength = results.length;
       assert(resultsLength === 1, 'Did not find exactly one e-mail with license information');
       foundEmail = true;
      imap.end();
    }); 
  });
});
imap.connect();

// Wait for e-mail to be confirmed.
browser.waitUntil(function() { return foundEmail; }, 10000);

Note that the call to imap.connect() is asynchronous – so how do we “notify” our test that e-mail fetching has been successful? We use the waitUntil function and pass it a function that returns a variable. In the e-mail handling, we set the variable to true if we successfully fetch the e-mail.

The entire test script is available for download here.

Ghostlab 2 is here

We’re excited to announce that Ghostlab 2 is now available! Let us give you a quick tour of what’s new, and what’s the same.

As in version one, Ghostlab 2 focuses on helping you making responsive and multi-device testing much faster and more convenient. It lets you browse any site you’re interested in, keeping any number of connected clients in sync by propagating scrolling, clicking, and any other site interactions it discovers. This works for local browsers and mobile devices on your network. It lets you (validly!) fill out forms automatically within a fraction of a second, lets you keep track of all connected clients, and offers you the ability to identify problems and fix them on any of them.

At the same time, we’ve introduced many changes in version 2 – some of them significant, some of them more on the minor-but-useful side. We’d like to present three of them that we’re particularly happy to share with you.

Client inspection

In Ghostlab 1, we’ve included Weinre, an awesome project that allowed us to remotely inspect the DOM on any connected device. However, we wanted to do more than that, and decided we’d have to rethink how we provide remote inspection tools. In Ghostlab 2, we are including the developer tools you know from Google Chrome – with some tweaks of our own. You are still able to inspect any connected client – but in addition, you can now inspect all of them synchronously.

That means that when you remotely “fix” that CSS property that destroys your page in Chrome or Firefox, you’ll immediately also see what the consequences are for any other connected browser or device.

On top of that, we decided that inspecting CSS is often not enough – especially after some bad experiences finding JavaScript bugs on mobile phones. So we’ve come up with a solution that lets you remotely debug JavaScript on any client that is connected to Ghostlab – be it a browser, a tablet, a mobile phone or even a TV set. We’ve even been told it works on IE8!

Preprocessors

To allow Ghostlab to further integrate into the workflow, we’ve also added the ability to compile several languages. Initially, we’re shipping support for Sass / Less / Stylus, Jade / Haml, and CoffeeScript / Typescript. When you’re using Ghostlab with your local site, it will now be able to compile all these source files into browser-ready HTML/CSS/JS and then automatically refresh the page on your devices.

We’re not including binary compiler packages in our distribution, rather, Ghostlab will attempt to install them them should they not be present on your system. We’re using npm and gem for that purpose. This makes it easy to offer a wide range of preprocessors in the future while not making the Ghostlab distribution too heavy. While the initial set of precompilers may seem limited, we’ll constantly be on the lookout for interesting candidates to include.

Redirection service

We’ve been told by several customers that when working in a team, pointing mobile devices to the Ghostlab server can be cumbersome. Of course, in order to avoid having to enter the URL on every one of them each time (which is slow even using the provided QR Code), they had bookmarked the Ghostlab server URL on every device. but: which URL? Several developers means several computers means several IP addresses – so each developer has a different (and potentially changing) IP address and the device bookmark will ever so often point to the wrong URL.

The idea of the redirection service is simple. A redirection bookmark is just a URL, looking something like http://302.pm/ABC677. It redirects to an IP address in your local network, like http://192.168.2.3:8005. The redirection bookmark has an access token, and using that access token, Ghostlab is able to update the redirection bookmark to the current IP and port where Ghostlab runs. When on a device, you access the bookmark, you will always be redirected to the up-to-date Ghostlab address. This way, you can either share redirection bookmarks within a team (by sharing the access token), or simply create your individual and permanent Ghostlab URL for your devices to use.

Enough said. Give it a spin!

So much from us. You can have a look at the Ghostlab website, where you can download a full-featured and completely free 7-day trial!

Javascript Quiz – Win a FREE Ghostlab License

 

We recently worked with Chris Coyier at CSS-Tricks.com to create an in-depth tutorial on Javascript Events!

To further test your knowledge of Javascript we put together a quiz with two awesome prizes!

The quiz for a Ghostlab licence is now closed.

Two winners were chosen to win a free Ghostlab license and one winner was chosen for an upcoming blog interview:

1st place: Sergio Lopes

2nd place: Mark Nicholson

How to Proxy Requests in node.js

Recently we received quite a few requests from people wanting to use Ghostlab for their website and webapp testing from behind corporate proxies. Unfortunately, as Ghostlab’s server component is built on node.js and the HTTP client in node.js assumes that a direct connection to the Web can be made (i.e., node.js ignores the system’s proxy settings), we had to build our own support to proxy requests in node.

There are some other blog posts on how to do a request through a proxy for HTTP, so adding that was easy. But not for HTTPS, so that’s what this little write-up is mainly about.

HTTP

Let’s start with the easy part. Assume you want to request the content of https://www.vanamco.com/ghostlab. Requesting it through a proxy, we would do something like:

  • Connect to the proxy server (e.g. with telnet 192.168.5.8 3128 if 192.168.5.8 is the IP of your proxy server running on port 3128),
  • do the GET request using the full URL as the path:

GET https://www.vanamco.com/ghostlab/ HTTP/1.1

Or with node.js:


var Http = require('http');

var req = Http.request({
    host: '192.168.5.8',
    // proxy IP
    port: 3128,
    // proxy port
    method: 'GET',
    path: 'https://www.vanamco.com/ghostlab/' // full URL as path
    }, function (res) {
        res.on('data', function (data) {
        console.log(data.toString());
    });
});

req.end();

HTTPS

While the above code is the correct way to do a HTTP GET request through a proxy, it won’t work if you try to use HTTPS. For instance, if you replace the path by https://www.twitter.com, your proxy server might say something like
“Unsupported Request Method and Protocol. Squid does not support all request methods for all access protocols.”

In fact, you actually wouldn’t want the proxy to be able to do this; you wouldn’t want it to decrypt and send you back your unencrypted data. Instead, proxies support the CONNECT request, which will establish a tunnel to the remote server, so the data will remain encrypted.

When you open a telnet session (telnet 192.168.5.8 3128) and do a CONNECT request


CONNECT twitter.com:443

the proxy will answer with


HTTP/1.0 200 Connection established

This means, that the proxy set up the tunnel to the host. Now you can communicate with the host through the tunnel. I.e., you’ll have to start by sending your encrypted request (which is a bit hard to do with telnet, so the next code listing shows how to do it with node.js).


var Http = require('http');
var Tls = require('tls');

var req = Http.request({
    host: '192.168.5.8',
    port: 3128,
    method: 'CONNECT',
    path: 'twitter.com:443',
});

req.on('connect', function (res, socket, head) {
    var cts = Tls.connect({
    host: 'twitter.com',
    socket: socket
    }, function () {
        cts.write('GET / HTTP/1.1rnHost: twitter.comrnrn');
    });

    cts.on('data', function (data) {
        console.log(data.toString());
    });
});

req.end();

The node.js program does a HTTP CONNECT request to the proxy and listens for the connect event. The event handler is passed an instance of net.Socket which we can use as if we were directly communicating with the remote server. I.e., we need to start by sending the encrypted request and we’ll receive back the encrypted content.

To do the encryption and decryption, we use node.js’s TLS module. The tls.connect method accepts an options argument which contains the socket we received from the connect event handler. Make sure to also include the host in the options, even though the node.js docs say the host will be ignored when you’re passing a socket. If you don’t pass the host, an error will be thrown, “Hostname/IP doesn’t match certificate’s altnames”. As a second argument, tls.connect accepts a callback function (a listener to the secureConnect event, really), in which we do our HTTP request (manually, in this snippet) writing to the tls.ClearTextStream object returned by tls.connect.

A HTTPS Proxy Agent

To make life a little easier and also benefit from node.js’s HTTPS client implementation, we can wrap the CONNECT request and dealing with TLS into a https.Agent so we only have to do this when doing an HTTPS request via a proxy:


var Https = require('https');
var agent = new HttpsProxyAgent({
    proxyHost: '192.168.5.8',
    proxyPort: 3128
});
Https.request({
    // like you'd do it usually...
    host: 'twitter.com',
    port: 443,
    method: 'GET',
    path: '/',

    // ... just add the special agent:
    agent: agent
}, function (res) {
        res.on('data', function (data) {
        console.log(data.toString());
});
}).end();

In summary: in addition to a regular HTTPS request, you only need to instantiate an HttpsProxyAgent and pass the instance in the agent property of the request options.


 

To implement the HttpsProxyAgent, we can reuse most of node.js’s implementation of the http.Agent and https.Agent. But we need to overwrite two methods, addRequest and createSocket to handle the asynchronous nature of how we receive the socket (the original implementation assumes that createConnection returns a socket), and so we can emit an error event on the request object when the connection to the proxy fails.

Here we go (Gist):
Update (March 23, 2015): The code below was updated to work with node v0.12.0. See Gist for more information.


var Util = require('util');
var Https = require('https');
var Tls = require('tls');

function HttpsProxyAgent(options) {
    Https.Agent.call(this, options);

    this.proxyHost = options.proxyHost;
    this.proxyPort = options.proxyPort;

    this.createConnection = function (opts, callback) {
        // do a CONNECT request
        var req = Http.request({
        host: options.proxyHost,
        port: options.proxyPort,
        method: 'CONNECT',
        path: opts.host + ':' + opts.port,
        headers: {
            host: opts.host
        }
    });

    req.on('connect', function (res, socket, head) {
    var cts = Tls.connect({
        host: opts.host,
        socket: socket
    }, function () {
        callback(false, cts);
    });
    });

    req.on('error', function (err) {
        callback(err, null);
    });

    req.end();
    }
}

Util.inherits(HttpsProxyAgent, Https.Agent);

// Almost verbatim copy of http.Agent.addRequest
HttpsProxyAgent.prototype.addRequest = function (req, options) {
var name = options.host + ':' + options.port;
if (options.path) name += ':' + options.path;

if (!this.sockets[name]) this.sockets[name] = [];

if (this.sockets[name].length < this.maxSockets) {
    // if we are under maxSockets create a new one.
    this.createSocket(name, options.host, options.port, options.path, req, function (socket) {
        req.onSocket(socket);
    });
} else {
    // we are over limit so we'll add it to the queue.
    if (!this.requests[name])
    this.requests[name] = [];
    this.requests[name].push(req);
}
};

// Almost verbatim copy of http.Agent.createSocket
HttpsProxyAgent.prototype.createSocket = function (name, host, port, localAddress, req, callback) {
    var self = this;
    var options = Util._extend({}, self.options);
    options.port = port;
    options.host = host;
    options.localAddress = localAddress;

    options.servername = host;
    if (req) {
        var hostHeader = req.getHeader('host');
        if (hostHeader)
            options.servername = hostHeader.replace(/:.*$/, '');
    }

    self.createConnection(options, function (err, s) {
    if (err) {
        err.message += ' while connecting to HTTP(S) proxy server ' + self.proxyHost + ':' + self.proxyPort;

        if (req)
            req.emit('error', err);
        else
            throw err;

    return;
}

if (!self.sockets[name]) self.sockets[name] = [];

self.sockets[name].push(s);

var onFree = function () {
    self.emit('free', s, host, port, localAddress);
};

var onClose = function (err) {
    // this is the only place where sockets get removed from the Agent.
    // if you want to remove a socket from the pool, just close it.
    // all socket errors end in a close event anyway.
    self.removeSocket(s, name, host, port, localAddress);
};

var onRemove = function () {
    // we need this function for cases like HTTP 'upgrade'
    // (defined by WebSockets) where we need to remove a socket from the pool
    // because it'll be locked up indefinitely
    self.removeSocket(s, name, host, port, localAddress);
    s.removeListener('close', onClose);
    s.removeListener('free', onFree);
    s.removeListener('agentRemove', onRemove);
};

s.on('free', onFree);
s.on('close', onClose);
s.on('agentRemove', onRemove);

callback(s);
});
};

Image credit: hiroze