Nightwatch gotcha

Nightwatch is a really amazing tool for in-browser testing. A clean interface for the underlying Selenium API. Working with it is as good as it will ever get when creating those kinds of tests.

Even though it’s much nicer it has its own possible pitfalls.

I come across an interesting problem. My test was skipping an assertion. The code in question had the following structure:

module.exports = {
  'Test example': browser => {
    browser
      .url('http://www.example.com')
      .waitForElementVisible('body', 1000)
      .tableContent('table', rows => {
        browser.assert.equal(rows[0], ['1', 'one']);
      })
      .end();
  }
};

(.tableContent is my custom command which extract all text from an table and returns it as an array of arrays.)

The test starts normally: a lot of calls to nightwatch’s methods. Near the end, there was a call to my custom command and to assert the result is what I expect it to be. The code inside of the callback was ignored.

After reading the documentation (several times), combing through the internet and poking the code in the debugger I managed to find the root cause.

Test execution mechanism is the culprit. When the test is run, the calls to nightwatch’s are queued and executed when the results are coming. The execution is not synchronous, as it often needs to wait for the result of selenium calls (the clue behind it is that all selenium calls take a callback as an argument). It’s definitely a clever solution to the problem, but it comes

All this scheduling and execution makes it non-obvious that by the time .tableContent fires its callback the whole test will be over because it reached the call to .end. Nightwatch has no way of knowing that my custom command is still running.

Fortunately, there’s a method .perform which gives an ability to inject arbitrary code into the command queue. Because one of the arguments to the callback is done function, nightwatch can know exactly when that custom code will finish.

Restructuring the code the final form is following:

module.exports = {
  'Test example': browser => {
    browser
      .url('http://www.example.com')
      .waitForElementVisible('body', 1000)
      .perform((client, done) => {
        client.tableContent('table', rows => {
          client.assert.equal(rows[0], ['1', 'one']);
          done();
        });
      })
      .end();
  }
};

The test case execution will wait until my custom command and associated assertions are run. Exactly what I wanted in the first place.

To clean up that code will require building a custom assertion. This extra complication can be encapsulated behind a cleaner interface consistent with the rest of the built-ins.