Structured End-To-End Tests With Protractor

The Angular team provides a tool set for creating end to end tests (e2e) called protractor http://www.protractortest.org based on WebDriver from Selenium. It’s extended by a deferred element finder and support for the ng-model notation, so it could be the ideal partner for testing AngularJS applications. But if wishes were horses, beggars would ride.

E2E tests in real life projects tend to be messy. In my opinion this is caused by two reasons. At first, when using a back-end service in this kind of testing, there’s the problem how to reset it’s state, most commonly a data base, to a well defined starting condition. Second, I observe that changes on user interfaces occur more often than in the back-end. So the task of writing user interface navigation in your tests gets similar to pushing Sysiphus’ stone up the hill only to watch it rolling down when someone in your team changes the HTML in a minor way, e.g. embedding additional <div> or changing an id. In this article only the later issue will be discussed. Protractor might be the wrong tool for solving the first issue. Though not analyzed, it may be easier using the Java based Selenium Web-Driver, not because it’s better, but resetting the data base to defined state using the back-end tool set might be more efficient than re-implementing it using node.js, as long as node.js isn’t used as back-end technology of course.

Instead of writing another protractor tutorial, the official one is used, but structured differently by using something I call an user interface API. There might be a better or an official name, but let’s just call it UI API. The intention should be clear. Instead of developing only the user interface for the end user, it should be considered to develop and deliver an API to access all portions of the UI. Especially in an environment with multiple back-end systems it’s easier to abstract to common language than to speak in different “languages” of the UI. For example the UI of a car dealer might consist of three modules: security, product configuration and financing, developed by different teams with different technologies. So instead of writing (in pseudo code) something like the following:

the developer of the login user interface could just provide login and business functions to abstract the details away. Even more important, there is contract between the tester and the developer how to access a specific portion of the UI. In this example “Model Foo” depends tightly on the locale. The developer can’t know, which locale the tester uses. Therefore the developer might put more information on it’s own getting the page more testable.

But why not using Selenium IDE instead of this abstract contract? After spending several months using the Selenium IDE in former project the following happened:

1. The IDE did not record the events we wanted to, so we had to change it a bit

2. Test implemented, everything o.k.

3. Several days later changes in the UI broke the test. Tried replay the test, but we forgot what exactly we were typing in the forms. So from now on we had Excel lists which described what and how to test.

4. Recorded the test again but in the meantime we forgot how we changed the output

5. Instead of using the Selenium IDE abstractions of the UI actions were written, like login, addPartner, addContract…

6. … oh, there they are …

So soon or later you tend to create these UI verbs. But in my humble opinion the UI developer should take care of them, instead of the integration tester.

Extract and define the UI contract

The following example should demonstrate extracting the “verbs” into one or more self-contained module. It is available on Github, of course: https://github.com/dzuvic/structured-protractor-example.

To use Protractor it needs to be configured. The bare minimum with a base URL looks like this:

Protractor supports several test frameworks, but we’ll concentrate on Jasmin, because it’s Protractor’s default test framework. In your own tests you should take care of the base URL. In some projects there’s a class of parameters, which are configured centrally. The base URL is typically one them and should be handled appropriately. In this example, the base URL is just defined in Protractor, so the helper module depends only on that.

When we take a look at the tutorial example of Protractor it looks like this:

All we have to do is to extract the defined functions in an own helper module. First of all a helper module, that contains the “UI verbs” has to be defined. In the end it looks like this:

The Jasmin test uses this helper module:

Unfortunately the helper needs objects that Protractor initialized, so the module has to be initialized with them as well. This initialization happens each time when the page has to be loaded. A minor remark: The helper module is loaded with a relative path, which isn’t ideal. This could be avoided by creating it’s own npm module and publishing in a private repository.

To use this tests just execute the following:

Conclusion

At this point it should be clear, that writing End-To-End tests gets ugly and hardly to maintain really fast if there isn’t a kind of contract between the UI and the tests. Putting this “contract” into an exportable module, that the UI developer is responsible for, should keep the test code clean. In fact, this pattern isn’t new at all and is called “Separation of concerns” and best described by none other than Edsger W. Dijkstra.