How to Choose an Automation Framework
It’s pretty common to use automated tests to enhance the quality and speed of any project you may be working on. But often the right question is: which framework is better for my project?
The problem in choosing an automation framework
Choosing the right framework is not a one-size-fits-all solution. You have to look at the various factors and project requirements to ensure you are implementing a framework that leads to an efficient result. I’ve weighed all the pros and cons of each framework below so that you can make an informed decision for yourself and your project.
The Comparison:
Protractor
Pros
- Inbuilt support for AngularJS element identification strategies. It’s better to go with a protractor if the application is developed using AngularJS.
- It has support for different unit test frameworks like (Jasmine, Mocha, Cucumber etc.)
- A good reporting mechanism (HTML, XML, etc.)
- Compatible with Continuous Integration
- TypeScript support is available which will help Angular 2 developers to write unit tests very easily
Cons
- It does not have support to automate mobile apps
- It is implemented as a wrapper to the WebDriverJS, which means there is one more layer added between the Selenium server and the Protractor. If there is an issue with WebDriverJS, the Protractor team should wait for the WebDriverJS team to fix that issue.
WebDriverIO
Pros
- WebDriverIO is a custom implementation of W3C WebDriver API. This gives it an edge to have full control over implementation rather than depending on WebDriverJS implementation.
- It is fully extensible and is written to be as flexible and framework-agnostic as possible. It can be applied in any context and serves not only the purpose of testing.
- It has a command-line interface, ‘wdio,’ which makes test configuration as easy and simple as possible so that a non-programmer can configure the setup.
- It has support for most BDD and TDD test frameworks.
- It has good support and enthusiastic developer community and end-users, which gives it an edge over NightwatchJS.
- It can be used with ‘webdriverCSS’ to compare CSS stylings of an element on the webpage.
Cons
- Since it is a custom implementation, it is also a disadvantage as it deviates from generic syntax which may confuse selenium developers coming from other languages.
- It can be used for automating AngularJS apps but it is not as customized as Protractor.
NightwatchJS
Pros
- Similar to WebDriverIO, it is also a custom implementation of W3C WebDriver API.
- As all other JS libraries, it is easily extensible–one can write custom commands, assertions extending it.
- It has its own testing framework and assertions mechanism which solves the programmer’s headache of choosing a testing framework.
Cons
- It does not have many choices for unit test frameworks as it has its own testing framework and also supports Mocha.
- Slightly lesser support than WebDriverIO and Protractor.
TestCafe
Pros
- 1-minute setup
- Web mobile supported
- Device emulator
- Multiple in-built selector strategies (React, Angular, AngularJS, Vue, Aurelia)
- Set test speed
- Compatible with Continuous Integration
- Typescript support
Cons
- Multi-tab, multi-window not supported
- No support for mobile native apps
- Unable to set Explicit waits
As you can see, there is not an easy answer for which is the best automation framework. But I’m sure you can find an answer to which is the best fit for your project. I hope this document will help you choose the one that matches your requirements best.
Need an easy start and run?
These are proofs of concept using the Page Object Model architecture implementation-specific for each framework, basically, plug and play. Please feel free to download the following Github repositories and play with the different frameworks so you can see the difference between them. You can also fork it and start with your own project or playground. Let’s do it!
Written by Edgar Figueroa, Senior Software QA Engineer at Wizeline (Originally Published in The Pulse)
