.png){kind=icon}
Test automation is one of the most discussed topics in the QA industry. We asked Nick, our Lead Automation engineer, about Test Automation: how to get started, how to choose a programming language, and the best tools and strategy to implement a successful test automation process in your project development cycle.
Starting Test Automation from Scratch
Choose a Language and Become Familiar with it
Choose a programming language you and your team are familiar with. If there is no preference or if you are not familiar with a language, python is a good option. Python is an easy to read intuitive language with a large number of maintained and documented libraries, as well as an extensive collection of getting started material and help articles.
You’ll want to make sure you’re comfortable with the language. Become familiar with the different ways to store and access data (search “[my language] data structures” to see documentation), how to separate functionality into small functions or methods, and how to add external libraries to your code, probably using some sort of package manager. Once you feel comfortable creating a basic card game in your language, you are ready to automate.
Choosing a Framework
When it comes to choosing a framework, it’s important to consider what best fits your team’s needs. Things like how results are saved and presented, how well documented the framework is, how popular the framework is, how much “work” the framework does for you in setting up tests will all need to be discussed with your team.
Working with proof of concepts is very useful. One strategy you can adopt is to create several different frameworks so that you and your team can weigh the pros and cons of each and choose the best one for the job.
Setting up goals
Before you and your team set automation goals, you’ll want to have a good understanding of the best use cases of automation, and the limitations of automation. Test automation is very good at automating things like UI flows which do not change or only change slightly, such as listing a new product on a website. Test automation is usually not good at things like verifying the visual layout of elements. You also need to think about time to refactor tests to keep them up-to-date with the testing target. If a site is changing rapidly, refactoring automation tests can be time-consuming and be more work than it’s worth.
When creating automation goals, it’s best to tie them to user stories or requirements, just as you might with manual test cases. You may want to focus on goals that provide the greatest value, such as adding automated tests for critical flows or reduce the greatest number of manual testing hours.
Test Execution
The great thing about automated tests is that they can be run as frequently as needed. While some tests can be very simple and fast, others can be costly to run. If you are using a service like Saucelabs or Browserstack, or if they require a powerful machine to run, tests can quickly become expensive. Those sort of long-running, expensive tests should be limited to only what would provide valuable feedback.
As a general rule, tests should be run only as much as the results of the tests can be analyzed by a person, anything more and you may just be wasting resources, any less and you may miss bugs.
The frequency also depends on how the results of the tests are communicated. Some automated tests give detailed results which must be analyzed by a person, while other critical tests could quickly communicate results, maybe through an alert in a slack channel. Tests which can quickly communicate results could reasonably be run more frequently.
Test Automation Tools
For UI web app tests, Selenium is the most popular and established tool. It allows the user to automate browsers using keyword phrases. Selenium describes a set of commands that a browser accepts, using a standalone application to issue commands to the browser that is being implemented. This means that the library which issues Selenium commands can be implemented in any language. There are official language bindings for Ruby, Java, Python, C#, Javascript, and non-official bindings for almost any other language. Selenium also allows for automation of all popular browsers including Firefox, IE 11, Safari, Opera, Chrome, and Edge.
Selenium is a tool that can be used in automated tests however it is not a testing framework.
Manual vs Automated Tests
Automated testing is excellent at verifying that specific flows or functionality of a website are working properly. These automated tests could be kicked off with each new build of a website, quickly giving valuable feedback to the developer. Whereas manual testing may take days to schedule and complete. Automated tests can also be used to set up alerts that some critical functionality is broken.
Manual testing is much better at testing new components or flows, since manual tests could be done faster than the time it takes to automate. Manual tests are also better when it comes to exploratory testing and usability testing.
Conclusion
Test automation is an extremely powerful tool which can reduce manual hours and give critical real-time feedback to developers. We have been able to create automation suites for clients which plug directly into the CI/CD pipeline, as well as longer running scheduled tests. Both of these types of automation give our clients valuable information, and give developers greater confidence in their code changes.
If you’d like to know more about our test automation services, contact us today!