Robot Process Automation As A Power Tool For Testing

By Jesper Ottosen

Business automation tools called Robot Process Automation (RPA), provide a new powerful tool for testers and business users doing testing, especially in the context of implementing large scale standard systems like SAP, Microsoft Dynamics 365, Salesforce etc. [16]

While there are other power tools for web and API testing, the RPA tools are a class of their own, as RPA tools allow for codeless automation macros on the desktop. RPA tools can do some very handy things. They can be used for both test data and regression testing. In this article, we’ll walk through a real testing example and show how you can get started using RPA.

As RPA makes especially UAT and end user testing more effective than before, especially when implementing commercial off-the-shelf business systems, this seems to challenge the base of the testing pyramid [5]. In the testing pyramid model GUI test automation is expensive and should be done less, compared to unit and integration testing - which there should be more of. Using RPA as a test tool, it is possible to do a lot of GUI test automation as the tool can capture repeated IT system interactions.

Robot Process Automation For Business Flows

RPA is currently a hot topic [7] in automating activities for the business users. As many business users often have to interact repeatedly across multiple systems to do their work. RPA tools aim to automate simple business processes that are repeated many times on a well-known set of applications, as described in “A Brief History of the Rapid Expansion of RPA” [6]:

“RPA is pragmatically automating existing interfaces and taking the burden of maintaining them. With this shift, RPA is suited for automating the iceberg portfolio of legacy applications, and with that, decreasing resource intensiveness.”  

The tools allow for both scheduled and “batch” running of activities but are quite as often used to automate a manual process on demand. It’s much like an advanced macro that can both execute, set, and capture data across applications.

Caption: Here is a macro that checks on the web if it’s Friday.

In my experience, there is a distinct difference between a business analyst and a business user. The business analyst works alongside developers and testers in an IT organization while a business user is often the actual user of the application. Business users could be nurses in a hospital context, sales and order entry staff, or logistics and public servants.

RPA tools help the business users to specify their system interactions graphically and then run them on demand, or recurring on a server. It’s all about automating activities around data collection and data entry. For instance, placing orders in ERP (enterprise resource planning) systems, setting up data in the warehouse system or managing staff rosters.

Another example is “new hire” activities. Before a new employee starts, they need to be created in the user directory (AD), have groups assigned and registered in various systems. These activities are things the human resources department does in 30 minutes with varying accuracy. While the RPA tool can perform these tasks as an assisted process or an unassisted process, the RPA supported activities are faster and can have fewer errors, compared to HR staff “copy-pasting” [11] for the same tasks.

Getting Started With RPA

Automating the business process has long been a vision for many testing tools. The RPA is rather successful in supporting automating repeatable activities in IT systems.

Previously, HP tried adding reusable business components to their test management suite but it never caught on. It seems the RPA tools are aiming to fulfil some of the promises of making it easier for business analysts to prepare and run their testing.  

I have been running a test trial with the RPA testing tool “LeapTest” [12]. LeapTest has a default video capture of screen activity during the test run, which is a very effective way of logging the test execution. Also, it connects equally well to Citrix applications and Virtual machines, which is important in an enterprise business context. Other RPA tools are UIpath, “Automation Everywhere”, and Automic Software (formerly UC4). Tricentis, the software test tool vendor, also offers an RPA solution [8].

The business people in your organisation might already have an RPA initiative. You will want to choose an approach and tool that align with those initiatives. There is clearly a benefit in aligning with them, as they might already have put RPA and automation of the management functions on a roadmap or agenda to address at some future date.

Trying Out LeapTest

LeapTest has a 14 day free trial with full functionality. Try out a demo of the tool first. Investigate how well it integrates into your application under test and into your test management tool. When closing your trial period, you should have gathered enough examples and video files of test runs, so that you can make a decision to continue. These tools have a price point in the same range as Microsoft MSDN subscriptions [15], so they can be pricey.

Plan your request for tools to management around when they are doing budget planning. Pitch your request for funding based on how you will implement the tool in your daily work, and what problems it will solve, in addition to how much money and time it could save. It might both speed up your testing and increase your test coverage. Also consider preparing knowledge sharing materials for other team members, and other people testing around you.

Using RPA As A Tester

Below is a screenshot of a LeapTest “script” that gets the current application subversion (besides subversion 8.7). Login into the application is done under the “login” block, which is described as a reusable building block.

Caption: Sample RPA script to: find the minor version number (8.7.x).

While evaluating LeapTest, I also set up an additional script. Given a VM connection started the desktop application on the VM, the script logs into the application and finds a person in a given search criteria and adds a business activity to the person. The person with the given business activity is the starting point of additional tests. The script runs 1-2 minutes automatically. If I did it manually it would take me at least 5 minutes to perform the same operations.

After needing this kind of test data a few times, I found myself running the RPA script instead of traversing the steps myself. I also found myself using the script to log into the application instead of doing that myself too.

Using the graphical editor lets me easily reconfigure the tool to find other persons or initiate other business activities. Creating test data is only a matter of spinning the proper script. Similarly, I can prepare regression suites to run on demand. Some of the tools also allow for the scripts to run on a schedule, so that each morning I have fresh test data, or I know that basic functionality on the application is available.

Similar to my IXO screwdriver, a cordless battery powered screwdriver that is very effective in many configurations and powers, RPA gives my testing more power than individual “manual” screwdrivers. It even has bits for configurations where my available manual screwdrivers fall short.

Caption: The small IXO cordless screwdriver replaces the “manual” screwdrivers

Summary Of RPA Testing

The Testing Bits website has a blog post on testing and RPA that summarizes it this way [10]:

  • Code-less: No need to memorize any syntax.
  • Simplicity: Easy to create a process through simple drag and drop.
  • Scalability: It can be achieved by assigning work to multiple workstations.
  • Cost saving: Huge reduction in cost as very minimal workforce is required.
  • Accuracy: As the tasks are performed by the bots.
  • Productivity: As it is robotic, productivity will be very high.
  • Flexibility: Test process does not depend on the type of software under test, whether it is web based, desktop application or mobile application.

Myths Of RPA Testing

There are already some myths around RPA and testing, these are addressed here “Robotic Process Automation [RPA], Test Automation – Myths and Facts” [9]:

  • RPA is similar to Test Automation.
  • Testing with RPA is just like Test Automation.
  • Testing Tools like Selenium could be used for RPA.
  • RPA may result in job losses.

Reversing The Testing Pyramid

RPA tools are challenging the model of the testing pyramid by enabling business focussed GUI (graphical user interface) automation both on-demand and scheduled in nightly builds. Some of the tools I have seen have solved challenges of GUI automation such as testing if an icon was highlighted or not, or CSS elements having locations other than previously specified.

The TestBash Belfast presentation, “A Test Pyramid Heresy” [5], described that there are business tests, QA tests, and development tests. Often projects about implementing standard systems only have the business and the implementation partner. Companies and organisations that implement standard solutions, like Dynamics 365 or SAP, seldom have the need or the know-how to test the configurations or code below the GUI.

I know from a specific case that clinical hospital staff has used RPA tools to automate their testing without coded UI [19] maps or writing one line of code in a testing framework. No test engineers or SDET’s (Software Development Engineers in Test) had to take BDD [18] scenarios and auto-generate code for this to happen. First of all, because the hospital as an organisation never had SDET’s in the first place, they usually only did user acceptance testing on their previous solution. For the big replacement project, as this was, they wanted to test all the hospital processes and configurations implemented. With a large set of testing to do on the end application, they looked for a test automation tool to support the business subject matter experts and their testing.

While the business user persons test a lot, their primary input to the testing activities are still the business knowledge they have accumulated. They have little interest in learning technical test automation terms, or other testing terms for that matter. For business users, testing isn't their primary job, and they function more like project owners rather than testers. They like to be able to check boxes and move quickly to the next necessary task.

This project specifically found that RPA scripts were highly effective for a regression test suite, for test scenarios with many variations and for managing test data. They could build RPA building blocks that reduced 40 clicks to one “shared function” that could then be parameterized based on the specific test case.

Compared to previous testing with no RPA tool, the tests were faster to run and prepare already for the second run. Usually, a rule-of-thumb has been that the tests should run 4 times before break even. Tests with many variations saved time within the first execution using RPA as compared to having a business subject matter experts run the execution of the test again. And why should they, if we can empower business users with better tools to support their testing?

While some would say this is bad for testers, I prefer that testing is done close to the business user and by people with the best available domain knowledge. Sometimes this is a professional tester that lives and breathes with their application, sometimes it’s IT operations people, and in many enterprise and public organisations it is the end users [13].

While Chrome developer tools and similar tools do help the professional tester, it’s not something I would ask the business end users to look too much into (unless they are on a path to becoming testers - that happens). The ease of use and graphical display of RPA tools makes it easy for business users to describe the things they want tested in the user interface.

It’s a more business driven approach to test automation, compared to Gherkins [20], and Microsoft coded UI practices [19]. Especially when implementing standard systems where the project has very little  development code, but is mostly configurations and customizations of the commercial standard system (like SAP, dynamics etc [16])

Power Tools For Testing

RPA can power business testing, and that can affect the amount of testing done by the professional testing team either on the business side or the development side. Some testers might have to rearrange their work when RPA powers the business testers to do more.

One way for a tester to aid business users, while they are using RPA to support their testing, could be as a designated “tool coach”. The tool coach supports and facilitates business users to use the tool effectively. One way to do this would be for the tool coach to setup the shared building blocks, shared repositories, and guidelines on how to solve various testing challenges in the tool or application under test. It could be that the “tool team” schedules the nightly test runs.

There are many other power tools for the more interested tester. The article “6 Technical Testing Skills that Aren’t Automation” [1] highlights how to get started with things like source control, build-in testing tools in Chrome, and understanding logging. If your landscape has more integrations and web, then look into Fiddler (a web proxy [3]) or Postman. There is a great guide to Postman on GitHub by Danny Dainton [2]. Learning a proxy tool or learning general scripting language like Python [4] will also enable powerful testing into the inner details of an application.

In the context of commercial business applications, development is mostly a configuration of a solution developed by the vendor. While some tools exist that can automate SAP flows (like MicroFocus Unified Functional Testing tool [17]), these tools require coding skills that the business end users involved in testing seldom have. There hasn’t been a good solution until RPA, for automating large scale end user acceptance testing.

RPA tools can also improve and make your testing more effective with the same arguments as made above about business end user testing. There is no reason to see this as a threat to your testing, but a way to skill up that is perhaps less technical than usually considered.

The key story of RPA in testing is that it is “Automation in Testing” [14], yet it’s a class of tools to help us test setup test data, run regression test, and help us traverse 40 clicks to get to what is really interesting from the testing (not confirming) perspective.

Author Bio:

Jesper Ottosen is a Senior Test Manager at NNIT A/S, an IT outsourcing company that provides IT consultancy and IT services to Danish and international customers. There is no commercial tie into any RPA tools, besides that, we use them for some of our testing. Follow Jesper on twitter @jlottosen or on the blog.


  1. 6 Technical Testing Skills that Aren’t Automation by Jess Ingrassellino 

  2. All things Postman by Danny Dainton

  3. Fiddler by Telerik

  4. Getting Started With Python by Josh Grant 

  5. TestBash Belfast 2017: A Test Pyramid Heresy: a fresh look at test automation strategies by John Ferguson Smart    

  6. A Brief History of the Rapid Expansion of RPA by Roger Berkley

  7. RPA often starts out like a teenage romance: a lot of enthusiastic fumbling around that ends quickly, frequently leading to disappointment by Phil Fersht

  8. Tricentis

  9. Robotic Process Automation [RPA], Test Automation – Myths and Facts 

  10. Robotic Process Automation and The Testing Future by Swapnil Bhukan

  11. How RPA Can Help Companies Rethink HR Tasks by Nick Ostdick

  12. Leaptest 

  13. The domain expert is the tester by Jesper Ottosen

  14. Automation in Testing by Richard Bradshaw

  15. Microsoft MSDN pricing 

  16. SAP, MS dynamics 365O, SalesForce  

  17. Micro Focus Unified Functional Testing - the former HP QTP product 

  18. BDD

  19. Microsoft Coded UI 

  20. Gherkin