Better Software, Faster: A Tester’s Report On The Quality Assistance Model

¨Under the Quality Assistance Model, you’re always on the lookout for the next challenge. As a tester and engineer, I have a lot of freedom, support, and bandwidth to experiment that I try to use wisely … You must be vigilant and aware of all of your team’s workstreams, and you should spot opportunities to improve and experiment, closing gaps whenever you see them. And finally you learn continuously so that you can adapt when needed.¨

The pace of technology development speeds up every day. To cope, companies strive for excellence in every aspect, but most likely their main goal is speed and time to market. You can use the best technology available, hire the best people, and have an amazing barista serving you morning coffee, but if you don't figure out how those things work together effectively, your business is unlikely to succeed.

When I joined my current company as a test engineer, our team followed an established quality assurance process. However, product delivery speed needed to keep pace with the company’s growing revenues. As a result, the tried-and-true quality assurance process the company used was showing signs of wear. And we testers in particular needed to re-evaluate the way we worked. 

Thankfully, during that evaluation, we came upon the Quality Assistance Model, which supports business to scale and keeps quality high without the need to hire more test engineers.

What Is The Quality Assistance Model?

As originally defined by Atlassian, the Quality Assistance Model promotes the ownership of quality and testing by every engineer on the team, regardless of their role. This allows each engineer to develop code, test it, and release it to production without gatekeeping by a quality team or quality engineer. 

Then what do testers do? Under the Quality Assistance Model, you use your testing expertise to guide and educate the team, empowering and influencing them to produce high quality software. 

Sharing quality and testing responsibilities across the team spreads the testing workload more evenly and gives testers time to:

• Build testing infrastructure
• Fill gaps in test automation
• Explore new tools and techniques 
• Monitor production performance
• Conduct exploratory testing, ensuring sapient investigation of your product that automated tests cannot provide
• Support developers on their team, or even on other teams

The Quality Assistance Model Process

In this video Atlassian lays out the Quality Assistance Model. 

Following A Phased Approach To Quality Assistance

As a company, we wanted to be able to release any important front-end or back-end work to production multiple times daily. The schedule for native app release, in contrast, was weekly. Everyone understood the goal, but they also needed to know what their role was and how to play it. Ultimately, our adoption of the Quality Assurance Model was not so much a transformation as it was a mindset shift, in which we committed to owning quality collectively while ensuring the quality bar remained high.

We carefully thought through the transition plan and divided it into three phases. We knew that we would not succeed if we transitioned to a completely new way of working overnight. We needed a phased approach to define and achieve reasonable goals, identify emergent problems, and develop solutions. And we developed tools and metrics to measure success at the end of each phase.

Modelling A Troubled QA Workflow

In the development lifecycle we sought to change, development and test were out of sync. The critical symptom: mountains of open tickets in the Test column in Jira. To add to the testing bottlenecks that plagued our team, testers were releasing code to production at the same time that developers were adding new features at the other end of the process. The result was an inefficient ping-pong game, and the whole team was frequently stressed out.

A Model of The Quality Assurance process that the team sought to change.

The Transition Begins: Implementing The First Phase

In Phase One of our implementation of the Quality Assistance Model, we sought to replace old, suboptimal practices with new ones. For example, we didn’t have a specific place for testing in Jira, so when exactly testing should occur was anybody’s guess. We needed to establish clear, easy-to-follow processes to ensure that testing was happening in the right place and at the right time.

We decided on these objectives for Phase One:

• Establish example mapping, feature kickoffs, and feature demos as new practices
• Ensure that teams were consistently practising new techniques
• Ensure that tickets were not getting stuck in the Jira Test column

Example mapping. In the Jira Test column, activity was divided into two parts. In the first part, teams were to run an example mapping session (similar to a Three Amigos session) before development began. These sessions would help ensure that everyone understood the requirements, the unknowns, and testing that needed to be done. The tester on each team planned the sessions, working with the product manager and designer to make sure all requirements were captured, and then running the session themselves or taking notes. At the end of the session, the tester led a discussion about testing, writing down all examples and suggested test scenarios. At the end of the session, the team would have a good deal of confidence about what to do next, what to build, and how to test it. Our aim was to reduce or eliminate ambiguity.

Feature kickoffs and feature demos. The right side of the Jira Test column represented feature kickoffs and feature demos. Feature kickoff is a short catch-up between developer and tester before development begins to walk through final ticket requirements. This ensures everyone understands the requirement thoroughly.  Feature demo occurs when the developer is done building the feature, to ensure the ticket is done as expected. Initially, the tester conducted both kickoffs and demos, but as Phase One continued, the tester trained developers on how to run the sessions with their peers so as not to create a silo. In this way, the process would scale up easier and teams did not depend on the tester being available to get important work done. This practice embodied the goal of ownership of quality by everyone, not just the testers.

Phase One was potentially the most challenging. The teams were still doing some of their old process tasks while adopting new ones. Testing was still mainly done by testers as the team slowly worked to shift ownership of testing to the whole team. The beginning of a shift in mindset can be the most difficult for many people.

A diagram of Phase One of the team's implementation of the Quality Assistance Model 

Phase Two: Redistributing Testing Work And Changing The Role Of Testers

In Phase Two, we continued to establish new practices:

• Developers would write automated testing for new features and bug fixes, not just unit tests
• The whole team, not just testers, owned the process of releasing to production
• Testers helped keep focus on how testing is done; stories were rarely assigned to testers

Developers write automated tests for new features and bug fixes. In our previous development lifecycle, developers wrote unit tests, which are and continue to be a pivotal part of feature development. But unit tests by themselves are not enough to ensure that a feature is ready to go to production. And if we wait for testers to build out automation on their own, more bottlenecks emerge. So our new model sought to guarantee that developers wrote as many TYPES of tests as needed to ensure uneventful releases to production. 

Developers take part in production releases. In our old way of doing things, testers released code to production while developers worked on new features. This ensured an ever-growing disconnect between developers and testers. In Phase Two, we moved to joint ownership of releases to production. To guarantee joint ownership, we added to the definition of ¨done¨ for each feature a requirement that the code be moved to master AND released to production. 

Testers educate the team on how testing should be done and support testing efforts. Meanwhile, testers worked with developers and the team on how testing should be done for individual features while also evaluating how testing was done overall. Testers did take on some of the work to bridge gaps in test automation and to help developers increase their test coverage. Happily, none of this required the addition of more testers to the team! Sharing testing responsibilities ensured that we moved fast and efficiently and that the entire team knew a lot about good testing practices.

How Phase Two went: Asking developers to write integration and UI tests was not a quick switch. We often debated over responsibilities and ownership. We all knew that delivery speed might be a bit slower due to the process of increasing automation coverage, but we all knew it would help the team in the future. 

At the end of Phase Two, our team was far more aligned on process and goals than we were before. Development and testing were in sync, more bugs were caught before production releases, and test coverage was well on the rise, building our confidence that regression defects wouldn't show up in production. Communication and collaboration were greatly improved, and we had a broader base of common knowledge.

A diagram of the changes made in Phase Two

Phase Three: Fortifying Our New Practices And Establishing A Culture Of Learning

In Phase Three we sought to build on our progress in Phase Two:

• Developers owned test automation
• Developers assisted other developers with review, instead of relying on testers to do it
• Testers took on the task of monitoring production

Developers own test automation. In Phase Two, developers began to write tests for features and bug fixes. In Phase Three, they began to take on the ownership of test automation and its maintenance. Owning test automation means that you are accountable for all levels of testing for features you develop. That includes both your own code and potential impacts to other code in the project. Developers were called on to broaden not only their technical skills, but also the awareness of business logic and full accountability for the team’s deliverables, not just their own. 

Developers assist other developers with review. While testers were often available to support the review of tests the developers wrote, developers were to assume that role as often as possible so as not to create bottlenecks. 

Testers tackle production monitoring. You might be asking: where did QA fit in after the developers took on so much work? Apart from working with developers on daily challenges, the big task for testers was to build production monitoring. At that time, we didn't have a cohesive approach on how to monitor the impact of released features, nor did we know enough how to build systems that could uncover defects due to behaviours that were not easy to spot during testing. 

Since then we have created a good level of monitoring and observability. However, I think we still have to figure out a way to make it consistent for each team, and find an optimal way to get quick feedback on not only critical features, but also for other features we can learn more about.

A diagram of Phase Three of the their implementation

How Did We Do?

At the end of each phase, the teams measured their successes and failures. Each phase lasted at least three months to ensure we were comfortable with the new processes and that they were working as intended.

Testers’ role in the transformation was huge. Not only did they drive the changes in each phase, but they also coached developers on approaching testing earlier in the cycle. Clearly defined objectives for each phase helped testers to know where they were heading. 

Acting as influencers was challenging. Imagine having to shape the thinking and practices of others after years of following an established process, all the while making sure that delivery speed and quality remained intact. Happily, we testers had a lot of support and encouragement from our team members. Many people volunteered to facilitate meetings and take on new responsibilities so that testers could trim their overflowing task lists.

These were challenging times, but I can say that I grew as a tester, as a team member, and as a person.

How Testers At My Company Work Now

Under the Quality Assistance Model, you’re always on the lookout for the next challenge. As a tester and engineer, I have a lot of freedom, support, and bandwidth to experiment that I try to use wisely. I use my observational, analytical, and technical skills to discover where the next solution for a problem can bring impact and value not only in my squad, test discipline or technology, but also across the business. It's not an easy task. It means to face a lot of ambiguity, experimentation, failures and sometimes giving up on ideas that don't work.

My job has three main focuses: daily support for my team; leading the test engineering discipline; and mentoring junior engineers. I have to look ahead to the future while keeping a close eye on the work of the day ahead of me. On any given day, I act as an expert tester, an analyst, a product manager, or DevOps. You must be vigilant and aware of all of your team’s workstreams, and you should spot opportunities to improve and experiment, closing gaps whenever you see them. And finally you learn continuously so that you can adapt when needed.

Not surprisingly, we rebranded! We testers used to be referred to as ¨Quality Assurance,¨ but we are now known as ¨Test Engineering.¨ And we’re always thinking of how the role should change and grow.

To Wrap Up…

Any initiative that requires people to change the way we work is difficult. You are asking people to change their behaviour and habits and to alter their expectations. Not everyone likes to change. Even fewer people like to change quickly.

The transition took us a long time, possibly longer than we were expecting. But the model serves us well today, since we move fast, fix forward, and reiterate reliably. It also gives us room to fail safely, because we have tools and support to identify failures and fix them quickly. As a reflection of our success in establishing efficient, reliable processes, we have maintained a ratio of eight developers to one test engineer in the team.

The Quality Assistance Model won’t work for everyone. It requires not only technical skills to be able to navigate different stacks, but also the agility and open-mindedness to reinvent your role. You have to be proactive and disciplined. It won’t work for engineers who want a lot of guidance and who don’t like ambiguity. But a team whose members are willing to grow and change can benefit greatly by adopting the Quality Assistance Model. 

For More Information

• Shift Left or Shift Right? Why Not Both!, with Arlémi Turpault
• Discussion: Driving a Culture of Quality, with Ben Dowen, Caleb Crandall, and Parveen Khan
• Exploring the Dual Roles in Software Testing: Quality Advocacy vs Hands-On Testing, forum post by Richard Forjoe