A Guide to Bug Refinement in Software Testing: Streamlining Your Workflow

"The most heartwarming moment for me was when one of our regular attendees said that the bug refinement session is their favourite meeting of the week. It’s not just about refining bugs, it’s about working as one team to create quality software, and to have fun doing so!" 

Moving From Bug "Triage" To Bug “Refinement”

When I joined my current company, there were more bugs in the backlog than anyone could keep track of. Everyone knew there was a bug problem, but they didn’t know where to start. To top it off, many folks joined and left the development team over a short period of time. We saw changes in many key positions, such as our VP Product, Principal Developer, Principal QA Engineer and Tech Lead. To add to the confusion, the core development team shrank greatly in size, meaning some of us had to wear many new hats. 

As a new team member and a tester, my goal was to support the team in improving the quality of the product. But I could do this only if we figured out how to work as the team we had now become. 

We'd had regular bug triage meetings in the past, but nobody seemed eager to resurrect these sessions, for various reasons. However, I noticed that the team felt that feature refinement sessions helped us deliver higher quality work on the code for user stories. So we thought of applying that approach to bug triage meetings. And this was the way that the bug refinement session was born!

What Is A Bug Refinement Session?

During a bug refinement session, we aim to create prioritised, workable bug tickets. At a minimum, newly filed bugs are reviewed, replicated, refined, and then prioritised. After each bug goes through the refinement process, it is assigned to a specific developer to fix or it is dropped into the backlog for a team member to pick up at a later date. Using this approach, bug tickets are treated with the same love as a feature task, with clear acceptance criteria defined for the fix. 

We usually hold these sessions at least once a week, sometimes more frequently depending on project need. The structure of the meetings may vary; sometimes we use the session to review the validity of the bugs in the backlog. For releases that were buggier than normal, we'll use the session primarily for firefighting. The goal is always the same though: curate a high quality bug backlog containing workable tasks with measurable outputs any one of which can be picked up and easily worked on by the development team.

Who Should Attend Bug Refinement Sessions?

Typically, the bug refinement session includes people in the following roles:

• Project manager
• QA representative
• Developer representative

The project manager advocates for customer needs and ensures that the proposed fix to any bug will satisfy these needs. They provide important information on prioritisation, acting as a voice for the business, and they assist with finding the most suitable person to work on the bug fix.

The QA representative helps the group understand the context of each bug, expanding on the reproduction steps and test data setup and highlighting the impact of the bug and associated risks to the customer and business. They also contribute to discussions about the technical approach taken to fix the bug and how it will be tested.

The developer representative brings their technical perspective to the table, assessing how much effort it may take to fix the bug and highlighting any dependencies which might affect the work. They participate in root cause analysis by doing a deep dive into the code and investigating what led to the defect.

Some companies include a representative from their Customer Success team in the meeting as well. They bring a fresh, customer-focused perspective, as well as a deep working knowledge of the system, which helps keep the meetings focused on the real goal: building a better quality product for the customer. 

Our company has seen a huge difference in the quality of the fixes we provide because of the diverse mix of people who attend the meeting. The most striking difference is that the developers now understand the impact to the customer of the choices they make during the development process.

How Does A Typical Bug Refinement Session Go?

For bug refinement sessions to be productive, the bug backlog has to be well organised. If your team is just starting out with refinement sessions, the focus should be on clearing the bug backlog, closing old bugs which are no longer relevant due to system changes, and rejecting tasks that lack appropriate details such as full reproduction steps. 

After the existing bug tickets meet the new quality standards, a typical meeting flow might be: 

Triage of urgent new bugs. First off, attendees should be asked if they have brought any urgent bugs with them to the meeting. These will typically be bugs with a high enough priority and severity to warrant being pulled into the next release. The context around the bug will be confirmed, the group will agree (or not) that it is an urgent bug, then a full reproduction will be performed and the success criteria for a fix will be agreed upon.

Bug verification and root cause analysis. In this phase, the team determines whether a bug is a legitimate bug, or whether it is due to user error, bad data, or some other sort of misunderstanding. This quick assessment should be led by product management and QA, who have more experience in this area. Once the team confirms that the bug is legitimate, one or more of the code-literate attendees could try to track down the root cause in the product code. While they do that, someone else may have a browser open with developer tools loaded, to inspect network requests for potential data errors, or use tools to investigate frontend components. They may delve into the system logs to find any useful error messaging. The aim is to quickly identify the root cause of the issue.

Selection of technical approach. If root cause analysis is successful, the team then discusses the appropriate technical approach to correct the error.  Sometimes it might just be a one-line fix in a stored procedure, sometimes it requires a full overhaul of the way a whole feature will work. All team members participate in this discussion whether or not they are full-time coders. If the team cannot decide on a solution, all data from the investigation is recorded in the task, as well as any additional speculation on root cause, to aid the person who eventually picks up the work. Should a technical approach be agreed upon, that too must be recorded where the developer can find it easily (Jira story, for example).

Prioritisation. Your company might use a different methodology for prioritisation than ours does, but the common elements are that there is a clear agreement from the attendees on how soon the fix should be undertaken based on risk, the impact to the business and customer of the bug's existence, and the importance of the fix related to the current work in progress. This will be noted down on the task somewhere using an agreed notation, such as high / medium / low or P0-P5.

Assignment. Depending on the size of the company and their development processes, the bug may get added to a backlog for a single team or a cross-team effort. Or, it may get assigned out directly to the developer who has the most appropriate skillset and / or domain experience.

Cover all unrefined tickets. Unrefined tickets should be ordered from newest to oldest, and the above process should be repeated to cover any new bugs raised since the last session. This provides a high quality, workable bug backlog from which developers can choose tasks as their schedules and workload permit.

What Are The Benefits Of Bug Refinement?

Wherever a company is in their development lifecycle journey, running bug refinement sessions can bring a number of benefits. For less organised teams, it can be a great way to kickstart the collaboration necessary for further improvements. For more established departments, it can add another layer of efficiency to existing processes. It can even just be a great way to get to know various members of the team better, learning how they think and who they are as people. 

Efficiency And Speed

Bug refinement sessions align well with the popular Shift Left approach, since they bring the debugging, scoping, and design forward in the process. This approach can prevent the rework entailed by waterfall development processes, where work gets passed back and forth from developer to QA, often with communication limited to what's in the bug tracker. And, just as a refinement session roots out missing requirements in a user story, a bug refinement session highlights areas of impact that might otherwise go unnoted.

Developers can get frustrated because they spend time investigating a bug task they've been assigned, only to find out that it wasn't a bug at all and instead was a data or configuration issue in a test environment. A good bug refinement session results in tasks like this being rejected before they can take the developer's time. It's a great way to ensure that only relevant and workable tasks are on the backlog.

Relationship Building

Bug refinement sessions can help build relationships among people no matter what their role at the organisation. By working closely with colleagues from different disciplines, you get to develop a real sense of respect and understanding which might not have been there before, and this helps you work together more effectively. These strong relationships lead to benefits outside of the meetings, where you may see developers, testers, and customer success agents collaborating on  their work and sometimes even becoming friends. 

Shared Understanding

Details can often get lost when work passes from a person in one role to a person in another. Working together on tasks from the beginning can cement a shared understanding of the domain's language and concepts that may not have been there before. This means that all involved parties can easily communicate about the work, ensuring the understanding of the work is retained all the way from the customer raising an issue to the fix being deployed. This can lead to fewer instances where a fix is pushed out, only to find that the customer is still having issues.

Challenges

It’s not all smooth sailing when a group tries to introduce new processes in an organisation. As testers, we’re all probably familiar with the common pushbacks of “We just don’t have time for that” or “We’re already sick of meetings, how will yet another one help?” 

One challenge we had at our company was that the benefits of the bug refinement sessions weren’t immediately obvious to the leadership team. They ended up frequently questioning their value, leading to reduced attendance from members of their departments. Clear communication of the meetings’ output via clear and reliable reporting can be helpful in this situation, as can being curious about what is fuelling any resistance to the meetings. Sometimes, those who object are having trouble understanding something… but since you don't know what you don't know, they can't articulate what that is.  

Another challenge can be that repeated attendance at any type of meeting, bug refinement sessions included, can lead to people falling into a rut, getting bored, or just going through the motions. At this point, it’s suggested to rotate your line-up to give folks a break, meaning you’ll get some fresh points of view and more enthusiastic participation.

To Wrap Up

Creating high-quality, workable bug tickets brings tangible benefits to engineering departments, ranging from better relationships between colleagues and departments to faster fixes being pushed out to customers. It also creates a rare space for folks from different disciplines to explore new perspectives and learn from each other. 

The most heartwarming moment for me was when one of our regular attendees said that the bug refinement session was their favourite meeting of the week. So it’s not just about refining bugs, it’s about working as one team to create quality software, and to have fun doing so!  

For More Information

• When Testers Deal With Process Debt: Ideas to Manage It And Get Back To Testing Faster, Melissa Eaden
• Benefits Of A Bug Bash - And How To Run One, James Espie
• Experience Report: Introducing Change To An Organisation As A Test Consultant, Beren Van Daele