A Quick Start Guide To Learning Performance Testing

Brought to you by The PerfBytes Community

Of all the myriad types of testing in the non-functional realms, performance testing will likely be one of the most technically challenging. Whether you’re new to the vast architectural exposure, the deep system knowledge required, or the nuanced experience of real-world traffic modeling, performance testing can be limitlessly intimidating to new testers. It can also be the most rewarding and fun testing you ever get to do!

That’s why the performance geeks from the PerfBytes community put together all of our best suggestions to help you. In the sections below you will find different ways to learn about performance testing and practice performance activities. The recommendations are loosely ordered by level of difficulty. If performance testing is a totally new thing for you, then we would suggest the items mentioned first for each list. As you make progress, be sure to let us know how it’s going on The Club! Best of luck and remember to have fun!

Reading

Perhaps you are familiar with the acronym: RTFM. That stands for “read the fabulous manual” and that’s exactly what you should do: read the fabulous books about performance listed below. In our experience, people like different styles of writing, so there are different sources represented in the list below. You should not only be familiar with each publication, but also take some time to find the best author to fit your style of learning about performance. Below are suggestions from different leading authors in performance testing and engineering. Start with Chapter 1 for each book, and if you like what you are reading keep going, but don’t take my word for it . . . start reading!!

Measuring

One of the essential critical thinking skills to practice, as a performance tester, is measurement. The measurement of time elapsed during an action or step in your testing actions helps you to become aware of potential performance failure, from the end-user’s perspective. The measure of volume or size will help you to become aware of the capacity limits of a system and it’s potential overutilization. How many blueberries can fit in my breakfast bowl? In your mental practice, begin measuring things outside of a software program or computer system.

For instance, every day measure...

  • how long it takes to brush your teeth

  • the number of people entering/existing a space for 5 mins

  • the speed you travel in a vehicle

  • how long it takes for your computer to restart

  • how many animals you see in 15 mins

You should learn the basics of calculating the minimum, maximum and average values for your measurements. And then work on more complex calculations like standard deviation, mode, median and percentiles. What is the average # of blueberries in my breakfast bowl?

Everyone loves pretty graphs. Now that you’ve mastered the calculations, it’s important to know how to visualize the data in graphs with pretty colors and lines. Take the measurements you’ve collected over the last several days; pull the data into a spreadsheet, and start making graphs.

Listen to a Podcast

There are times when you might need to learn by osmosis. Well, at least with your ears. Whether you’re driving, exercising or trying to ignore the other people talking too loudly next to you on the train, here’s an awesome list of podcasts about performance:

Writing

One of the most valuable things you can do with everything you’ve learned about performance so far from: reading, measuring, calculating, graphing, and listening to countless hours of podcasts is to start documenting stuff. Write about your experiences and how you apply them where you can with work projects, presentations, or even blogs. Below are some suggestions on what you can write. 

  • Performance User Story: Write a user story about end-user functionality and give performance attributes to it for response time and throughput objectives or success criteria.

  • Performance Issue Write-up: Research one existing, known performance issue in your application and document how that behavior could be measured and monitored. Consider how to search through application logs, monitoring system resources, or end-user behaviors. How would you prevent the issue from impacting end-users? Record your findings and present it to your project manager or DevOps department.

  • Prerequisites For An Application: Document the prerequisites for a performance test of your application - what needs to be in-place for data, configuration, environment, accounts, tools.

  • Transaction Flow Diagram: This is an image that illustrates how data comes into a component, what happens to the data inside the component and what comes out of the component (Concept Draw).

  • Network Diagram: This is an image that includes the front-end clients, middle-tier servers/apps and the back-end data sources (Network Diagram 101).  

Collaborating

Performance testing is a great way to make friends and influence people. Some of the best ways to learn new concepts or ideas can be to seek out experts or facilitate discussions with them on things you’ve read, listened to, or wrote about. Use the examples below to start your own collaboration.

Ask Performance Questions:

  • Ask a performance question in a meeting where only functional requirements are being discussed - for instance, “Do we know how many people will use this function?” or “How fast should this application respond for function X?”

  • Ask a complete stranger if they ever complain about computers being too slow. Write down what they say qualitatively and quantitatively.

  • Ask the network people why they always get blamed for poor performance - write down how they respond about specific known limitations in the network.

  • Ask a project manager for their list of tasks associated with performance testing or performance engineering projects. Write down how they respond, what tasks they list and the timeframes they think each task will take.

  • Sit down with your architect to understand the dependencies and relationships between tiers and components. That will help you better understand causation vs simple correlation of random metrics.

  • Ask someone why you are/aren't doing performance testing on your project - that is, if you aren’t. If you are doing performance testing on your project, do you know why? Ask a colleague why they think performance would be important.

  • Ask if you can help troubleshoot a production performance issue: with systems, applications, database or network.

Performance 1:1’s:

  • Have a 1:1 interview with a performance tester that has more than 5 years experience, ask them about how they got started.

  • Have a 1:1 interview with an end-user about their experience of the application’s performance - write down what they say, both qualitatively and quantitatively.​

Pairing and Mobbing:

  • Arrange a pairing session with another team member to dig into a performance problem, analyze performance trends or discuss new objectives for performance. You can conduct exploratory performance testing in the session.

  • Conduct a session of mob performance testing where you take turns driving at the load testing and/or monitoring tools. Discuss what you see and learn during the session to determine new objectives or opportunities for performance optimization.

Lunch & Learns:

  • Facilitate a performance party where you get people together and have them do a manual load test of the same web page or same application function.

  • Invite your OPS team or DBA’s out for beers and ask them about the 10 worst performance issues with the system and applications.

Tools & Skills

Sometimes people start with tools first, trying to learn how to use them before they understand the principles behind the tool. Hopefully, you have worked on some of the previous sections and have a better idea of what performance testing is and isn’t. Hopefully, you have an idea of how you’d like to use some of the following tools, concepts, and skills listed in this section. If you already work with a tool, try something different and do a comparison. Practice and time will decide what you ultimately become comfortable with, but finding out what’s available and what a tool can do is the first step.

Learn:

 Create:

  • Create a load test with 10 concurrent users creating 10 transactions per second; measure the min, max, average and 90th percentile for response times.

Investigate:

​Download:

Explore:

Contributors: Mark Tomlinson, Perze Ababa, Eric Proegler, Rafal Szypulewski, Andreas Grabner, James Pulley, Cliff Benedict, Melissa Eaden

 

PerfBytes Bio

What is PerfBytes? It's a live internet radio show about performance testing and performance engineering, which is hosted by Mark Tomlinson and James Pulley. We got our start back in 2012 during a series of early-morning breakfast meetings at the International House of Pancakes in Beaverton, Oregon while were were both working an engagement for a customer nearby.

As we consumed copious amounts of pancakes, omelets (made-to-order) we also engaged in several serious debates about hot topics in performance testing and engineering such as, what's the difference between a load test and a stress test? And, why don’t testers understand the negative impacts of over-provisioning virtual machines?

Of course, each morning It took about an hour and a half to agree that we hadn't come to an agreement on how to answer these questions, so we scheduled another breakfast meeting for the next cold and rainy Tuesday morning. Over the course of those six months we met with several other performance gurus, kept meeting for breakfast and kept hashing over the contemporary practices, learnings, techniques, pitfalls and tools used by load testers. And that led us to create a podcast to share those discussions with the greater world. Now almost every episode we cover those things that aspiring performance professionals want to learn:

  • to increase their awareness of performance subjects about the science behind performance testing

  • to implement performance practices as advised by the real stories from industry practitioners, vendors and consultants

  • to translate these improved performance practices into value for their organizations