Responsive Matters: A Primer On Responsive Design Testing

by Gem Hill

The web is always changing. Coding standards, design trends, the way people interact with sites have changed drastically since the web’s inception.

Another thing that’s changing is how people access the internet. Screens are getting both larger and smaller. Consider the rise of internet enabled Smart TVs, and people accessing the internet using games consoles. Also, the fact that it only took a month for someone to hack a browser onto an Apple Watch, while the Samsung Gear S shipped with Opera Mini.

Spending a small fortune on various devices, not to mention a large amount of time testing your website or app on everything available, isn’t realistic. How can you test a responsive design and make sure your users are getting a good experience? Below are some tips, hints and practices to help guide you through responsive design testing.

Research Your Options

You need to balance catering to everybody and catering to the existing customer base of a site or application. You may know your existing customer base favours windows-based PCs, but does that mean you want to make mobile devices a lower priority, or try to get more mobile users? Your test strategy should take your business strategy into account. Test in the direction you are going as a company, but maintain what’s necessary for the current customer base.

Common Terms and Definitions

Breakpoints: Breakpoints are the point at which your sites content will respond to provide the user with the best possible layout to consume the information. Most of the time this is one for desktop, one for transitioning to a tablet layout, and one for a mobile layout, and these are set at pixels (a mobile breakpoint may be 320px). This may be fine for you. You may need to choose specific breakpoints for your mobile and tablet. Maybe you want to look at an in-between one for “phablets”, or a smaller one for smart phones.

Fluid layout: Instead of using pixels as mentioned above, a fluid layout uses percentages. So instead of a component being 70px wide on a desktop, 50px on a tablet and so on (known as a fixed layout), it will be 70% width on a desktop, and 50% on a tablet, or whatever numbers make sense. You can read about the pros and cons of these approaches here

Glanceability: This is a heuristic that refers to presenting important and relevant information to users in a way they can easily understand and act on at a glance. It covers many aspects of user experience; presentation, filtering, and context awareness. It means you want to consider what your users need to see in a smaller space and when. It’s been suggested that most people will only ever glance at a smartwatch for 3 seconds at a time. If you people to engage with your product on a smaller screen, you need to consider time as well as space.

Graceful Degradation: Nothing looks good on Internet Explorer 8 or below any more. The web has changed, and some browsers just cannot support things that are expected in modern web design (Internet Explorer 8 is pre HTML5, so none of those things are supported by it). Graceful degradation is when you design and build a site so it looks good on modern browsers, and is still a decent experience for older browsers, even if it doesn’t quite look as pretty. A lot of Android phones come with stock browsers that will have differing levels of support for various bits of HTML/CSS/jQuery/etc. You may want to ensure your site still looks okay and works well on these browsers.

Progressive Enhancement: This is the opposite to graceful degradation. Instead of designing and building for a modern browser then working backwards you start with your less modern browser, build something good, and then add bits of design - using HTML5 for example - for browsers that support them. If you’re working with a client that typically uses older tech (think government or healthcare), this may be the way to go for you.

Haptic Feedback: You know when you’re typing on your phone, and there’s a little vibration every time you tap a letter? That’s haptic feedback. It’s useful for people who are blind or have other vision issues as they can tell they’re tapping a button as opposed to inactive areas of the screen. Apple patented their haptic feedback for multitouch screens, meaning that you can get haptic feedback while pinching to zoom in or out. On both Android and iOS10 devices you can add access the haptic engines so your web app can trigger haptic feedback when appropriate.

Screen readers: The vast majority of mobile devices have screen readers, which will work both on apps installed on the device and websites accessed on the device. It’s always good to check that screen readers can make sense of your product on both desktop and mobile devices.

Domain Knowledge

Domain knowledge can come into play here as well. For example, the UK government didn’t start transitioning from Windows XP until 2015, and some healthcare and education institutions may use tablets for both their relative cheapness and portability. Macs may be popular with students due to Apple’s discount scheme, or Chromebooks may be prevalent due to their simple OS and Cloud based apps.

There are a few different options out there. If there is an existing site, check out marketing information and analytics. If they’re using Google Analytics, then this  will have details on browsers and devices that visit the site, as well as if those customers bounced off the site.

When looking for an overview of global statistics, Statcounter will give you graphs and numbers for device/software/browsers used to access the internet. The information gathered from statistics and domain knowledge can also inform your testing and where to focus your time and energy looking for responsive design issues.

Other Influences On Responsive Design

Localization greatly influences designs, so anything that is going to be aimed at a global audience needs to both be localized and responsive. This includes making sure any changes made due to translations are still responsive and possibly reviewing the device market in those countries.

Accessibility again greatly influences design, so consider how you’re going to make your website accessible across devices.

Mobile Lab Options

From here you can decide your next steps. If your organisation has the space and capital to build a device lab, using or creating a mobile device matrix will help narrow down the number of devices needed. Additionally, creating a strategy to keep your device software up-to-date, charged, and when to replace old devices with newer ones should be part of the overall design and setup of the lab.

If it doesn’t make sense for you to purchase devices then you can use emulators as a good approximation of most mobile devices on the market. Browserstack has phones with various browsers available, Chrome’s web dev tools contain an emulator and also has the added bonus of being able to throttle the network, so you can check responsive sites on slow internet and cellular connections. AWS offers a real device farm for testing you web application or site on devices. Xcode is great at emulating iOS devices and interactions like screen rotation.

Using a combination of emulators and real devices can keep costs down and still give you a great deal of information about your application or site. It’s good to review emulator and device choices at least once a year to make sure old devices are recycled and new devices, emulators, or browsers are being added to your testing tool box.

Responsive Begins With Design

Testing responsive websites starts way back in design phase of the product or feature lifecycle. Once you know what you need to design for, you can consider design options, what devices and browsers will best support the design and how to best test the design based on the domain and the target audience of the product or feature.

It’s worth remembering that the mobile experience of a site is no longer something that can be half done and may be the only way some people access the site. Research shows that increasing numbers of people are using things that aren’t desktop computers to access the internet.

Smartphone Usage

While smartphones are easily the most popular way to access the internet, especially with growing numbers of people not having traditional broadband connections in their home at all. Connected TVs and Smart TVs are also a growing trend. A recent US study showed that an estimated 71.4% of the US population will use the internet via a Connected TV at least once a month. Connected TVs don’t necessarily mean Smart TVs. They could be TVs hooked up to games consoles or computers which can connect to the internet, or set top boxes such as Apple TV. These trends should influence the responsive design chosen to implemented for a app or feature.



Design Choices Matter

When envisioning how a site looks and feels, most people envision a larger screen, a laptop, or desktop computer size. Very few think about the mobile view first unless it’s a core tenant of the design approach. Choices made for a mobile device display can inform the full design.

When talking with clients about their responsive sites, something like the following can often happen:

Product Owner: We want a carousel, but we’ll have to hide that on mobile

Tester: If we’re hiding it from mobile users, is there really a need to have it at all? Either it’s important enough to show all users or it’s not important enough to be shown to any.

This can help trigger a thought process of catering to mobile users in their own right as opposed to just paring down the ‘full’ design until it fits on a small screen.

Design elements like contrast, size, and simple labels become more important on small screens, making the site more accessible. If you work with a UX or UI designer, they can help with steering the product owner to more mobile friendly choices, especially if that’s the focus of what you are building. Being aware of what kind of design choices are best when implementing a responsive design can inform how you test the product and where issues might appear on certain devices or screen sizes.

Things to Consider While Testing

Are you testing the back end or Content Management System of the website/application or just the front end (or both)?

  • Admin user journeys may be more complex or fiddly than customer journeys, so if admins want or need to be able to manage their site or app from a smaller device, you need to take this into account when designing and building this section

  • Expectations, devices, and experience levels will be different for different admin users, so you may need to take this into account - this ties back into domain knowledge, if all the admin users are using company provided Blackberrys then you need to make sure you’re testing the front and the back end on these devices.

  • Internal users are users as well, and you want them to be able to do their job as efficiently as possible. The tenets of user experience, glanceability, and mobile considerations apply here as well. Even just given your admin users a dashboard that fits in important information on a smaller screen could greatly increase their engagement with the project.

Look for your complex user journeys and make sure they can be done on smaller screens and keyboards

  • Payment gateways.

  • Account management.

  • Two Factor Authentication

  • Downloading or uploading documents

Can you get an automation framework setup to test these user journeys on every build or deployment of the code?

  • Some automation frameworks will take screenshots or video of the tests running so you can check these to ensure elements are displayed correctly.

What breakpoints are you using?

  • The most common are 320px and 480px for mobile, 768px and 1024px for tablet and then anything bigger than that, but given the audience you’re aiming for, you may have different ones to work with.

Think about elements that will not show on these devices, and how this will affect the look and feel of the site.

  • Menus can be tough. If you have a nice hover effect on desktop, what happens on mobile? Tap to open the menu is the normal response, but what if that top menu item is a link to a landing page? The user selects the link, but instead of the menu opening, the landing page opens and the user has no easy navigation signposts

  • Elements may also need to get bigger on mobile devices; fingers being less specific than mouse cursors. Chrome’s dev tools turns the usual arrow cursor into a round cursor that covers a slightly larger area to emulate a finger, so you can test there if needed.

What about popups, animations, videos, anything that moves?

  • A site that you can’t scroll down while using a mobile device because the map is too big and you end up scrolling the map instead of the page is annoying to the user. Avoid it by instead linking to the map or having the map closed until the user interacts with it.

  • Google is cracking down on mobile sites that use popups. Reconsider how popups are used in an application and if they should be used at all. Google’s announcement about this back in August 2016 said: “Pages that show intrusive [popups] provide a poorer experience to users than other pages where content is immediately accessible. This can be problematic on mobile devices where screens are often smaller.”

What about accessibility?

Where are people going to be using the site?

  • By using Chrome emulated throttled networks, you can simulate what kind of experience users will have with different kinds of data access. This can influence elements of the responsive design and the site can be modified even further to recognize data speed limitations and only load what is necessary for the user in a timely manner.

  • If the site is likely to be accessed on 2/3/4G, then it may be worth making sure that the site can load with slower networks.

Portrait or landscape?

  • Rotate your devices! Make sure it works no matter how the person is holding a device.

Automation Considerations For Mobile Devices

If you want to automate your testing, there are platforms to do that, such as Calabash, or Galen for layout testing. If you’re leaning towards using actual devices, something like Ghostlab will allow you to test on multiple devices at once. AWS device lab also allows for automation against many devices and could give you access to hard-to-find devices or old operating systems.

Tips For Remembering Responsive While You Test

This sounds like a lot of things to remember and juggle, but it needn’t be overwhelming. A few tips to keep in mind when approaching this stuff.

Depending on how your development cycles work, you may find it easier to designate a day of responsive testing. Grab your devices, your vms, your emulators and throw them all at what you’re testing for a few sessions and see what comes out.

Alternatively, you may want to test on a story or task basis. In that case, maybe make sure you’ve got responsive testing on any to-do or checklists that you may have, so it doesn’t fall off your radar if you get distracted.

Don’t forget stock browsers. A lot of Android users will have Chrome, but Samsung devices come with the stock browser called ‘Internet’. Some Samsung devices won’t or may not be able to use Chrome as an alternative.

Responsive testing is a good way to get yourself out of your comfort zone. As a Mac user at work, testing on our Windows machines can be a challenge if I’ve forgotten how that works. Throw a Surface at me and it’s like I’ve forgotten how computers work.

This is also good for checking that things make sense as well. When you’ve been testing things for a while you can get used to the quirks inherent in the system. Responsiveness at its core is about making good and efficient use of space provided, and making sure things make sense in that space.

About Gem Hill

Gem Hill is a web tester at CTI Digital, specialising in Drupal testing. She has a testing podcast called Let's Talk About Tests, Baby, which covers all areas of testing. She lives in Manchester, with her partner Mark, and far too many comics and board games. She can be found on twitter at @gem_hill or @LetsTalkTests