We release updates to the Monzo iOS and Android apps every couple of weeks. Those updates are now reaching hundreds of thousands of customers so we have to be careful that they work as smoothly as possible, with as few bugs as possible. In this article, Priscila Minks Zanuzzo outlines how we manually test the apps before releasing them.
Breaking things can be stressful, frustrating and inconvenient at the best of times. But here at Monzo it’s a huge part of how we improve and iterate. In order to offer products our customers can trust, we need to test features to their limits and at times, as many of our cardholders will know, the community helps us do this! Before we ask them to help though, there’s lots of internal testing for me to work through.
Every new feature starts life as a ‘kick-off’ session, where designers, developers and product managers meet to establish the purpose of the feature and how it should work. As a product tester, I go to that session to try to understand the critical path — what are the essential requirements necessary for this feature to deliver the expected outcome? When testing sending money to a friend for example, being able to view my contacts in the app is essential for me to complete my payment and so I need to focus on ensuring that works in as many situations as possible.
Checking this flow of actions works perfectly is the priority — exploring the variety of related edge cases (much less common bugs which only a small number of users may encounter) comes later! It’s really important for us to strike a balance between shipping updates often whilst having confidence they will work as they should, something which requires plenty of consideration when prioritising which bugs we have time to fix before the next release.
I use Jira to share bug reports detailing the sequence of actions leading up to the crash, so that engineers working on the feature can replicate the situation and investigate further. Identifying the cause of a crash requires some detective work! We’ll often need to repeat the same process numerous times in an attempt to pinpoint the source of the problem. We’ve added a special menu in the staff only version of the app that stores easily shareable error logs and provides some helpful pointers.
There are a variety of ‘staging’ environments we use to simulate account activity in order to test early versions of new features. We can recreate almost everything you might see or use in the app, from outbound transactions to spending reports and also see how it looks on our end, so we can better understand how certain actions in the app interact with the internal tools we use to solve customer support queries. This is particularly helpful when revamping things like our identity verification process, which requires interaction from both the customer and our customer operations team.
You may have seen our weekly tweets calling for Monzonauts to take part in our user testing sessions. If you haven’t yet and fancy a preview of something exciting we’re working on, get involved (we’ll even throw in lunch)! User testing is the closest we can get to real-life usage and it’s super useful when trying to polish the rough edges of new features.
Our approach to planning user testing sessions and selecting tasks depends on what we’re trying to find out. In cases where we’re trying to understand how users interact with a certain feature, we focus on letting people play with prototypes. We’re looking to see how customers navigate and react to each stage in the process. Other times we’re looking for more qualitative insights, to understand our users expectations and so we’ll have a more general conversation around the topic.
User testing sessions are valuable to different teams in different ways. For product managers, they help understand the real priorities and problems our users have. For designers, user testing sessions are the driving force behind a series of changes and revisions to the design which will ensure the end result is as intuitive as possible. Any other team at Monzo can also join in to test things they’re working on in the real world.
One of our biggest challenges is reaching a sample of volunteers who accurately reflect our customer base. We often find the attendee group is not as diverse as we’d like it to be, and this is something we’re actively trying to fix. Unconscious bias can sometimes affect the quality of responses and makes it harder for us to make relevant and valuable improvements. If you know anyone who isn’t yet using Monzo and would like to be involved in our user testing we’d love to hear from them!
Pressing the button
Before an update is released, we need to be confident there have been no regressions. This is where a change to the app breaks something that was previously working fine. We’ll test ‘backwards’ to check this, looking at all the other parts of the app which could potentially be affected by the update and making sure they still work. From there, we can release the app to staff to test, followed by users in our Android Beta and iOS Testflight channels and then finally launch to the public!
As you may have noticed, the testing doesn’t stop there! Each release contains a bunch of bug fixes as a result of additional testing and reports from the community — another shining example of our customers’ contribution to making Monzo the best bank it can possibly be!