This article initiates a series of articles about benefit hypotheses testing. Please look forward for next ones 🙂
Before I became a PO, I had spent most of my career working in software testing domain. But don’t be afraid – this post is all about agile product owners and how to deliver amazing products that customers will love:)
Through my testing journey, I have learnt multiple different techniques. Although, I haven’t used most of them in my current product owner role. However, it seems that I’m still under an influence of one software testing method. It’s called the V-model. I think that every product oriented person should be aware of this technique, despite the fact it has originated before agile methodologies. Let’s see how it changed my view on the product craftsmanship.
Firstly, we should start with a satellite overview of the V-model.
V-model in software testing
The V-model, as I have mentioned, was created in the ‘pre-agile’ era. Test managers designed it as an upgrade to testing principles from waterfall methodology. It consist of two ‘axes’ as on the picture below.
On the left line, you can see different types of requirements basing on particular level of abstraction. You should start reading it from the top to bottom – you will start from high level designs and, ultimately, get to the development itself of the application. At every step you visit, you need to perform two actions:
- define requirements / design architecture / develop the code as defined on the stage.
- plan out testing activities for those requirements.
So that at every step of product / feature development, you will have corresponding verification or validation stage. You can find these on the right line. Horizontal arrows connecting left and right nodes clearly visualize relation between different levels of development life cycle and tests related to them.
During the product delivery, you would execute all steps from left to right, so the testing would take place after the whole analysis and development – it is still sequential model rather an iterative one.
Why was this model so good?
People considered this approach far ahead of its time. When you designed each of stages, you already invested and planed testing actions at the corresponding level. By introducing your testing SMEs early in the process you ensure that:
- you increase transparency in your team. You achieve that because everyone is aware what shall be tested when they design / develop the step they are responsible for.
- the product your are building is testable = can be tested. Developers know exactly all tests that are planned, so they can build the product in a way that support that testing.
- you plan out all crucial requirements – testers are masters in out of the box thinking so it’s great to always keep them in the loop.
In the end, teams that used this method achieved higher quality products with less number of defects.
Agile V-model
I think that in current era of agile methods, you can easily find the V model’s legacy in current software development method. Let’s not treat this model literally but more as a concept. Here is how I would visualize this:
The main difference between these two versions of V-model are:
- agile V-model act as a cycle for each of Product Backlog Items (PBIs) rather than as development sequence of the whole product.
- requirements are stored as different artifacts than previously.
- each of highlighted sections take place much quicker and more frequently.
In contrary, you can still see that requirements definition is sequential, from top to bottom. Same applies to planning tests.
How can Product Owners benefit from the V model?
Let’s drive back to our product oriented realm to check what value can Product Owners have from this method.
Testing first
Firstly, whenever you work with your team defining product backlog items, look for testing related traits. According to most of agile methodologies, it is your duty, as a PO, to either accept or reject the increment if it may not address customer needs. Plan upfront how you are going to do this to make those test smoother. You can also help with arranging user acceptance tests for releases.
I can imagine that you may not have direct impact on testing methodology at the technical level. However, please support all upfront test design initiative. This will both increase quality of your product and will increase transparency of requirements for the whole team. Everybody will know exactly what you and testers will expect and how they will validate the product.
The WHY
Secondly, as you can see, this whole diagram describes technical aspects of the product being built. It focuses only on how to efficiently deliver high quality product and doesn’t mention anything about the context of the product. Simon Sinek would describe that as product’s: HOWs and WHATs. However, the model misses fundamental question about: WHY do we build the product in first place? So let me expand this ‘V’ shape a bit.
Basing on the point above, I have added one level of testing – benefit testing. Before we even start thinking about the solution itself, you need to state the problem that you plan to address. So, in the left bubble you will work on artifacts related to problem definition and in the right one you can conduct your benefit tests.
Product Owners frequently tend to not focus enough on these activities – I actually consider that as one of their deadly sins (you can find the full list in my previous blog post)
How can Product Owners implement extended Agile V-model in their role?
Let’s check the left arm of the Agile V-model.
Problem statement
Start with writing down the problem that you want to resolve with your product / feature. As you are about to plan your next increment, I presume that you know why you are building the increment. However, if you struggle to get to the problem level of your product, please let me know about your case – I’m planning to cover this topic in my next article.
Benefit hypothesis
When you have the problem defined, you can move to the benefit hypothesis statement. In that sentence / paragraph, you need to name what outcome the user can expect if they start using your product. Please keep in mind that we are not talking about an output. The main difference between those two words is that the output defines what you receive (e.g. a car) and the outcome defines how you benefit from the output (e.g. I save one hour every day in commute time).
Benefit hypothesis examples
| Problem statement | Benefit hypothesis |
| Every week I have to report back list of tickets closed in the system. I have to manually visit five different pages and create an excel file with results. This takes me whole Friday, every week. | If we provide a dashboard that will summarize list of all tickets closed in the system, our users can save 80% of the time they spend in preparing such reports. |
| When I buy a meal in the drive-through, I often spill the drink on the floor of the car. | If we add a cup holder in our car design next to driver’s seat, we can increase customers satisfaction by 2% and reduce number of complains in social media by 3%. |
| I love taking pictures of interesting things that I see when I’m on a walk. Frequently, I cannot get to the camera app in my phone active quickly enough to make that picture and I lose the chance for an amazing picture. | If we add shortcut (app/button) for taking instant photos in our phone, we can increase Net Promoter Score (NPS) by 0.5. |
| As an archiving team in our company, we have a thousand documents a day that we need to scan and upload to the system. It takes a minute to do so for each of them. | If we provide automatic scanner for our archiving team, we can save a million dollars a year for our company. |
MVP
When Product Owners know the problem and what you would like to achieve for your customers, now focus on the smallest possible increment that can cover that need. Break down your work and prioritize only necessary features so that you can reach to the experiment as soon as possible. To do this efficiently, you can try checking my top prioritization techniques.
To conclude, here is the output that you should have for this whole step:
- stated problem and the benefit you would like to achieve for your users.
- experiments to verify that expected benefit.
- MVP scope to enable mentioned experiment.
By making this change, you not only define testable benefit hypothesis. You also plan out, from day one, that you have to perform relevant benefit testing and know how to quickly get to the experimentation phase with your MVP.
What about the benefit hypothesis testing itself?
I consider that the right arm of this Agile V-model is huge subject by itself and I’ve decided to cover different techniques how you can efficiently conduct experiments in follow up posts.
If you found this article interesting and would like to make sure to not miss next ones, feel free to sign up for my newsletter bellow.