When Products Fail — The Product Post Mortem

Failure in Product Management is common. A recent study conducted by George Castellion and Stephen K. Markham found that 39% of software products fail. That’s close to 1 in 3.

Failure should be welcomed. If you don’t fail, you are simply not moving fast enough. When products fail, taking the time to reflect and learn can provide incredible leverage to improve how you build products going forward.

This Product Post Mortem template provides a summary of how PMs should conduct these meetings to effectively maximize learning and growth.

How products are built

When conducting the Post Mortem, it’s important to track down where in the process something broke down. This starts with understanding the process by which products are built.

1. Finding the right problem.
2. Defining the right hypothetical solution.
3. Building the prototype.
4. Targeting the customer segment.
5. Testing & measuring the impact.

When running the Post Mortem, play through each of these steps in reverse order. Where in the process did you go wrong? And how do you know you did?

📈Step 1: Measuring impact

Did your product actually fail?

When launching a product, clearly define your metrics of success that will validate your hypothesis, and implement objective way of measuring it. Once the product is launched, someone should be responsible for measuring and reporting back results objectively. A good metric is measurable, leading (vs. lagging),

Common mistakes here:

1. Not having a success metric (oh boy).
2. Defining the wrong metric (e.g. output metrics, lagging metrics).
3. Over relying on qualitative data vs. quantitative behavior.
4. Inaccurate data (e.g. not measurable, biased, or incomplete).
5. Non statistically significant results (e.g. low sample size, changes within margin or error).

🎯Step 2: Targeting the customer segment

Did you ship to the wrong customer?

When launching a product, clearly define your target segment — focusing on the segment that you think has the biggest need for this solution. After all, market is equally important to product to find product market fit. Post launch, segmentation data should be used to further validate your hypothesis.

Common mistakes here:

1. Going after the wrong customer.
2. Going after a diverse set of customers that renders results insignificant (e.g. all traffic).
3. Not segmenting results by sub-segments (staying high level).
4. Not being able to define or implement segments (no customer data).

🛠Step 3: Building the prototype

Did you build the wrong product?

Building good products is hard. You might have identified the right problem / segment / solution, but what you actually build is wrong. The best method to identify if this is to measure leading metrics that would lead to the lagging metric you are looking for to validate your hypothesis. This will help you identify where in the prototype you got it wrong.

Common mistakes here:

1. Building a poor or confusing user experience.
2. Not solving the job to be done better than incumbent solutions.
3. Overcomplicating the product or shipping too late.
4. Not sharing early prototypes with users for feedback.

🔍Step 4: Defining the hypothesis

Was your initial hypothesis wrong?

Look — it happens. We are not always right in our hypothesis. The point of testing is to validate. A good product manager will often be right (this is the hardest thing to teach). But when they are wrong, they will reduce the cost of being wrong by shortening the cost of testing. By doing so, they can maximize the cycle of hypothesis to validation, and therefore learn faster.

Common mistakes here:

1. Having the wrong hypothesis.
2. Not clearly defining your hypothesis before building.
3. Not basing your hypothesis in any data or facts.
4. Taking up too many resources to validate your hypothesis.

🤝Step 5: Identifying the problem

Was the initial problem incorrect?

The vast majority of products fail because they fail to solve real problems. This happens when PMs jump to conclusions and lean on intuition rather than research and analysis. To avoid this from happening, invest in an external research function outside of the product management organization.

Common mistakes here:

1. Not executing proper customer research.
2. Not identifying real or material pain (e.g. “this would be nice” not “this is a huge pain”).
3. Not identifying the right root cause of the pain (ask the 5 whys!)
4. Letting your confirmation bias skew your conclusions.

The Post Mortem Template

Here are the section of a successful Post Mortem:

1. Description: Short description of the issue, 1–2 sentences.
2. Root Causes: Analysis of the underlying reason why this happened.
3. Resolution: Steps taken as a result of the issue (e.g. roll back, pivot).
4. Impact: Business impact of this issue (e.g. a sprint of development work).
5. Timeline: List dates and times relevant to the issue (e.g. analysis, decisions, specs, etc.).
6. Learning: What went well? What didn’t go well?
7. Action Items: To be filled in the postmortem with clear owners and timelines based on the discussion.

How to run the Product Post Mortem

1. Be constructive. A Post Mortem is not about assigning blame or putting people down. It’s not about assessing an employee’s performance. It’s about reviewing why, as a team, the product failed — and identifying learnings that can be put in place to increase the odds of success next time.
2. Be factual. Facts are more important than opinions. Detail the sequence of events, key decisions and rationale, data used to validate, etc. You can assign names to these events, but the focus should be less on who and more on what. Teams succeed and fail, together.
3. Be inclusive. A Post Mortem should include everyone that was involved in the product so they can have their say. Make sure everyone has their say.
4. Keep solutions for the end. The majority of the time in Post Mortem should be spent going throughfacts, identifying where the breakdown happened, and potential solutions.
5. Come ready. The Post Mortem should be driven by the main PM responsible, and the template should be filled in with the basic information (minus action items) to make the meeting more efficient.

Closing thoughts

Remember that all failures are opportunities to learn. Focus on understanding what forces caused the failure to materialize, and what key learnings to take away from it. Hindsight is 20/20, so don’t dwell too much in the past, and focus on what you will do differently next time — there is always something you can improve. By creating a culture where folks feel safe enough to open up, share failures, reflect, and take action — you gain a significant advantage. Over time, companies that exemplify growth mindset will outperform those that do, and win.