How to start with Key Metrics, Assumptions and Tests.

1. Pick ONE Key Metric to focus on.

We believe the MOST important thing is to focus on ONE thing at a time.

“A startup can focus on only on metric. So you have to decide what that is and ignore everything else.”  (Noah Kagan, Founder of AppSumo)

In a startup there are always going to be more things to do than time available.  That’s just a fact of life in a startup and its never going to change.  

So, to be successful, you have to decide WHAT you need to focus on right now, and as far as possible you should ignore everything else.  That will give you the time to concentrate all your energy and resources on your most important Key Metric.

Once you’ve improved one part of your business, you’re then going to pick the NEXT Key Metric, and like before, you’ll aim to ignore everything else and totally focus all your energies on that Key Metric.  When you’ve improved it, you’ll pick the next Key Metric.  

An so the cycle will repeat as you systematically work through your business, eliminating risks and improving your product market fit and your sales funnel.

So, in summary, here’s the process from 30,000 feet:

1. Pick ONE Key Metric.
2. Ignore everything that’s not related to that ONE Key Metric.
3. Focus ALL your work on improving that ONE Key Metric.
4. You’re going to test a bunch of assumptions around that Key Metric.
5. When you’ve improved that Key Metric, pick the NEXT most important Key Metric.
6. Then, repeat from Step 2.  Again, again and again.  

You’re constantly focusing and learning.  Success in your startup is 99% perspiration.   You are going to apply a rigorous disciplined scientific focus on the most important things.

So, here’s how we’re going to apply that process to the Startup Dashboard:

First, like we said,  you’re going to pick ONE Key Metric to focus on.   You’re going to stop and write down what you want that Key Metric to be. Its going to be your goal.  And it will display right at the top of your Dashboard.  

Now, EVERYTHING you do from here on has one single aim and that is to move that ONE Key Metric towards the Goal.

Next you’ll start defining your Assumptions and how you will Test those assumptions.

What’s an Assumption?

An assumption is something we don’t yet know the answer to.  You can call it a Hypothesis.  Really - these are just fancy words for “guess”.  BUT - and here’s what’s special.  You aren’t just randomly guessing at things.  You’re going to make a guess, and you’re going to run a specific test check whether our guess was right.  And then, you’re going to run that test and measure the result and LEARN whether your guess was right or wrong.  

It doesn’t matter whether your guess was right or wrong.  What DOES matter is that you LEARN from the test and that you are able to use that learning and share that learning across your team, and also with anyone else who’s a stakeholder in your business.  That could be your advisors, mentors, investors, etc.

An example assumption:

For an ecommerce site I was working on, there were lots of products on sale and we felt that we should be making more sales. (Our conversion rate was less than 0.5%). We “guessed” that if customers could find the products more easily, they would be more likely to order and we would sell more.  ASSUMPTION: “If we make it easier to search for products, we’ll increase sales.”

How do you use the learning?

As you learn, you are able to change what you do in your business and also to come up with new assumptions (guesses) based on your learning.  As you keep learning what does and doesn’t work, you can keep doing MORE of what works and LESS of what doesn’t work.

Example Metrics by Funnel Stage:

Some example metrics for different stages in the sales funnel:

Acquisition Metrics:

• Cost of customer acquisition - The cost of getting someone to buy something;
• Unique visitors -  The numbers of potential customers who only visit once;
• Fraud rates - The amount of fraud per channel.

Activation Metrics:

• Conversion rate - The number of visitors who buy something;
• Top keywords driving traffic to the site - those terms people are looking for associated with your business;
• Uptime and reliability - Number of complaints, problems and outages the company has;

Retention Metrics:

• Purchases per year - The number of purchases made by each customer per year;
• Abandonment - The percentage of people who start to make a purchase and do not finish the process;
• Stickess - this is how much the customers use the product, eg: one time per week;

Referral Metrics:

• Average shopping cart size - The amount invested in a purchase by a customer;
• Virality - word of mouth advertising and sharing by each of the visitors;

Revenue Metrics:

• Revenue per customer - The lifetime value of each customer. That is how much value you get from each customer;
• Upselling - The cause that makes a customer increase their buying and how often does it happens;
• Ads - Promotional content in return for cash;

Example Metric Values:



Here are some example Conversion rates for an e-commerce business*:


Schwan’s - 40.6%

woman within - 25.3%

Blair.com - 20.4%

1800petmeds.com - 17.8%

vitacost.com - 16.4%

QVC - 16.0%

ProFlowers - 15.8%

office Depot - 15.4%

Amazon - 9.6%

Tickets.com - 11.2%

Ebay - 11.5%

SOURCE * http://www.marketingcharts.com/direct/top-10-online-retailers-by-conversion-rate- march-2010-12774/

 http://www.conversionblogger.com/is-amazons-96-conversion-rate-low-heres-why-i-think-so/

2. PRIORITISE: Test the MOST risky assumptions first. 

"Building a successful product is fundamentally about risk mitigation."  
(Ash Maurya, Running LEAN)

Risky doesn’t always mean the most important, or the biggest. Its the ones that put your entire business model at risk. They are the ones that you should tackle first. 

It is important to define priorities on the assumptions we want to validate. But, often we don’t know what to test next. So, here’s our view on defining the most risky assumptions on your model. 

If you already have a product launched with a market/fit:

1. Define the flow, as a customer, from the awareness to getting you revenue (or, more briefly, define your sales acquisition funnel);
2. Is there any part of that process where customers get usually stuck? Are any part of your funnel that you “believe” it’s not performing well? Maybe it’s time to understand what’s happening on your model to a customer arrival (Acquisition) to a point where you’re being paid (Revenue);
3. Define an assumption you wish to validate that you believe it’s the reason your funnel is not working as you wish; If that part of the process you’re aiming, suddenly stopped working or didn’t exist, would your product stop working or your customers leave? If you change something, does you model may grow instantly?
4. That’s where you should start.

3. Define "falsifiable" tests

Ok, now you’ve defined the most risky assumption. You need to test that. 

You need to create a quantitative way to define it as true or false (or validated and invalidated). In other words you need to turn that an assumption into a testable form.

The Lean Startup methodology is based on the scientific method, which is a process where you define an hypothesis (assumption) and converts to a falsifiable hypotheses (test). 

"A falsifiable hypothesis is a statement that can be clearly proven wrong." 
(Ash Maurya, Running LEAN)