Reading Testing with Humans gave me new tools and led me to rethink how I approach experimentation. As a follow-up to their earlier book, Talking to Humans, Giff Constable and Frank Rimalovski offer a detailed framework for decision-making through practical, well-designed tests. In this article, I’ve pulled together the main takeaways and how they apply to building a successful business.
Table of Contents
- Introduction
- 1: Prioritize What Could Break You
- 2: Create a Simple Financial Model to Expose Blind Spots
- 3: Build Experiments That Are Small but Robust
- 4: Match Your Test to the Truth You Need
- 5: Choose the Right Kind of Test
- 6: Set Clear Criteria Before You Begin
- 7: Make Decisions, Not Excuses
- 8: Fight the Bias Volcano
- Final Thought
Introduction: My Flawed App
Not too long ago, I spent two years developing a financial literacy app called Money Merlin. I thought it was what college students needed. It had challenges, rewards, and lessons on budgeting and saving. My team and I launched it with confidence.
However, I quickly found out that most users didn’t stick around. The features I believed in didn’t matter to them. I had built based on my vision, not on real-world evidence.
That experience taught me the value of testing first and testing often.
Before you spend months building a product or launching a business, it’s worth asking: What if your idea is wrong? The book Testing with Humans by Giff Constable and Frank Rimalovski offers a practical answer: run experiments.
In this article, I draw from that framework and break it down into eight powerful principles. These principles would have helped me with validating and developing my mobile app. Yet, they can also help any entrepreneur, team, or innovator make confident, data-informed decisions.
1. Prioritize What Could Break You
Every idea comes with a set of assumptions. Some are harmless, while others can be potentially fatal. These assumptions are the beliefs we hold about how people will respond to what we’re offering. We might assume that our target audience has a certain problem. We might assume they’ll want our solution, be willing to pay for it, return to use it again, or even recommend it to others. But assumptions are not facts. The more we act on them without testing, the more we risk building something that misses the mark.
The first step is to write down all the assumptions you’re making. Think about your customers, their needs, your solution, how they’ll find you, and how your business will operate. Once they’re on paper, you can begin to sort them out. Focus on the assumptions that carry the most weight. If proven false, these are the ones that could break your idea. Those are the ones you need to test first. Catching them early can save you months of work and help you move forward with more clarity and confidence.
Once you’ve listed your assumptions, a great test to evaluate each one is the Riskiest Assumption Test or the RAT framework. I created a spreadsheet to assess your assumptions (view here). After doing the test, the assumptions with high consequence scores and high likelihood of being false represent your riskiest assumptions. Those are the ones to test first.
2. Create a Simple Financial Model to Expose Blind Spots
Before testing anything, build a lightweight financial model, even if your business is still just an idea. You don’t need a complicated spreadsheet with endless tabs. A simple outline is enough to give you a basic picture of how you think your business will function financially. This process helps you turn vague hopes into specific assumptions you can examine.
The numbers don’t need to be exact. The point isn’t to get everything right but to surface the parts of your plan that are based on guesswork. A simple financial model makes it easier to see which assumptions matter most. Once you’ve laid it out, you can identify which inputs feel the most uncertain and which ones would have the biggest impact if they turned out to be wrong.
This isn’t about precision. It’s about uncovering what you don’t know and forcing clarity around key drivers.
3. Build Experiments That Are Small but Robust
You don’t need a perfect product to start testing. Trying to get everything right before launching a test can slow you down and waste resources. What you need is a focused, well-designed experiment that teaches you something valuable with minimal effort.
A strong experiment begins with a clear goal. You’re not trying to prove that your entire business will succeed. You’re trying to learn whether a single key assumption is true or false.
Keep the experiment small. Limit the number of variables. Don’t try to test five different features at once. Instead, zoom in on one hypothesis and design an experiment that gives you a direct signal about that one thing.
Here is a Validation Experiment Design Worksheet you can use to create your own experiment plan.
Always ask: How can I learn just as much with half the time and effort?
4. Match Your Test to the Truth You Need
Some tests tell you more than others. Interviews and surveys are quick and useful early on, but they aren’t always reliable. If you want stronger evidence, you need to simulate real customer behavior. The key is to match your test to the kind of truth you’re looking for at that stage of your idea.
Here’s the spectrum your testing and truth fall on:
You don’t need to start big. You just need to be intentional. Ask yourself, What kind of evidence would I trust if I were an investor or co-founder? If you only need to know whether people understand your idea, a mockup might be enough. If you need to know whether people will pay for it, then a pricing test or pilot might make more sense.
The further you go toward real customer action, the more meaningful your learning becomes. So match your test to the truth you’re seeking. Build just enough to learn what matters next.
5. Choose the Right Kind of Test
There’s no one right way to validate an idea. What matters is choosing the method that fits what you need to learn. In Testing with Humans, it outlines a wide range of experiment types. Each one is designed to test specific assumptions, such as demand, behavior, usability, or pricing, without requiring you to build a full product. Below is a table of the test types mentioned in the book:
Each test type varies in effort and provides a different level of credibility in what you learn. Surveys and mockups are fast but less reliable. Concierge tests and paid usage offer stronger signals but take more work.
The key is to test behavior, not just opinions.
6. Set Clear Criteria Before You Begin
One of the biggest mistakes you can make when running an experiment is deciding whether it “worked” only after seeing the results. Without clear criteria, it’s easy to misinterpret what happened, especially if you’re hoping for a certain outcome. To stay objective, you need to define success and failure before you begin.
Start by asking yourself:
- What are we trying to learn?
- What outcome would give us confidence to move forward?
- What result would make us stop and reconsider our idea?
This is where pass/fail metrics come in. They don’t have to be perfect or scientific, but they should be specific. For example:
- If you’re running a Landing Page Test, decide ahead of time how many sign-ups will count as a win.
- For an Ad Test, set a target click-through rate or conversion rate.
- If you’re doing Pre-Selling, define how many people need to commit to buying before you consider it validated.
You can also include qualitative criteria. Maybe you’re running a Concierge Test, and your goal is to hear customers describe the problem in their own words. In that case, success might mean at least five people expressing a clear pain point that your solution addresses.
Setting these criteria upfront gives you something to measure against and helps you avoid common cognitive traps like confirmation bias or sunk cost fallacy. It also makes team decision-making easier because everyone is aligned on what success looks like.
7. Make Decisions, Not Excuses
Experiments are only valuable if you use the results to make real decisions. It’s easy to fall into the trap of running tests, gathering data, and then hesitating, waiting for more proof, ignoring inconvenient results, or telling yourself you’ll decide later. But delay is its own decision. If you’re not moving forward, pivoting, or stopping, you’re drifting.
At the end of every experiment, you should be able to make one of four clear choices.
Each decision helps you focus your time, energy, and money where it matters most. Don’t let uncertainty become a hiding place. Even small decisions, made quickly and based on what you’ve learned, will move your project forward faster than endless analysis or perfectionism.
Speed matters, but so does clarity. Don’t linger in indecision. Make the next move based on what you’ve learned.
8. Fight the Bias Volcano
Even with a well-designed experiment, your brain can still lead you in the wrong direction. That’s because we all carry cognitive biases, mental shortcuts that can distort how we interpret results. If you’re not careful, these biases can make you see what you want to see instead of what’s actually true.
Here are four common biases that could erupt the bias volcano and destroy your results:
-
Anchoring: You fixate on the first piece of feedback or data you receive and give it too much weight. For example, one enthusiastic comment early on might blind you to a dozen critical ones later.
-
Confirmation Bias: You focus on the information that supports your original idea and ignore or downplay the rest. This is why setting clear success criteria before running a test is so important.
-
Bandwagon Effect: You start to believe something is working just because others around you are excited or optimistic. Group enthusiasm can feel like validation, even when the data says otherwise.
-
Sunk Cost Fallacy: You keep pushing an idea forward just because you’ve already invested time, money, or emotional energy into it, even if the evidence says it’s not working.
The best way to cool these biases down is to build guardrails into your process. Write down your hypotheses, set pass/fail targets ahead of time, and involve others in reviewing the results. Share your data with someone who isn’t emotionally attached to the project and ask for their honest opinion.
Bias doesn’t mean you’re weak or foolish. It means you’re human. Yet, if you stay aware of how bias blow up your results, you’ll be better prepared to make decisions based on reality, not wishful thinking.
Final Thought
Skipping testing because you’re “pretty sure” you’re right is like jumping out of a plane with a parachute you designed in Microsoft Paint. It might work… but wouldn’t you rather try it on the ground first? Preferably without the splat.
That’s exactly what I did with Money Merlin. I built what I thought people would want, not what they actually needed. If I had tested earlier, I could have learned in one week what it took me two years to discover
So, take my advice. This approach saves time, money, and frustration. More importantly, it builds confidence. Each test gives you a clearer view of what works, what doesn’t, and what direction to take next. You stop relying on hope and start relying on real signals from the people you’re trying to serve.