Creating hypotheses for experimentation
When planning for experiments and shaping a Growth roadmap, testing good ideas matters just as much as testing them accurately. In the early stages of a vaguely defined project, designers often rely on intuition to guide early decisions. While intuition is a powerful tool, it’s most effective when paired with structure, evidence, and strategy. This module explores how to channel intuitive thinking into testable ideas by building clear hypotheses rooted in team goals and user insights.
Evidence-Based Design Decisions
When working on a vaguely defined project, relying heavily on intuition is common. Designers develop intuition over time, influenced by their knowledge of products, past experiences, and exposure to diverse perspectives. This intuition acts as a guiding force, helping designers navigate uncertain paths and make initial decisions with creativity and insight. However, combining intuition with evidence adds rigor to the decision-making process. While intuition is a valuable tool for designers, it should complement, not replace, research, data, and established best practices.
Bridging Goals and Hypotheses
As shown in the diagram, the process starts with a goal and based on the gathered insights, the team can start defining ideas and hypotheses. As a designer, your role is pivotal in facilitating this conversation, ensuring that creative insights are grounded in strategic and data-driven thinking.
01. Understanding Your Business Objectives and Metrics
a. Start with why
Similar to the non-experimental design process, the foundation of effective experimentation begins with understanding the core problems (the experimental design process will be covered in the next chapter/module). Collaborate with your product team to define clear goals and identify the metrics that reflect user engagement throughout their journey.
For example, a goal might be to increase the Activation* rate for Atlas Customers or decrease the Churn rate
b. Linking to Strategic Objectives
Explore how experiments must be aligned with and drive the company's strategic goals. It’s essential to define measurable goals for your experiments that support broader business objectives, ensuring that every design decision contributes to the company's success.
02. Gathering insights
Work closely with User Research and Analytics teams to gather informative inputs that shape your metrics. Taking Activation as an example, identify the key factors that influence the activation rate, and delve into user behavior during these critical tasks.
a.Studying the inputs
Examine the factors influencing each metric and work with researchers to ask questions that offer deeper insights. For example:
What does the user need to do or see before activating?
What layers of service can influence this action?
What are the pain points and opportunities related to each metric?
b. Anticipating Future Needs
Anticipate the foreseeable future needs of your users by looking at the trends and signals or asking questions like:
How will their relationship with your product evolve?
What new needs might arise?
How are competitors adapting their experiences?
How will the characteristics of future customers differ from those of your current target audience
How are our users' or customers' usage patterns changing over time?
03. Defining the opportunity
Creating the statement
Create a clear picture of how the identified issues negatively impact the customer experience. Use the framework below to articulate and communicate the opportunity or problem you're designing for:
Issue/opportunity:______ Cause:______ Impact:______
Issue/Opportunity: What is the core problem or opportunity?
Cause: What is driving it?
Impact: What is the measurable effect on the business or user experience?
04. Building a Targeted and Measurable Hypothesis
Multiple hypotheses can be generated for every problem/opportunity statement. A good hypothesis is specific, measurable, and based on evidence. Use phrases like “I think,” “We should,” or “If we” to propose changes.
Components of a Strong Hypothesis
To create a hypothesis that is both measurable and actionable, ensure it includes:
The specific change you are testing.
The expected impact (how much change and in what timeframe).
The target audience or users who will be affected.
05. Prioritizing Hypotheses
Experimentation allows you to test bold ideas without risking a significant impact on metrics, as tests can be conducted in a controlled environment. However, to use resources effectively, you should prioritize hypotheses based on their potential impact, investment, and risk.
Prioritization framework
Evaluate your hypotheses using a prioritization framework that considers both the value and the risk associated with each test. This ensures that you focus on the most impactful experiments first.
As you implement these steps, remember that the true power of hypothesis-driven design lies in its ability to combine intuition with evidence. By prioritizing and testing hypotheses, you can iterate with confidence, knowing that your design decisions are not only creative but also strategically aligned with business goals.
Your role as a designer extends beyond crafting visually appealing interfaces; you are a facilitator of thoughtful experimentation, ensuring that every design decision contributes meaningfully to both user satisfaction and business success. You may use Hypothesis building framework as a facilitation framework in your conversation. View every experiment as a learning opportunity. Whether a hypothesis proves correct or not, the insights gained will inform your next steps and lead to more informed design decisions.