Path: Foundation (all roles) | Time: 12 min
<aside> 💡
TL;DR: An experiment is not just "set up a test and check results." It is a structured process with seven phases. Skipping any phase introduces risk: bad hypotheses, wrong metrics, unreliable results, or wasted learnings. Every role has a job at every phase.
</aside>
Every experiment, from a simple button color test to a complex pricing algorithm redesign, follows the same lifecycle. The phases exist because each one catches a category of error that would otherwise propagate forward and compromise the result.
This is where experiments are born. A product manager notices a drop in conversion. An analyst spots a pattern in user behavior. A designer proposes a new layout. A developer suggests a technical optimization.
The output of ideation is a business question: "Would changing X improve Y?" At this stage, the question does not need to be precise. It needs to be real. The worst experiments start from solutions looking for problems ("we built this feature, let us test it") rather than problems looking for solutions.
Who leads: Product Manager
Who contributes: Anyone. The best experiment ideas come from all roles.
The business question becomes a testable hypothesis. This is where vague ideas become precise, falsifiable statements. A hypothesis has a specific structure: if we do [action] for [audience], then [metric] will change by [amount], because [rationale].
The hypothesis must specify what "success" looks like before you see any data. This is not optional. It is the single most important discipline in experimentation, and we will cover it in depth in F3.
Who leads: Product Manager with Data Scientist
Who contributes: The hypothesis must be written down and reviewed before proceeding. No written hypothesis, no experiment.
The hypothesis is translated into an experiment design. This includes: