Test one thing at a time

The problem

A creator changes the hook and the CTA in the same next version. It wins. Which one won? You don’t know. If it had lost, which one lost? You don’t know that either. You spent a slot and learned nothing.

This is the oldest rule in testing, and it is the most broken one.

That feels slow. It is not. Two clean tests teach you two things you can use forever. One messy test teaches you nothing you can trust.

Here is how to set up a clean test:

1. Name the leak. “This video leaks at Payoff.”
2. Write a guess. “I think people leave because the hook promises a before-and-after and the next line is a product name.”
3. Make one change that matches the guess. Cut the product name from line two. Show the before-and-after first.
4. Ship the next version and wait until it has enough views to read the funnel. Too early, and the numbers bounce too much to trust. Lesson 9 covers when that is.
5. Compare Payoff to the original. Better? Your guess was right. Write it down. Worse? Also write it down. That is a thing you now know not to do.

That last line is where the money is. A team that runs a hundred tests and records none of them will re-run the same losing idea next month. A team that records every one has a hundred things they know.

The example

Two briefs for the same leaking video.

“Test one thing” does not mean “test small things.” Change the whole hook, not one word in it. Big, single changes give you clear answers. Tiny changes give you noise.

Do this in Verdict

1. 1
   Open Learnings. Every verdict is a guess. Every next version is the answer. The record holds both.
2. 2
   When you brief the next version, write the guess in the note, in one sentence, so the answer has something to be compared to.
3. 3
   When the next version has enough views, log the outcome. The next time you see the same leak on the same kind of product, the answer is already sitting there.