The problem
A good strategist can usually think of several ways to fix an underperforming video. That is not the hard part. The hard part is deciding which one deserves the next test.
You are rarely choosing between an obvious right answer and an obvious wrong one. A weak result has several plausible causes, each cause suggests several reasonable changes, and every one of them sounds defensible in the room. You are choosing between competing guesses, not spotting a mistake.
The numbers tell you where the video lost people. They do not tell you why, and the same drop is consistent with a lot of different stories. Take a video that holds attention all the way through and then loses people at Convert.
| Plausible cause | What it would make you brief next |
|---|---|
| The hook pulled in the wrong person | A narrower opening, aimed at someone likelier to buy |
| The offer never actually appeared | The same video with the offer stated plainly |
| There was no reason to buy today | A deadline, a bundle, a stock line |
| Nobody but the creator vouched for it | A testimonial or a second voice |
| The product page does not match the promise | Nothing. This one is not a creative job at all |
All five are reasonable. They lead to five different videos. Four of them are wrong, and you will not know which four until something gets tested. Meanwhile each test costs a slot, a creator’s time and a piece of the account’s distribution, so you cannot run all five and see.
The example
Here is what makes this harder than it needs to be. Most teams keep the outcome of a test and lose the thinking that produced it.
| What survives | What disappears |
|---|---|
| Which video won | What we thought was wrong with the one before it |
| Which creator performed | Which of the plausible causes we picked, and why that one |
| The numbers, still sitting there | What we expected to move, and whether it actually moved |
The experiments happened. The learning did not. So a strategist meets a familiar-looking result with no way of knowing that the same team tried three answers to it two quarters ago, and starts the same search from the beginning. The field of five never gets any narrower, on any product, ever.
This is the quiet cost of working at volume. Production scales faster than memory, so a team can run hundreds of tests and still make decisions out of whatever the people in the room happen to remember. Lesson 30 is the habit that fixes it: write the guess down before the test, so the answer has something to attach to.
One caution, because the opposite mistake is just as expensive. Past results are not rules. Creative is too contextual for that: an angle can fail on a product with few reviews and a weak offer, and work on the same product three months later. So an old failure is not a veto on trying something again. It is information about what to expect and what to change first. Lesson 31 is about not re-running dead tests, and it is worth reading as "know what happened last time", never as "if it failed once, it is closed".
Do this in Verdict
- 1When the Leak Finder marks a step, treat the suggested fix as the first candidate rather than the answer. Lesson 16 is the habit of looking one step before the leak, which usually removes two of the five.
- 2Check Learnings before you pick. If this product or this leak has been worked before, the field you are choosing from is already smaller than it looks.
- 3When two candidates both look reasonable and you can only afford one, take the one worth more money rather than the one that looks worst. Lesson 17 covers that read.
- 4Write down which cause you picked and what you expected to move, not just what you changed. That sentence is the thing that disappears, and it is the only part that makes the next search shorter.