On this page
A thumbnail test variant is different enough when a viewer at feed size could say in one word what changed, and when you can say in one sentence what a win would teach you. Both halves matter. The first is what lets the test resolve; YouTube's help page on testing titles and thumbnails says a test resolves faster when the variants are more different. The second is what makes the result worth having. This article is a ladder for choosing how different to make the second cover: four rungs, each a larger change than the last, each paired with the kind of hypothesis it can answer and the rule for when to climb.
One lever, all the way, at the right rung
The A/B testing guide gives the rule: change one lever and change it completely, so the variants are not twins and the result names the lever. The small-channel testing piece adds that a channel with few impressions needs the difference to be bold. Those are the same rule applied at different rungs of one ladder, and the decision that most testing advice skips is which rung to stand on.
The ladder has four rungs. Each rung changes one thing and holds the rest, and the thing it changes is larger than the rung below.
| Rung | What changes | What must stay | A Winner teaches | Performed the same teaches |
|---|---|---|---|---|
| 1. Expression | The emotion on the face | Layout, headline, colour, object, crop | Which feeling this audience trusts on this format | Emotion is not the lever here; climb |
| 2. Headline | The words, or their lever (number, word, none) | Picture, face, layout, colour | Which words the picture needs | The picture carries it; climb, or test the picture |
| 3. Layout | The arrangement (face-led, object-led, split, big face) | The promise, the emotion, the palette | Which structure this audience reads fastest | Structure is settled; climb |
| 4. Concept | The promise itself (outcome, contradiction, specific) | The video, and honesty about it | Which angle on this video the audience wanted | The video's appeal does not depend on the angle |
The lower rungs are refinements: you already have a cover that works and a hypothesis about one part of it. The top rung is exploration: you do not yet know which promise to make. The one-lever rule and the bold-variant rule describe the bottom and top of the same ladder.
The four rungs, and the confound to hold still on each
Every rung has one trap, which is that changing the named lever tends to change something else with it. Holding that second thing still is most of the craft.
Rung 1, expression. Same layout, same headline, same colour, same object; only the face's emotion changes. Shock against doubt, doubt against calm, delight against flat. The confound is the face itself: a different expression often arrives with a different crop, gaze or face size, and then the test is partly about size. Keep the crop and the gaze fixed, so the only thing a viewer could name is the feeling. Which expressions fit which promises is the list to choose the two ends from.
Rung 2, headline. Same picture; the words change lever. A number against a word, a word against no headline at all, a stakes word against a curiosity word. The confound is composition: three big words and one big number do not occupy the same space, and a headline that shrinks or moves also changes how much of the face shows. Fix the text block's position and size before you change what it says, and if the number needs less room, leave the room empty rather than letting the face grow into it.
Take a budgeting video, say. The picture is fixed: a doubtful face on the right at half the height, a shoebox of receipts on the left, a dark blue ground. Variant A carries "$412" in yellow. Variant B carries "HIDDEN FEES" in the same yellow, in the same block, at the same size. A Winner for A says this audience wants the stake priced before it clicks; a Winner for B says the word does more than the figure on this format. The confound is that four characters and eleven do not fill a block the same way, so if B's block grew and swallowed half the shoebox, a viewer could name two differences and the result is partly about the receipts. Fix the block. Then change the words.
Rung 3, layout. Same promise, same emotion, same palette; a different arrangement. Face-led against object-led, a split against a single frame, a big face against a face beside a prop. Here the confound is unavoidable: a layout change moves the face's size, and that is part of the lever, not a leak. Accept it, and write the hypothesis about the structure rather than the face.
Rung 4, concept. A different promise from the same video. The three concept types are the natural pair or trio: the outcome cover against the contradiction cover, or either against the specific one. Everything visible may change, because the promise changed. What must hold is honesty: both variants have to describe the video that exists. This is the rung where the judge matters most, since different promises attract different viewers with different expectations, and a cover that wins clicks can lose on watch time share if its viewers arrived for a video they did not get.
Before a rung 4 test, write the "viewers who click this expect" sentence for each variant, as the watch-time-share explainer suggests, and be stricter than usual. On the lower rungs a mismatch costs you a lesson. On the concept rung it costs you a test that resolves quickly, decisively, and in favour of the cover that promised least.
When to climb and when to descend
The ladder is a route, not a menu. Where you start and which direction you move depend on the last result and on how many impressions the video will get.
- Start where your hypothesis is. If the question is "does this audience want doubt or shock on reviews", that is rung 1. If the question is "which angle on this video", that is rung 4. Do not start low because low is tidy; start where the uncertainty is.
- Climb after a flat result. Inconclusive or Performed the same on two well-separated variants means the lever you tested does not move this audience on this format. Log it as a finding, then climb one rung on the next video of the same format. What to do with an inconclusive test covers the case where the variants were twins rather than the lever being dead; that is a re-run at the same rung, not a climb.
- Descend after a Winner. A concept win tells you which promise to make; the next test on that format refines it one rung down, at layout, then headline, then expression. Each descent narrows the question and each result compounds on the last.
- Start high on a small channel. Few impressions cannot separate an expression change. Begin at rung 3 or 4, where the difference is large enough to resolve, and descend only when the channel's impressions grow.
- Never change rungs inside a test. A three-slot test with a control, a new expression and a new concept is two tests in one, and the concept slot will win or lose for reasons that swamp the expression slot. Three slots belong to one rung: the control, one direction, and the other direction.
We would start every new format at rung 4 even on a channel with impressions to spare, and even though that spends a precise instrument on a coarse question, because until the promise is settled the lower rungs are refining a cover you may throw away.
The channels that learn fastest from testing move down the ladder over a year. Their first tests are concept tests, because they do not yet know what their audience wants from them. A dozen tests later they are running expression tests, because the promise and the layout have become house rules and the remaining uncertainty is small. A channel still running concept tests after two years has not been writing the lessons down.
Pairing a hypothesis to a rung
A hypothesis is one sentence with four parts: the format, the lever's two ends, the expected direction, and the viewer's reason. "On tutorials, a calm face beats a shocked one, because the viewer wants to be told this is easy." Each rung has its own hypothesis shape, and writing the sentence before making the second cover is what stops you testing two favourites.
| Rung | Hypothesis shape | Example | Not a hypothesis |
|---|---|---|---|
| Expression | On [format], [emotion A] beats [emotion B] because the viewer wants [feeling] | On reviews, doubt beats delight because the viewer wants a verdict, not a fan | "The smiling one looks friendlier" |
| Headline | On [format], a [lever] headline beats a [lever] headline because [what the picture cannot show] | On finance stories, the number beats the word because the picture cannot show the stake | "Try the shorter text" |
| Layout | On [format], [structure A] reads faster than [structure B] because [what the viewer looks for] | On unboxings, object-led beats face-led because the viewer is checking the product, not me | "Split screens are trending" |
| Concept | On [format], the [type] promise beats the [type] promise because [what the viewer wants to know] | On experiments, the outcome cover beats the contradiction cover because the viewer wants to know whether it worked | "Let's try something different" |
The "because" clause is the part that transfers. A result that confirms the clause gives you a rule about the viewer, which applies to covers you have not made yet. A result without a clause gives you a fact about two pictures.
Write the hypothesis in the log before the test starts, not after the result arrives. A hypothesis written afterwards always fits the result, and a log full of confirmed hypotheses that were written in hindsight is a log of nothing. The decision log template has a field for it in the right place.
Making each rung on the bench
The second variant should cost minutes, not an evening, or the ladder will not survive a busy month. Each rung maps to a different route on the bench, and each route changes the one thing the rung names while holding the rest.
- Rung 1: change the expression on the finished result. Two options from the same saved face, two credits, nothing else in the render moves.
- Rung 2: Tweak the finished render with a new headline. The result becomes the source, so the picture, the face and the style are held and only the words change. Where the object rather than the words is the lever, draw a box in the editor and change only that region.
- Rung 3 and 4: Remix the render with a different style or a new scene description and the same saved face. A layout change wants a new style; a concept change wants a new scene. The iteration habits that keep a lineage readable are in iterating one change at a time with Tweak.
Whichever route you use, put the two covers side by side at thumb width before the test and ask a person who has not seen either to name the difference in one word. If they cannot, the variants are twins whatever the rung says. If they name two things, you have a confound to fix.
What to do next
- Write the hypothesis sentence for the next test, with the "because" clause, and read off which rung it belongs to.
- Make the second cover by the route for that rung, and check that nothing else moved.
- After the result, climb on a flat result, descend on a Winner, and write the lesson in the log with the rung named.
- When three results on one format agree, stop testing that rung on that format. It is a house rule now.
