✦ Task type · Research and checks
Ad creative feedback, before the media budget votes
In-platform tests find the winner and never say why. That is fine when you have budget to burn learning it, and expensive when the losing variant was losing for a reason you could have fixed.
The question a media platform answers is which creative performs. The questions it cannot answer are what the creative said, to whom, and whether anything in it will cause a problem later.
What to put in the public brief
- That it is written feedback on advertising creative
- The rough category, so people can self-select
- How many variants they will see
- That the ad has not run yet
Keep the creative and the brand private. Unreleased advertising in a public brief is a leak.
What to put in the private instructions
Show the variants in a fixed order and say so
Order effects are real. If you want to control for them, run two campaigns with the order reversed rather than letting each worker choose.
Ask what is being advertised before asking anything else
If they cannot say, nothing else in their response matters, and you have found the problem.
Ask which one would stop them scrolling, and why
The reason is the deliverable. A preference without a reason is a vote.
Ask what could be misread
Phrased as an invitation, not a challenge. This is where you find the claim that reads as a promise, or the image that reads differently in another market.
Proof worth requiring
Ask for
Written answers per variant, and one line on whether the category is relevant to them. Relevance decides how you weight the rest.
Do not ask for
A rating per variant. You will average the ratings, and the average of three comprehension failures is a number that looks like an opinion.
When this task type is the wrong tool
If the creative is already live and you want performance, the platform's own data is better and cheaper. This is a pre-flight instrument.
If what you actually need is to know whether the underlying offer works rather than how it is presented, concept feedback asks that question directly, and it is a different question.
Common questions
Why not just A/B test in the ad platform?
Do both, in that order. The platform tells you which variant won and gives you a number; it cannot tell you that the losing one was read as a different product entirely. Human pre-testing answers comprehension and risk, which are the two things a click-through rate cannot separate.
Should I show one ad or several?
Several, and ask them to compare. Comparison produces reasoning; a single ad produces a verdict, and verdicts from strangers skew polite. Three variants is the useful number, because two invites a coin flip and five invites a ranking nobody thought hard about.
What reward is fair?
Five to twelve dollars for a considered comparison across three variants. Below five you get preferences without reasons, which is the same information the platform gives you for free.
What should I ask?
What is being advertised, who it is for, what the ad is asking them to do, and which of the three they would stop scrolling for and why. The last clause is the whole point; without it you have a poll.
Thinking of taking this task?
The same job, from the other side. Rewards are set by the poster, so treat these as the range this task type usually sits in rather than a promise.
- Usually pays
- $5 to $12
- You need
- Nothing beyond written attention
- Proof expected
- Written answers for each variant
You are comparing, not scoring. The reason one would stop your scroll is the deliverable, and none of them is a legitimate answer.
Create a worker accountTest your creative
Set the reward and how many responses you need. Workers reserve a spot; your creative stays private until the spot is theirs.
Post this task