✦ Task type · Research and checks
Surveys on a task marketplace, and what the sample really is
A marketplace is not a representative panel and never will be. Knowing precisely which claims that rules out is what separates useful survey data from a chart nobody should have made.
Marketplace surveys are fast and cheap, which is exactly what makes them dangerous. The data arrives looking like panel data, and the temptation is to treat it as such.
The honest use is real and worth having: open-ended reasoning, message comprehension, and direction on questions where you currently have nothing.
What to put in the public brief
- Honest completion time, not an optimistic one
- The topic in general terms
- Any eligibility that matters, stated up front
- That partial responses are not paid, if that is your rule
Keep the survey link and any brand identification private, or you will get responses from people who looked at the brand before answering.
What to put in the private instructions
Give the link and repeat the time estimate
If the real time exceeds the estimate, workers who took it in good faith are underpaid, and the marketplace remembers.
Put an attention check early and another late
Not adjacent to each other. Early catches the people who never intended to read; late catches the ones who stopped.
Ask the open question before the closed ones
Once you show options, you have supplied the vocabulary and the unaided answer is gone.
Say what happens to their answers
Whether responses are anonymous, how long you keep them, and whether they leave the platform. This is a data-protection obligation, not a courtesy.
Proof worth requiring
Ask for
The completion code your survey tool issues at the end. It ties a payment to a finished response without you handling the response contents.
Do not ask for
A screenshot of their answers. It defeats the anonymity you probably promised, and it puts personal data in an approval queue that was not designed to hold it.
When this task type is the wrong tool
If you need a representative sample or a claim you will publish, use a research panel with known demographics and accept the cost.
If you want depth rather than counts, concept feedback gets you reasoning from thirty people that no survey of three hundred will produce.
Common questions
Is this sample representative?
No. It is people who register on a task marketplace, filtered by whatever eligibility you set, and it skews by country, age and comfort with online work in ways you cannot correct after the fact. Use it for directional signal, message testing and open-ended reasoning. Do not use it for population estimates, and do not put a margin of error on it.
How do I stop straightlining?
Two attention checks, placed unevenly, and at least one reversed item. Straightlining is answering everything down the same column, and it survives most survey tooling untouched because the responses look complete. A reversed item catches it without insulting anyone paying attention.
What reward is fair?
Roughly a dollar a minute, honestly estimated. Underpaying a survey does not reduce responses; it reduces care, and careless responses are worse than no responses because you will analyse them anyway.
How many completions?
Fifty to two hundred for directional work. Below fifty a subgroup breakdown is noise you will read as a finding, and that is the specific failure this sample makes easy.
Thinking of taking this task?
The same job, from the other side. Rewards are set by the poster, so treat these as the range this task type usually sits in rather than a promise.
- Usually pays
- About a dollar a minute
- You need
- Nothing beyond honest attention
- Proof expected
- A completion code from the survey tool
Attention checks are in there. They are not tricks; they are how a poster separates careful answers from straightlining, and careful answers are what you are paid for.
Create a worker accountField a survey
Set the reward, the number of completions and who qualifies. Workers reserve a spot; the survey link stays private until they do.
Post this task