Task type · Research and checks

Surveys on a task marketplace, and what the sample really is

A marketplace is not a representative panel and never will be. Knowing precisely which claims that rules out is what separates useful survey data from a chart nobody should have made.

Marketplace surveys are fast and cheap, which is exactly what makes them dangerous. The data arrives looking like panel data, and the temptation is to treat it as such.

The honest use is real and worth having: open-ended reasoning, message comprehension, and direction on questions where you currently have nothing.

What to put in the public brief

Keep the survey link and any brand identification private, or you will get responses from people who looked at the brand before answering.

What to put in the private instructions

  1. Give the link and repeat the time estimate

    If the real time exceeds the estimate, workers who took it in good faith are underpaid, and the marketplace remembers.

  2. Put an attention check early and another late

    Not adjacent to each other. Early catches the people who never intended to read; late catches the ones who stopped.

  3. Ask the open question before the closed ones

    Once you show options, you have supplied the vocabulary and the unaided answer is gone.

  4. Say what happens to their answers

    Whether responses are anonymous, how long you keep them, and whether they leave the platform. This is a data-protection obligation, not a courtesy.

Proof worth requiring

Ask for

The completion code your survey tool issues at the end. It ties a payment to a finished response without you handling the response contents.

Do not ask for

A screenshot of their answers. It defeats the anonymity you probably promised, and it puts personal data in an approval queue that was not designed to hold it.

When this task type is the wrong tool

If you need a representative sample or a claim you will publish, use a research panel with known demographics and accept the cost.

If you want depth rather than counts, concept feedback gets you reasoning from thirty people that no survey of three hundred will produce.

Field a survey

Set the reward, the number of completions and who qualifies. Workers reserve a spot; the survey link stays private until they do.

Post this task