The Onboarding Quiz Isn't Really Asking For Your Data

Why the best apps interview you — and when the interview backfires.

Jones Beach, New York, 1972. A researcher spreads out a towel, turns on a radio, and after a while walks off. A confederate strolls up and steals the radio.

When bystanders had just been minding their own business, four out of twenty did anything about it. But when the researcher had first asked one question — "would you watch my things?" — nineteen out of twenty intervened. Some literally chased the thief down the beach. (Those are the raw counts from Moriarty's 1975 study, and they're worth stating raw: twenty people per condition is a small sample, and honesty about that is part of the point.)

Nobody was paid. Nobody was persuaded. They were just asked — and the asking changed what they did.

That mechanism — questions as interventions — is the real reason the best apps in the world open with a quiz. It's also why most onboarding quizzes are built completely backwards.

What you'll get from this piece

Three things. First, the science of asking: what happens in someone's head when they answer a question about themselves, including a number from an experiment spanning more than 40,000 households. Second, the trap: why a quiz that works at three questions becomes actively destructive at fourteen — there's a specific psychological cliff, and we'll locate it. Third, the real world: we sampled 35 onboarding flows from top iOS apps ourselves, and we'll use what we found to rebuild a bad onboarding into a good one, screen by screen.

Asking is an intervention

Here's the cleanest version of the effect ever measured. In the early nineties, Morwitz, Johnson and Schmittlein got access to giant consumer panels — over 40,000 households analyzed. Some households were asked a single question: "Do you intend to buy a car in the next six months?" No ad, no pitch, no follow-up. Among households that weren't asked, 2.4 percent bought a car. Among households that were: 3.3 percent — roughly a 35 percent relative increase in actual car purchases, from one survey question.

It's called the mere-measurement effect, and it has since been meta-analyzed across 116 experiments: overall effect size d+ = 0.24 — small, consistently real, and strongest (d+ = 0.29) when people predict their own behavior.

That self-prediction detail is the key to onboarding. In 1980, Steven Sherman called residents and asked them to predict whether they'd volunteer three hours for the Cancer Society. People over-predict their own virtue, so most said yes. Called back days later with the real request, 31 percent of the predictors actually volunteered, versus 4 percent of people who were simply asked directly. Sherman called it a self-erasing error: people make an inaccurate prediction about themselves — and then behave until it's true.

Now look at any modern goal-selection screen with those numbers in mind. When your user taps "Run a 5K" or "15 minutes a day," that's not a database write. That's a self-prediction — the exact instrument from these studies, deployed at scale.

This is the part almost everyone gets wrong about onboarding quizzes. Builders think the quiz's value is the answers — the personalization payload. But the research says a huge part of the value fires the moment the question is answered, whether or not you ever use the data. The quiz is the commitment, not the data.

And to be straight with you: no public controlled study has ever isolated "personalization quiz → retention" in an app. The strongest evidence is the psychology of asking itself — small, real, and meta-analyzed — plus one company's A/B program we'll get to shortly.

The completion cliff

There's a second force in every quiz, and it has a failure mode.

In 2012, Norton, Mochon and Ariely had people assemble IKEA boxes, fold origami, build Lego — then measured what people would pay for their own creations. Builders bid 63 percent more for a box they had assembled than others would pay for an identical pre-built one. Amateur origami folders valued their crumpled work near what other people would pay for an expert's. The IKEA effect: labor leads to love.

But read the paper to the end, because the authors ran the destructive version too. When people built something and then didn't finish — or watched it get taken apart — the effect vanished. Their words: labor leads to love "only when labor results in successful completion."

Apply that to your onboarding. Every question a user answers is invested effort, which converts into attachment only if they finish and see what their effort built. A quiz abandoned at question fourteen isn't a partial win. It's worse than no quiz: you charged the user effort and handed them nothing.

So every question you add does double damage. It raises the odds of abandonment — survey telemetry shows completion falling steeply as question counts climb into the teens — and it raises the stakes of abandonment at the same time. That's the completion cliff.

Which means the payoff screen — "Here's your plan, built from your answers" — is not decoration. It's the completion artifact: the moment invested effort becomes ownership. An onboarding quiz without a visible payoff screen is an IKEA box you make the user assemble and then never let them see.

The real world: Duolingo, Noom, and our 35 flows

The best real-world numbers come from Duolingo, because they actually publish their A/B results. Their most famous onboarding test wasn't adding a question — it was deleting the signup screen. Moving account creation from the front of the flow to later, after you'd already done a lesson, Duolingo reports raised daily active users by about 20 percent. Softening the wall further — turning "sign up or lose your progress" into a quiet "Later" — added another 8.2 percent DAU. (Both figures are company-published, with no confidence intervals disclosed — and note the metric is DAU, not signups.)

Notice what that means. The questions that survived at the front of their flow are the commitment questions — your goal, your daily minutes. The question they evicted was the extraction question: "give us your email." Commitment first, extraction later, after you've experienced value.

To see how widely the lesson has landed, we ran our own audit.

What 35 onboarding flows from top iOS apps actually do — our sample, Aug 2026

Three takeaways. One: the hostage sequence lives on mostly in wellness, while commerce and media apps let you straight in — the account as save-mechanism, not toll booth, is how confident products behave. Two: inside the quizzes, skipping is rare, and the long quizzes have started interleaving persuasion between the questions — the quiz has become the sales letter. Three: the raw permission ambush is nearly dead; the floor has risen, and the differentiator now is what you promise before you ask.

And the cautionary tale: Noom rode the mega-quiz to a billion-dollar funnel — and to a $56 million settlement. To be precise: the settlement was about auto-renewal and cancellation friction, not the quiz itself. But one behavioral scientist's documented teardown of that flow — roughly 45 minutes of questions before meaningful value — makes the deeper point. Past a certain length, a quiz stops being personalization and becomes a filter that only the desperate survive. Filters produce great-looking outcome stats. They just don't produce great products.

Teardown: rebuilding Stride

Let's rebuild one. Fictional running app — Stride. The "before" version is a collage of everything we just measured going wrong. (Explore the interactive before/after: demo.html)

The signup wall. Before: name, email, password, confirm password — a toll booth before the user has seen anything. That's the exact screen Duolingo deleted for a 20 percent DAU gain. After: the app opens straight into the quiz, and the account ask moves to the end, reframed as "Save your plan." Signup as save mechanism. Same fields, opposite meaning.

The question tunnel. Before: fourteen questions, no progress indicator, no skip — including a gem like "What's your favorite running shoe brand?", a question whose answer changes nothing. After: three questions, a visible "1 of 3," and skip on every screen. Each question passes the test: something visibly changes based on the answer.

The goal question. Question one is the commitment question: "What's your goal?" — run a 5K, get consistent, clear my head. The user taps it themselves. Research on active choice suggests a choice you physically make sticks harder than one made for you — so this is the one screen where preselecting a default is the wrong move. That tap is Sherman's self-prediction. Let them make it.

The payoff screen. Before: the quiz ends at "You're all set!" and dumps into an empty dashboard. Effort collected, receipt never issued. After: "Your Week 1 plan" — three runs, on the days they chose, at the level they chose, with their goal printed at the top. This is the completion artifact. The IKEA effect needs a finished box; this is the box.

The notification ask. Before: a raw iOS dialog with no context — the ambush that's nearly extinct among top apps. After: a primer first — "Tuesday, 7am — want a reminder before your first run?" It quotes the user's own answers back to them. A permission request that's downstream of the user's stated goal isn't an interruption; it's service.

Count it. Before: seventeen screens, zero skips, zero payoff, account demanded upfront. After: three questions the product visibly uses, one self-prediction, one receipt, one earned permission — and the account ask waiting politely at the end, holding something worth saving.

The rule

The quiz was never really about your data. Asking changes the asker — one question moved car purchases 35 percent; one question turned bystanders into thief-chasers.

So here's the rule: ask questions you'll visibly use, and show the receipt.

Three questions that shape the product beat fourteen that shape a marketing profile. And nothing you collect matters if the user never sees what their answers built.

Open your onboarding tonight. For every question, ask: what changes based on the answer — and does the user see it change? Every question that fails both tests is just friction wearing a lab coat.

Sources

• Moriarty (1975), "Crime, commitment, and the responsive bystander," Journal of Personality and Social Psychology 31(2) — scribd.com/document/622928083/moriarty1975
• Morwitz, Johnson & Schmittlein (1993), "Does measuring intent change behavior?", Journal of Consumer Research 20(1) — papers.ssrn.com
• Wood et al. (2016), question-behavior effect meta-analysis, Personality and Social Psychology Review 20(3) — pubmed.ncbi.nlm.nih.gov/26162771 (open access: eprints.whiterose.ac.uk)
• Sherman (1980), "On the self-erasing nature of errors of prediction," Journal of Personality and Social Psychology 39(2) — psycnet.apa.org
• Norton, Mochon & Ariely (2012), "The IKEA effect: When labor leads to love," Journal of Consumer Psychology 22(3) — hbs.edu
• Cioffi & Garner (1996), active vs. passive choice, Personality and Social Psychology Bulletin 22(2) — journals.sagepub.com
• First Round Review — Gina Gotthilf on Duolingo's A/B testing program — review.firstround.com
• SurveyMonkey completion-rate telemetry — surveymonkey.com
• Noom class-action settlement, S.D.N.Y. 2022 — journals.law.unc.edu; FTC dark-patterns enforcement policy (Oct 2021) — ftc.gov
• Jason Hreha, Noom onboarding critique — thebehavioralscientist.com
• Mobbin onboarding audit — our sample, n=35 flows from top iOS apps, Aug 2026

A note on evidence tiers: the Moriarty, Morwitz, Wood, Sherman, Norton, and Cioffi & Garner findings are peer-reviewed; the Duolingo figures are company-published without disclosed methodology; the SurveyMonkey completion data is vendor telemetry; and the 35-flow audit is our own sample, reported with raw counts.