← Smart Scale Systems

What makes a quiz question hard — without making it unfair

1 August 2026 · Smart Scale Systems

Most quiz questions fail in one of two directions. They are too easy, so nobody feels anything when the answer appears. Or they are too obscure, so the viewer never had a chance and feels cheated rather than beaten. The questions that work sit in a narrow band between those failures.

We have written and curated 406 questions across 20 topic packs for OmniFrame Quiz Studio — a single pack on AI and technology runs to 120 questions on its own, roughly eight times any other. What follows is what we have learned about that narrow band, and how to aim at it deliberately rather than by luck.

The target: just out of reach

The strongest questions produce a specific feeling: I almost knew that. Not "I knew it" and not "I've never heard of that." Almost.

That feeling is doing several jobs at once. It keeps the viewer watching, because the answer feels retrievable and they want the few seconds needed to retrieve it. It makes the reveal satisfying instead of humiliating. And it produces comments, because a viewer who almost got there wants to explain their reasoning in a way that someone who was never close does not.

Everything below is a technique for manufacturing that feeling on purpose.

1. Make the intuitive answer wrong — but keep it available

The most reliable way to put an answer just out of reach is to place a plausible wrong answer directly in front of it. The viewer reaches for the obvious response, finds it, commits, and only then discovers the obvious response was the trap.

A price rises 50% then falls 50%. Are you back where you started?

A 50% drop from the higher price leaves 75% of the original.

"Yes, exactly" is not a throwaway option. It is the answer almost everyone forms before they finish reading. The question works because that instinct is available, confident, and wrong — and because the explanation takes one sentence to overturn it.

Compare that to a question where the wrong answers are simply unrelated. There is no trap to fall into, so there is no moment of correction, so there is no reason to keep watching.

The explanation must name the trap, not just verify the answer

This is the refinement we arrived at last, and it changed how we write explanations.

Take the most famous question of this type. For years our explanation read: "If the ball is 5 cents, the bat is $1.05, totalling $1.10." That is true, and it is useless. It confirms the answer for someone who already has it, and tells the person who guessed wrong nothing about why they were wrong.

A bat and ball cost $1.10. The bat costs $1 more. How much is the ball?

The tempting 10 cents makes the bat $1.10 and the total $1.20; only 5 cents leaves a gap of exactly $1.

The second version names the wrong answer the reader almost certainly gave, and shows precisely where it breaks. That is the difference between an explanation that closes the loop and one that just states the key.

The test: does your explanation mention the wrong answer? If a question has a dominant trap, and the explanation never acknowledges it, the explanation is talking to the wrong reader.

The corollary: distractors are the question

Writers tend to labour over the question and then dash off three wrong answers. This is backwards. In a multiple-choice format, the difficulty lives almost entirely in the distractors.

Look at the answer set for a probability question:

Two fair coins are flipped. Given at least one head, what is the chance both are heads?

The remaining equally likely outcomes are HH, HT, and TH; one of the three is HH.

Every wrong option here is a real mistake somebody actually makes. "One half" is what you get if you ignore the conditioning. "One quarter" is the unconditional probability of two heads. "Two thirds" is the same error inverted. Each distractor corresponds to a specific reasoning path — which is why the question separates people who understand conditional probability from people who half-remember it.

A useful test when writing: for each wrong answer, can you name the mistake that produces it? If you cannot, replace it.

2. Hide the assumption, not the information

There is a difference between a question that withholds facts and one that withholds a frame. The first is unfair. The second is a puzzle.

A man asks a bartender for water. The bartender points a gun. The man says thank you. Why?

The scare cured the hiccups, so he no longer needed water.

Every fact needed is present. What is hidden is the assumption that the gun is a threat rather than a treatment. The viewer has all the information and still cannot see it, which is exactly the "almost knew it" position — and why the reveal lands rather than annoys.

The same structure drives the classic surgeon riddle in our lateral-thinking pack: nothing is missing except the reader's assumption about who surgeons are. Questions built this way age well, because the trap is a habit of mind rather than a fact someone either has or lacks.

3. One sentence must be enough to explain it

This is the constraint we enforce most strictly, and the one that quietly does the most work.

Across our 406 questions, 403 have an explanation of exactly one sentence. The median is nine words. Not because short is inherently better, but because a question that needs a paragraph to explain was never fair to begin with. If the reasoning cannot be compressed, the viewer had no realistic path to it in the seconds available.

The three exceptions are instructive. Two are mental-arithmetic shortcuts — "100 × 47 is 4700. Subtract 47 to get 4653." — where the method genuinely has two steps and compressing them into one sentence would obscure the technique being taught. The third weighs two options against each other. In every case the second sentence carries new work, rather than restating the first.

That is the real test. Not sentence count, but whether the extra words do anything. If your explanation is running long and it is not teaching a multi-step method, the question is the problem.

Try writing the explanation before the question. If it runs long, the question is too complicated — not too hard, which is different. Hard is good. Complicated is a different failure.

This constraint also matters in short-form video specifically, where the reveal has a few seconds at most. A clean one-line explanation is a design requirement, not a stylistic preference.

4. Only one answer may survive scrutiny

The fastest way to lose an audience is an answer they can argue with. Someone will always try.

A woman buys a parrot that repeats every word it hears, but it never speaks. Why?

It cannot hear words to repeat.

Note the wording: repeats every word it hears. That phrasing is load-bearing. It rules out "asleep" (it would still repeat words later) and "a toy" (the premise says it repeats). Deafness is the only condition consistent with both halves of the statement.

Before publishing, argue against your own answer. If a determined viewer can make a second option work, either tighten the wording until they cannot, or cut the question. Ambiguity reliably produces the wrong kind of comment section.

5. Vary where the right answer sits

This one is mechanical, easy to get wrong, and invisible until someone exploits it.

Writers unconsciously favour certain positions — often placing the correct answer second or third, because first feels too obvious and last feels like an afterthought. Do that consistently and a regular viewer stops solving your questions and starts reading your habits.

Across our 406 questions the correct answer lands in each of the four positions almost exactly a quarter of the time: 95, 100, 105 and 106. That distribution is maintained deliberately. It costs nothing and it removes an entire class of shortcut.

Worth auditing if you have a back catalogue. Count where your answers sit. Most people are surprised.

6. Difficulty and virality are separate dials

We tag questions on two axes, and treating them as one axis is a common mistake.

These come apart constantly. A question can be brutally hard and completely unshareable — obscure trivia does this. A question can be fairly easy and enormously shareable, because the pleasure is in watching someone else fall for it.

The 50%-rise-then-fall question is not hard for anyone who slows down. It is highly shareable, because the trap is recognisable and the sharer knows exactly who will fall for it. Optimise for whichever dial your format actually needs. For short-form video, shareable usually beats hard.

7. Use domain knowledge people almost have

In technical topics, the sweet spot is a concept your audience has heard of but has not had to name.

A chatbot remembers your first message, then suddenly forgets the document you pasted earlier. Nothing crashed. What invisible limit did you hit?

The model can only keep a limited amount of text in active context.

Anyone who uses AI tools regularly has experienced this. Many could not name it. That gap — lived experience without vocabulary — is the richest source of questions in any technical field, and it is why "nothing crashed" is in the question. It eliminates the mundane explanation and forces the viewer toward the concept.

The same structure produces harder questions further up the same subject:

A disease affects 1 in 1000. A model predicts nobody has it and scores 99.9% accurate. Why is it useless?

With rare positives, always guessing "no" scores well while catching none of the real cases.

Note what the question does: it hands you a number that sounds excellent, and the whole difficulty is noticing that the number is measuring the wrong thing. "Overfitting" is there because it is the term people reach for when a model disappoints, whether or not it applies.

A checklist

Before a question ships:

  1. Can you name the specific mistake behind each wrong answer?
  2. Is the question itself short enough to read at a glance? Ours run a median of 12 words.
  3. Does the intuitive answer appear as an option?
  4. Does the explanation fit in one sentence?
  5. Have you tried to defend a second answer — and failed?
  6. Does the explanation name the trap, rather than only confirming the key?
  7. Is every fact needed actually present in the question?
  8. Across your last twenty questions, is the correct answer spread evenly across positions?
  9. Would someone want to send this to a specific person?

Rule three is the one that fails most often, and it is the easiest to talk yourself out of. If the explanation keeps growing, the question is not ready.

Try it

The OmniFrame Quiz Studio is free and runs entirely in your browser. It ships with the packs quoted above, including the deceptive-maths and lateral-thinking sets, and script mode turns your own questions into video. No signup, no watermark.

If you write a question you are proud of — or you think one of ours is ambiguous — tell us. Ambiguity complaints are the most useful mail we get.

Home · All question packs · Quiz Studio · About · Contact · Privacy & Disclosure · Terms