Sampling and inference on the SAT
A study can only speak for who it studied and how it was designed. Random sampling lets you generalize, and random assignment lets you claim cause.
How to solve sampling and inference questions
- Check who was randomly sampled. Conclusions apply only to that population.
- Treat estimate ± margin of error as the plausible range.
- Remember that a larger sample generally gives a smaller margin of error.
Quick tip: Results only generalize to the group that was randomly sampled. Only random assignment shows cause.
Worked example
A researcher randomly selected 400 adults in a city and found that 62% support a new park, with a margin of error of 4%. Which conclusion is best supported?
- ABetween 58% and 66% of adults in the whole country support the park.
- BExactly 62% of adults in the city support the park.
- CThe percent of all adults in the city who support the park is plausibly between 58% and 66%.
- DThe park will be built because most adults support it.
Show the answer
Answer: C · The percent of all adults in the city who support the park is plausibly between 58% and 66%.
- Who was sampled?. Random adults from one city, so conclusions apply to that city.
- Build the range. 62% ± 4% gives 58% to 66%.
- Use plausible language. The true value is likely in that range, not exactly 62%.
- A. Only adults in one city were sampled.
- B. A sample gives an estimate, not an exact value.
- D. The survey says nothing about whether it will be built.
The common trap
Plausible is not certain. Any value inside the interval is reasonable, and the true value is not guaranteed to be at the center of it.
Volunteers chose whether to drink coffee each morning. The coffee drinkers reported more energy. Which conclusion is supported?
- ACoffee has no effect on energy.
- BAmong these volunteers, coffee drinking is associated with more energy, but cause can't be shown.
- CCoffee causes more energy for all adults.
- DCoffee causes more energy for these volunteers.
Show the answer
Answer: B · Among these volunteers, coffee drinking is associated with more energy, but cause can't be shown.
Random sampling lets you generalize. Random assignment lets you claim cause. This study had neither.
- A. The study found a difference. It can't rule out an effect.
- C. There was no random sample of adults and no random assignment.
- D. People chose for themselves, so something else could explain the difference.
Remember
- Conclusions apply only to the population that was randomly sampled.
- Cause needs random assignment.
- estimate ± margin of error is the plausible range, not a guarantee.
Practice questions
Try each one, then open the answer to check your work.
In a random sample of 50 residents of a town of 1000, 30 support a new park. Based on this, about how many residents support it?
- A30
- B1200
- C400
- D600
Show the answer
Answer: D · 600
The sample rate is 60%, and 60% of 1000 is 600.
- A. 30 is the count in the sample. Scale it up to all 1000 residents.
- B. That doubles the rate. 30 out of 50 is 60%.
- C. That's the number who don't support it.
A random survey estimates that 60% of voters favor a measure, with a margin of error of 3 percentage points. Which is the most reasonable conclusion?
- AThe true percentage is plausibly between 57% and 63%.
- BExactly 60% of voters favor the measure.
- C3% of the people surveyed gave false answers.
- DThe true percentage is certainly greater than 63%.
Show the answer
Answer: A · The true percentage is plausibly between 57% and 63%.
The margin of error gives a plausible range around the estimate: 60 ± 3.
- B. A sample can't pin down an exact percentage. The margin of error gives a range.
- C. Margin of error measures how much a random sample can vary. It says nothing about false answers.
- D. The plausible range tops out at 63%, so higher values aren't supported, let alone certain.
A researcher surveyed customers who answered an online review form and generalized the results to all of the store's customers. What is the main flaw?
- AThe survey should have had a margin of error of 0.
- BThe sample was not randomly chosen from the population.
- CThe sample is too large to analyze.
- DThe researcher should have used a median.
Show the answer
Answer: B · The sample was not randomly chosen from the population.
A result only generalizes to a population that was randomly sampled. Customers who answered an online review form are not a random sample of all of the store's customers.
- A. No sample survey has a margin of error of 0. The problem is the sample wasn't random.
- C. A large sample is easier to trust, not harder. The problem is who was sampled.
- D. Using a median doesn't fix a sample that wasn't chosen at random.