What does a poll's "margin of error" mean?
A description of "margin of error "
During the summer of 2017, many national polls of adults reported that more than 60% of U.S. adults supported gay marriage. But this number by itself is not precise.
A "true" value exists out there. However, it could be only measured with absolute precision if every U.S. adult was polled. In national polls, this would require a polling agency to contact all 245 million adults, which is clearly impossible. Most polls actually involve only from 1,000 to 50,000 adults, and are thus only an estimate of the true value. The margin of error indicates how precise the estimate can be expected to be.
Assuming that the people who were actually polled were a true random sampling of U.S. adults, then:
"A margin of error of plus or minus 3 percentage points at the 95% confidence level means that if we fielded the same survey 100 times, [asking different people each time,] we would expect the result to be within ~+mn~3 percentage points of the true value 95 of those times." 1
Such a poll would require asking the opinion of about 1,070 randomly selected individuals.
- Sometimes, a polling agency will say that the margin of error is "X" percent. What they really mean is that it is ~+mn~X percent.
- Estimates of the margin of error lose some accuracy when public opinion is strongly for or against.
There are many ways in which polls can be designed to give inaccurate results. For example:
- The agency could phone people at home only during normal working hours, and thus obtain a surplus of retired persons in their survey.
- They could dial only cell phones and thus include a surplus of young adults in the survey.
Thus it is important to assess whether the polling agency is trustworthy.
Example of a typical poll:
On 2017-AUG-03, Quinnipiac University in Hamden, CT conducted a poll among 1,125 randomly selected U.S. adults. They started the poll eight days after President Trump tweeted that transgender service members were to be forcibly expelled from the military. They found that:
- 68% of U.S. adults agreed with the statement: "Transgender people should be allowed to serve in the military."
- 27% disagreed.
- 5% had no opinion or refused to respond -- an unusually small percentage for such a controversial topic.
60% of subjects involved in the poll who were Republican voters opposed transgender persons in the military while 91% of Democratic voters and 72% of Independents supported them.
- The poll's margin of error is ~+mn~3.4 percentage points. However, the margin of error for the subsets of persons polled who were either Republican or Democratic or Independent voters would be higher, because those values would be based on fewer than 1,125 adults.
Those polled also felt by a margin of 89% to 8% that it should be illegal for employers to discriminate against transgender employees.
The following information sources were used to prepare and update the above
essay. The hyperlinks are not necessarily still active today.
Andrew Mercer, "5 key things to know about the margin of error in election polls," Pew Research Center, 2016-SEP-08, at: http://www.pewresearch.org/
"Margin of error Calculator," American Research Group, Inc., 2017, at: http://www.americanresearchgroup.com/
"Release Detail," Quinnipiac University Poll, 2017-AUG-03, at: https://poll.qu.edu/
Copyright © Ontario Consultants on Religious Tolerance
Author: B.A. Robinson
Originally posted on: 2017-AUG-16