Home —> Education/Resources —> Election Polling Resources

Online Panels

An online panel is a sample of persons who have agreed to complete surveys via the Internet. Although a few online panels in the United States and Europe recruit their members from probability samples (randomly selected addresses or telephone numbers), most online panels use other methods of recruitment.

The vast majority of online panels in the U.S. are made up of volunteers who were recruited online and who receive some form of compensation for completing surveys, such as small amounts of money or frequent flyer miles.

The recruiting information presented to prospective panelists often also emphasizes the opportunity for the panel member to affect new products or policies or the sheer fun of taking part. People join a panel by going to a website and providing personal and demographic information that is later used to select potential respondents for specific surveys.

These volunteer panels (also called opt-in or access panels) are very common, and several firms make members of their panels available to market and other researchers. The panels are often quite large, with hundreds of thousands or even millions of members, but the response rates to any specific survey request are often very low, suggesting that the number of active panel members at any given time is much lower than the advertised number. The probability-based panels generally have many fewer members than the nonprobability panels that dominate online research.

Another method of recruiting online samples is generally referred to as “river sampling,” so named because respondents are invited to participate in a survey on a flow basis as they visit specific websites. Once a respondent agrees to do a survey, he or she answers a few qualification questions and then is routed to a waiting survey. Sometimes, but not always, these respondents are offered the opportunity to join an online panel.

The great virtues of online panels are that they produce results very quickly and at much less cost than other methods for conducting polls or surveys. For example, several thousand responses can be obtained in a few days. Because no interviewers are involved, the data collection costs are often quite modest.

There are two major concerns about online panels. The first is that the panel members are not representative of the larger population—such as the voting age public. There are three aspects of this issue. Even though most people in the U.S.—more than 80 percent—have access to the Internet, the ones who don’t differ systematically from those who do. People without access to the Internet tend to be older, less educated, and poorer than those with Internet access, and this difference can bias the results of a survey or poll. (For example, older persons are more likely to vote than younger ones.) Some of the probability-based panels deal with the issue of noncoverage by supplying computers and Internet access to panel members who don’t already have them, but most online panels simply omit those without Internet access. Another issue is that Internet surveys often have very low response rates, often in the single digits. Many are invited to complete the survey, but few choose to do so. (This is also a problem with many telephone surveys.) As a result, the sample respondents may be an unrepresentative subset of the panel members initially invited for the survey. The final aspect of this issue of representativeness involves the use of volunteer samples rather than probability samples. Non-probability sampling and its limitations are covered in a separate briefing.

Two strategies are often used to reduce the biases associated with the poor representativeness of online panels. The panel members invited to complete a specific survey may be carefully selected to match the make-up of the intended population on age, sex, education level, or other variables. Or the responding sample may be weighted so it matches the population on a set of relevant demographic or political variables (such as partisan affiliation). Studies of the effectiveness of various weighting schemes suggest they reduce some (30 to 60 percent) of the error introduced by unrepresentativeness but not all of it. See The Science of Web Surveys, Table 2.4 for a summary. Still, the problem of unrepresentativeness is not unique to online panels. Even probability samples (for example, those interviewed by telephone) are likely to require weighting adjustments to compensate for the effects of survey nonresponse and other problems.

The other big concern about online panels is that panel members may rush through the surveys to obtain their rewards and don’t respond thoughtfully. So far, though, the quality of the answers obtained from online panels does not seem to be worse than that from more traditional methods of data collection and, in some cases, may be better.

Online panels are a popular tool for data collection and they have proven their value for many purposes. In fact, online surveys outperformed live telephone polls in forecasting the outcome of the 2012 election. Although they did not do so well in 2014 they again were on par with live telephone in 2016. In addition, rigorous evaluations have found that volunteer online panels are less representative of the general population than probability samples. Still, online panels will doubtless play a growing role in political polls done in the next few years.

References
Baker, R., Blumberg, S. J., Brick, J. M., Couper, M. P., Courtright, M., Dennis, J. M., ... & Zahs, D. (2010). Research synthesis AAPOR report on online panels. Public Opinion Quarterly, 74(4), 711-781.

Tourangeau, R., Conrad, F. G., and Couper, M. P. (2013). The Science of Web Surveys. New York: Oxford University Press.

This information was developed by AAPOR as part of a comprehensive online journalism polling course created in partnership with NewsU, a project of the Poynter Institute and funded by the Knight Foundation. The course launched September 2007.

Download PDF Version