Home —> About Us —> History —> Presidential Addresses —> 2011 Presidential Address

2011 Presidential Address

2011 Presidential Address from the AAPOR 66th Annual Conference

Taking Aapor's Mission To Heart
by Frank Newport

I would like to offer three propositions regarding the central topic of this address—taking AAPOR's mission to heart.

First, the proposition that the American Association for Public Opinion Research (AAPOR) not lose its focus on the powerful purposes behind public opinion research, most importantly the magnificent conviction that measuring, assessing, analyzing, summarizing, and understanding public opinion is valuable because it provides a fundamental basis for making decisions that move society and culture forward.

Second, the proposition that AAPOR needs the courage to do more to provide leaders, decision–makers, and others with actual assessments and summaries of what public opinion is, rather than leaving that to others.

Third, the proposition that AAPOR and its members need flexibility and open-mindedness in determining which methods to use to achieve these goals—lessening the chance of being left behind in the constantly changing research environment in which we live.

Let's look at these one by one.
1. AAPOR, in my opinion, has as its highest calling the science of measuring, analyzing, and summarizing public opinion on key topics—for a number of important purposes.

We need to ensure a continuing and primary focus within AAPOR on these powerful purposes.

Fundamentally, the question we need to ask is: “Why are we measuring public opinion—no matter how we may do it?” More specifically, not to put too blunt an edge on it: “Why, ultimately, are we here today?”

I think there are at least four answers to that question.

First, we measure public opinion because it provides basic data for science. AAPOR's focus is on the virtue and benefit of measuring large groups of people's attitudes, opinions, and behaviors. The resulting data provide the grist for the mill of the sociologists, political scientists, communication researchers, psychologists, and economists who are striving to explore individual and social behavior.

Second, we measure public opinion to provide estimation of important population values. As our past president Bob Groves will be the first to testify, we cannot measure every person in the United States all the time and enumerate all of the variables in which there is potential interest. We spend billions to do that every ten years. Between the decennial censuses, we have to use other methods to estimate these population parameters. Many of these methods fall under our particular branch of social science.

Third, we measure public opinion because it is of interest to audiences. One of my favorite scholars, social psychologist Leon Festinger, postulated that humans have a fundamental drive and need for social comparison. I believe he was right. Public opinion research gives us the ability to understand what the people around us think and feel about key issues of the day—something we humans apparently like to know.

I don't think today's news producers and editors and Web content providers are highly conversant with Festinger's theories. But these gatekeepers are fairly coldblooded in their efforts to increase ratings, circulation, and hits. And they apparently believe that public opinion data and polls are of great interest to their audiences.

Then, there is a fourth use of our data, which I believe is in many ways AAPOR's highest calling. This is the conviction that public opinion is not valuable just because it can be used as a data source for social scientists. And the conviction that public opinion is not valuable just because it can provide estimations of population characteristics. And the conviction that public opinion is not valuable just because it provides humans with feedback on how their fellow humans are thinking and feeling.

But the conviction that measuring, assessing, analyzing, summarizing, and understanding public opinion is valuable because it provides a fundamental basis for making decisions that move society and culture forward. The collective views of a large group of individuals put together provide insights and guidance that do as good a job of helping direct the evolution of our societies and culture as any other source of insight or wisdom that can be conceived.

I am more convinced than ever that the collective views of the people of a region, state, and country are absolutely vital to the decision-making that ultimately affects them. And it's not just me. Anecdotally, it appears that we are seeing more and more public officials talking about and referencing public opinion. Public opinion has become a major part of the current discourse of public policy in this country.

We had the unusual situation last fall of both Barack Obama and John Boehner saying that they needed to pay attention to the public, echoing the sentiments of deposed California Governor Gray Davis who said eight years ago: “I didn't stay in touch with the people. That's clearly my biggest regret. Voters are the source of all wisdom. You have to conduct an ongoing dialogue with them” (Broder 2003).

This is evident around the world as well. Who can miss the significance of public opinion as we watch the upheavals in countries like Egypt and Tunisia and Libya in which, like Gray Davis, the autocratic leaders were clearly not “staying in touch with the people”? Secretary of State Hillary Clinton recently said: “Even in the most authoritarian regimes around the world, people are listening to the opinions of their publics…And so there's a growing effort to make sure that your views and your actions at home and abroad are aligned with what public opinion is” (Rose 2011).

There is of course great debate on the principles of just how and to what degree public opinion should be taken into account in governance decisions, particularly in a representative democracy. Our new AAPOR Task Force on Public Opinion and Leadership is now in the middle of parsing in depth this debate, calling on the resources of some of our country's most learned scholars of public opinion.

But few doubt or challenge, I think, the virtue of the idea that it is, in the broadest sense, good to give people a voice in the decisions that affect them—the idea that, collected together, aggregated views distill more wisdom and better insights than do individual smart people themselves.

And in large, complex societies, doing this is directly in line with AAPOR's reason for being.

Listen with me here to the words of our newly adopted AAPOR mission statement:

We strive to “Encourage and disseminate systematic analyses of public opinion on the major issues of the day…to give people a voice in the decisions that affect their daily lives” (AAPOR 2010).

This is the rationale that so enraptured the pioneers in our field, and the founders of this association, so many years ago. According to AAPOR's history as recounted in A Meeting Place, at least one version of the original constitution of AAPOR included the goal to “promote the utilization of public opinion research in democratic policy formation” (Sheatsley and Mitofsky 1992).

I stand here before you today as a true-believing disciple of this part of the mission of AAPOR—the big-picture emphasis on the value of making sure that public opinion is a part of the decision-making—the democratic policy formation—that affects all of us. This is truly a higher calling—a significant contribution that we as professionals can make to enhance the evolution of our culture and our society.

Here we are, sitting at the annual conference of a group dedicated to the goal of “disseminating systematic analyses of public opinion on the major issues of the day” and working to “give people a voice in the decisions that affect their daily lives” to be used for “democratic decision-making.” This is a tremendous responsibility and a thrilling opportunity. We are professionals entrusted with handling this dynamite, powerful information.

The goal that we must not lose sight of is the promotion of excellence and quality in this process—so that people can be given an accurately represented voice in the decisions that affect their daily lives.

2. It follows that AAPOR needs the courage to deal more directly with the issue of what our data actually tell us about public opinion, encouraging and sponsoring judgments, assessing quality, and providing leaders, decision-makers, and others with summaries of what public opinion is, rather than leaving it to others to make those judgments.

Just how can AAPOR carry out its higher calling of helping ensure that leaders and others can use public opinion as a core basis for decision-making and democratic policy determination? In answer to that question, I imagine a continuum.

At one end of that continuum is what I would call AAPOR's traditional path—wherein AAPOR says to the world that we will help train and educate professionals, and—and this is the key—push for full disclosure of how research is done in order to make it easier for others to make the judgments about the quality of public opinion data and summary assessments of what it all means.

AAPOR in this sense becomes a scientific facilitator—making sure that the transparent details of research are provided—and then lets others make use of the data as they will.

This position embodies a great tradition. The push to encourage, push, pull, or cajole any practitioner or entity to make the basis for their conclusions crystal clear is a core requirement of science.

It is also in many ways a safe course, and in many ways the least controversial and least likely to offend.

But it is unclear to me that this traditional path maximizes the influence that AAPOR can have in attempting to reach its mission relating to public opinion.

There's a wonderful novel called The House of God (Shem 1978), written by physician Samuel Shem, in which he dissects life and death in a Boston hospital. One of his clever locutions was the concept of TURFing patients. Shem uses the word TURF to represent the efforts by the ER and other doctors to shift the responsibility for a patient, usually an elderly one, to someone else. Shem's book recounts the efforts on the part of the overworked residents to TURF patients somewhere else—anywhere else—so that they don't have to deal with them…as in: “I TURFed the patient to Urology, but she bounced back to me.”

We are not, for the most part, dealing with elderly patients, but with efforts to measure public opinion in order to make it a part of decision-making.

But AAPOR has, to a degree, historically adopted a TURF mentality. We make sure that full information about the methods involved in efforts to represent public opinion is available in great detail. Then, as an organization, we TURF the responsibility for making judgments and summaries of what the data show to someone else.

My friend Diane Colasanto, who stood here at this podium in 1997, referenced the AAPOR censure of researcher Frank Luntz in her presidential address (Colasanto 1997). Luntz, as some of you will remember, promulgated data relating to the Contract with America, but without any supporting evidence of where it came from. He stonewalled, then AAPOR censured. Excellent.

But suppose for a moment that AAPOR had been successful in prodding Luntz to provide every intimate, precise detail on how his study was conducted. AAPOR would have applauded him for his transparent revelations.

But, and this is the key point, this would have TURFED the responsibility of making sense out of Luntz's data to someone else. The hapless citizen or reporter or decision-maker, interested in understanding public opinion on the Contract with America, would still be adrift, reliant on his or her own resources to evaluate Luntz's research and to put it in context with other data.

I am often asked: “How can I, the layperson, reach a conclusion about public opinion on a topic with so many differing public opinion surveys and polls I stumble across?” I hesitate, because there is no easy answer. I can't refer them to AAPOR, because all AAPOR as an organization does is make sure that the promulgator of the research makes all details public. I often end up saying “make some judgments about the brand name of the entity promulgating the data and/or the outlet in which it is carried.” Or “go to a trusted and reliable public opinion specialist to make those judgments for you.”

Well. If there is one thing I have learned over the years, including 20 at Gallup monitoring public opinion data on essentially a daily basis, it is that analyzing and summarizing public opinion is a difficult challenge indeed.

Everyone in this room is aware of how tricky this can be.

I think, however, that this is the point. If we as professionals find this tricky, just how tricky it must be to leaders, decision-makers, and others who may in good faith want to use public opinion as an input as they go about making policy and decisions that affect, sometimes profoundly, our society.

Consider the Practice Guidelines and Quality Standards that come from the American College of Cardiology. These are “a comprehensive collection of evidence-based clinical documents developed by consensus of leaders in the field of cardiovascular medicine. These documents provide recommendations for the practice of cardiology as well as strategies for cardiovascular quality improvement” (American College of Cardiology 2011). I'm sure that some of those recommendations are controversial. Cardiovascular surgeons may have strongly different opinions about the value of and indications for coronary artery bypass grafting surgery than do non-interventional cardiologists. Yet this association still seems to be able to review and summarize evidence in its field of research. I don't see any reason why our association can't think about doing the same.

So, the question is: To what degree can or should AAPOR shift as an organization more to the other end of the continuum I outlined above—moving beyond its traditional TURF procedure and more toward the goal of “Providing Assessments of Public Opinion Ourselves”?

The more we move toward this goal, of course, the more controversy arises. That is, in part, because of the highly emotional, political, ideological nature of much of the material with which we deal.

But a consideration of this type of risky shift is warranted. My call today is that AAPOR as a body needs to do more to help in this process, to accept more responsibility for the use of public opinion, as is in our mission statement and calling.

In short, my proposition is to take seriously the challenge of building off our excellent track record of encouraging full disclosure of public opinion research methods, and to add in more efforts to assess the results of these methods—by making judgments and interpretations of that public opinion.

All these considerations led to the formation of AAPOR's Task Force on Public Opinion and Leadership, as mentioned, chaired by myself and Bob Shapiro. We are now in the middle of taking a hard look at these issues.

But some clear ideas are already emerging.

First of all, we certainly as an association are in an excellent—in fact, the best—position to provide these types of answers.

As Peter Miller indicated in his presidential address last year, “We alone [meaning AAPOR] offer the dispassionate, comprehensive, informed perspective that can mold understanding of polls and surveys” (Miller 2010, p. 605).

So. What exactly can AAPOR do? To quote Elizabeth Barrett Browning, “let me count the ways” (Browning 1954 [1850]).

AAPOR should of course continue the needed push for transparency in how public opinion research is done. An integral part of any summary or analysis of public opinion is an evaluation of the quality of the data from which it is derived—which requires full disclosure.

AAPOR can use the tools it has available to encourage members of the association to do their own summaries of public opinion. Many of us already do. There are good examples from this conference here in Phoenix: “Public Opinion on Healthcare Policy and Reform,” which follows this lunch today. Or the session “Public Perception of Illegal Immigration Policy—Local, State, Regional, and National Perspectives,” which is coming up a little later this afternoon.

AAPOR can empower the Communication Committee and communication specialists within our management firm to publicize what members have done in terms of summaries, and to create a database of summaries on key issues of the day so that we can direct opinion leaders and decision-makers to use them. We can expand a formal focus on communicating summaries of public opinion, and on educating journalists and others on how to evaluate public opinion. This tracks the concerns of medical scientists, two of whom in a recent review focused on “the gross inadequacy of current mechanisms for finding, extracting, and delivering the best possible information to where clinical decisions are made—the weakest link in the chain of research evidence” (Davidoff and Miglus 2011).

AAPOR can develop guidelines for how to conduct summaries of public opinion. Right now on our website we have “Questions to Ask When Writing About Polls” (AAPOR 2011b), but these focus mainly on sampling and methods. We have Frequently Asked Questions (AAPOR 2011a), but these focus mostly on sampling issues: “Do response rates matter?,” “What is a random sample?,” “Do cell phones affect survey research?,” and “Opt-in surveys and margin of error.” We need more FAQs on how to determine, integrate, and summarize public opinion on key issues of the day.

Finally, AAPOR can consider the idea of promulgating actual assessments and summaries of public opinion under AAPOR's aegis, and promote their distribution.

With regard to this last point, let me interrupt for a moment and read an example of an integrative summary of public opinion on immigration, of great interest to us here at this conference in Arizona: “The majority of Americans continue to espouse status-checking policies obliging everyone in the United States to carry identification cards. There has been a general decrease in the public's support of security measures entailing the building of fences and walls. The majority of Americans espouse some form of policy allowing illegal immigrants the option of remaining in the United States. There appears to be increasingly extreme and clearly negative attitudes as to whether the government and public servants can effectively implement these policies” (Segovia and Defever 2010, p. 393).

This summary, although involving data a few years old at this point, is an example of integration of existing data on this volatile topic that would or could represent the type of thing AAPOR could promote.

But! This summary of attitudes toward immigration is, in fact and in reality, already an official AAPOR endeavor! It is, as some of you may recognize, a summary review of public opinion on immigration published in Public Opinion Quarterly (POQ) last summer and written by Francine Segovia and Renatta Defever.

In fact, the more I look into this, the more I realize that AAPOR is already involved in the process of making judgments and summarizing public opinion on key issues—albeit on a limited basis—at its conferences and also in POQ.

Rummaging back through previous issues of POQ, the interested reader finds summaries of public opinion on such topics as immigration; attitudes about the American dream; support for the United Nations; the U.S. residential housing market before, during, and after the housing bubble; reforming the presidential nomination process; tolerance; confidence in law enforcement; North Korea; evolution, creationism, and the teaching of human origins in schools; energy policy; labor unions; global health threats and infectious diseases; global warming; the Iraq war; and government actions to prevent terrorism.

Wow. Some of these summaries are directly relevant to the policy issues of the day: labor unions…Wisconsin; energy policy…$4-a-gallon gas; the presidential nomination process…the 2012 primary season; terrorism…the events of a week and a half ago in Pakistan. And so on.

Yet I'm not sure that many leaders and elected officials are aware of these summaries, nor that they are taken into account as they ponder policy decisions for the future.

But they are there, and they represent a start.

What I'm saying is that AAPOR has, in some areas, already taken on the responsibility of assessing public opinion on key issues of the day. My proposal is to, as they say, take this to the next level—and not to TURF that responsibility to someone else.

I believe the world will be a better place if AAPOR takes on more responsibility for routinizing and systematizing and expanding efforts to analyze and summarize public opinion as an association, and disseminating these more widely.

If AAPOR is not at the forefront of helping make public opinion available and usable by leaders, policymakers, and decision-makers, who will be?

3. AAPOR needs to promote flexibility, not dogmatism, in determining whichmethods to use to achieve its objective of measuring, analyzing, and summarizing public opinion.

Traditional survey research is of course at the core of AAPOR's history. From its beginning, the founders of AAPOR were enthralled by the discovery and application of the miracle of random sampling—the discovery that estimating what every person in a large and growing population was thinking could be accomplished on a relatively low-cost basis by using the principles of random sampling.

George Gallup was captivated by the writings of Englishman James Bryce, who had envisioned a time in the future when the Fourth Phase of Public Opinion could be reached: “…if the will of the majority of the citizens were to become ascertainable at all times.” Bryce didn't know how that was possible, lamenting “How, without the greatest inconvenience, can votes be frequently taken on all the chief questions that arise? No country has yet surmounted these inconveniences” (Bryce 1995, [1888]).

George Gallup knew. He and the other pioneers of our industry figured out that sampling done right created accurate representations.

That thrill of being able to use small samples to understand what everyone was thinking has driven AAPOR over the years since. As Cliff Zukin said in his presidential address five years ago: “Our methodology is built on the notion—and science—of sampling. That is, we select and interview a small group of people to represent an underlying population” (Zukin 2006, p. 428).

Our profession has historically focused on shifts and refinements and tweaks in exactly how that sample-to-population process can take place. These refinements have seen the change from in-person to telephone interviewing, to the use of random digit dialing, more recently to the impact of cell phones, and now the move back to address-based sampling.

This has been good.

I'm often asked: “Don't you know that you pollsters are missing the people who are talking on cell phones?” I respond: “Oh? Thanks for sharing!” No, I actually am very proud to say that my professional organization has pioneered research into the cell-phone phenomenon for years now. AAPOR was focused on cell phones before cell phones were cool.

This is one of our organization's great strengths—the pulling together of great minds and disparate perspectives to focus on the constantly changing challenges inherent in figuring out how to accurately represent the attitudes and opinions of a very large and ever-changing population.

These challenges continue unabated—but in new ways.

We are faced today with the paradox that Americans appear to be more difficult to reach and less likely to want to talk when we select them—yet, apparently, are more and more willing to express their opinions to others when they are not asked to. This is a fascinating state of affairs. We are interested in the public's opinion. The public, apparently more than ever,wants to give its opinion, just not necessarily in the way we want them to. And new technology has allowed the public to express its opinion publicly in many new ways.

This outpouring of unsolicited public opinion has not gone unnoticed. And it should not go unnoticed by AAPOR.

I have no doubt that our existing paradigm of focusing exclusively on researcher-originated, probability-based respondent selection and interviewer-generated questions will change in the years ahead.

Just recently, I've seen research that:

Shows how Twitter activity accurately reflected the share of the vote in the 2009 German elections (Tumasjan et al. 2010).

Shows how “likes” on Facebook are correlated to the 2010 midterm election outcomes (U.S. Politics on Facebook 2010).

Reports on the work of computer scientists at Carnegie Mellon who concluded: “In this paper, we find that a relatively simple sentiment detector based on Twitter data replicates consumer confidence and presidential job approval polls” (O'Connor et al. 2010). Those were Gallup polls, by the way.

There's a recent post from the Daily Beast’s Howard Kurtz: “Between February and April, according to an analysis for Newsweek by General Sentiment, a company that tracks and measures online content, posts involving Sarah Palin fell 38.3 percent over the past 30 days. Social-media mentions dropped in lockstep, down 32 percent over the same period, to 135,421…It is hard to escape the conclusion that her influence has peaked” (Kurtz 2011). Note that Newsweek did not commission a poll to reach that conclusion.

And a paper being delivered today here at this conference, in fact, concludes that “…a carefully executed opt-in Internet panel produces estimates that are as accurate as a telephone survey and that the two modes differ little in their estimates of other political indicators and their correlates” (Ansolabehere, Fraga, and Schaffner 2011).

AAPOR's Executive Council is taking these new approaches to estimating public opinion seriously. Our Task Force on Opt-In Online Panels, led by Reg Baker, released a comprehensive report (Baker et al. 2010) last year. We have now established a new Task Force on Non-Probability Research to broaden our reviews. This reflects our understanding that if there is one thing we have learned over and over again, it is that nothing stands still; everything changes. The traditional model of interview-generated, sample-to-population estimating is no exception.

There is the danger of spending so much time analyzing and reviewing the traditional survey sample methods our profession has used that we lose sight of the possible potential of newly emerging methods.

We may be guilty of putting the cart before the horse—method before objective. Focusing on our goals first allows us more flexibility in picking, choosing, and analyzing whatever methods we believe will work best to achieve those goals. Objective first, methods second.

In a way, this resembles the thinking of another AAPOR president, Warren Mitofsky, in his presidential address (Mitofsky 1989). Warren basically, as I read it, argued that this association needed to encourage flexibility in assessing methods—as long as the researcher discussed fully the reasons why he or she felt that the methods produced legitimate results for a given situation. This, said Warren, could include non-probability samples. Warren, it seems to me, was arguing something similar to what I'm saying today—that the methods must be selected and used to fit the goals of a research situation, diminishing the value of a blind allegiance to one and only one method for approaching the research task.

Conclusion

All in all, I'm calling today for a renewed emphasis within AAPOR on what is our unique niche and positioning—the focus on providing sound, scientific measures and summaries and understanding of public opinion on the key issues of the day—in order to be the primary organization focused on moving culture and society forward by harnessing the incredible power of the collective wisdom and opinions of the people, in this country and around the world.

Part of this call may result in a renewed focus within AAPOR on goingbeyond disclosure and doing more of the actual assessing and summarizing ourselves. If AAPOR is not involved in doing this, it is left to others who may not have our professionalism and scientific orientation and experience.

And I am calling on AAPOR to be flexible in its evaluation of the methods used to estimate public opinion.

As George Gallup said in his presidential address given at the 1955 AAPOR conference: “Although the Voice of the People cannot be taken as the Voice of God, the soundness of their judgment and their willingness to be led is impressive…As Talleyrand said: ‘the only thing wiser than anybody is everybody’” (Gallup 1955, p. 242).

We at AAPOR have been entrusted with the job of measuring the opinions of that “everybody.” This is truly an exciting challenge, and one that all of us in this room this afternoon should be vastly excited about taking on.

Member Login

Join AAPOR Donate

Donate