Your first experience with an opinion poll likely came in elementary school, passed to you in the form of a note, containing a single question: Do you like me? Check Yes or No.
Looking back, it’s easy to become nostalgic about an artifact from a simpler time. But there’s nothing simple about the “Like Me” note. In fact, many of the same factors that make public opinion polls dangerous can be found in those lines scribbled on notebook paper.
There’s the ambiguous use of terms (“What level of romantic attachment is meant by ‘like’?”), the binary choice that excludes context and complexity (“I like her but not in the way she is thinking . . .”), and the shaping of an opinion that comes from forcing a response to the query (“I hadn’t thought about it before, but if she likes me, maybe I should like her too.”).
If you’re reading this and you’re still in elementary school, I don’t want to discourage you from taking such polls to gauge romantic interest. But if you’re no longer in elementary school I do want to discourage you from taking most other forms of opinion polling seriously. As I will argue, such polls tend to make us smug and dumb. The main example we’ll be considering is the recent poll which, as The Atlantic reported, contends that, “White Evangelicals Believe They Face More Discrimination Than Muslims.”
The following argument is extremely long because I’ve tried to be thorough. Fortunately, you don’t need to read the entire article. Once you’ve read enough to be convinced such polls are useless—if not outright dangerous—feel free to skip to the bottom and read the conclusion.
Superior to the Believers in Lizard People
Some polls ask the same question year after year to gauge long-term changes in public opinion. (You’ll rarely see these polls mentioned on the news, much less on social media.) The majority of polls, though, are ad hoc and based on what the polling company believes will still be newsworthy by the time the results are tallied.
To capture the public’s attention the poll results must either vary from expectations or provoke outrage (usually by varying from expected norms).
All opinion polls are examples of what the late historian Daniel Boorstin called “pseudo-events”—events planted primarily (not always exclusively) for the immediate purpose of being reported or reproduced. But not all pseudo-events get reported on, and not every opinion poll is “newsworthy.”
A poll showing 90 percent of people who identify as transgender also support expansion of transgender rights would not be newsworthy. However, if a poll showed that 90 percent of Bible-believing evangelicals supported expansion of transgender rights every news outlet in America would report the results. The latter is newsworthy, the former is not.
Pollsters understand this reality and so shape their polls to “uncover” these unexpected results. We’ll talk more in a moment about how polls are framed by pollsters and the media. But for now think about why this latest poll focused on the results of white evangelicals. Why did the news not report another finding from the poll, such as that 63 percent of white mainline Protestants say there is a lot of discrimination against Muslims?
The reason, of course, is because that is the presumed “correct” answer to the implied question. It’s not controversial (hence newsworthy) for white mainline Protestants to say that Muslims are frequently discriminated against, because that is the answer the pollsters and the media believe is the right opinion to hold.
For an opinion poll to be newsworthy it must be non-neutral, and deviancy from the expected norm is what will be reported. Take, for instance, a 2012 survey by National Geographic, which found that 77 percent of Americans think aliens have visited Earth. The 77 percent figure is likely higher than you would expect (and there is reason to be skeptical of the results for the reasons cited below). If you already believed aliens have visited Earth, you might take comfort in knowing others agree with you. And if you think the idea is nonsensical it gives you a reason to feel smug and superior to those who hold a bizarre opinion.
We are not required to take every opinion found in a poll seriously or pretend there is a possibility they could be correct (I’m fairly confident that the 4 percent of Americans who believe lizard people control politics are in error). But we already know that many people hold wrong beliefs—sometimes dangerously wrong beliefs with eternal consequences. Even if such polls are well intended they rarely have much use other than tempting us to be prideful. We smirk and scoff at the rubes who hold such silly opinions—even when they are our brothers and sisters in Christ. Rather than showing why they err, we pass judgment on them in a way that can makes us stumbling blocks (Rom. 14:10;13).
Next time you read an opinion poll, ask yourself why this is considered newsworthy. When you find the answer you’ll know what opinion you are expected to hold by the media—and who the media expects you to distance yourself from.
Aggregators of Ambiguity
Imagine being randomly called and asked whether 77 degrees is hot or cool. What answer would you give? What would be the “correct” answer?
There is no correct answer, of course, because I’ve stripped away all context from the question and relied on three ambiguous terms: it, hot, and cool.
The respondent might be thinking I meant Celsius rather than Fahrenheit. They might also assume I’m asking about the temperature of their surroundings. If they live in Antarctica (or in a conference room in an office building), 77 degrees might seem “hot,” while the same temperature in Texas in July might seem “cool.” Similarly, if you’re in a swimming pool where the water temperature is 77 degrees you may describe it as cool. And if your body temperature is 77 degrees, you are not just cool, but dead.
For an opinion poll to accurately reflect a consensus opinion there must first be a consensus agreement on what the terms mean. You’ll rarely, if ever, find an opinion poll that defines the terms, much less develops consensus about their use. This reason alone is enough to make almost every opinion poll worthless.
Consider the question asked in the PRRI poll: “Just your impression, in the United States today, is there a lot of discrimination against [INSERT ITEMS; RANDOMIZE], or not? And is there a lot of discrimination against [INSERT NEXT ITEM], or not?”
The two key terms in that question are “discrimination” and “a lot.” What do those terms mean? It depends on the person being asked. Your answer is likely to differ considerably based on many factors, such as whether you apply a whole-number standard or a fractional-number standard to the vague term “a lot.”
Is That a Big Number?
For example, let’s say that 1.5 million people in Group X have been discriminated against. Is that “a lot”? Under a whole-number standard, 1.5 million may indeed be a large number of people. If you use the fractional-number standard, though, you would need additional information.
Let’s say that Group X is the “American Muslim population.” Since there are only about 3 million Muslims in America, this would mean one out of every two Muslims (1/2) has been discriminated against. Most everyone would agree that if 50 percent of a population is affected, it could be described as “a lot.”
But what if we say that Group X is the “American Christian population.” Since there are approximately 223 million Christians in American, 1.5 million would only be 1 out of every 148 Christians. If less than 1 percent of Christians face discrimination should someone using the fractional-number standard describe it as “a lot”?
The meaning of “a lot” depends on many factors, including the standard of measurement we use. But it can also differ based on the way we apply our standard of measure to our meaning of the term “discrimination.”
Imagine you’re a Seventh-day Adventist and the first thing that comes to mind when you hear “discrimination” is employer-based religious discrimination. This is something your denomination is likely to be familiar with. As legal scholar Eugene Volokh noted last year, “there are apparently about half as many Seventh-day Adventists as Muslims in the United States, and yet Seventh-day Adventists account for six of the 54 [religious discrimination] cases, which is to say about half as many as Muslims.”
If the poll question were asked of Adventists who associate “discrimination” with employer-based discrimination they might assume that they face as much, if not more, discrimination than Muslims. Since they are merely a subset of Christians, it would be a reasonable extrapolation for them to assume that Christians face more discrimination than Muslims. The Adventists could have warrant for their opinion even if their belief were in error.
Taking an opinion poll using vague terms is akin to asking 1,000 people to measure their kitchen table using different-sized rulers—some with 6 inches to the foot and others with 18 inches to the foot. We should expect to get inconsistent results even if they all had the same-size kitchen tables (which they don’t). The same is true for opinion polls that ask for “impressions” using measuring tools that vary based on the individual.
In a previous article I discussed how, in stripping away context, the daily news makes us dumb. Opinion polls have a similar effect. By removing all context and attempting to convince us that group consensus can be built on individual ambiguity, these polls cause us to hold false beliefs about reality.
Problem of Priming
If you heard about this “Muslim-evangelical discrimination” poll, it was probably from a news report and not from an examination of PRRI’s actual poll. But even those who looked closely at the question used can overlook how the results could be skewed by “priming.”
In psychology, priming refers to activating particular representations or associations in memory just before carrying out an action or task, such as answering a question or providing a response. For example, if a person is asked to name a fruit after hearing or seeing the word “yellow” he will be more likely to give the response “banana.” As Psychology Today explains, this happens because yellow and banana are closely associated in memory.
Now let’s take another look at the PRRI poll question: “Just your impression, in the United States today, is there a lot of discrimination against [INSERT ITEMS; RANDOMIZE], or not? And is there a lot of discrimination against [INSERT NEXT ITEM], or not?”
There is a potential for priming if the first item has an association that could affect the second. Here are the items used in that question: Gay and lesbian people; Transgender people; Immigrants; Blacks; Muslims; Whites; Christians.
Because the terms are randomly distributed, a majority of respondents would see the terms “gay and lesbian people” and/or “transgender people” before the word “Christians.” Is there any reason white evangelicals might be primed to associate the terms “gay and lesbian people” and “transgender people” with discrimination against Christians? Any reason you can think of? Any reason at all?
Now you may believe that white evangelicals should not associate discrimination with news stories about people losing their businesses, Christian schools losing funding, and being called “bigots” for refusing to support same-sex marriage or opposite-sex people in public bathrooms. You may think that everyone should share your definition of what counts as legitimate discrimination—a definition that always excludes Christians. But the reality is that many evangelicals do make that connection and are likely to answer accordingly when primed to respond that way.
Term Switching
Unfortunately, ambiguity and priming affect not only poll respondents but also affects those who hear about the poll. How many people, for instance, substituted the word “persecution” for “discrimination”—even if only in their own minds—when reading the poll or commenting about it on social media?
We are primed to think of all claims of discrimination as forms of persecution, and since persecution is something that only happens to Christians in other countries, any claim that Christians in America face discrimination/persecution can be summarily dismissed.
Mishearing Leads to Misunderstanding
Take another look at the headline from The Atlantic: “White Evangelicals Believe They Face More Discrimination Than Muslims.”
Technically, there is nothing inaccurate about the headline. But it does give a skewed impression of what the poll results actually found.
The poll question itself is not structured in an either/or—neither/nor format. It is not asking, “Do Christians face more discrimination than Muslims?” It’s asking whether there is “a lot of discrimination” against each of seven groups. It does not ask if there is more discrimination for one group compared to another. That’s an important distinction.
When asked those two separate questions—Do Muslims face a lot of discrimination? Do Christians face a lot of discrimination?—a respondent could have answered yes to both, no to both, or yes to one but no to the other. Based on the results, almost half of evangelicals answered no to one or both. While 56 percent said that Muslims do not face a lot of discrimination (or responded that they don’t know), 43 percent said the same was true about Christians. That’s a 13 percent difference—not as radical as it first appears. Taken as a whole, almost half of evangelicals think that both groups face a lot of discrimination.
Compare that to the response gaps for white mainline Protestants (33 points), non-white Protestants (35 points), white Catholics (38 points), or unaffiliated (a whopping 54 point difference).
The result could be, as many people interpret the results, that white evangelicals are out of touch and deluded about the level of discrimination faced by Christians in America. But isn’t it also possible that, depending on how we define discrimination, white evangelicals are merely more cognizant of the problem?
When Are Impressions Plausible?
Some people responding to the poll may take offense by the claim that Christians face discrimination while Muslims do not. However, I suspect the true complaint is with that fact that 57 percent of white evangelicals think Christians face a lot of discrimination—more than do white Catholics and white Protestants combined. Why is there such disagreement?
Let’s consider a possible explanation: That evangelicals are basing their “impression” on their subjective in-group experiences, while the other groups are basing their opinion on what they consider reasonable expectations and assumptions.
I myself have had more contact with and personally know more Muslims than does the average white evangelical in America. I also suspect I know more Muslims than the average white Catholic or white mainline Protestant. Yet if you asked me if I thought Muslims were discriminated against “a lot” my answer (“Yeah, they probably do.”) would be based on assumptions rather than on direct knowledge. I simply don’t recall hearing much about discrimination from my Muslim friends. Like most people, I assume that since minorities have historically tended to face a lot of discrimination it would be logical to assume that religious minorities do too.
But if you asked me if Christians faced a lot of discrimination, my answer would be based on a type of knowledge—the subjective experiences of people I know or know about indirectly. For example, I know some Christians who are academics and could not get tenure at a public college or university if their peers knew they were Bible-believing evangelicals. And yet (I believe) a Qur’an-believing Muslim would not face the same type of discrimination at those same schools.
I've also had interactions in social and work settings where people mocked Christians in a way that they would never do to a Muslim. Is that discrimination? Maybe not. But I assume those people who openly mocked my faith would discriminate against me if given the chance. I may be wrong in thinking that way, but it certainly shapes my impression of whether Christians are discriminated against. And if a pollster were to ask, it would color the answer I gave.
Whether white evangelicals like me are correct in our impressions is debatable. Whether or impressions are plausible should not be controversial. Scoffing at the bumpkins who think Christians in America face discrimination is its own form of ignorance and parochial thinking.
Conclusion
My goal in this article is not to convince you that Christians in America face “discrimination” (whatever that even means). I merely use this latest poll as an example to highlight how opinion polls shape how we think, make us feel superior if we agree with the unstated “correct” answer, and attempt to make us feel foolish if we are associated with those on the “wrong” side.
My intention in this overlong article is to show that by removing essential context and making us think we learned about reality when we did not, the example poll (and others like it) made us slightly dumber.
Opinion polls can even make pollsters dumber. “Polling is merely an instrument for gauging public opinion,” pollster George Gallup claimed. “When a president or any other leader pays attention to poll results, he is, in effect, paying attention to the views of the people. Any other interpretation is nonsense.”
On the contrary, it’s Gallup’s claim that is nonsense. Polls are not gauges of public opinion—at least not reasoned opinion. The truth of the matter, as Winston Churchill noted, is that, “There is no such thing as public opinion. There is only published opinion.” Indeed, these published opinions are nothing more than public perceptions produced by pollsters. These are products—aggregations of our confusion and ignorance about other people’s subjective impressions—packaged to be sold to us by the media.
Why then do we even pay attention to opinion polls? They rarely have any positive and productive use. Their chief effect, albeit modest, is to fool us into falsely believing we’ve gained some useful tidbit of knowledge or insight. And all too often they have the deleterious, and longer-lasting influence, of leading us to dismiss the impressions and experiences of our brothers and sisters in the faith.
We Christians in America don’t seem to be able to agree on much nowadays. But if polled there should be 100 percent agreement on the need for more unity and love within the body of Christ. If they are causing unnecessary divisions within the church, perhaps it’s time we paid less attention to opinion polls.
Is there enough evidence for us to believe the Gospels?
In an age of faith deconstruction and skepticism about the Bible’s authority, it’s common to hear claims that the Gospels are unreliable propaganda. And if the Gospels are shown to be historically unreliable, the whole foundation of Christianity begins to crumble.
