Also, researchers, like all people, have their own biases and preconceptions and these show up in their research. Ideally, in this type of work researchers should be completely neutral and simply go where the data take them, but unfortunately this is often not the case. I’ve aimed for this neutral approach with this book, but I don’t know that I’ve been entirely successful. Social researchers’ beliefs color our research in ways that we’re not even aware of—even when we’re studying “objective” facts.
Finally, we should be suspicious of social statistics because they tend to mutate when they are passed along. (I illustrated this mutation process in chapter 1.) As such, even if a researcher is highly skilled and completely neutral, his or her work might become more inaccurate with each retelling. Often this mutation results in statistics growing more dramatic over time.
As I write this concluding chapter, I have a nagging worry that you the reader won’t believe that you have both the ability and the need to critically evaluate statistics. You’ve had a lot of training in school and day-to-day life about accepting facts from experts. Well, to make it as easy as possible, I’m going to deputize you. In the old Westerns, after a bank robbery or some other heinous act, the town sheriff would deputize town citizens, and they would ride off in a posse to catch the bad guys. I’m not going to give you a horse or a gun, but I will give you an official deputy-sociologist badge:
Cut this badge out and put it in your wallet or purse. It gives you the right to do the following with any statistic about Christianity:
Question whether it’s accurate
Question the motives of the person writing
Disagree with the conclusions
Judge the statistic in light of your own experiences
Not believe it for any reason, including just being in a cranky mood
It’s official: Go forth and think for yourself about the portrayal of Christianity.
APPENDIX 1
Identifying Evangelical Christians
Survey researchers typically use one of three methods for identifying Evangelical Christians. The first and most commonly used method is to measure denominational affiliation. Here researchers ask people which type of church they affiliate with. For example, the General Social Survey asks respondents: “What is your religious preference? Is it Protestant, Catholic, Jewish, some other religion, or no religion?” Those who identify themselves as Protestant are then asked: “What specific denomination is that, if any?”
Respondents’ religious affiliations are classified into several categories. When possible, I have used Steensland, et al.’s (2000) RELTRAD classification method, which identifies seven religious groups in the United States: Evangelical Protestant, Mainline Protestant, Black Protestant, Catholic, Jewish, Other Religions, and Religiously Unaffiliated.
Evangelical affiliations include, but are not limited to, the following: Southern Baptist Convention; Independent Baptist in the Evangelical Tradition; Nondenominational; Lutheran Church, Missouri Synod; Presbyterian Church in America; Assemblies of God; Church of Christ; Church of the Nazarene; Free Methodist Church; and Seventh-day Adventist.
Mainline Protestants include: American Baptist Churches in USA, United Methodist Church, Evangelical Lutheran Church in America (ELCA), Presbyterian Church USA, Episcopal Church in the USA, and United Church of Christ.
Black Protestants include: National Baptist Convention, African Methodist Episcopal, Church of God in Christ, as well as African-American participants in other Baptist denominations.
Other religions include: Buddhism, Hinduism, Islam, Mormonism, Jehovah’s Witnesses, Christian Science, and Unitarian-Universalist. This classification coding scheme does not imply that these other religions are similar in content; rather, there are relatively so few of their members in this country that they typically cannot be analyzed separately, and so they are grouped together into this leftover category. Also, some of the “other” religions identify themselves as Christian. As such, the definition of Christian used in this book includes only the Protestant and Catholic traditions. I’ll leave to others the discussion of whether some of these other religions are Christian in the theological sense.
The religiously unaffiliated include atheists, agnostics, and those who have strong religious and spiritual beliefs but do not affiliate with any particular religion.
A second approach to defining Evangelicals asks respondents if they label themselves as Evangelical. For example, the 2000 General Social Survey asked respondents: When it comes to your religious identity, would you say you are a Pentecostal, Fundamentalist, Evangelical, Mainline, or Liberal Protestant, or do none of these describe you?”
A third approach asks respondents various questions about their beliefs and practices, and the research decides who is Evangelical on the basis of respondents’ answers. Perhaps the best known example of this approach is used by the Barna Group. They define bornagain Christians as those who say: (1) They have made a personal commitment to Jesus Christ; and (2) believe they will go to heaven because of having confessed their sins and accepted Jesus Christ as Savior. Among born-again Christians, Evangelicals are those who agree with seven more theological points: (1) Their faith is very important, (2) they have a responsibility to share their faith with non-Christians, (3) Satan exists, (4) salvation is gained through faith alone, (5) Jesus lived a sinless life, (6) the Bible is accurate in all that it teaches, and (7) God is the perfect and powerful Creator of the world. An individual must agree with all nine of these points to be labeled Evangelical.
Significantly, the Barna Group Research’s definition of being born-again leaves out many Catholics and Mainline Protestants, and in some research reports, the Barna Group Research has labeled as “non-Christian” the Catholics and Mainline Protestants who do not meet the born-again criteria.[1] Furthermore, the Barna Group Research does not ask questions about affiliation, so one could be defined as an Evangelical Christian without ever attending church.
These three measurement approaches yield different findings; in fact, Hackett and Lindsay (2008) found that the estimated number of Evangelicals in the United States ranges from 5% to almost 50%, depending on which measurement approach is used. I recommend reading their article if you want to learn more about these measurement approaches and their implications for research.
APPENDIX 2
Data Sets
Here are brief descriptions of the main data sets analyzed in this book.
America’s Evangelicals
The America’s Evangelicals Study was collected in 2004, and it was sponsored by Religion and Ethics Newsweekly and U.S. News and World Report. Respondents, ages eighteen and over, were contacted nationwide for a telephone interview, and the final sample size was 1,610 respondents. This included an oversample of White Evangelical Christians, allowing for more in-depth analysis of this group. These data can be accessed from the American Religious Data Archive.
ARIS
The American Religious Identification Survey is a large-scale three-part study of religion in the United States. It was collected in 1990, 2001, and 2008, with the 2001 and 2008 studies replicating the earlier 1990 study, thus documenting changes over time. Respondents were selected using random-digit-dialing in the forty-eight contiguous states. Sample sizes were roughly 113,000 in 1990; 50,000 in 2001; and 54,000 in 2008, and respondents were interviewed by phone. In each interview respondents were asked the open-ended question, “What is your religion, if any?” Responses to this question are coded into a taxonomy of religious traditions and denominations. Full reports of the study’s findings are available at americanreligionsurvey-aris.org.
General Social Survey
The General Social Survey is an ongoing national survey about a wide range of social values, attitudes, and behaviors. It is collected by the National Opinion Research Center at the University of Chicago. From 1971 to 1993 it was collected annually (except for the years 1979, 1981, and 1992). Since 1994, it has been collected every other year.
The General Social Survey collects a full probability sample of all English-speaking, noninstitutionalized adults over age eighteen in the United States. Starting in 2006, Spanish speakers were added to the target population. Interviews are conducted in respondents’ homes, and the survey has a high response rate because it makes numerous callbacks. Sample sizes range from about 1,500 to 4,500. The data can be accessed through the Inter-University Consortium for Political and Social Research.
Monitoring the Future
Monitoring the Future is an annual study of the beliefs, attitudes, and behaviors of high school students, college students, and young adults. I’ve analyzed data from the annual survey of twelfth graders, which has been gathered since 1975 by the Institute for Survey Research at the University of Michigan. Each year Monitoring the Future interviews 16,000 high school seniors from 130 randomly selected public and private schools nationwide. In smaller schools, all the seniors might be interviewed; whereas, in larger schools a random or other unbiased sample is taken. The questionnaires are administered in the classroom. The data can be accessed through the Inter-University Consortium for Political and Social Research.
National Longitudinal Study of Adolescent Health
The National Longitudinal Study of Adolescent Health is a longitudinal study of American adolescents. It started with a nationally representative sample of seventh to twelfth graders sampled in 1994 to 1995, and they have been interviewed several times since. The initial sample was taken from several hundred American high schools and middle schools. The first wave had over 20,000 respondents, and Waves 2 through 4 have had about 15,000 respondents. Surveys were collected both in the classroom and at home. Direct inquiries about the data to the Carolina Population Center at the University of North Carolina, Chapel Hill.
National Survey of Family Growth (2002)
The National Survey of Family Growth, Cycle VI, was collected in 2002 by the National Center for Health Statistics. It sampled men and women ages 15 to 44 from throughout the United States, and it interviewed a total of 12,571 respondents. The data can be accessed through the Inter-University Consortium for Political and Social Research.
National Study of Youth and Religion, Waves 1 and 2
The National Study of Youth and Religion is a nationwide study of American youth. Its first wave was collected in 2003, in which 3,370 English-and Spanish-speaking teenagers and their parents were interviewed. At the time of Wave 1, the teenagers were ages 13 to 17. Three years later, at the time of Wave 2, in 2006, the respondents were ages 16 to 20. Both data sets were collected by the University of North Carolina at Chapel Hill. These data can be accessed from the American Religious Data Archives.
Pew U.S. Religious Landscape Survey (2008)
The Pew U.S. Religious Landscape Survey is a large-scale, nationally representative study regarding religion and public life. It was collected by the Pew Foundation in 2007 with the data being published in 2008. Adult respondents were sampled from the Continental United States, and a total of 35,556 respondents were interviewed, mostly by phone. Reports on these data are available at the Web site of the Pew Forum on Religion and Public Life: www.pewforum.org.
Social Capital Community Survey (2006)
The Social Capital Community Survey was collected by the John F. Kennedy School of Government at Harvard University in 2006. This survey had two components: A nationwide sample of 2,741 adults and twenty-two community studies of another 9,359 adults. In this book, I analyze only the nationwide sample. Respondents were interviewed by telephone, and the data set is available from the Roper Center for Public Opinion Research.
APPENDIX 3
Bivariate vs. Multivariate Analysis
This book mostly examines bivariate relationships, i.e., those between two variables, without controlling for other variables as one does in multivariate analysis. As an example, in chapter 6, I examine the relationship between religion and crime, and overall religious people are arrested less often and commit less crime than the religiously unaffiliated. This finding is open to various causal interpretations. Among them, it could be that women are more likely to be religious, women commit less crime, and so the observed association between crime and religion might only be caused by these two correlations. In statistical language, gender might make spurious the correlation between religion and crime.
Sounds simple, right? Well, as seems to always happen with issues of causality, things start to get complex. Even if the relationship between crime and religion disappears completely when controlling for gender (which it doesn’t, as I show below), there could be a more elaborate causal story. Perhaps the role of women in society, especially as it relates to not committing crimes, is influenced by religious principles. If so, religion influences women’s behavior, which in turn affects crime. From this perspective, the social roles associated with gender become a causally mediating variable linking religion and crime rather than being an extraneous control variable. It might explain the impact of religion rather than explaining it away.
In my analysis, I could control for gender, but why stop there? Criminologists have linked criminal behavior to many other factors, including race, social class, age, geographical region, personality characteristics, attitudes, social ties, employment, education, and past experiences with the criminal justice system. If we’re to conduct a proper multivariate analysis, we should control for these other factors as well. This approach, however, considerably increases the complexity of the analysis, and one could easily write a book about religion, crime, and gender alone.
Multivariate analysis certainly has a place in academic research, and I have used it in my own scholarly publications, but for the purposes of this book, I fear that it would take the analysis far beyond the interest level of the non-academic reader. In order to examine a wide range of outcome variables, I put aside issues of causality simply to clarify the bivariate relationships of religion.
In case you were wondering, however, here is the relationship between religion and crime, controlling for gender. To simplify the presentation, I will compare Protestants to the religiously unaffiliated.
Protestant vs. Unaffiliated
Outcome In Whole Sample Males Only Females Only
Arrested 9% vs. 15% * 17% vs. 22% * 3% vs. 6% *
Damaged Property 7% vs. 12% * 11% vs. 17% * 4% vs. 7% *
Stolen > $50 3% vs. 5% * 4% vs. 6% * 2% vs. 3% *
Hurt Someone in Fight 5% vs. 7% * 10% vs. 10% 2% vs. 3% *
* Difference is statistically significant at p = .05. Data from Wave 3 of Add Health
APPENDIX 4
Statistical Significance
Statistical inference is a key feature of survey research, for it allows us to know what kinds of conclusions we can make about a population of people simply by studying a sample of them.
Here’s an example: Suppose we want to predict who will win the next presidential election. We could interview every single American and ask them if they will vote, and if so, who will they vote for? This would give us a reasonably accurate prediction (to the extent that people know ahead of time for whom they will vote), but it would take a lot of money and time. Instead, we would probably draw a sample of Americans. Supposing that we took a random or near-random sample, statistical inference tells us how certain we can be that our sample reflects the population as a whole. Generally speaking, assuming appropriate sampling procedures, larger samples do an overall better job of representing the population than do smaller samples.
With regard to this book, issues of statistical inference come up most acutely in comparisons of different religious groups. For example, Figure 6.1 reports that those Evangelical Christians who have ever been married are less likely to have been divorced than the religiously unaffiliated. This difference does exist among respondents in the General Social Survey, but does that mean that we can generalize to Americans as a whole (assuming the General Social Survey is an accurate representation of the American population)? Sociologists answer this
type of question by testing whether the difference between the groups is statistically significant. As is commonly done, this means using statistical analysis to test if we’re 95% sure that the differences we observe in a sample really do exist in the population. In this case, the difference in divorce rates between Evangelicals and the religiously unaffiliated is statistically significant, meaning that we can be reasonably certain that these two groups have different divorce rates in the American population.
It’s not entirely clear what is the best way to present statistical significance tests in a book like this, which is aimed at a general audience. If I were writing for fellow sociologists, I would report all significance tests for each analysis, but this would create dozens and dozens of tables just crawling with coefficients, standard errors, and z-scores. Instead, I will present a table summarizing key significance tests on my Web site, so if you are interested, you can check it out at brewright.com.
NOTES
CHAPTER 1
[1]. The concept of statistics mutating is discussed in persuasive detail in Best, 2001.
Christians Are Hate-Filled Hypocrites...and Other Lies You've Been Told Page 16