Reviewing Robert Wuthnow’s Inventing American Religion: Polls, Surveys, and the Tenuous Quest for a Nation’s Faith, Alan Brill examines the ways that surveys about the state of religious life can mislead and the specific implications for the Jewish community of drawing false conclusions from them:
When in 1988 George H.W. Bush beat Michael Dukakis in the polls (and in the actual election), people did not go around saying that the future is Republican or that the Democratic party is dying. . . . However, when it comes to polls about religion, we find pundits . . . and ordinary people assuming that any given trend will continue without accounting for changing times. . . . Almost all of the discussions [within the American Jewish community about] the future of Orthodox or Conservative Judaism, the Pew study, the Jewish renewal movement, or assimilation are predicated on assuming that the answers to [survey] questions at a given point in time can be predictive. . . .
[A]lways remember that polls have a very low response rate. Most of them, whether about religion or politics, have an 8-percent response rate now. . . . [According to] Wuthnow, even when we are reassured that a poll is trustworthy—for example, it claims to have a margin of error of 3 percent—the margin of error is likely closer to 20 percent. Even then most of [the results are skewed by poorly formulated questions and the like].
[Moreover], whereas political polls face occasional reality checks—elections actually happen, and pollsters can [subsequently] adjust weighting factors so that the data are closer next time—polls about religion have no such checks. So if we hear that a certain percentage of the public is not really Catholic even though they say they are, . . . we can only ask ourselves, “Well, does that make sense with what we know from other sources, and from talking with our neighbors?”
Read more on Book of Doctrines and Opinions: https://kavvanah.wordpress.com/2016/05/21/religion-polls-and-judaism-robert-wuthnows-inventing-american-religion/