The problem with Scottish focus groups

So the dust has settled, and Scotland will be a member of the United Kingdom for a wee bit longer. This may come as a relief or disappointment to partisans who saw the polls dead even up till last week’s election. But it’s not surprising to the bookies, who use another metric to predict results.

We last looked at this phenomenon during the 2012 U.S. presidential elections. (WordPress is not cooperating at the moment; there should be a hyperlink to https://www.otismaxwell.com/learn-voters-expectations-intentions/ ) Obama vs. Romney was predicted to be neck and neck, but four days before the election I said Obama would win easily and explained why. When they asked how they plan to vote, voters become cheerleaders. They endorse a position even if they have no plan to make it to the polls—or they have a secret prejudice or preference which can be exercised in the privacy of the voting booth.

But ask voters who they think will win, and you get a much more accurate result: the voter as pundit. They’re happy to trade in their personal baggage for the chance to speculate based on their circle of friend and acquaintances.

Which brings me to focus groups as they are used in evaluating direct marketing creative or messaging. Every direct marketer has had the experience of a focus group rejecting a creative platform because “I’d never fall for that” or some such, when we well know, and will subsequently prove, that those very same newly-minted marketing mavens will behave completely differently in the privacy of their web browser or mail pack.

Polls, and focus groups, don’t lie. The moderators and analysts make adjustments to equalize those who are trying to game the system. But they’re not reliable either, and the Scottish referendum result shows why.