Margin of error critical in interpreting polls


The final Irish Times/MRBI opinion poll before voting proved to be a remarkably accurate indicator of the ultimate outcome of the election, asserts Ian McShane

One of the salient features of the 2002 election campaign has been the high level of media coverage afforded to the national and local opinion polls.

As always, once the results of the election are clear and the dust has settled, the same media bodies will turn to the polls, and the degree to which they provided solid indicators of the performance of the various parties at the actual election.

For MRBI and The Irish Times, this election was unique insofar as a decision was taken to amend the method of questioning from which party voting intentions are extrapolated.

The new technique was used with a view to counteracting the historical propensity of opinion polls in Ireland to overestimate levels of support for Fianna Fáil.

Bespoke research conducted by MRBI for The Irish Times had previously indicated that one in eight of all potential voters either could not identify the party to which a TD they could name belonged, or identified their named TD with an incorrect party. Among the latter group, stated support for Fianna Fáil was well above average.

On the basis, therefore, of strong evidence to suggest an overstatement of Fianna Fáil support and an understatement of support for Fine Gael - with the smaller parties and the ever-growing body of Independents linked to this confusion between party and candidate allegiance - the method of questioning was altered.

Rather than ask respondents to which party they would give their first-preference vote, therefore, the historical party support question was dropped altogether, with survey participants instead presented with a simulated ballot paper during the interview.

The ballot paper listed the candidates standing in the constituency by name, along with their party affiliation where appropriate. Respondents then allocated their first, second, third preferences, etc., as they would when actually casting their votes.

The table sets down the first-preference voting intentions as measured on Monday, May 13th, four days before polling day, alongside the actual first-preference vote achieved. The third column notes the variation between voting intentions and actual vote achieved, and should be viewed within the context of the statistical margin of error of 3 per cent, pertaining to a sample size of 1,000.

The margin of error is a critical element in interpreting opinion-poll findings, and one which is often ignored, or misunderstood, by media commentators.

Essentially, the sampling error allows for the fact that the views of 1,000 people on a given day provide an estimate of the views of the electorate.

We can in fact be 95 per cent confident that the result, if all 2.95 million electors in the State had been interviewed on the same day, would lie within three percentage points either side of the poll figure. The first point to note is that the only party support figure which exceeds the margin of error when compared with the actual vote achieved is Fianna Fáil's, and even then by just a fraction of one percentage point.

The second point to make is that the opinion poll was carried out a full four days before people actually cast their votes.

It would be naïve not to expect people's opinions to shift and consolidate during the final days of the campaign.

It would not be unreasonable to assume, for example, that Michael Noonan's debate victory over Bertie Ahern, and the PDs' ferocious anti-Fianna Fáil majority campaign, may have had some positive effect on support for Fine Gael and the PDs, with a corresponding decline for Fianna Fáil.

It is interesting to note that the variation between voting intentions and actual behaviour for Labour, Sinn Féin, the Green Party and Independents is negligible.

It is our contention that if we had not replaced the party support question entirely with the ballot paper, support for Fianna Fáil would have been grossly overstated, with a knock-on effect all the way down to the Independents.

While absolute first-preference voting intentions are obviously of critical importance in providing indicators as to the likely outcome of the election, the true value of the opinion poll lies in the intuitive interpretation of what the figures are really telling us.

A number of key conclusions were drawn in the MRBI survey commentary reported in The Irish Times on Wednesday, May 15th, and written on the evening of Monday 13th.

First, it was stated that Bertie Ahern seemed almost certain to emerge as the next Taoiseach, and that if Fianna Fáil required support from outside the party, the most likely arrangement appeared to be Fianna Fáil along with the PDs and/or a number of like-minded Independents.

Second, it was stated that Fine Gael appeared set to lose seats, with the precise extent of its losses impossible to quantify as it would be fighting for a range of seats on the final count.

This, too, has come to pass, with the full extent of the party's losses only apparent early on Sunday morning. It was stated that Labour could make but modest gains on its current standing of 21 seats. In fact the party has stood still on precisely 21.

From Sinn Féin's perspective, it was pointed out that four days before the election the party's target of three seats seemed probable, with four or more a distinct possibility. It has increased its representation to five seats.

As an overview, it was concluded that as the Green Party and Independents also appeared likely to increase their number of seats, the long-term outcome of the election could well represent a realignment of Irish politics, with the right-of-centre ground belonging to the Fianna Fáil-PD bloc, left-of-centre defaulting to Labour, and Sinn Féin, the Green Party and socialist Independents representing the more radical left, with Fine Gael facing some difficult decisions.

Political commentators on yesterday's early-morning radio shows were echoing precisely this sentiment.

It is thus quite clear that the Irish Times/MRBI pre-election poll of Monday, May 13th, was remarkably accurate in gauging the material outcome of the election itself; a fact undoubtedly facilitated by the unique and innovative manner in which the ballot-paper technique was employed during the survey.

Ian McShane is managing director of the Market Research Bureau of Ireland