Federal Election Polls

[angrypenguin]

Thank you.

It appears Angus Reid was the best with average 1% error and EKOS was the worst. Somehow the latter doesn't surprise me though I was hoping nanos comes first, though they too did well with 1.4% error. If we exclude EKOS then they all were within 2% of each other. Nothing like now which is a 10% difference.

I was surprised too. I used to think Nanos was the most accurate. Guess I was wrong.

[dialamah]

I am dumbfounded. Nanos polls barely move The Signal, and yet the other two polls released today made the Liberals go UP in the Signal? http://signal.thestar.com/

I am at a loss for words. I'm not statistician but these last few polls have me so confused.

Maybe they're using Ouiji boards to weight the different polls?

[angrypenguin]

Maybe they're using Ouiji boards to weight the different polls?

HAHAHA. Ok, you made my night.

[cybercoma]

Not to mention the erratic swings we've seen from Nanos. The only reason I see to take them ahead of the others is that their data is more fresh. Even then, Ipsos shows a closer race than Nanos, giving me pause anyway.

Nanos hasn't seemed erratic to me, maybe I didn't notice. Can you point to an example?

In any case, Nanos is about 11.5 points off on the CPC numbers from Angus, Ipsos, and EKOS. That should give folks pause. Since the others are in agreement, it makes me wonder if Nanos is the outlier. Especially considering he's only polling 400 people at a time, then aggregating those results.

[angrypenguin]

Nanos hasn't seemed erratic to me, maybe I didn't notice. Can you point to an example?

In any case, Nanos is about 11.5 points off on the CPC numbers from Angus, Ipsos, and EKOS. That should give folks pause. Since the others are in agreement, it makes me wonder if Nanos is the outlier. Especially considering he's only polling 400 people at a time, then aggregating those results.

So in your professional opinion, Nanos is the one that is farthest off? I mean as you said, for them it's all within the margin of error, no?

[cybercoma]

Seems to be a consensus building here that Nanos stands alone with his Liberal lead. This implies Conservatives at 35% is likely closer and, therefore, adding in a given 3-5% for more efficiently getting out it's vote, etc. and we have another 4 years of Stephen Harper.

Generally Nanos, but if you look at the other pollsters, it's still not that cut and dry. As of 9pm EDT Oct. 6, these are all of the polls over the last 7 days (click image for source).

8/15 polls have the Liberals ahead. Out of those 8 polls, 6 are Nanos, 1 is Innovative and 1 is Léger. Innovative has no other poll to compare. The second Léger poll flipped the Liberals and Conservative; however, they have them within a single point, so any change is within the margin of error. They're in a statistical tie.

The earlier EKOS, Forum and Angus Reid polls have the Tories with roughly a 7 point lead. Mainstreet gives them an 8 point lead. Meanwhile, Nanos has the Liberals with a 4 to 4.5 point lead in most polls except yesterday's.

So what's going on here?

I'm telling you that I think it's social desirability bias. The CPC's tactics are being labelled as racist. Former Progressive Conservative premier Danny Williams even said as much in an interview. When an interviewer calls people up and talks to them personally on the phone, the respondents may be more likely to hide their intent to vote for the CPC from the interviewer, so as not to appear to buy into what's being labelled as racist. The other polling firms are controlling for this unintentionally by having impersonal web panels and IVR robocalls. Respondents are more likely to be honest when there isn't another person on the line potentially judging them.

[angrypenguin]

Generally Nanos, but if you look at the other pollsters, it's still not that cut and dry.

---SNIP---

Respondents are more likely to be honest when there isn't another person on the line potentially judging them.

Thanks for your opinion. i respect it wholly. However, do you really think Tom Dick and Jane know about stuff like that essentially real time? Most people could give a damn about the thing that unites all of us (politics).

Edited October 7, 2015 by Charles Anthony
[---SNIP---]

[cybercoma]

do you really think Tom Dick and Jane know about stuff like that essentially real time?

Know stuff like what? I don't know what you're asking here or why you're asking it.

Edited October 7, 2015 by cybercoma

[angrypenguin]

Know stuff like what? I don't know what your asking here or why you're asking it.

Sorry, allow me to elaborate please. You're saying that the person picking up the line doesn't want to seem....[insert stereotype here]. Most voters would not know what stereotype they are trying to avoid because they wouldn't know one exists, no?

I mean we all follow polls and the news regarding this election near real time, but for it to resonate to most...that doesn't seem like something that is reflected on a nightly basis.

My $0.01 CDN...

[cybercoma]

The other side of the coin is that telephone interviews are still the best way to get a representative sample. Some people may be more willing to participate with a live interviewer than a computerized polling system. Amongst pollsters, this seems to be the understanding too. They get better responses from live interviews. It seems that people who opt out of robocalls may share common characteristics, reducing the randomness of the sample that's left over to respond. It's also the case that there are more people with access to a landline or cellphone than there are people who have access to computers to be a part of an online panel. More importantly, folks have to opt into online panels, which makes their representativeness questionable at best. That's why the margins of error in online panels are not supposed to be reported and indeed they're presented as the margin of error "in a similar sized poll." That is to say a poll that does have a random sample. The way these companies then weight the responses is important because that will affect the end results as well. That's why census information is so important. The weighting is only as effective as the census.

Lots of things to consider. It will be interesting to see which pollsters are closest after the election and to hear what they have to say about their methods. There's definitely going to be some soul searching after this one in order to understand what worked and what didn't.

[angrypenguin]

Is the rejection rate higher for phone (live interviews) versus online?

[cybercoma]

Sorry, allow me to elaborate please. You're saying that the person picking up the line doesn't want to seem....[insert stereotype here]. Most voters would not know what stereotype they are trying to avoid because they wouldn't know one exists, no?

It doesn't have to be most people. It just has to be enough people to introduce bias. In a poll of 400 people if 120 of them are saying they'll vote Conservative, but really 40 people say they're voting Liberal because they don't want to be judged by the interviewer for voting Conservative, then that's enough to destroy your results and change the outcome by a considerable amount.

My personal opinions of the Conservatives and their tactics aside, it's pretty hard to deny that there isn't a large number of people and a number of news reports that is painting their tactics as racist, bigoted, or intolerant. Someone who supports the Conservatives' positions may not want to be judged that way, so they might lie about where their vote is when conducting a person-to-person interview. They don't need to know that these are the mechanisms at play. They just need to be willing to vote for the CPC, but be worried that people will judge them for it.

[cybercoma]

Is the rejection rate higher for phone (live interviews) versus online?

Yes, but that doesn't matter as much as the representativeness of the leftover sample. Online panels are not a random sampling of Canadians.

[angrypenguin]

Thanks @cybercoma for your opinion. It's vastly appreciated by me.

[dialamah]

I'm telling you that I think it's social desirability bias. The CPC's tactics are being labelled as racist. Former Progressive Conservative premier Danny Williams even said as much in an interview. When an interviewer calls people up and talks to them personally on the phone, the respondents may be more likely to hide their intent to vote for the CPC from the interviewer, so as not to appear to buy into what's being labelled as racist. The other polling firms are controlling for this unintentionally by having impersonal web panels and IVR robocalls. Respondents are more likely to be honest when there isn't another person on the line potentially judging them.

Maybe, but I'm not persuaded they consider it anything but common sense, and they're more likely to dismiss such claims as lies or exaggerations, so why would they be embarrassed?

But I've no better idea so I guess the social desirability theory works as well as any.

[cybercoma]

It's also important to note that younger voters are less likely to respond to telephone polls. They most often text each other and don't answer phone calls from strange numbers. The selection bias here, I would argue is somewhat mitigated by the fact that younger voters, while less likely to answer the polls, are also less likely to turn out to vote. So having that in mind also makes the Nanos numbers even more peculiar. Younger voters are seen as more progressive, yet they're less likely to participate in a phone interview. Meanwhile, the telephone interviews from Nanos show the progressive share of the vote much higher than the conservative one.

It's all very interesting, if you have no life like me.

[angrypenguin]

It's also important to note that younger voters are less likely to respond to telephone polls. They most often text each other and don't answer phone calls from strange numbers. The selection bias here, I would argue is somewhat mitigated by the fact that younger voters, while less likely to answer the polls, are also less likely to turn out to vote. So having that in mind also makes the Nanos numbers even more peculiar. Younger voters are seen as more progressive, yet they're less likely to participate in a phone interview. Meanwhile, the telephone interviews from Nanos show the progressive share of the vote much higher than the conservative one.

It's all very interesting, if you have no life like me.

Well apparently young people don't vote for the Conservatives. So...how do you explain that for online polls?

[cybercoma]

Well apparently young people don't vote for the Conservatives. So...how do you explain that for online polls?

Again, I think the bigger factor is social desirability bias. Online polls are impersonal. So if someone thinks admitting that they're going to vote for the Conservatives is unpopular, they're more likely to tell the truth in an online poll than they are with an interviewer who could be silently judging them.

[cybercoma]

Oh and these online polls are not like polls setup by websites. People join online panels and they're sent questionnaires on a wide variety of topics from market research to social policies to election intentions.

[dialamah]

Again, I think the bigger factor is social desirability bias. Online polls are impersonal. So if someone thinks admitting that they're going to vote for the Conservatives is unpopular, they're more likely to tell the truth in an online poll than they are with an interviewer who could be silently judging them.

Wouldn't they be more likely to say undecided, rather than claim an intent to vote for a different party altogether?

Also, as the furor over it dies down would these people stop denying their voting intention, so over the next few days the polls will even out?

[cybercoma]

Wouldn't they be more likely to say undecided, rather than claim an intent to vote for a different party altogether?

Also, as the furor over it dies down would these people stop denying their voting intention, so over the next few days the polls will even out?

Either way it would still skew the results. Saying they're undecided or refusing to respond would still reduce the difference in votes between the CPC and LPC in these polls.

Also, I don't think this is exclusive to the niqab issue. I think the niqab and immigration issues are really big obvious drivers of social desirability bias. Nobody wants to appear racist or have to explain themselves to someone who might think they're racist, especially if they don't believe that they are. So that's a big one. However, there's enough anti-Harper sentiment out there--I would argue it's angrier and a lot more aggressive than the anti-Trudeau or anti-Mulcair sentiment--that it may still come into play even after the niqab and refugee issues are long forgotten.

[angrypenguin]

Oh and these online polls are not like polls setup by websites. People join online panels and they're sent questionnaires on a wide variety of topics from market research to social policies to election intentions.

I disagree with you. My wife takes part of these online polls and is paid for them. I'll have to ask her which polls in particular but she has been asked to fill them out. (for $)

Edited October 7, 2015 by angrypenguin

[angrypenguin]

It doesn't have to be most people. It just has to be enough people to introduce bias. In a poll of 400 people if 120 of them are saying they'll vote Conservative, but really 40 people say they're voting Liberal because they don't want to be judged by the interviewer for voting Conservative, then that's enough to destroy your results and change the outcome by a considerable amount.

My personal opinions of the Conservatives and their tactics aside, it's pretty hard to deny that there isn't a large number of people and a number of news reports that is painting their tactics as racist, bigoted, or intolerant. Someone who supports the Conservatives' positions may not want to be judged that way, so they might lie about where their vote is when conducting a person-to-person interview. They don't need to know that these are the mechanisms at play. They just need to be willing to vote for the CPC, but be worried that people will judge them for it.

Also, does the CPC voter really care what people think in a phone poll though?

I mean, look i'll be frank. I am a CPC voter, I could give a shit what the pollster was asking me. I realize I'm pretty die hard but I don't think any other CPC supporter would care...

[cybercoma]

If the policies are being editorialized as racist and intolerant, most people would not want to be associated with that, even if they were intent on supporting the Conservatives. Most people who vote Conservative--as opposed to many of the radicals on this forum--just want a smaller government that's going to tax them less. They believe that giving industries tax breaks and goodies will make Canada a better more prosperous place. All of this garbage about the niqab and whatnot, they probably couldn't care less about and are frankly embarrassed to hear it. If the party that they support because they believe it has the best fiscal policies and economic plans is being painted as knuckle-dragging bigots, they're not going to want to be associated with that. Sure, there's a number of them that wouldn't care. But we're talking about polls of 1200 or so people and only 350 or so are responding Conservative. You don't need many people to hide their intentions for it to skew the results. I'm arguing that it's more likely that a Conservative would hide their intentions because they wouldn't want to be associated with bigotry and racism in this case. Those things are socially unpopular. So much so that people who make racist arguments will aggressively deny it even when it's plain as day. There's a very good chance that respondents when talking to someone person-to-person is not going to want to be associated with the party that people are calling bigoted and intolerant.

[angrypenguin]

If the policies are being editorialized as racist and intolerant, most people would not want to be associated with that, even if they were intent on supporting the Conservatives. Most people who vote Conservative--as opposed to many of the radicals on this forum--just want a smaller government that's going to tax them less. They believe that giving industries tax breaks and goodies will make Canada a better more prosperous place. All of this garbage about the niqab and whatnot, they probably couldn't care less about and are frankly embarrassed to hear it. If the party that they support because they believe it has the best fiscal policies and economic plans is being painted as knuckle-dragging bigots, they're not going to want to be associated with that. Sure, there's a number of them that wouldn't care. But we're talking about polls of 1200 or so people and only 350 or so are responding Conservative. You don't need many people to hide their intentions for it to skew the results. I'm arguing that it's more likely that a Conservative would hide their intentions because they wouldn't want to be associated with bigotry and racism in this case. Those things are socially unpopular. So much so that people who make racist arguments will aggressively deny it even when it's plain as day. There's a very good chance that respondents when talking to someone person-to-person is not going to want to be associated with the party that people are calling bigoted and intolerant.

Thanks for this. My argument would be...if the average right voter were stupid (which is what has been implied on this board), why would the right voter answer the phone at all in the first place? I think in your assumption you are assuming that the voters as part of N know what's going on day to day. I don't think so.