Now, about this profile. I deliberately spent very little time with it, in order to insure that the on-line "person" I created for Angus Reid would be demographically incoherent. So for example:
1) For my name, I gave the hero in one of my (many) unpublished novels, but listed my sex as female.
2) I provided the phone number on the back of my Visa card (their help line), and a postal code from a nearby LCBO.
3) I gave a relatively correct age, but listed my occupation as student.
4) I provided a hotmail account as my e-mail address.
I lied outrageously in ways that escape me right now, but they accepted my application anyway.
So yesterday I received my first survey on "green issues", which also asked about voting intentions towards the end. Again, I answered as quickly as I could, without putting much care into the overall coherence of my responses (the wife and I were going out for wings). So for example, when it came to voting intentions, I punched in Tory, though I had indicated the BQ as the party most likely to act decisively on environmental issues. And so on.
My verdict is: the Angus Reid on-line survey system is pathetically easy to prank (or FREEP, as the youth of today apparently call it), unless some kind of filters are in place that I am not seeing. You could, for example, create as many on-line profiles with Angus Reid as you have hotmail accounts, and unless they are tracking IP addresses and tying each account to a specific address, it is impossible for them to verify even the most basic pieces of demographic information (like the province you live in). At least with phone surveys they can tie you to a relatively precise physical location and presumably tell if you are lying about your gender. Further, even if they are tracking IP addresses, you could work from a wireless hot-spot or a public library computer to create multiple profiles. The only real limit is the amount of time a person is willing to spend filling out on-line forms (this one survey took me about fifteen minutes, and whomping up the initial profile about the same length of time).
But if you're a political party that is already willing to buy headlines from Bourque, or if you're a Blogging Tory with no life outside the frat house, it would not be difficult to manufacture several dozen Tory responses via the Angus Reid system.
Or Liberal responses, if you were that way inclined, or whatever.
Therefore, treat Angus Reid survey results with a great dollop of skepticism.
16 comments:
If someone from the company phoned your house to get demographic information during a study, you could lie through your teeth too and there's nothing that can be done (unless you provided absolutely ridiculous answers).
The online polls are just as vulnerable to mistakes or "pranks" as the telephone studies.
An interviewer does have to use some discretion but you have to realize that faulty information can be provided over the phone too - though likely if you used super hero names the interviewer would likely clue in! :)
Personally, I hope that online poll skeptics from all parties follow your suggestion and waste an inordinate amount of time setting up multiple email addresses and online accounts. Then I hope they all respond to every survey. Then I hope that it's a while before they realize that they are all just cancelling each other out. But in the meantime hopefully it takes some time away from their writing posts about how they've figured out how to take on online polls.
Whooee! I reckon the big difference is that with phone polls, the pollsters call pollees. With the online sign-up thing, the pollees make the first move by enlisting. Darren's on the money whe he sez the pollees can prnak the phone interviewer as easily as the online prankster. BCL's on the money when he sez the difference is that the online pollee can create multiple identities but most of us only have one phone number.
It ain't rocket science t' only allow one response per IP,though. Pergressive Boogers only allow one vote per IP in their set up.
Ma is on Ispos's survey list an' does some bigass surveys about once a week or so. I ain't sure how she got started but I'm sure it wasn't from her volunteerin' but from them askin' her t' do it.
JimBobby
Right, JimBobby.
I guess the next thing to do would be to test whether they are doing 1 vote 1 IP. Not that that's so hard to get around.
And, Clark, at the moment I don't think they would cancel out. Blogging Tory types seem to move en masse on polls they want to screw over. I don't see similar behavior from the other segments of the on-line population.
When they phone your home - they have your correct phone number and probably already have your name, etc.
Online is a joke. Remember the poll naming the best PM that never was - obviously crammed with Tory submissions.
I wish people would stop confusing online polls conducted by firms with the one's people run on their own websites, a la Bourque. They are totally different beasts.
I think one needs only read the comments from Cherniak's second post about extrapolation to see how off the mark he was. As are most folks who's intuitions may seem right but aren't in the end.
anon 9:57,
Yes but pranking the one is only a little more ambitious than pranking the other.
I mentioned this at Jason C's, but in the UK, the firm "YouGov" has used online polling, and the results of their polling for the last couple of general elections there were no worse than the traditional pollsters, and arguably better.
It's Wiki write-up (yeah, it's Wiki, so treat with caution) gives some indication of its sample-size, and the fact that it's used by some fairly heavy-hitting media outlets (Daily Telegraph, Economist).
I don't know if Ipsos/Angus Reid is as "good" as YouGov, but it would seem that you can get polling via on-line sampling that's no worse (or better) than the telephone method.
Oh dear, there's trouble a-brewin' in Leftyland today.
http://www.NationalNewswatch.com/
It is revealed Dijon isn't the Kyoto champion he claims he is, and he's also suing that old country-wrecker Gilles.
Why can't they just get along? Think of the children.
I wish to thank the Liberal Party of Canada from the bottom of my cold, black, evil paleo-con heart, for electing Dijon as your leader. Oh, I'm mean and harsh, too.
I mentioned this at Jason C's, but in the UK, the firm "YouGov" has used online polling, and the results of their polling for the last couple of general elections there were no worse than the traditional pollsters, and arguably better.
Maybe someone should look into whether the UK has a surplus of illiterate mouth-breathing rightwingers (like the anonymous above) who have nothing better to do with their time than gum up serious inquiry with irrelevant nonsense.
I don't think so; In the UK, those types are usually down the pub or brawling at football matches.
Well, I certainly think its possible to set up an on-line survey/voting system that is good enough to prevent the worst kinds of pranking (I actually had to do some research on this topic a couple of years ago). Angus Reid's just seems weak in this area (although doing the survey itself is kind of fun).
Since the American "election" of 2000, I've become a Luddite with regard to online population surveys. There's nothing a liar loves more than plenty of unfathomable technology.
BCL:
"Blogging Tory types seem to move en masse on polls they want to screw over. I don't see similar behavior from the other segments of the on-line population."
are you suggesting that ignatieff suppporters are blogging tories? ;-) cause that on-line mcleans poll showing 99% support for iggy sure pops in my mind. as wells said, online polls (the garden variety, not the pollsters variety) are only good for judging which campaign is the m,ost insecure.
Any online polling is for fun only, since it's so easily manipulated.
Another poll that went whacko and blinked out.
:)
http://no-libs.com/?p=1494
I registered too and played it "straight"...as is my wont.
I noticed as an Albertan when they asked what provincial party I voted for last time they did not have my party listed. They had the Conservative Party but not the Progressive Conservative Party...so I said "Other."
There is a huge difference I can assure you.
Post a Comment