Fill Out this Survey…
I was picking up some last minute supplies the other day at the corner Rite Aid. It is not my usual stop but was on the way to another place I was going. I got my items to the counter and was checked out by a pleasant, enough, woman. When she handed my my receipt commented that I had a special survey receipt.
She instructed me to call the toll-free number and rate the store on a scale from 1 to 5. Good research would stop the conversation there but she continued. She told me that 1 was the worst and 5 was the best and I should give the store all 5′s and that would be nice.
Now, I might not be in my research statistics class right now, but it gave me an interesting example of bias in research. A long time ago I assisted on an economics project assessing the socio-economic underpinnings of my city. The research was designed by professors of the college and reviewed for bias and accuracy by other members of the faculty. One would think this was enough rigor; it was not. The students conducting the survey had the opportunity, in asking the questions to add spin and bias to the answers given by respondents. Only through rigorous training and oversight could the research be deemed relatively bias-free. If a student skewed responses by so much as a wink or a smile, the survey was junk.
These toll-free surveys are commonplace and each one offers the respondent a chance to win $10,000 or something like that. It serves to both rate the store but more likely to gather important customer data that is worth more than the 1 prize given annually. I’ve called them and found out. Hey, I am a curious sort of guy.
This recent example shows how survey data can be skewed by the innocent or not-so-innocent comments made at the time of the survey. By telling me that all 5′s would be nice, the store clerk attempted to HALO the responses, that is make them all shine. Why? Well, what are the incentives in place? We don’t know. One could venture a guess that the stores are ranked across the district and managers get some bonus for # of surveys completed and highest survey score. That is the most likely scenario but one that is not confirmed.
Bringing this back to my research, if my work is conducted by others, say volunteers or company representatives, I need to monitor the execution closely. Most likely a survey will not be the tool of choice. That said, I might use it to help clarify and identify important relationships that my quantitative analysis can prove or disprove. It is imperative that statements like “rate this highly for me” be eliminated early on to make the research more valid.
At Rite Aid, despite the potentially high rating that store gets each month, I would be asking the question of how they do it. Somewhere, some one is asking that question and they only need to shop there once or twice to find out.
Your comments are most welcome (hey, how do you like the theme by the way?)
Want another Doc Blog post on interesting stats?
Other Doc Blog posts about my RES/722 class adventure:
Lies, Damn Lies and Statistics
Disappointed
Happy
Fill Out This Survey…
Navigating the Schedule Change
LOL you have not changed the theme that much Mike
. But is better. Mmmm thinking with your sense of humour perhaps you were checking whether I was really reading your posts ROFL.
Wow better stop picking on you as you have put me on your link roll.
Ok, now after finishing 31 Day to Build a Better Blog, and you have sort of asked for advice by asking if we liked your new template. I will jump right in and offer advice, without being asked – the about page is very important. Understandably you may not want people to know more about you given you are currently study – so ignore this advice it is not suitable. The more personally you can connect with your readers, the better you will be as a blogger. On your About page it is important to tell your readers, concisely, more about you as a person first, and then what your blog is about. If you want people to subscribe to your blog this is very important.
I would also put the disclaimer on a separate page at the top.
Saying all that – you write really well! And your use of pictures really grabs attention. Great analysis of how researchers can bias results.
Sue
Like or Dislike:
0 
0
It’s great that you noticed this and had the curiosity to investigate further – it really shows how research can be biased! I also think what you’ve discussed would be important reading for any research student who is writing up their own survey.
Looks like you’ve got a great blog here
I’ll be checking it out more often.
Cheers,
Kate
Like or Dislike:
0 
0
[...] Well, a new visitor, Kate from waxlyrical.net came by and dropped off a nice comment about bias in research. Her comment about my curiosity was spot on and made me chuckle a bit. I am curious, and I find myself down the rat hole on many things. In fact, a colleague of mine and I discuss our little addictions to new ideas and information from time to time. [...]
Like or Dislike:
0 
0
Hi Mike perhaps check out http://betterblog.ning.com/ and consider joining us? We are the group (led by Sue and Frances) who got together with the added motivation of chocolate for a reward. Fortunately we all realised it was much more than that. Better still I’m the teacher of a class of hard working cute kids – and we won! Please drop in and say hi.
It’s a wonderful idea to start the blog challenge on the first day of the month. The support and solidarity of a group/network also helps enormously. The Better Blog Ning site will also benefit from your experiences and expertise. Cheers, Al
Like or Dislike:
0 
0
Actually technically I should clarify Al’s statement. Michelle Martin is our leader. She decided to join the 31 Day Project and invited her readers to join her. Thought about it for a few days, I have not time, but decided to join anyway. Frances thought about it for a few days, then joined but challenged me to loser buys 1 kg chocolate. I then said why stop there – lets open up the winning to the World.
So our team joined with Michelle’s team. Michelle is the person who has set up Better blog community at Ning. As I said to her a leader is important. And we are all grateful to Michelle.
Sue
Like or Dislike:
0 
0
Here again are some insightful comments from new readers.
Sue wrote some great comments on how to make the Doc Blog better. Incorporated into the improving site are Sue’s thoughts.
Kate commented about the need to researchers and students to be curious. I couldn’t agree more. The nature of schooling in the US is to not question, limit exploration of curiosities, and to not challenge with alternatives. Critical thinking is sorely missing and it is the informal, edublogging world that is bringing it back.
Al welcomed me to the fold of better blogging challengers. I am now in that fold and looking for ways to make the Doc Blog better and more effective.
Like or Dislike:
0 
0
Love the new theme!
Like or Dislike:
0 
0
Joel,
Thanks. Part of my 31-day better blogging challenge. I admit that I was inspired by your theme and looked into others from the same developer.
Mike
Like or Dislike:
0 
0
I think I like this design better…more contrast to the colors. I’ve been considering changing themes and establishing a better title.
Like or Dislike:
0 
0
[...] Fill Out This Survey… [...]
Like or Dislike:
0 
0