Amazon Mechanical Turk Undergoes Through A Lot Of Massive Panic

Max Hui Bai, a student at the University of Minnesota recently posted a question on Facebook which threw Amazon MTurk into a panic. The question was simple: “Have anyone used Mturk in the last few weeks and notice any quality drop?” A great many people use the Mechanical Turk of Amazon widely for research purposes. They carry online surveys and questionnaires there which many people fill for low income. The researches conducted there have been important for many publishers and researchers.

The Situation Of Low Quality Of Amazon Mechanical Turk

Bai further elaborated the situation in his blog in which he told how exactly the quality went low. He said that he noticed that most of the open-ended questions got a non-serious response which hindered his research. He had to disregard more than half of the data he had collected as it was not reliable. This cost him a lot of time and effort which he could use in other research work. He also noticed that some respondents had duplicate GPS locations. This further rose doubts regarding the reliability of his research.

In response to his Facebook post, many other people said that they had noticed a poorer quality too. They complained about similar issues and showed concern regarding the integrity of their researches too. The suspected culprits are bots – the software which copies human behavior. It is not the actual human beings threatening the research, which raises questions regarding Mturk’s security and privacy.

Psychologists And Publications Are Worried

The discussion soon escalated and now psychologists all over the world are worried due to the problem. New Scientists published an article in this regard “Bots on Amazon’s MTurk Are Ruining Psychology Studies.” This indicates how big of a problem this is. The psychologists are primarily sweating over this because psychological researches revolve around human behavior. The researches affected by bots are not acceptable at all so they are looking for sites now.

Panos Ipeirotis from New York says, “It might be bots, it might be human-augmented bots, or it might be humans who are tired of taking the survey and are just randomly clicking the buttons.” So the lower quality could be a result of many factors which we cannot be sure of at the moment. There have been rumors and complaints regarding the privacy of users on Mturk in the past too. Social scientists are hoping that Amazon authorities look into the matter as most of their researches depend on it.