Please use this identifier to cite or link to this item:

Is Quality Control Pointless?

File Size Format  
0524.pdf 584.23 kB Adobe PDF View/Open

Item Summary

Title:Is Quality Control Pointless?
Authors:Krause, Markus
Afzali, Farhad Mohammad
Caton, Simon
Hall, Margeret
Keywords:Crowd Science
Knowledge Innovation and Entrepreneurial Systems
Crowd labour, Crowdwork, Quality control, human computation, NLP
Date Issued:08 Jan 2019
Abstract:Intrinsic to the transition towards, and necessary for the success of digital platforms as a service (at scale) is the notion of human computation. Going beyond ‘the wisdom of the crowd’, human computation is the engine that powers platforms and services that are now ubiquitous like Duolingo and Wikipedia. In spite of increasing research and population interest, several issues remain open and in debate on large-scale human computation projects. Quality control is first among these discussions. We conducted an experiment with three different tasks of varying complexity and five different methods to distinguish and protect against constantly underperforming contributors. We illustrate that minimal quality control is enough to repel constantly underperforming contributors and that this is constant across tasks of varying complexity.
Pages/Duration:10 pages
Rights:Attribution-NonCommercial-NoDerivatives 4.0 International
Appears in Collections: Crowd Science

Please email if you need this content in ADA-compliant format.

This item is licensed under a Creative Commons License Creative Commons