Could you please pay attention?’ Comparing in-person and MTurk Responses on a Computer Code Review Task

Simple item page
dc.contributor.authorGibson, Anthony
dc.contributor.authorAlarcon, Gene
dc.contributor.authorLee, Michael
dc.contributor.authorHamdan, Izz Aldin
dc.date.accessioned2020-12-24T19:51:42Z
dc.date.available2020-12-24T19:51:42Z
dc.date.issued2021-01-05
dc.description.abstractThe current study examined the differences in data quality across two environments (i.e., in a laboratory and online via Amazon’s Mechanical Turk) on a computer code review task. Researchers and practitioners often collect data online for the sake of convenience, as well as for obtaining a more generalizable sample of participants. The lack of social contact between the researchers and participants, however, may result in less effort dedicated to the experimental task resulting in poor quality data. The results of the current study showed that data quality—at least when measuring the individual difference variables—was drastically worsened when the experimental task was presented online. In contrast, we observed little differences in the experimental task perceptions across the two samples. Rather, participants spent significantly less time examining the computer code when completing the experiment online. The current study has implications for the effects of using online platforms (like MTurk) to collect experimental data.
dc.format.extent10 pages
dc.identifier.doi10.24251/HICSS.2021.504
dc.identifier.isbn978-0-9981331-4-0
dc.identifier.urihttp://hdl.handle.net/10125/71121
dc.language.isoEnglish
dc.relation.ispartofProceedings of the 54th Hawaii International Conference on System Sciences
dc.rightsAttribution-NonCommercial-NoDerivatives 4.0 International
dc.rights.urihttps://creativecommons.org/licenses/by-nc-nd/4.0/
dc.subjectCrowd-based Platforms
dc.subjectcareless responding
dc.subjectcode review
dc.subjectmturk
dc.titleCould you please pay attention?’ Comparing in-person and MTurk Responses on a Computer Code Review Task
prism.startingpage4148

Files

Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
0408.pdf
Size:
451.63 KB
Format:
Adobe Portable Document Format
Download

Collections

Crowd-based Platforms