Artifact Sampling: Using Multiple Information Technology Artifacts to Increase Research Rigor

Abstract

Researchers in many scientific disciplines routinely conceptualize information technologies (IT) as antecedents or outcomes in theoretical models. The ongoing theorizing of IT leads to a novel methodological challenge termed instantiation validity (IV). In this paper, we contribute to research on remediating IV challenges by proposing and advocating the methodological practice of artifact sampling, whereby multiple artifacts are sampled from the population of all possible artifacts (the instantiation space). Artifact sampling extends the practice of employing multiple research subjects or survey respondents, routinely used in social sciences, into the IT artifact design space. Artifact sampling is an important methodological practice that stands to increase the rigor of research dealing with software artifacts. As it is currently not being adequately undertaken in the aforementioned research, many studies may result in biased or unjustified conclusions.