Black Box or Open Science? Assessing Reproducibility-Related Documentation in AI Research

Simple item page
dc.contributor.authorKoenigstorfer, Florian
dc.contributor.authorHaberl, Armin
dc.contributor.authorKowald, Dominik
dc.contributor.authorRoss-Hellauer, Tony
dc.contributor.authorThalmann, Stefan
dc.date.accessioned2023-12-26T18:36:21Z
dc.date.available2023-12-26T18:36:21Z
dc.date.issued2024-01-03
dc.identifier.doihttps://doi.org/10.24251/HICSS.2024.081
dc.identifier.isbn978-0-9981331-7-1
dc.identifier.otherbc5847b2-2911-455f-8b1e-d119f0f2d791
dc.identifier.urihttps://hdl.handle.net/10125/106458
dc.language.isoeng
dc.relation.ispartofProceedings of the 57th Hawaii International Conference on System Sciences
dc.rightsAttribution-NonCommercial-NoDerivatives 4.0 International
dc.rights.urihttps://creativecommons.org/licenses/by-nc-nd/4.0/
dc.subjectOpen Science Practices in Information Systems Research
dc.subjectai documentation
dc.subjectopen science
dc.subjectreproducibility
dc.titleBlack Box or Open Science? Assessing Reproducibility-Related Documentation in AI Research
dc.typeConference Paper
dc.type.dcmiText
dcterms.abstractThe surge in Artificial Intelligence (AI) research has spurred significant breakthroughs across various fields. However, AI is known for its Black Box character and reproducing AI outcomes challenging. Open Science, emphasizing transparency, reproducibility, and accessibility, is crucial in this context, ensuring research validity and facilitating practical AI adoption. We propose a framework to assess the quality of AI documentation and assess 51 papers. We conclude that despite guidelines, many AI papers fall short on reproducibility due to insufficient documentation. It is crucial to provide comprehensive details on training data, source code, and AI models, and for reviewers and editors to strictly enforce reproducibility guidelines. A dearth of detailed methods or inaccessible source code and models can raise questions about the authenticity of certain AI innovations, potentially impeding their scientific value and their adoption. Although our sample size inhibits broad generalization, it nonetheless offers key insights on enhancing AI research reproducibility.
dcterms.extent10 pages
prism.startingpage682

Files

Original bundle

Now showing 1 - 1 of 1
Thumbnail Image
Name:
0067.pdf
Size:
392.72 KB
Format:
Adobe Portable Document Format
Download

Collections

Open Science Practices in Information Systems Research