Black Box or Open Science? Assessing Reproducibility-Related Documentation in AI Research
Files
Date
2024-01-03
Contributor
Advisor
Department
Instructor
Depositor
Speaker
Researcher
Consultant
Interviewer
Narrator
Transcriber
Annotator
Journal Title
Journal ISSN
Volume Title
Publisher
Volume
Number/Issue
Starting Page
682
Ending Page
Alternative Title
Abstract
The surge in Artificial Intelligence (AI) research has spurred significant breakthroughs across various fields. However, AI is known for its Black Box character and reproducing AI outcomes challenging. Open Science, emphasizing transparency, reproducibility, and accessibility, is crucial in this context, ensuring research validity and facilitating practical AI adoption. We propose a framework to assess the quality of AI documentation and assess 51 papers. We conclude that despite guidelines, many AI papers fall short on reproducibility due to insufficient documentation. It is crucial to provide comprehensive details on training data, source code, and AI models, and for reviewers and editors to strictly enforce reproducibility guidelines. A dearth of detailed methods or inaccessible source code and models can raise questions about the authenticity of certain AI innovations, potentially impeding their scientific value and their adoption. Although our sample size inhibits broad generalization, it nonetheless offers key insights on enhancing AI research reproducibility.
Description
Keywords
Open Science Practices in Information Systems Research, ai documentation, open science, reproducibility
Citation
Extent
10 pages
Format
Geographic Location
Time Period
Related To
Proceedings of the 57th Hawaii International Conference on System Sciences
Related To (URI)
Table of Contents
Rights
Attribution-NonCommercial-NoDerivatives 4.0 International
Rights Holder
Local Contexts
Email libraryada-l@lists.hawaii.edu if you need this content in ADA-compliant format.