Context Changes and the Performance of a Learning Human-in-the-loop System: A Case Study of Automatic Speech Recognition Use in Medical Transcription

Files
0305.pdf (575.42 KB)
Date
2023-01-03
Authors
Mucha, Tomasz
Seppälä, Jane
Puraskivi, Henrik
Contributor
Advisor
Department
Instructor
Depositor
Speaker
Researcher
Consultant
Interviewer
Annotator
Journal Title
Journal ISSN
Volume Title
Publisher
Volume
Number/Issue
Starting Page
3121
Ending Page
Alternative Title
Abstract
The paper presents how organizational practices enable the improvement and maintenance of task performance in a learning human-in-the-loop system exposed to a wide range of context changes. We investigate how the case company tripled the efficiency of medical transcribers by leveraging its machine learning-based automatic speech recognition technology. We find that the focal system operated across stable, drifting, and jumping contexts. Despite changes, it continued to improve or maintained performance thanks to two sets of organizational practices aligning it with the context: extending and refining. This paper makes two key contributions: It shows the importance of considering context changes in the design and operation of learning human-in-the-loop systems. Our empirical findings help with resolving some contradictory outcomes of the recent conceptual work. Secondly, we show that context alignment practices are situated at the sociotechnical system level and, thus, are not just technical solution nor can be detached from social elements.
Description
Keywords
Leveraging IT, AI and Data Science for Healthcare Beyond the Hospital: Learning from Scientific, Operational, and Business Perspectives, artificial intelligence, human-in-the-loop, machine learning, task performance
Citation
URI
https://hdl.handle.net/10125/103014
Extent
10
Format
Geographic Location
Time Period
Related To
Proceedings of the 56th Hawaii International Conference on System Sciences
Table of Contents
Rights
Attribution-NonCommercial-NoDerivatives 4.0 International
Rights Holder
Local Contexts
Collections
Leveraging IT, AI and Data Science for Healthcare Beyond the Hospital: Learning from Scientific, Operational, and Business Perspectives
Email libraryada-l@lists.hawaii.edu if you need this content in ADA-compliant format.