Applications of Human-AI Collaboration: Insights from Theory and Practice
Permanent URI for this collection
Browse
Recent Submissions
1 - 8 of 8
-
ItemVero: A Method for Remotely Studying Human-AI Collaboration( 2022-01-04)Despite the recognized need in the IS community to prepare for a future of human-AI collaboration, the technical skills necessary to develop and deploy AI systems are considerable, making such research difficult to perform without specialized knowledge. To make human-AI collaboration research more accessible, we developed a novel experimental method that combines a video conferencing platform, controlled content, and Wizard of Oz methods to simulate a group interaction with an AI teammate. Through a case study, we demonstrate the flexibility and ease of deployment of this approach. We also provide evidence that the method creates a highly believable experience of interacting with an AI agent. By detailing this method, we hope that multidisciplinary researchers can replicate it to more easily answer questions that will inform the design and development of future human-AI collaboration technologies.
-
ItemThe Effect of AI Advice on Human Confidence in Decision-Making( 2022-01-04)As artificial intelligence advances, it can increasingly be applied in collaborative decision-making contexts with humans. However, questions on the design of different collaborative environments remain open. In the context of AI-assisted human decision-making processes, we analyze the influence of AI advice on human confidence in the final decision. In a laboratory experiment, 458 subjects performed an image classification task. We compare their confidence over three treatments: i) a baseline case where subjects do not receive any AI advice; ii) where subjects receive AI advice; and iii) in addition to AI advice subjects also see the certainty of AI for its choice. Our results suggest that while AI advice can increase human overconfidence, this effect can be mitigated by augmenting the AI advice with its certainty. Our result not only contributes to the growing literature of human-AI collaboration, but also bears important practical implications for the design of collaborative systems.
-
ItemThe Cognitive Effects of Machine Learning Aid in Domain-Specific and Domain-General Tasks( 2022-01-04)With machine learning (ML) technologies rapidly expanding to new applications and domains, users are collaborating with artificial intelligence-assisted diagnostic tools to a larger and larger extent. But what impact does ML aid have on cognitive performance, especially when the ML output is not always accurate? Here, we examined the cognitive effects of the presence of simulated ML assistance—including both accurate and inaccurate output—on two tasks (a domain-specific nuclear safeguards task and domain-general visual search task). Patterns of performance varied across the two tasks for both the presence of ML aid as well as the category of ML feedback (e.g., false alarm). These results indicate that differences such as domain could influence users’ performance with ML aid, and suggest the need to test the effects of ML output (and associated errors) in the specific context of use, especially when the stimuli of interest are vague or ill-defined.
-
ItemSupporting Online Customer Feedback Management with Automatic Review Response Generation( 2022-01-04)The growing amount of online reviews plays a significant role in a business' image and performance. Businesses in the hospitality industry often lack necessary resources to organize and manage online customer feedback and are therefore likely to search for alternative ways to handle this. AI-based technologies may offer valuable solutions. However, there is currently little research on if and how AI solutions may support the process of responding to online customer feedback in the hospitality industry. This paper presents and evaluates a concept for assisting customer feedback management with automatically generated responses to online reviews. Our solution contributes to ongoing investigations into text generation applications for supporting human authors and also proposes new approaches and potential business models for managing online customer feedback.
-
ItemHuman-Machine Hybrid Decision Making with Applications in Auditing( 2022-01-04)The decision making process in a variety of organizations faces substantial changes, largely as a result of advances in information technology and artificial intelligence (AI). A considerable number of decisions that were traditionally made by humans, are now made by machines. Consequently, many jobs that were held by experts in some fields are now occupied by data scientists who can build AI algorithms. In this paper, we address this change in work environments and suggest an innovative process for hybrid decision making between humans and machines. We focus on the auditing profession, but our method can be used in other human-intensive and critical fields such as healthcare, financial services public sectors, and humanitarian organizations.
-
ItemHuman-AI Collaboration – Coordinating Automation and Augmentation Tasks in a Digital Service Company( 2022-01-04)Organizations are increasingly turning to artificial intelligence (AI) to support service development and delivery. Both AI and human action need to be organized and coordinated. Recently, the automation-augmentation paradox has been discussed in literature. Automation implies that machines take over a human task, whereas with augmentation humans and machines collaborate closely to perform different tasks. In this paper, we investigate how the collaboration between humans and AI unfolds in different organizational coordination mechanisms. Using Mintzberg’s coordination mechanism (1989), we analyzed the division of labor between human and AI in a case company offering personalized recipes of vegetarian dishes. Our findings suggest that certain primary coordination mechanisms (direct supervision and standardization of norms) need to be in place for the AI to perform properly. We find that AI can take control over service scaling and service personalization (augmentation), whereas humans are in control of service improvement (automation).
-
ItemHow Can Organizations Design Purposeful Human-AI Interactions: A Practical Perspective From Existing Use Cases and Interviews( 2022-01-04)Artificial intelligence (AI) currently makes a tangible impact in many industries and humans’ daily lives. With humans interacting with AI agents more regularly, there is a need to examine human-AI interactions to design them purposefully. Thus, we draw on existing AI use cases and perceptions of human-AI interactions from 25 interviews with practitioners to elaborate on these interactions. From this practical lens on existing human-AI interactions, we introduce nine characteristic dimensions to describe human-AI interactions and distinguish five interaction types according to AI agents’ characteristics in the human-AI interaction. Besides, we provide initial design guidelines to stimulate both research and practice in creating purposeful designs for human-AI interactions.
-
Item