Applications of Human-AI Collaboration: Insights from Theory and Practice

Permanent URI for this collection

Browse

Recent Submissions

Now showing 1 - 5 of 8
  • Item
    Vero: A Method for Remotely Studying Human-AI Collaboration
    ( 2022-01-04) Hohenstein, Jess ; Larson, Lindsay E. ; Hou, Yoyo Tsung-Yu ; Harris, Alexa M. ; Schecter, Aaron ; Dechurch, Leslie ; Contractor, Noshir ; Jung, Malte F.
    Despite the recognized need in the IS community to prepare for a future of human-AI collaboration, the technical skills necessary to develop and deploy AI systems are considerable, making such research difficult to perform without specialized knowledge. To make human-AI collaboration research more accessible, we developed a novel experimental method that combines a video conferencing platform, controlled content, and Wizard of Oz methods to simulate a group interaction with an AI teammate. Through a case study, we demonstrate the flexibility and ease of deployment of this approach. We also provide evidence that the method creates a highly believable experience of interacting with an AI agent. By detailing this method, we hope that multidisciplinary researchers can replicate it to more easily answer questions that will inform the design and development of future human-AI collaboration technologies.
  • Item
    The Effect of AI Advice on Human Confidence in Decision-Making
    ( 2022-01-04) Taudien, Anna ; Fügener, Andreas ; Gupta, Alok ; Ketter, Wolfgang
    As artificial intelligence advances, it can increasingly be applied in collaborative decision-making contexts with humans. However, questions on the design of different collaborative environments remain open. In the context of AI-assisted human decision-making processes, we analyze the influence of AI advice on human confidence in the final decision. In a laboratory experiment, 458 subjects performed an image classification task. We compare their confidence over three treatments: i) a baseline case where subjects do not receive any AI advice; ii) where subjects receive AI advice; and iii) in addition to AI advice subjects also see the certainty of AI for its choice. Our results suggest that while AI advice can increase human overconfidence, this effect can be mitigated by augmenting the AI advice with its certainty. Our result not only contributes to the growing literature of human-AI collaboration, but also bears important practical implications for the design of collaborative systems.
  • Item
    The Cognitive Effects of Machine Learning Aid in Domain-Specific and Domain-General Tasks
    ( 2022-01-04) Divis, Kristin ; Howell, Breannan ; Matzen, Laura ; Stites, Mallory ; Gastelum, Zoe
    With machine learning (ML) technologies rapidly expanding to new applications and domains, users are collaborating with artificial intelligence-assisted diagnostic tools to a larger and larger extent. But what impact does ML aid have on cognitive performance, especially when the ML output is not always accurate? Here, we examined the cognitive effects of the presence of simulated ML assistance—including both accurate and inaccurate output—on two tasks (a domain-specific nuclear safeguards task and domain-general visual search task). Patterns of performance varied across the two tasks for both the presence of ML aid as well as the category of ML feedback (e.g., false alarm). These results indicate that differences such as domain could influence users’ performance with ML aid, and suggest the need to test the effects of ML output (and associated errors) in the specific context of use, especially when the stimuli of interest are vague or ill-defined.
  • Item
    Supporting Online Customer Feedback Management with Automatic Review Response Generation
    ( 2022-01-04) Katsiuba, Dzmitry ; Kew, Tannon ; Dolata, Mateusz ; Schwabe, Gerhard
    The growing amount of online reviews plays a significant role in a business' image and performance. Businesses in the hospitality industry often lack necessary resources to organize and manage online customer feedback and are therefore likely to search for alternative ways to handle this. AI-based technologies may offer valuable solutions. However, there is currently little research on if and how AI solutions may support the process of responding to online customer feedback in the hospitality industry. This paper presents and evaluates a concept for assisting customer feedback management with automatically generated responses to online reviews. Our solution contributes to ongoing investigations into text generation applications for supporting human authors and also proposes new approaches and potential business models for managing online customer feedback.
  • Item
    Human-Machine Hybrid Decision Making with Applications in Auditing
    ( 2022-01-04) Hooshangi, Sara ; Sibdari, Soheil
    The decision making process in a variety of organizations faces substantial changes, largely as a result of advances in information technology and artificial intelligence (AI). A considerable number of decisions that were traditionally made by humans, are now made by machines. Consequently, many jobs that were held by experts in some fields are now occupied by data scientists who can build AI algorithms. In this paper, we address this change in work environments and suggest an innovative process for hybrid decision making between humans and machines. We focus on the auditing profession, but our method can be used in other human-intensive and critical fields such as healthcare, financial services public sectors, and humanitarian organizations.