Generative AI or Real Users? Investigating the Relative Impact of Generative AI vs. Humans on Online Review Quality

Abstract

Online reviews matter for customers, firms, and platforms increasingly. The recent advancement of generative Artificial Intelligence (AI) techniques makes it possible to generate online reviews automatically. However, the relative impact of generative AI vs. humans on online review generation is unknown. On the one hand, generative AI can generate high quality reviews because they are trained on diverse and high-quality data. On the other hand, generative AI hallucinates and may generate fabricated content, threatening the quality of the generated reviews. Using data from one of the biggest online review platforms, Yelp.com, we apply fixed effect models to understand the relative impact of generative AI vs. humans on the quality of generated reviews. We find that reviews from generative AI averagely have bigger ratings, a higher level of inconsistency between rating and sentiment, shorter, harder to read, and more positive and subjective content. Our study has both theoretical and practical implications.