The Trending Customer Needs (TCN) Dataset: A Benchmarking and Automated Evaluation Approach for New Product Development

Abstract

In recent years, there have been many studies which summarize User Generated Content as lists of ranked keyphrases representing customer needs for the purposes of New Product Development. However, methods for the evaluation of keyphrase lists do not robustly assess solutions for these purposes. Therefore, in this paper we present the “Trending Customer Needs” (TCN) dataset of over 9000 top trending customer need keyphrases organized by month from 2007-2021 which spans 37 product categories in the area of Consumer Packaged Goods (e.g. toothpaste, eyeliner, beer etc.). TCN is a curated dataset for the benchmarking of supervised machine learning approaches in the prediction of customer needs using User Generated Content. We describe the process of curating TCN while ensuring its quality. Finally, we demonstrate its utility via a case study of Reddit discourse as a potential predictor for future customer needs in Consumer Packaged Goods.