Evaluation and Optimal Calibration of Purchase Time Recommendation Services

Abstract

Price Comparison Sites enable customers to make better – more informed, less costly – buying decisions through providing price information and offering buying advice in the form of prediction services. While these services differ to some extent, they are comparable regarding their prediction target and usually monitor every arbitrarily small price decrease. We use a large data set of daily minimum prices for 272 smartphones consisting of 198,560 daily price movements from a Price Comparison Site to show that the standard prediction setting is not optimal. A custom evaluation framework allows the maximization of the achievable savings by altering the calibration of the forecasting service to monitor changes that exceed a certain threshold. Additionally, we show that time series features calculated in a calibration period can be used to obtain precise out of sample estimates of the saving optimal forecasting setting.