Not Enough Data to Be Fair? Evaluating Fairness Implications of Data Scarcity Solutions

Abstract

This study explores the implications of the use of data scarcity solutions on fairness in machine learning, specifically in consumer credit interest rate prediction. We develop a comprehensive taxonomy of Data Scarcity Solutions (DSS) by analyzing academic literature, data science competitions, and practical implementations. We identify six distinct DSS clusters: Data Extension, Pre-Training, Public Data Inclusion, Data Sharing, Federated Learning, and Active Learning. Our evaluation shows that most DSS enhance both performance and fairness, with minimal negative correlation between the two. Notably, approaches incorporating external or synthetic data significantly improve fairness. This research contributes to understanding DSS beyond algorithmic performance, providing a framework for evaluating their societal impact. Furthermore, it offers practitioners a taxonomy to select the right method for tackling data scarcity and addresses fairness concerns in real-world scenarios.