Use of Conventional Machine Learning to Optimize Deep Learning Hyper-parameters for NLP Labeling Tasks

Gu, Yang
Leroy, Gondy
Journal Title
Journal ISSN
Volume Title
Deep learning delivers good performance in classification tasks, but is suboptimal with small and unbalanced datasets, which are common in many domains. To address this limitation, we use conventional machine learning, i.e., support vector machines (SVM) to tune deep learning hyper-parameters. We evaluated our approach using mental health electronic health records in which diagnostic criteria needed to extracted. A bidirectional Long Short-Term Memory network (BI-LSTM) could not learn the labels for the seven scarcest classes, but saw an increase in performance after training with optimal weights learned from tuning SVMs. With these customized class weights, the F1 scores for rare classes rose from 0 to values ranging from 18% to 57%. Overall, the BI-LSTM with SVM customized class weights achieved a micro-average of 47.1% for F1 across all classes, an improvement over the regular BI-LSTM’s 45.9%. The main contribution lies in avoiding null performance for rare classes.
Data, Text, and Web Mining for Business Analytics, deep learning, design science, machine learning, natural language processing
Access Rights
Email if you need this content in ADA-compliant format.