Leveraging Financial Data with Big Data Tools or Generative AI

Permanent URI for this collectionhttps://hdl.handle.net/10125/112435

Browse

Recent Submissions

Now showing 1 - 3 of 3
  • Thumbnail Image
    Item type: Item ,
    Beyond Zero-Shot: Enhancing LLM Financial Complaint Classification with Relevancy-Driven RAG-Based Few-Shot Prompting
    (2026-01-06) Pradhan, Manaranjan; Vemprala, Naga; Gudigantala, Naveen
    Large language models (LLMs) have shown significant promise in natural language processing (NLP) tasks, yet their efficacy in real-world consumer complaint classification without fine-tuning remains a challenge. Zero-shot classification offers a valuable solution for categorizing consumer complaints, particularly for handling new and dynamic financial issues, as it allows models to classify data without prior labeled training. However, the nuanced and often overlapping nature of financial complaint categories makes this task particularly difficult. This study explores both zero-shot and a novel few-shot prompting approach for classifying consumer complaints submitted to the Consumer Financial Protection Bureau (CFPB). We compared traditional zero-shot prompting with two few-shot methods: one using randomly selected classified examples and another leveraging the top 5 most relevant classified examples with semantic similarity for in-context learning. Our results consistently demonstrated superior performance with our relevancy-driven, retrieval-augmented generation (RAG) prompting. To validate these findings and ensure they weren't due to chance, we replicated our experiments across several leading LLM models, including GPT-4o, QWEN, Deepseek V3, and Anthropic Claude Sonnet 4.0. Across all tested models, the relevancy-based few-shot approach yielded consistently better results, which we rigorously validated using accuracy, precision, recall, and F1-score. Furthermore, when benchmarked against traditional machine learning models including a fine-tuned RoBERTa, SVM, and logistic regression, our relevancy-driven few-shot approach demonstrated markedly superior performance, validating its effectiveness for this complex classification task. This research highlights the significant potential of carefully curated, relevant examples in enhancing LLM performance for complex text classification tasks in the financial domain.
  • Thumbnail Image
    Item type: Item ,
    FA-RAG: Financial Advice System Using RAG Based on LLMs
    (2026-01-06) Miwa, Julian; Li, Da; Kumamoto, Tadahiko; Kawai, Yukiko
    This study presents FA-RAG, a Retrieval Augmented Generation (RAG) framework leveraging large language models (LLMs) to improve the quality of automated financial advice. The system is constructed from 940 expert-authored financial consultation cases, which include household income and expenditure data, demographic attributes of the consulter, and professional advice from financial planners. Both textual and numerical data are transformed into vector representations and stored in a Pinecone vector database, enabling efficient retrieval of relevant cases. When new consultations are processed, the system queries this database to supply the LLM with contextually relevant information, thereby producing more accurate and personalized advice. For evaluation, FA-RAG was applied to 100 consultation cases, with multiple insurance premium levels examined to analyze how recommended reduction rates vary under different financial conditions. The results demonstrate the potential of integrating domain-specific data and retrieval mechanisms with LLMs to support reliable and tailored financial planning assistance.
  • Thumbnail Image
    Item type: Item ,