Information Extraction from Financial Tables - Combining Multimodal Large Language Models and Domain Knowledge

Large Language Models (LLMs) are increasingly used to extract structured information from financial tables in annual reports. However, LLM outputs are prone to systematic errors such as unit and scale mismatches, which reduce the reliability of the extracted data. This thesis investigates whether domain-specific, rule-based validation can improve the accuracy of LLM-based financial table extraction.

Kakkanattu, Kevin, 2026

Type of Thesis Master Thesis
Client
Supervisor Hanne, Thomas
Views: 1 - Downloads: 0
Building on the extraction pipeline developed by Dimmler (2025), a modular validation service was designed and implemented following a Design Science Research methodology. The service follows the Mixture-of-Rule-based Experts (MoRE-LLM) architecture described by Koebler et al. (2024), in which the LLM serves as the primary reasoning engine and six independent rule-based expert groups perform post-hoc checks on the output. A manual error analysis of 288 incorrect baseline answers identified the dominant error patterns and guided the rule design.The service was evaluated on a corpus of 90 companies using Claude Sonnet 4.5, with each experiment executed three times to account for LLM non-determinism. Mean accuracy increased from 41.1% to 58.8% (+17.6 percentage points), with 157 answers improving per run on average and zero regressions across all 2,670 question–answer pairs. All corrections originated from a single rule group, the units expert, through two deterministic rules: scale expansion and percentage format correction. An ablation study confirmed that disabling this group eliminated all corrections, while disabling any other group had no effect.
These findings demonstrate that a simple, rule-based validation layer targeting predictable format errors can produce a substantial and reproducible accuracy improvement in LLM-based financial information extraction.
Studyprogram: Business Information Systems (Master)
Keywords
Confidentiality: öffentlich
Type of Thesis
Master Thesis
Authors
Kakkanattu, Kevin
Supervisor
Hanne, Thomas
Publication Year
2026
Thesis Language
English
Confidentiality
Public
Studyprogram
Business Information Systems (Master)
Location
Olten