Evaluation of LLM-based textual AI services for Digital Trust
This Master's thesis addresses how Digital Trust in LLM-based textual generative AI services can be systematically and comparably assessed.
Hilber, Sebastian, 2026
Type of Thesis Master Thesis
Client
Supervisor Härer, Felix
Views: 1 - Downloads: 0
Since the release of ChatGPT in late 2022, adoption has accelerated while concerns persist regarding security and resilience, data protection, and harmful content outputs. Existing red-teaming and safety testing tools provide partial coverage but do not support an integrated, comparable assessment across services.
Using a Design Science Research approach and scoped to the technically measurable dimensions of cybersecurity and resilience, data protection, and content safety, this thesis developed (1) a Digital Trust framework specifying dimensions, criteria, evidence types, and scoring rules, and (2) a proof-of-concept benchmark that operationalises the framework by synthesising existing tools. The work is informed by a systematic literature review across DBLP and Scopus; database searches returned 4,068 records; following title and abstract screening, a final corpus of 16 studies informed the framework design.
The benchmark was applied to three LLM-based textual AI services, producing differentiated comparative Digital Trust scores and demonstrating the feasibility of integrated, practitioner-oriented trust assessment.
Studyprogram: Business Information Systems (Master)
Keywords
Confidentiality: öffentlich