Current methods such as retrieval-augmented generation and human-in-the-loop validation are reaching their limits in terms of scalability, resource efficiency and error minimization.

The investigations in this work were conducted in the context of a retrieval-augmented generation system. In particular, the errors of hallucinations, excessive summarization, and document retrieval errors were detected. To overcome these challenges, this study introduces a two-phase agentic system comprising different roles, including a reviewer and a reviser, which jointly improve LLM-generated content. Implemented within the RepoChat system, utilizing extensive documentation from Nagra on radioactive waste management, this study demonstrates the practical application of an agent-based verification process.

Empirical evaluations confirm measurable improvements in semantic completeness and precision through the reduction of unsubstantiated claims, even if the improvements were marginal. In addition, a retrieval evaluator agent was implemented that was able to successfully distinguish relevant from less relevant document parts. However, variability in results highlighted the complexity of fully automating semantic assessments. These outcomes form a basis for future developments aimed at strengthening trust in AI-generated results and increasing their reliability.