This summary presents a master’s thesis that delves the performance evaluation of ChatGPT as a chatbot/digital employee within the financial department of the Motorfahrzeugkontrolle Basel-Landschaft (MFK BL).

The study employs a range of research techniques, including mixed-method research and a controlled experiment, to establish a robust framework for investigating the study’s objectives. By evaluating performance across diverse complexity levels, the thesis offers a clear perspective on Artificial Intelligence (AI)’s potential to enhance or surpass human expertise in handling financial inquiries. The key findings of the study reveal that ChatGPT demonstrated an adequate accuracy rate in handling financial inquiries of the MFK BL. While it performed well with easy inquiries, its accuracy decreased with ones that are more complex. The responses were generally well-structured and user-friendly, with a majority rated as good or very good. Moreover, ChatGPT needed minimal follow-up interactions. ChatGPT passed the complexity handling metric in many cases but failed in others, with the failures more noticeable in medium and hard inquiries. The necessity of combining AI tools like ChatGPT for routine tasks with human expertise for more complex issues is evident.

This approach could optimize operational efficiency and maintain high standards in customer service, highlighting the irreplaceable value of human input in complex financial inquiries. The study proposes several recommendations for the future, like investing in further training of ChatGPT with more specialized financial data to improve its performance in handling complex inquiries.