The study begins by examining the primary ways ChatGPT can be manipulated to support social engineering attacks, including techniques such as jailbreak, reverse psychology, and prompt injection. It then proposes and develops countermeasures to detect and prevent these AI-driven threats. A comprehensive literature review identifies existing research gaps and informs the development of a novel risk identification process tailored to mitigate AI-enhanced social engineering risks.

Methodologically, the research follows the five-step design science research (DSR) process, encompassing awareness, suggestion, development, evaluation, and conclusion phases. Through this process, the study develops a practical framework for identifying and mitigating cybersecurity risks associated with ChatGPT. The artifact’s effectiveness is tested through qualitative and quantitative evaluations.

The findings highlight the necessity of integrating human judgment with technological solutions to effectively counteract AI-driven social engineering threats. The research provides actionable recommendations for enhancing cybersecurity strategies and calls for a proactive, interdisciplinary approach to address advanced AI technologies' ethical and security challenges. This thesis aims to equip cybersecurity professionals with the tools and knowledge to anticipate and mitigate the risks associated with the misuse of ChatGPT in social engineering attacks.