The Impact of Chatbot Modalities on Critical Thinking, Digital Trust, and User Attitude in Higher Education

This study investigates how chatbot interaction modalities—text, voice, or avatar—affect university students’ critical thinking (CT), digital trust (DT), and user attitudes (UA) within an educational context. Building on a Design Science Research approach, an experimental chatbot artifact was developed, incorporating a hybrid dialogue system (Dialogflow CX and GPT-4) within a content-neutral, debate-oriented scenario to ensure rigorous experimental control and pedagogical relevance. A cross-sectional experiment was conducted with 90 university students, who were randomly assigned to one of the three chatbot modalities.

Doganci, Faruk, 2025

Type of Thesis Master Thesis
Client
Supervisor Schneider, Bettina, Jäger, Janine
Views: 1 - Downloads: 0
Quantitative analysis using one-way ANOVAs revealed no statistically significant differences in students’ CT or DT across the modalities, despite the debate-focused, Socraticstyle dialogue designed to promote critical engagement. This suggests that the robust underlying dialogue structure and content may have been the dominant factors influencing CT, overshadowing the interface modality. Similarly, the consistency in DT across modalities implies that trust may be more dependent on perceived technical reliability and the transparency of data handling rather than superficial social cues, reinforcing the concept of "appropriate trust" in AI interactions. In contrast, a statistically significant and practically meaningful difference was found for UA, with text-based interactions leading to more positive attitudes compared to both voice-based and avatar-based modalities. This outcome diverges from some literature suggesting a warmer reception for richer modalities, indicating that for academic debate tasks, the perceived efficiency and psychological safety of text may outweigh the social presence benefits of voice and avatar, which can be affected by technical limitations (e.g., speech recognition issues, "uncanny valley" effect) and novelty effects.
These findings contribute to theoretical understandings, particularly regarding Media Richness Theory in Chatbot Design by demonstrating that the impact of interface richness on higher-order cognitive processes and trust is not always direct or positive, particularly when pedagogical design and practical usability factors are prominent. Practically, the study recommends prioritizing dialogue logic and content over modality choice for CT and DT, while acknowledging text-based interfaces as potentially optimal for UA in structured academic settings.
Studyprogram: Business Information Systems (Master)
Keywords Critical Thinking, Digital Trust, User Attitude, Chatbot Modalities, Chatbot Interaction, AI in Education, Avatar in Debate, Human-AI Interaction, Educational Technology, Media Richness Theory, Dialogue Design, Hybrid Chatbot
Confidentiality: öffentlich
Type of Thesis
Master Thesis
Authors
Doganci, Faruk
Supervisor
Schneider, Bettina, Jäger, Janine
Publication Year
2025
Thesis Language
English
Confidentiality
Public
Studyprogram
Business Information Systems (Master)
Location
Olten
Keywords
Critical Thinking, Digital Trust, User Attitude, Chatbot Modalities, Chatbot Interaction, AI in Education, Avatar in Debate, Human-AI Interaction, Educational Technology, Media Richness Theory, Dialogue Design, Hybrid Chatbot