Beyond Grammatical Fluency: A Comparative Analysis of Pragmatic Competence in AI-Generated Urdu–English Speech Act Translation across Google Translate, ChatGPT, and Claude

Abstract

Although AI-based translation systems have achieved impressive levels of syntactic fluency, their capacity to preserve pragmatic meaning across languages remains poorly understood, particularly for culturally rich, low-resource source languages such as Urdu. This study addresses that gap through a qualitative comparative analysis of how Google Translate, ChatGPT, and Claude translate three pragmatically sensitive speech act categories — requests, promises, and apologies from Urdu into English.

Authentic Urdu utterances were drawn from the Pakistani television drama Pari Zad and translated through all three systems. Translations were evaluated against the original utterances using an integrated analytical framework combining Austin's (1962) and Searle's (1969) Speech Act Theory, Grice's (1975) Cooperative Principle, and Intercultural Pragmatics, assessing illocutionary force preservation, politeness strategy retention, emotional sincerity, honorific equivalence, and cultural appropriateness.

Findings demonstrate that all three systems produce syntactically fluent English output (92–97% grammatical accuracy), yet diverge sharply at the pragmatic level. Google Translate consistently exhibited the highest rate of pragmatic failure, converting polite emotional requests into authoritative commands, rendering promise illocutionary force implicit, and stripping honorific politeness markers from apologies. ChatGPT demonstrated the most balanced pragmatic reconstruction across all three speech act categories, while Claude produced pragmatically acceptable outputs that occasionally intensified emotional register beyond the source utterance. The study concludes that syntactic fluency is a necessary but insufficient criterion for translation quality, and identifies four structural causes of AI pragmatic failure with direct implications for AI development, translator training, and multilingual communication policy.