ChatGPT Is Powerful, but Does It Have Power Distance?: A Study of Culturally Imbued Discourse in AI-Generated Essays

ChatGPT Is Powerful, but Does It Have Power Distance?: A Study of Culturally Imbued Discourse in AI-Generated Essays

Andrew Schenck
Copyright: © 2024 |Pages: 17
DOI: 10.4018/IJAET.338219
Article PDF Download
Open access articles are freely available for download

Abstract

Power distance (PD), a cultural value denoting acceptance of asymmetrical power relationships, influences the force of rhetoric used by a writer to address their reader. However, AI technologies such as ChatGPT lack an explicit awareness of PD, which could affect the quality of AI-generated persuasive texts used for language learning. To investigate this issue, 200 persuasive essays written by ChatGPT were compared to 200 essays written by L1-English university learners. Three elements of formulaic language related to PD were examined: stances, modals, and pronoun deixis. Differences in stances (z = -3.411; p = .001) and modals (z = -2.100; p = .036) were both significant according to the Wilcoxon signed ranks formula, whereas differences in pronoun deixis were nearly significant (z = -1.917; p = .055). Overall, language of ChatGPT appears generic and incomplete, suggesting that consistent and uniform expressions are being borrowed from an LLM training corpus to mimic aspects of PD. Limitations of AI highlight a need for pedagogical emphasis of culturally imbued discourse.
Article Preview
Top

Introduction

Social media outlets like Twitter and TikTok have recently been abuzz with posts about AI platforms like ChatGPT (Haensch et al., 2023; Li et al., 2023). Despite concerns expressed by some educators concerning possible violations of academic integrity and adverse consequences to learning outcomes (Li et al., 2023; Yan, 2023), attitudes about AI technology remain largely positive (Haensch et al., 2023; Liu & Ma, 2023). The potential of AI to enhance English instruction has also been acknowledged (Jeon & Lee, 2023; Mohamed, 2023). ChatGPT may serve as an interlocutor, teaching assistant, content provider, or evaluator (Jeon & Lee, 2023). It may also allow students to create texts from diverse perspectives, enhancing inductive forms of learning.

LLMs may have a plethora of applications in writing instruction. Recent research suggests that ChatGPT is a valuable tool for correcting surface-level errors related to grammar or structure (Algaraady & Mahyoob, 2023). The Modern Language Association and the National Council of Teachers of English also point out that teachers may use LLMs in the following ways to support aspects of style and rhetoric:

  • To enhance students’ rhetorical knowledge, critical thinking, and knowledge of conventions.

  • To offer a practical demonstration of some critical rhetorical concepts that have influenced writing and rhetoric studies, especially related to questions of process, praxis, and the construction of meaning.

  • To provide modals of written prose that can be used to highlight differences in genre, tone, diction, literary style, and disciplinary focus. Teachers can use LLMs to offer new processes for students developing multimodal writing genres since LLMs can process multimodal inputs and outputs. (MLA-CCCC Joint Task Force on Writing and AI, 2023, pp. 8-9)

As suggested above, LLMs like ChatGPT have the potential to help students learn more sophisticated techniques for constructing rhetoric, which may lead to responses that vary in genre, grammar, and style. However, little research has been conducted to ensure that LLMs live up to these lofty expectations. More research is needed to understand how teachers may effectively use AI technology to promote better writing (Jeon & Lee, 2023; Mohamed, 2023). Albeit limited, some research suggests that ChatGPT cannot adequately construct or interpret complex rhetoric, which may hamper efforts to integrate technology in English writing classrooms (Algaraady & Mahyoob, 2023; Fan & Jiang, 2023). These preliminary findings are intriguing, yet more research is needed to concretely identify and address the limitations of AI technology, thereby ensuring that the benefits of LLMs like ChatGPT can be fully realized.

Top

Literature Review

While LLMs can potentially transform teaching and assessment of writing, the complexity of human discourse or rhetoric cannot easily be replicated by AI technologies. Using an essay prompt that examined the difference between corpus linguistics and discourse analysis, a professor from the University of Murcia put ChatGPT to the test, finding that AI-generated essays were well structured and showed a cogent argument yet lacked clear appeals to authority (Pérez-Paredes, 2023). Inadequacies of ChatGPT were further exposed by Ian Bogost (2022) in his article entitled “ChatGPT is Dumber than You Think”; he found the enthusiasm for ChatGPT to be “misplaced,” writing that:

ChatGPT cannot truly understand the complexity of human language and conversation. It is simply trained to generate words based on a given input, but it cannot truly comprehend the meaning behind those words. This means that any responses it generates are likely to be shallow and lacking in depth and insight. (para. 2)

Complete Article List

Search this Journal:
Reset
Volume 15: 1 Issue (2024)
Volume 14: 1 Issue (2023)
Volume 13: 4 Issues (2022): 2 Released, 2 Forthcoming
Volume 12: 4 Issues (2021)
Volume 11: 4 Issues (2020)
View Complete Journal Contents Listing