Beyond Efficiency: Rethinking Literacy in the Age of AI

When did efficiency become the measure of successful communication? If genAI can summarize what we read and improve what we write, are we necessarily becoming better readers and writers? Or are we beginning to mistake the products of intellectual activity for the intellectual activity itself? For the last few days, I’ve had the pleasure of being engaged in these questions with the 2nd International Applied Linguistics Research Conference (AppLing 2026). This international conference, affiliated with the Australian Journal of Applied Linguistics, brought together researchers to reconsider methodological approaches in applied linguistics under the theme “Rethinking Research Methods in Applied Linguistics.”

Rationale for the PBDE Methodology

My interest in analyzing generative AI output grows out of two areas of my academic background: cultural studies and science and technology studies (STS, which considers the history and social study of technology). Cultural studies taught me to look beyond the apparent meaning of a text and consider how language reproduces ideologies. STS encouraged me to question the idea that technologies are neutral instruments or that technological development follows a single, inevitable path. Both perspectives seemed particularly relevant when AI chatbots started to capture the public imagination – and the imagination of my students and myself.

Bringing these interests into applied linguistics raised a set of questions. How could I distinguish a recurring discourse pattern from my interpretation of an isolated response? How could I construct a body of evidence that would allow other researchers to examine my conclusions?

These questions led me to investigate how scholars in applied linguistics and other fields analyze the discourse features of genAI output. Building on established approaches to pragmatic elicitation, algorithmic auditing, and discourse analysis, I started to call this kind of work prompt-based discourse elicitation (PBDE). My aim was to make the generation of texts and the reasoning behind my interpretation more transparent.

Because I had conducted a few PBDE studies previously, I enjoyed developing the framework for a short paper in the conference proceedings, titled “Revisiting Discourse Analysis for AI Literacy and Ethics: An Emerging Framework for Evaluating Generative AI Output.” Then, preparing the conference presentation gave me an opportunity to take the argument further. I made more explicit connections to established approaches in applied linguistics, including pragmatic elicitation and critical discourse analysis, and drew on examples from several of my recent studies to illustrate how the framework works in practice.

PBDE allows researchers to explore historically and culturally informed questions through systematically elicited chatbot output.

Impressions of the Conference

I found the conference both exciting and a little intimidating. Among the speakers and participants were editors of leading journals, including TESOL Quarterly and System. It was an unusually valuable opportunity to present a developing methodological framework to researchers who help shape the field of applied linguistics.

One particularly relevant keynote was Jim McKinley’s “What is a ‘Methodological Contribution’? Rethinking Innovation, Rigour and Publishability in Applied Linguistics.” His discussion of methodological rigor and transparency resonated with my own efforts to make qualitative analysis of genAI output more systematic and open to scrutiny. In particular, I appreciated his attention to accountability, or ensuring that the methods, findings, and conclusions are in alignment. A research methodology should provide an account for the origin of the data, how coding and evaluation proceeded, and even limitations to the findings. For instance, how might a different methodology change the questions we could ask?

I was also intrigued by Peter I. De Costa’s keynote, “Introducing Autoethnography in Applied Linguistics: Prospects and Possibilities.” Prof. De Costa came to my university a few years ago to talk about critical multilingual awareness; both then as well as now, he was very generous in sharing the lessons he has learned as an educator and editor. As De Costa pointed out, autoethnography is not a new methodology, and studies like participant-observer research have been around for 50 years. However, autoethnography has become increasingly used in applied linguistics. Although this is not directly relevant to my present studies, I can think of a few students who will benefit – and it is something I might try in the future. De Costa pointed out that autoethnography by applied linguistics professionals in the global south could provide important insights to the field.

Mirjam Hauck’s keynote, “Positionality in the Age of AI: Towards a Relational Ethics of Applied Linguistics Research,” resonated with my interests. Drawing on research involving virtual exchange and multilingual contexts, she questioned the neutrality of AI tools and emphasized researchers’ responsibility for their interpretive decisions. These concerns connect with my interests in science and technology studies and the importance of keeping human interpretation and accountability central to research. Her discussion of ubuntu and ren, two traditions emphasizing the relational nature of human existence, offered a way of thinking about research ethics beyond procedural compliance. I was particularly struck by her questions about whose voices AI tools amplify but also cui bono? As she pointed out, movements toward “slow AI” and localized / small AI offer different relationships that may have different sponsors than large commercial models. The Slow AI Project’s manifesto emphasizes uncertainty, deliberation, and the importance of preserving human judgment rather than delegating consequential decisions to automated systems. This resonated with my background in science and technology studies, where technological progress is not necessarily understood as inevitable or universally beneficial. This also resonated with my growing conviction that clarity and immediacy are not necessarily the highest goals of communication. Sometimes complexity, ambiguity, and making an effort to listen are more important.

The speakers in parallel sessions were also inspiring. The two papers on my panel were intriguing. Marco Antonio Espinoza wrote about his study of a speaker of Pewenche, an indigenous language in Chile, and Simon Arsa Manggala and Wisnu Pradana took an applied linguistics perspective to Indonesian folklore. On an earlier panel, Kyria Finardi talked about generative AI in the global south, considering developments in Brazil, which made an effective connection to Prof. Hauck’s talk. To me, papers like these represent some of the most interesting work in applied linguistics because they point out the use of language in multilingual contexts. A lot of work in English language teaching presumes a bilingual model, but it is the case here in southern Thailand and many other places around the world that people speak three, four, or more languages.

What Happens When AI Does the Reading?

The closing panel ranged across many topics, but I was struck by how frequently genAI entered the discussion. Mirjam Hauck’s emphasis on relationships and positionality, and Peter De Costa’s discussion of autoethnography, offered important alternatives to the depersonalized production of knowledge that AI can encourage.

Glenn Stockwell, one of the conference chairs, raised a concern that particularly resonated with me: the growing tendency to delegate reading to AI. I have been thinking about how we teach reading and writing, particularly in an educational environment where summarizing and extracting information are often treated as the principal goals of reading—tasks that can now be easily automated. To me, reading is not a simple matter of obtaining information. It is also a process of making connections and the unexpected insights that pop into your head when thinking about someone else’s ideas.

Walter J. Ong (Orality and Literacy, 1982) argued that writing, as a technology, creates a productive form of alienation: it separates the knower from what is known, allowing us to read our own thoughts almost as though they belonged to someone else. Writing and reading are therefore more than ways of recording or communicating ideas. Reading our own writing can change how we think, enabling distinctive forms of reflection and self-examination.

As Stockwell observed, writing is often taught in support of other learning objectives, such as vocabulary acquisition, rather than being valued as an intellectual practice in its own right. This is something I will consider further. These exchanges left me thinking that AI literacy must involve more than learning how to use AI tools effectively or recognizing their limitations. We also need to consider what we lose when we delegate reading, writing, and interpretation to them. This recalls my earlier research on Vannevar Bush’s memex, a proposed technology that offered possibilities for personal reflection and associative thinking, but whose subsequent technological legacy has often been framed in terms of efficient information processing. Perhaps one of the most important goals of literacy education is to help students cultivate the supposedly inefficient intellectual experiences that reading and writing make possible.

These concerns inform my research on chatbot discourse: if AI increasingly mediates how we encounter knowledge, we need ways to examine not only the accuracy of its outputs but also the assumptions they reproduce.

Overview of My Presentation

Based on a search for articles about discourse and genAI output, it has become clear to me that there are three important components of a PBDE study: a corpus created by standardized prompts, the use of multiple tools, and analytic schemes that allow for readers to retrace the connection between the elicitation and the analysis. In my proceedings paper, I noted that there are several studies that include one or two of these best practices, and noted three that combine all of them.

Based on my recent experiences, I offered an overview of a typical PBDE workflow. Starting with a scenario that meets the research questions, I design prompts and rubrics to produce and assess the corpus. The rubric is used to guide the discourse analysis, which can be refined inductively as in other research of this type. The results then guide quantitative and qualitative interpretation.

My studies have explored various questions: how chatbots advise prospective university students, how they evaluate academic writing, and how they explain scientific developments with complicated colonial histories. What united them was an interest in treating chatbot responses not simply as information to be checked for accuracy, but as discourse that could reveal assumptions about language, knowledge, authority, and social relations.

In order to make the process and benefits clear to the audience, I provided a quick overview of some illustrative studies. The three examples illustrate different ways of using PBDE: comparing responses across geographical settings, examining the assumptions behind automated writing advice, and using simulated-user interactions to explore how chatbot discourse changes through follow-up questions.

  1. Should I Major in English? In this study, I sent prompts to chatbots from six different physical locations. The results were somewhat surprising. For one, chatbots declined to offer a clear choice of choosing an English or chemistry major – but they framed English primarily as an employability skill and chemistry as a substantive field of study, a distinction I interpreted in relation to neoliberal discourses of higher education. Another finding was that the universality of the tools was challenged. In particular, the tools failed to offer appropriate advice to an LGBTQ+ person in Brunei. This offers insights into what topics are important when providing students with AI literacy.
  2. Steamrolling Scholarly Expression: This study originated in the observation of several researchers that genAI tools seem to be making some words more popular, while at the same time, stance and argumentation signals are becoming less common. I asked chatbots for two kinds of feedback regarding academic English and peer review. Comments about word choice and argumentation predominated, while other concerns, such as personal style and theory and scholarly contribution, seemed relatively scarce. This offers some insights into what kinds of help writers need.
  3. Discourse and Innovation: This study is important to me for two reasons. For one, it addresses ideological concerns that are important to me, such as the need to preserve local knowledge in the face of globalization. Secondly, it uses a type of audit known as a “sock puppet,” where researchers impersonate a user. The findings are worth considering further. The chatbots’ initial responses about innovation tended to present “great man” stories with milestones from the western world, but further probing revealed insight from postcolonial scholarship. This was a bit surprising; the fact that postcolonial perspectives emerged through follow-up questions suggests that the issue was not simply an inability to discuss them, but also how chatbot responses initially framed the subject. Again, this reveals a direction that would be needed for AI literacy.

These studies, along with the others I have conducted, have been interesting explorations of the epistemology of genAI. GenAI literacy should involve issues of ethics and fact-checking, but there is clearly room for much more. Although chatbots seem ready to take on any topic and their output seems knowledgeable, they seem to replicate dominant discourses. This means that pedagogies about mitigating bias and giving voice to underrepresented voices have gained more importance than ever before. I am also pleased by how PBDE is showing itself to be a methodology that allows for rigorous, systematic analysis.

My favorite question following my presentation related to Norman Fairclough’s three dimensions of critical discourse analysis. An audience member asked how the framework connects textual observations with the broader social practices and relations of power. That question helped me clarify an important distinction: I design research scenarios around theoretical interests, including questions about ideology and power, but leave the analysis open to unexpected findings. The scenarios are theoretically informed; the interpretations are not predetermined.

The conference left me with some new things to think about. How to make the movement from textual observations to broader social explanation more transparent is one of the issues I hope to explore further. Also, how can educators ensure that the growing efficiency of AI-mediated communication does not diminish the intellectual experiences through which we develop knowledge? I hope to pursue the first in a fuller methodological article on PBDE. The second seems likely to occupy me for much longer.

Leave a Reply

Your email address will not be published. Required fields are marked *