The unprecedented popularity of ChatGPT signifies a paradigm shift in how individuals interact with technology for educational purposes. This paper investigates the nascent skill of prompt engineering among English as a Foreign Language (EFL) secondary school students. While Large Language Models (LLMs) like ChatGPT offer immense potential to support writing development, their efficacy is contingent upon the user's ability to craft precise, effective instructions. This study captures the real-time, trial-and-error processes of novice users, analyzing the content, quality, and evolution of their prompts to complete a defined writing task. The findings reveal distinct behavioral pathways, underscoring the urgent need for structured prompt engineering education within EFL curricula to move students from inefficient experimentation to strategic collaboration with AI.
State-of-the-Art (SOTA) generative AI chatbots, epitomized by ChatGPT, represent a quantum leap from rule-based predecessors. Powered by neural network language models trained on vast corpora, they generate human-like text based on probabilistic predictions, enabling more flexible and context-aware interactions (Caldarini et al., 2022). "ChatGPT" is increasingly used as a generic term for this class of AI, setting a new performance standard.
Prompt engineering is the art and science of designing inputs to guide an LLM toward a desired output. It is not merely a technical skill but a form of computational thinking and meta-linguistic awareness. Effective prompts often require clarity, context, constraints, and examples (few-shot learning). For non-technical users, this presents a significant learning curve, often characterized by iterative guessing.
Research on AI in language learning has focused on automated writing evaluation (AWE) and intelligent tutoring systems. The interactive, generative nature of SOTA chatbots introduces a new dynamic—shifting the learner's role from recipient of feedback to director of a cognitive tool. This necessitates new literacies, blending traditional writing skills with AI interaction strategies.
The study involved secondary school EFL students in Hong Kong with no prior experience using SOTA chatbots. Participants were tasked with completing a specific writing assignment (e.g., an argumentative essay or descriptive paragraph) using ChatGPT. Primary data consisted of iPad screen recordings, capturing the complete sequence of prompts, ChatGPT's responses, and any revisions made by the students.
A qualitative case study approach was employed. The screen recordings were transcribed and coded along two primary dimensions: (1) Prompt Content (e.g., task specification, style requests, revision commands) and (2) Interaction Pattern (e.g., number of turns, adaptation based on output). Patterns were clustered to identify distinct user pathways.
Analysis of the screen recordings revealed four prototypical pathways, representing different combinations of strategic approach and prompt sophistication.
Based on observed patterns in the cohort.
These users input very brief, often single-sentence prompts mirroring the original task instruction (e.g., "Write an essay about climate change"). They exhibit low tolerance for iteration; if the initial output is unsatisfactory, they are likely to abandon the tool or submit the subpar result. This pathway reflects a tool-as-oracle misconception.
This group starts with a simple prompt but engages in a linear refinement process. Based on the AI's output, they issue follow-up commands like "make it longer," "use simpler words," or "add more examples." The interaction is reactive and incremental, demonstrating an emerging understanding of the AI's responsiveness to instruction but lacking an overarching plan.
A minority of students approached the task with a pre-meditated structure. Their initial prompts were comprehensive, specifying format, tone, key points, and sometimes providing an outline (e.g., "Write a 5-paragraph essay arguing for renewable energy. Paragraph 1: Introduction. Paragraph 2: Economic benefits... Use a formal tone."). This pathway yields higher-quality outputs with fewer turns, indicating advanced task decomposition and meta-cognitive planning.
These users treat ChatGPT as a dialogue partner. Instead of just issuing commands, they ask meta-questions ("How can I improve my thesis statement?") or request explanations ("Why did you choose this word?"). This pathway blends writing assistance with learning about writing, though it can meander and may not efficiently complete the core task.
The prevalence of the Minimalist and Iterative Refiner pathways highlights a critical gap. Left to their own devices, most students do not spontaneously develop sophisticated prompt engineering strategies. Their process is inefficient and often fails to leverage the full capabilities of the AI, potentially reinforcing passive learning habits.
The study argues for explicit prompt engineering education within the EFL writing classroom. This should include:
The goal is to cultivate students who are strategic directors rather than passive consumers of AI-generated text.
Core Insight: This paper delivers a crucial, often-missed truth: the democratization of AI tools like ChatGPT does not automatically democratize competence. The interface is deceptively simple, but the cognitive load of effective interaction is high. The real bottleneck in the "AI-augmented classroom" isn't access to technology; it's the lack of interaction literacy. The study brilliantly shifts the focus from the AI's output to the human's input, exposing the raw, unvarnished learning curve.
Logical Flow: The argument is methodical and compelling. It starts by establishing the problem (SOTA chatbots require skillful prompting), introduces the knowledge gap (how do novices actually do this?), presents granular empirical evidence (the four pathways), and concludes with a forceful call to action (education must adapt). The use of case studies grounds the theory in messy reality.
Strengths & Flaws: The major strength is its ecological validity. Using screen recordings of first-time users in a real task context provides authentic data that lab studies often lack. The four-pathway typology is intuitive and provides a powerful framework for educators to diagnose student behavior. The primary flaw, acknowledged by the authors, is scale. This is a deep-dive case study, not a broad survey. The pathways are illustrative, not statistically generalizable. Furthermore, the study focuses on the process, not rigorously measuring the quality of the final written product across pathways—a critical next step.
Actionable Insights: For educators and curriculum designers, this paper is a wake-up call. It provides a clear mandate: Prompt engineering is a core 21st-century literacy and must be taught, not caught. Schools should develop micro-lessons integrating frameworks like the Prompt Hierarchy Model, which moves from basic command prompts ($P_{cmd}$) to complex iterative reasoning prompts ($P_{reason}$). For example, teaching students the formula for a high-quality prompt: $P_{optimal} = R + T + C + E$, where $R$ is Role, $T$ is Task, $C$ is Constraints, and $E$ is Examples. EdTech companies should build these pedagogical scaffolds directly into their interfaces, offering guided prompt-building templates and feedback, moving beyond the blank text box.
From a machine learning perspective, a user's prompt $p$ serves as the conditioning context for the language model $M$. The model generates an output sequence $o$ based on the probability distribution $P(o | p, \theta)$, where $\theta$ represents the model's parameters. An effective prompt reduces the entropy of this output distribution, steering it toward the user's intended target $t$. The student's challenge is to minimize the divergence between the distribution of possible outputs and their goal, formalized as minimizing $D_{KL}(P(o|p, \theta) \,||\, P(o|t))$, where $D_{KL}$ is the Kullback–Leibler divergence. Novice users, through trial-and-error, are performing a crude, human-in-the-loop optimization of $p$ to achieve this.
Scenario: A student must write a persuasive letter to the school principal about starting a recycling program.
Minimalist Pathway (Ineffective):
Prompt 1: "Write a letter about recycling."
Output: A generic, bland letter.
Student Action: Submits output with minor edits.
Structured Planner Pathway (Effective - Using RTF Framework):
Prompt 1: "Act as a concerned 10th-grade student. Write a formal persuasive letter to a high school principal. The goal is to convince them to implement a comprehensive plastic and paper recycling program in the cafeteria and classrooms. Use a respectful but urgent tone. Include three arguments: 1) Environmental impact, 2) Student engagement/leadership opportunities, 3) Potential for cost savings or grants. Format the letter with a date, salutation, body paragraphs for each argument, and a closing signature."
Output: A well-structured, targeted, and persuasive letter.
Student Action: Reviews output, may ask for a refinement: "Make the third argument about cost savings stronger by adding a statistic."
This contrast demonstrates how applying a simple structured framework (Role: student, Task: write letter, Format: formal with specific arguments) dramatically improves the efficiency and quality of the AI collaboration.
The study's key results are qualitative, captured in the pathway descriptions. A hypothetical quantitative extension could yield a chart like: "Figure 1: Interaction Efficiency vs. Output Quality by Pathway." The x-axis would represent the number of prompt turns (inverse of efficiency), and the y-axis would represent the quality score of the final text (e.g., assessed via rubric). We would expect:
- The Minimalist to cluster in the high-efficiency (low turns) but low-quality quadrant.
- The Iterative Refiner to show medium-to-high turns with variable quality.
- The Structured Planner to occupy the high-efficiency, high-quality quadrant (low turns, high score).
- The Conversational Explorer to be in the low-efficiency (high turns) quadrant with variable quality, potentially high if the exploration is focused. This visualization would powerfully argue that the Structured Planner pathway represents the optimal target for instruction.
The implications of this research extend beyond the EFL classroom: