AI marking assistants: the future of assessment or not fit for purpose?

Picture it: a Sunday evening, 80 essays in the inbox and the feedback deadline looming on Monday morning. For most academics, this is not an occasional inconvenience, it’s simply the rhythm of the semester. Sixty-three per cent of female teachers in the US reported exhaustion in 2024, and more than 270,000 teachers are projected to resign every year for the foreseeable future, according to research. Feedback fatigue is not a metaphor. It is a staffing emergency.

Automated grading is not, strictly speaking, a new idea. The history of automated essay evaluation goes back to the 1960s, when early systems such as Project Essay Grade first attempted to replicate the judgement of a human rater. What is new is the speed, the ambition and the plausibility of the current generation of tools.

Agentic AI systems, which embed institutional rubrics directly into adaptive decision-making models, flag uncertain cases for human review and generate auditable scoring trails, now claim to shorten grading cycles by as much as 70 per cent. Purpose-built platforms such as CoGrader and Pregrade offer to process entire essay batches simultaneously, returning personalised criterion-referenced feedback to every student before the instructor has finished a first read.

• Police a losing battle or teach students to think clearly? The choice is ours
• As students become more ‘distant’, can feedback still hit the mark?
• GenAI has not broken assessment. It has exposed it

Even general purpose large language models such as ChatGPT and Gemini can be deployed as on-demand marking assistants at a negligible cost per submission. The efficiency gains are real, but so are the complications.

The grade inflation problem

The most persistent finding in the empirical literature on AI marking is also the most inconvenient: AI tools consistently award higher marks than human evaluators. A study found that when the same undergraduate assignment was submitted repeatedly to ChatGPT, scores varied by up to 24 percentage points across submissions, and Claude produced ranges of up to 33 points. Another study found that AI tools had a measurable bias against non-native English speakers – something that would have serious equity implications in internationally diverse student populations.

Why the generosity? Largely because LLMs trained on vast sets of human writing associate formal, grammatically tidy, well-structured prose with quality, regardless of whether the underlying argument actually holds up. A student who writes with surface-level fluency will likely score higher than their ideas deserve.

The rubric is the algorithm

Rubric vagueness was already problematic for human markers before the launch of AI. Even experienced assessors will end up applying ambiguous descriptors inconsistently. Students benefit most from rubrics when criteria are explicit and comprehensible, and it turns out the same principle applies to machines.

Vague rubrics that instruct LLMs to evaluate text as “helpful and factual”, for example, fail because they reward superficial but fluent answers that include irrelevant or non-existent facts. In computing education, rubric language that had been refined by iteration produced higher agreement between AI and instructor marks, according to a study.

Students who received AI feedback through rubric-aligned assessment tools showed gains in organisation, structure and writing quality compared with those receiving traditional feedback alone. The key word is “adaptive”: rubrics designed to work with the algorithmic properties of AI, not merely uploaded as-is to a prompt window.

What AI feedback does well and where it falls short

On criteria-based coverage, directiveness and accuracy, AI feedback broadly holds its own against human feedback. The problem is in prioritisation, the capacity to identify the single most consequential issue in a given student’s work, rather than distributing comments evenly across a comprehensive checklist of criteria. The experienced teacher reads a proposal as a unified communicative act and says: “This argument is fundamentally unclear, and everything else is secondary.” The AI reads the rubric and comments on everything simultaneously.

Students notice. English as a foreign language learners value AI as a useful companion but miss the dialogic quality of human feedback – in other words, the ability to ask a follow-up question, to be heard, to receive a response that felt genuinely addressed to them as a person. The defining pedagogical question is: can AI produce better writers or only better drafts?

Rubric-aligned chain-of-thought prompting, where the model is required to reason through its evaluation step by step before arriving at a score, has been found to substantially improve feedback transparency and educational alignment. A rubric-grounded automated pipeline can simultaneously generate auditable feedback and calibrated confidence signals that route uncertain cases to human reviewers, introducing genuine epistemic humility into the grading process.

Making human-in-the-loop mean something

Both the efficiency promises and the accuracy concerns converge on a single practical conclusion. Here is what effective AI-assisted assessment looks like in practice:

• Start with a pilot in high-volume, structured courses where rubrics are clear and performance criteria are observable. This is where AI adds the most value with the least risk.
• Train on your own previous assessments. AI models calibrated on historical examples of your own grading replicate instructor judgement far more accurately than off-the-shelf tools applied cold.
• Embed genuine faculty review at every decision point. A randomised trial found that AI-mediated feedback produced significantly higher-quality student revisions than human-only feedback, but only because teaching assistants actively verified, corrected and personalised AI suggestions before they reached students. The AI amplified human expertise; it did not replace it.
• Treat AI scores as a first-pass diagnosis, not a final verdict. Rubric design and faculty oversight could be crucial in this process. A rubric-aligned model found to produce statistically consistent and equitable outcomes by one study was most likely down to these two elements.

Human-in-the-loop AI feedback delivers on its promise only when human involvement means genuine critical engagement with AI output, rather than a rubber stamp on a spreadsheet of machine-generated scores. Anchoring bias is real: time-pressed instructors tend to adjust inflated AI baselines only marginally, instead of exercising their independent judgement.

AI marking tools are imperfect, and yet, when handled with appropriate care and rubric literacy, they can be useful. The case for their efficiency is compelling. The accuracy case is contingent on rubric quality and meaningful oversight. And the pedagogical case requires that we never mistake a faster turnaround for a richer learning experience.

The machines are not going to replace your judgement. But they might, in the most productive way possible, force you to articulate exactly what that judgement is worth and design your rubrics accordingly.

Aditi Jhaveri is senior lecturer and coordinator of scholarship advancement, and Eunice Tang is lecturer, both at the Center for Language Education at Hong Kong University of Science and Technology.

If you would like advice and insight from academics and university staff delivered direct to your inbox each week, sign up for the Campus newsletter.