Lydia Arnold

In my talk about assessment redesign in an age of AI last week at the HE Professional Assessment and Integrity Event , I encouraged a restless rethink where we explore assumptions and norms around assessment in an AI age. I argued this process should be uncomfortable as we make sense of complexity. Here are a few of the main threads from my talk.

Compound wicked problems

The talk made the case that we (as a community) should explore a set of ideas rather than rushing to a single solution or indeed any kind of fix, because assessment in the age of AI is too complex to be ‘solved’. I described AI in assessment as a compound wicked problem, because AI as a changeable, complex, society-shaking phenomenon sits on top of already complex issues: integrity, technology behaviours, environmental challenges, and a higher education context already marked by permachange. Wicked problems were conceived as a concept in the 1960s (in social planing) and 70s (in education), and the phrase doesn’t for me, capture the complexity of the layering of big issues.

Legitimate concerns and the right to object

As framing, I set out why staff and student concerns about AI and AI in assessment are legitimate and needed to be held, not dismissed. Bias, slop, safety, truth, environment, rights, and wellbeing are the right themes for universities to be asking about, while government and corporations seem to push technological boundaries. That is a critical academy doing its rightful job. This gap of concern must be worked through with curiosity and confidence that we will get through it. I have said elsewhere that we need to model respect in our debates, recognise different lived experiences, and concede that we may never reach consensus. I made a statement that I think, for now, while legitimate concerns play out, I don’t believe any student should be forced to take an assessment which uses AI unless it is linked to their course outcomes. Maybe that won’t age well, or I will shift, but this is about respecting people’s rights to live their values.

Questioning assumptions

An opening thread in the talk was the need to question the assumptions built into assessment. I described the current dominant assessment approach as objective, but faux objective may be better, product-based, scalable, criteria-led, pre-prescribed, and anonymous. Those features have their place, but I suggested that AI was forcing a reconsideration of whether some long-accepted assessment habits were now limiting creativity, voice, and more expansive learning. By example, I have previously talked about my concerns around criteria: predefining what good looks like can, in my view, erode creative headroom. It tells us how to be good before enabling us to think about different ways of creating good. Criteria can create convergence of performance when perhaps what the world needs is different ways of thinking (my response in practice has been to create criteria with headroom – but that’s a different blog post).

Why joy, audience, and purpose matter

I find myself talking a lot about joy lately, and in this talk it was a key concept. I need to acknowledge here that although I gave this particular talk, my work here is joint with Amanda Seys, and I need to level and say I am not always joyful (ha ha!).

I think joy, or an intrinsic valuing of what we learn, matters greatly. Pride in learning, or creating something, matters because it is what makes us want to engage. In an almost childlike way, having a sense of audience can strengthen this. Audience can bring meaning to one’s learning or performance. If assessment and curriculum feel flat or purely transactional, arguably shortcuts become more tempting. By contrast, tasks that create pride and a real reason to participate can bring energy back into learning and assessment. In that sense, joy was not treated as a soft extra, but as a serious condition for deeper engagement.

I gave examples from my own university of professional events and poster fairs where sharing and audience motivate, and in the past few weeks I also visited Staffordshire University’s GradEx showcase where students share their work with peers, industry experts and the local community (great job Staffs Uni). That sense of pride was palpable. At the start of my career, I remember even then, circa 2004, our online graduates finished their projects with a work-based learning exhibition in their place of work. My question then is: have we lost the joy and individuality in the search for perfect standardisation? Do we need to make assessment more of an event to bring in engagement energy and to motivate deep learning and creativity? It may not be through contrived novelty but through exhibition, audience and connection.

As well as audience, purpose matters greatly. I shared how I have had a ‘pass’ to access various perfectly good online learning platforms, but instead I learnt much more from Q&A on demand with AI – on everything from leadership theory, to coding and Excel data hacks, and from learning about the back story to a book I read to project planning approaches. Perhaps technology has shifted the concepts of curiosity, purpose and learning in to something more fundamentally needs driven than before when knowledge was curated. Wylde Scott’s Raising Dreamers suggests the future belongs to those who are curious and those who know how to learn. Curiosity is a difficult term since it may, sadly, be linked to privilege (exposure to ideas, learning to ask good questions, having time for a wondering mind, having experiences about which to be curious), but purpose is universal and purpose can drive curiosity. So can we create learning and assessment that allows and enables learners to work with their purposes (or, curiosities of our own kind)?

Joy, audience and purpose. They all matter.

Super-authentic assessment

The talk then moved into looking at authentic assessment, which has been touted as a partial response to AI and the risk of integrity. We need to be careful with that, because if you task me to make a podcast on a topic, I know little or nothing of, I can make a script and clone my voice and move from zero to completion in a very convincing way in about five minutes straight! While it isn’t a cure all for integrity then, it can address provide moments where students can be proud of their individuality and unique take.

As I have been saying for a while now, I think conceptions of authentic assessment have somewhat lost their way, so in my talk I mooted a reset of the idea of authentic assessment. Instead of treating it as more than a simple stand-in for “real-world” tasks, the idea was that authentic assessment should connect with future employment, disciplinary traditions, knowledge creation, the collective future, and the individual learner, so that it became genuinely relevant rather than merely work-like. That is where the notion of a super-authentic (or really-authentic) assessment sits though I am not sure super-authentic assessment will be the name I stick with for this. The AI age version of authentic assessment is linked to joy, exhibition, authentic self-expression, and the freedom to choose how to show learning. It is linked to transparence on how problems are approached, solved and iterated. It is not only about replicating work as it is, but about creating assessment experiences that are motivating, public-facing, meaningful, and aligned with a learner’s values and identity. I am not saying this should be the only type of assessment, clearly there are moments when a standardised test matters, or the ability to identify thing x or y rapidly really matters too (and many other considerations and scenarios). But when we are looking at creative assessment types, we need to manage our expectations: they are likely cheatable, and instead we can design them for motivation. I think in the questions I accidently committed to flesh this idea out in new resources (it’s now on my list!).

Why programme thinking mattered

The talk also argued that assessment redesign had to happen at programme level, not just module level. I have been on this soapbox for more than a decade. Students experience a journey, so change had to be coherent across the whole assessment pathway rather than introduced in isolated fragments. I advocate for programme planning, off-timetable design conversations, and a kind of pit lane approach where each programme can be reviewed. The aim is to create joined-up assessment journeys with the right mix of secure, oral, audience, and developmental moments. For insight in to this – in practice around AI – I would  recommend looking at Danny Lui’s work at University of Sydney, or for a process of design by programme I wrote up a process I was involved in called blueprinting.

Transparency as modelling

In the talk, I mentioned how important transparency was, especially staff being transparent about how they are using AI so that we can create an open and honest community. This goes beyond saying what tech was used, but making the tacit decissions about AI explicit so we model and normalise this. By example, in the actual creation of my slide deck and talk, I didn’t use AI, beyond creating images. And that’s because when I have to stand up and talk about something (audience), I want to be confident that I can answer questions and I’m clear of my own lines of thinking.  A shortcut wouldn’t have helped me to understand my own thinking anymore. So audience added my motivation to be thorough, AI would have helped me to create a product, but it wouldn’t help me to the deep thinking that I needed. Now, if I was creating a lecture, and I declared the same points, I would be modelling the thoughts that actually AI matters sometimes, but for some things, it just doesn’t get us where we need to. In another lecture I might say I used AI more heavily and here is why … Transparency done well helps model critical choices about AI,

A varied AI ecosystem

The conclusion didn’t create an AI Assessment utopia – instead it suggested an assessment system made up of a range of approaches. That may include some combination of these and other aspects:

• A two-lane approach.
• Clarifying guidance.
• Oral assessment.
• A reimagined idea of what authentic assessment is now.
• Respect and the right to object.
• Places to hold and debate legitimate concerns.
• Assessment and curricula that motivate, a return to joy.
• Staff and student upskilling.
• Programme-level planning.
• Transparency.
• Prioritising learning-to-learn to address purpose (and perhaps curiosity).

We are figuring out a way forward together, as we always have. How we respond now, collectively, is going to model how our students might respond to the next seismic change. We are modelling sitting with uncertainty. My own restless rethink is resulting in a growing list of connected considerations that we need to think about as we proceed to renegotiate assessment for a different kind of future.

AI Transparency Statement: I downloaded my audio talk, used Apple VoiceNotes to generate a transcript, then used Perplexity to condense it in to six headings I had in min. I took that as my starting point and strongly reworked most of it, therefore my words were the base text (source material), AI summarised, then I heavily edited, added, reworked and developed.