Advocates of anonymous grading argue that it provides a more equitable evaluation of student learning, however this approach breaks down when students use generative AI. In this episode, Cathy Elliott joins us to discuss how portfolio-based relational marking can provide a more direct measure of student learning. Cathy is a Professor (Teaching) of Political Science and Vice Dean of Education in the Faculty of Social and Historical Sciences at University College London. Cathy also co-founded the UCL Centre for the Pedagogy of Politics. Her educational interests include imagination, creativity, joy, and inclusive teaching Show notes and the transcript for this episode may be found at http://teaforteaching.com.
Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.
Transcript
From the show. Tap a time to play from there.
JOHN: Advocates of anonymous grading argue that it provides a more equitable evaluation of student learning, however this approach breaks down when students use generative AI. In this episode, we examine how portfolio-based relational marking can provide a more direct measure of student learning. [MUSIC] Thanks for joining us for Tea for Teaching, an informal discussion of innovative and effective practices in teaching and learning.
REBECCA: This podcast series is hosted by John Kane, an economist… JOHN: …and Rebecca Mushtare, a graphic designer… REBECCA: …and features guests doing important research and advocacy work to make higher education more inclusive and supportive of all learners. [MUSIC] JOHN: Our guest today is Cathy Elliott. She is a Professor [Teaching] of Political Science and Vice Dean of Education in the Faculty of Social and Historical Sciences at University College London.
Cathy also co-founded the UCL Centre for the Pedagogy of Politics. Her educational interests include imagination, creativity, joy, and inclusive teaching, which sounds like a really fun combination of topics. Welcome, Cathy. CATHY: Hi, thanks very much for having me. REBECCA: Today's teas are:… Cathy, are you drinking tea with us today? CATHY: Oh, I am drinking coffee, as a matter of fact, is that okay? REBECCA: Ah, we suppose. [LAUGHTER] CATHY: Even though I'm in London, sorry about that. [LAUGHTER] Yeah. JOHN: Yeah, if this was France, I could see it.
CATHY: I really apologize. [LAUGHTER] REBECCA: How about you, John? JOHN: I am drinking a Twining spiced apple chai tea. We've had it there for a while, and I figured I'd try it today. It's a nice fall day here. It's a little bit brisk. REBECCA: I have a random mix of green and black tea. JOHN: So, a dark green. REBECCA: It's a not describable kind of tea. [LAUGHTER] JOHN: Does it taste okay? REBECCA: It tastes fine. Yeah. JOHN: Okay. Good. [LAUGHTER] REBECCA: Kind of cleaning up the leftover. [LAUGHTER] It's Monday.
I don't know. CATHY: A delicious blend. REBECCA: Yeah. JOHN: We've invited you here today to discuss how you have used relational marking of student portfolio work in a political science class that you teach. Before we discuss how student work is assessed in the class, can you tell us a little bit about the focus of this class? CATHY: Yeah, so it is an undergraduate class, so level six, as we would call it in the UK, and it's on politics of nature. So students are learning about how nature shapes the political world, and
also how politics shapes the natural world in all sorts of different ways. So it's really fun. We spend quite a lot of time outside the classroom. We go to different gardens. We do some gardening. So it's a really lovely experience, I think, for the students. And we also engage quite a lot with art, so we do an art museum tour as well. JOHN: And so that brings in the creativity and the joy, which is nice. CATHY: Yeah. REBECCA: Sounds like a good combination to me. Sign me up. [LAUGHTER] CATHY: That's what everyone says. Yeah, no, it's really fun. I get a lot of guest hangers on at the garden trips. [LAUGHTER]
REBECCA: I'm sure. Can you also talk a little bit about what you mean by relational marking? JOHN: Yes. CATHY: So the approach to marking I take is really influenced by the ungrading movement, which will probably be much more familiar actually to you in the U.S. and Canada than it is to us in the U.K. In the U.K., often people haven't heard of it. But basically, all ungrading is really it's a whole series of practices. It's an umbrella for the idea that we might want to just take a skeptical look at practices of grading, where you give students either a letter grade or
a mark, and to focus our efforts much more on assessment as a way of learning, as a way of practicing, of doing some work, and then getting feedback. And the approach to ungrading that I take is really influenced by Susan Blum, and she writes about negotiated marking. And so students come to me having done some work continuously across the term, and they negotiate their final mark with me. Honestly, I would rather have a pass-fail module so that we just didn't have these conversations about marks.
But politically, that's not where we are at UCL. They have to have a number in the system at the end of the module in order to graduate. So that's the compromise that we have: that we negotiate that mark together as part of an ongoing process of thinking about both the final work that they've done, but also the effort and the engagement and so on. The reason that I call it relational marking rather than negotiated marking is because I really want to foreground the role of relationship in that process of deciding the final mark. So I'm trying to support students in their assessment literacy.
I'm trying to support them in understanding what assessment is for, and why we have this system of putting numbers in a system and how that's regulated and what that means, and all the evidence on what that does, and one of the things that I encourage them to think about is the impact that grading and marking have had on their relationship with other people. Because I think that often we don't mean to do it, but it was something that we remember being done by our teachers. And you'll often see people really unthinkingly say to students, “If you want to get a good mark, you should do this, and if you want to avoid getting
a bad mark, you should do that.” And it's kind of coercive, right? It's kind of putting a power dynamic into that relationship, which is unfortunate because actually teaching works best when it's about everybody being on the same side in the service of students being able to learn. It's also bad for relationships among students as well. So telling students you've got a better mark than the other student, and therefore you're better than them in some sense. It's not very healthy. It wasn't very good for me when I was a student, and like many academics, I was one of the students that did quite well. But yeah, this idea that some students are better than others, and they're
ranked and pitted against each other in competition, it's all a bit gross. So we have that conversation in class as a way of thinking about if we don't want those relationships, what relationships do we want, and how do we build really good, strong, trusting, responsible relationships within our classrooms? So that's why I call it relational marking. REBECCA: I really like that idea of those conversations about relationships and grading and marking that you mentioned. It seems really helpful. JOHN: And another thing is that it encourages students to focus on their grades rather than their learning, which is something we'd like to
move students away from in general. And we'll talk about that a little bit more later. But along the line of relationships, in a March 2025 paper in the Journal of Learning Development in Higher Ed, you note that a common practice is to use anonymous marking to help reduce awarding gaps and bias in grading. Yet awarding gaps persist when anonymous marking is used. How large are these awarding gaps, and why are they still persisting with anonymous marking? CATHY: Yeah, thank you. So yeah, obviously anonymous marking is the opposite of relational marking
because there is no relationship if you don't know whose work you're marking, or the relationship is a very strange one. So awarding gaps is something that really preoccupies in the UK, and the idea of awarding gaps is it's often the awarding gaps between white students and students of color. So white students are coming out with higher marks, better degrees, than students of color, but we also see awarding gaps, for example, amongst abled and disabled students, students from different social classes, and so on. But the one that I focused on in this particular article is the BME awarding gap, as we call it here, or the awarding gap between white and students of color.
In the UK, the gap between white students and students of color for a good degree, which in our system is defined as a two one or above was 8.8% in 2021, and between white and specifically black students, it was 18.4%. That is quite high. But when you look at first-class degrees, then the difference between white students and students of color was 9.5%. My institution has quite small awarding gaps, which we think is because we've already got very high entry qualifications.
So students that are coming here are already doing extremely well by that kind of yardstick. So the good degrees awarding gap at UCL was actually only 0.4%, but first-class degrees, so that's the highest possible mark in our system, the gap was 4.3%, rising to 15.3 between white and black students at that first-class level, and the question of why that exists is really complicated. It actually is still quite poorly understood. We don't really know, but one thing I think we do know is that it's probably not because markers, when they see a student's name, are biased against them.
That doesn't seem to be right, and there is some evidence of that, and the biggest systematic review of the literature, which looked into this, which was by HEFCE, so the Higher Education Funding Council for England, they did a big systematic review of all the evidence that currently exists. Although this is about 10 years ago, I think probably this is still the best evidence that we've got, and they said there are probably four overarching areas that we need to look at, although they're all very interlinked. So first of all, there is curriculum, including teaching,
learning, and assessment, and often the issue of hidden curriculum. So students that are less economically advantaged in various ways might not know how to write a good essay in the way that a student that went to a fancy school night, and so just that basic teaching students the study skills actually seems to make a big difference. So this is around pedagogies. Secondly, relationships between staff and students are really important, and students feeling like they belong seems to be a really important factor in reducing those awarding gaps.
Thirdly, it's students' social, cultural, and economic capital. So we can't do much about their material resources, but what we can do is support them with their cultural and social capital, so that they're able to use the support systems that are open to them and their peer networks, and engage in those kind of collaborative approaches to learning that seem to characterize the more successful attitudes of students that have more privilege, and finally, unsurprisingly, racism; students that are constantly on the end of microaggressions and negative micro interactions and assumptions about their
abilities are just less likely to do well. REBECCA: You started to hint at this, but you argue that relational marketing is more likely to reduce those awarding gaps than anonymous marking. Can you talk a little bit about this argument and why the relational marking might work better? CATHY: Yeah. So the paper makes two arguments. So the first is that anonymous marking really isn't helping. So in the UK, anonymous marking is the absolute set to norm, and this was the result of a really big student campaign about 20 years ago now, led by a student
leader who later became a very prominent politician in the UK, Wes Streeting. So UK listeners will recognize that name. But he really led this campaign. It was called "Mark My Words, Not My Name," and that was based on this idea that markers were biased. And so this is now absolute set to norm, and there is a fair amount of evidence that it doesn't make any difference at all. And it doesn't make any difference to awarding gaps, and there are some big studies that have looked at this, which is cited in the paper. But it also doesn't make any difference to students' perception of fairness. So David Boud and Naomi Winstone have done quite a nice paper where they've
compared students that had anonymous marking versus more relational marking, and the students actually, when they got marked anonymously, they didn't even feel that was any more fair. So anonymous marking has been the default answer in the UK to awarding gaps, but it isn't working. And so relational marking seems more likely to help because it addresses these four reasons why we think we see these awarding gaps. So first of all, the idea of taking the emphasis off the marks and putting it right firmly onto the feedback makes that hidden curriculum explicit.
So you are explicitly teaching students what the assessment criteria are. What do we think good work is? Once you've done some work, how can I give you some feedback to support you in improving that work? And we can have a conversation together about what those assessment criteria means, in which the student themselves has some voice. The student has some power because they're negotiating their grade. They're not just waiting passively for me to grade them. They're able to talk to me about why they think their work is good, and they might surprise me by thinking that things are good that maybe I
wouldn't have necessarily considered. Relational marking also gives them some freedom, and so they can address parts of the curriculum that are of personal interest to them, which might give them more intrinsic motivation and excitement and creative work and so on. So, lots of students of color do write about colonialism and the impact of colonialism on nature and botany and national landscapes and what the landscape looks like, and they do very well on that because it's a passion project for them. So that's part of it. And I think having those open conversations does build the relationship.
So it builds that belonging. It helps us challenge racism when it arises, rather than pretending it's something that we can get rid of… do a sort of technical fix, and it also clues students in to the ways in which they can access different areas of support because the feedback is continuous through the term and is part of the whole process, rather than just coming at the end when it's far too late to flag them to the support services available. So the whole process of building that relationship around the assessment and around learning, therefore, kind of attacks all four of those
potential causes of awarding gaps. And indeed, in my module, and I've now been doing it for five years in this way. I've had a look at the module statistics. If anything, there is a tiny, tiny little awarding gap in favor of students of color and in favor of women and non-binary students, but it's tiny and it's not statistically significant. In fact, there is really not much of an awarding gap at all, having implemented this pedagogy for quite some time now. JOHN: Could you walk us through the student experience in the class from the beginning through the end in terms of what types of things are they asked to do, what types of projects, and the development of that
relationship over the course of the term. CATHY: Absolutely. I'm just starting out on the term now, so the student journey is really fresh on my mind. So I'm very excited for Friday to meet my students for the first time. So today they will have had a message from me, so five days in advance, saying, "Come on to Moodle.” Do you have Moodle over there, a learning management system? JOHN: We do. I don't think it's as large as it used to be. The biggest things used here, I believe, are Canvas, Brightspace, and Blackboard, but Moodle is still up there, and many institutions do use it.
CATHY: Yes. So they're going to go on there. They're going to learn. There are various videos on there. So I do a flipped classroom, and the reason for doing that is mainly to free up the time so that we can do all this fun stuff, like go to gardens and go to art galleries and things like that. So all the information delivery is online. So hopefully they're working through that now, and they're watching some videos which are going to explain to them in slightly vaguer terms than I've put it to you what the assessment is and what I'm hoping they'll get out of it. So that's right from the beginning I'm sort of explaining to them what the assessment is, how I'm expecting them to engage in the module.
How I'm expecting them to learn. I have a little Padlet board there, which is full of advice from former students, which very helpfully says things like "get started early," [LAUGHTER] which they hear it better from each other than they might do from me. So that's terrific. And I also, and this is an idea that I nicked from Cate Denial. So I've never had an original idea in my life. I just steal from Susan Blum, Cate Denial, all these different brilliant people, and I've just put it into my module. So she has this idea of having a little questionnaire at the start of term. So I have my little questionnaire at the start of term, and that asks them, "What's your name?
How do you pronounce your name? What are your pronouns? We're going to be dealing with some quite difficult material this term. Is there anything you need to make that? Do you need a heads-up on anything? Is there anything you need to make this class accessible to you?” Those sorts of questions are signaling that there is a caring relationship here. Teaching is a caring profession. I am here as your teacher, not as a police officer, not as a counselor for the prosecution. [LAUGHTER] I'm here for you to support your learning. And sort of signaling all of that, and then getting into: What do you want to learn?
Have a look at the assessment criteria. Have a think about the learning outcomes for this module. What do you want to focus on? What are you interested in right now? That can change, but I'm just interested to know. And that also clues them into the fact that they might have some power and control, and that they can look at those assessment criteria at this point. For the first time this year, I'm asking them how do you use AI because I think actually we just need to have some open conversations about that in order to support their learning, and then just a little opportunity to ask any questions or say anything they want to.
So that's already before I ever see their face. They're already clued into the fact that I'm interested in them, and they can talk to me, and they can sort of raise these concerns, and many of them do, and they always express their appreciation. Also, there's a copy of the syllabus on there, which is in software that they can use to annotate it. So they've got questions, they can just flag that and ask their question. Looks very similar to Google Docs, but it's called Talis Elevate. And they have to do a little induction activity this week for the portfolio software that we use, which is Mahara, but we call it My Portfolio.
So they have to do that and work out how to use that, and then they come to class all ready for this. They've also done a little bit of reading about nature as well. Then they arrive in class on Friday, and the first thing that I do is learn their names. It's not a massive class. It's 40 students, so my memory can still just about do it. I make it into a little bit of a game, but I quiz myself every week, and it's always funny when I get one wrong and so on. But this is what I say to absolutely every educator: if there was one top tip I could give to you, there's one suggestion I could make that would make
your teaching a million times better, and it's relatively uncomplicated. It's totally low tech: learn your students' names. You can't have a relationship with someone whose name you don't know. And teaching is a relationship. And it is amazing to me how many students at a posh University in London come to me at the end of their degree and they say, "You are the only teacher that learned my name." That is really, really heartbreaking to me. And so I really, really make a point of that, and also I really publicly make a point of it, and I also try and get them to learn each other's names, which is also not taken for granted.
And so we have quizzes in the class on that, and then we start talking about the assessment. And to start with, students in the UK system were just absolutely flabbergasted at the idea that they might be able to have a say in what their overall mark would be for the module, and we would read an article by Susan Blum about why marking is really bad, and then we would talk about that. And quite often they would cry, and we would have an emotional opening session where we talked about like why is this bad and what are we going to do differently. And so they don't cry so much anymore because they know, word has gone
around, [LAUGHTER] it's a final-year module, and so they already know that mine is the hippie module, and it's a little bit different. [LAUGHTER] And one student said last year, and I really take this as a high compliment. They said “I took this module because I thought it would be an easy high mark, and it turns out to be one of the most difficult modules I've ever done, and I really worked hard, and I really learned a lot from it,” and so that's nice as well. But it's kind of a different vibe these days with students because inevitably you do something and it changes the conditions in which you're working. So that's week one, and then we go through the term, and every week they
have to write in a portfolio, and I have slightly changed it this year. So this year, I'm making it really, really explicit that they are writing about their learning journey every single week, and I give them some prompts and some questions and some things to think about. And I particularly want them to focus on how they're using AI, if they are using it, and talking to me about how they're using it and why they're using it and whether it's good and what ways it's blocking their productive struggle and so on. So they're writing about that every week, and they're handing that in. And then over the course of the term, they're writing two or three unessays.
So essays are boring, but you can write an unessay instead, which might be a blog, or a letter to the newspaper, or a video, or a podcast, or a chapter from your memoir, or a piece of nature writing, whatever it is you want, like go wild, be creative. You'll do two or three of those over the course of the term, and you'll hand in your best work. So you don't have to hand in all of them. You can have some false starts. You can experiment. You can play about. But you hopefully are working continuously and documenting that progress in your learning journal. Then through the term, they get feedback, and that is scaffolded
according to those assessment criteria. So hopefully that's really clear. But if they want to push back on the assessment criteria, if they want to negotiate that, that's fine. We can talk about how their work will be assessed as long as they've got a good reason for it. No problem. They're also giving each other feedback, and then right at the end of term, they hand in their work. They give me 2500 words of their best work plus that learning journal, and then they fill in a questionnaire where they tell me how they did. So how much effort did they make, and then how did they do against the assessment criteria?
How did they learn and develop? That's really important. So we're assessing process as well as product, and then finally, and hopefully de-emphasized, least important thing, what is the magic number that is going to go in the magic box? So long as they're in the right ballpark, then I allow them to have whatever magic number they choose. Obviously, sometimes that doesn't work out for reasons. By and large, so long as they're in the general right area, then that's fine, and they should know what right area they're in because obviously they've had the feedback. So they have the feedback before the point where the magic number goes in
the box, rather than after, which is really important to them, I think, and enables them to do their best work. So yes, it's a long journey over 10 weeks. JOHN: It sounds like a great experience, building in so many practices that we've talked about on past episodes of the podcast. But one thing I do want to note, and we'll include a link to this in the show notes, is that our friend Michelle Miller, who I think is our most frequent guest on the podcast, has a book on how to learn students' names, and we also did a podcast with her on that. So we'll include a link both to the book citation and to that podcast for those
who want to improve their ability to learn students' names because it is challenging. CATHY: Yeah, yeah, that book is fantastic. REBECCA: So you've talked a little bit about the feedback that students are receiving throughout the term. Can you talk a little bit about what format that comes in? You mentioned a rubric. Are there conversations that are ongoing? Is it written feedback? You mentioned peer feedback. Can you talk a little bit more in detail about what that feedback looks like? CATHY: Absolutely. So students have a form on Moodle that they fill in when they're ready for feedback, and they can have feedback on anything they want.
So they can have feedback on their learning journal. If there's something they haven't understood, they can write about that, try and puzzle it out in their learning journal, and then come to me with questions about it. Or they can have feedback on some of the unessays that they're hoping to hand in for their final work. And what they have to do when they fill that form, they have to tell me what I need to look at, and they have to tell me which assessment criteria they want me to pay particular attention to. And the thing that's useful about that is it then clues them in and reminds them that this is being assessed by criteria, not by some secret tacit assumptions in my head.
And those criteria are things we can have a conversation about and figure out what they mean. And then they tell me whether they would rather have written feedback, which would come as a comment on their portfolio, or whether they'd rather have a chat with me in my office. And it usually plays out 50-50, so half will go for the chat, half will go for the written feedback. You can often predict from the student's demeanor which one they might prefer, though not always. And so that's quite nice for me because it varies it up a little bit. I've got a choice of different things, and they are getting the feedback in the way that they want it and they find most helpful. And the trick that I discovered, and like most brilliant tricks,
it was totally by accident. So what I said to them was, “If you give feedback to somebody else, then you can have an extra piece of feedback from me.” And I was a little bit worried about that because I thought that makes it seem like my feedback is like the ultimate prize, and the peer feedback is like not so much. But the way I framed it is: look, academia works on favors. We all do each other favors. If you will do me the favor of giving feedback to somebody else, I will give a favor back to you and let you have some more feedback. The brilliant thing about that is they give each other feedback, but then
they never come back to me for the second piece of feedback [LAUGHTER] because they've already figured it out. Because they are getting feedback from each other, and that feedback is of high quality. And the reason it's of a high quality is: a] they have learned they need to use the assessment criteria, or else I won't owe them a favor. They can't just say, "Oh, your portfolio was very nice, and I like the pictures." Like they have to really engage seriously so that I will engage seriously with them, and b], they explain things so much better to each other because they're at the point where they're learning it. So they really got their heads around what these criteria mean.
Whereas for me, it's more of a… what one of my friends, Jason Davis, calls a threshold concept, where it goes far beyond it that it's almost hard to explain anymore. So the feedback they give each other is fantastic, and once they're into that culture of giving each other feedback, they almost never come back to me for the second piece. So that's a really nice labour-saving trick for you. And the ones that do come back for the second piece of feedback, they are the very keen beans, and they're often delightful students. JOHN: So, given the use of an unessay type format, do you use the same criteria for all assignments, or are they customized based on the specific
assignments that students are working on? CATHY: So they are the same criteria, and they're quite broad. So, for example, one of the criteria is demonstrate your understanding of how power and politics shape nature, and vice versa. So you can do that in a podcast, you can do that in video. Another of the criteria is creativity of your work on the portfolio, and what I say to them is, creativity can mean a lot of things. It can mean coming up with a killer argument. It can mean doing a beautiful video with like loads of visuals. Like creativity is lots of things. All I want you to do is argue for why you personally have been creative.
One of the criteria is ability to write for an audience, but you tell me who that audience is. So it's the same criteria for everybody, but they are more than welcome: a] to specify what those criteria mean for them in as narrow and as focused a way as they want, and b] to suggest their own if they think that that is appropriate. REBECCA: You talked about this focus on submitting a portfolio at the end. Can you talk a little bit about what's revealed by having a portfolio, rather than individual assignments throughout the semester? CATHY: Absolutely. So the portfolio is about trying to incentivize continuous
work, and that's partly for the formative feedback purposes. So if they're not working continuously, it's really hard to give feedback, and you're only ever giving feedback on just those contributions in class off the top of their heads rather than the work. So it's partly that, but it's also… and I think with AI, this is becoming increasingly important to all of us… you can see the process and not just the final product, because we all know some of the final products that the chatbots can just produce stories out of nowhere are really quite good, actually. And so, what I want to see is where they've engaged in that productive
struggle, including where they've engaged with asking AI to help them summarize things and generate text and all this kind of thing, what the portfolio does is it shows that level of effort and engagement across the term. And I am assessing that. I'm assessing their good faith attempts to learn just as much as I'm assessing the final product that they come out with at the end. I think it, especially for students that leave it all to the last minute, it's such good discipline because everybody does better work if they work continuously and they're getting feedback as they go along. JOHN: And that sounds like a pretty strong argument for moving away from
anonymous marking because with anonymous marking, you only have the product and you don't get to observe the process. So it sounds like, in a world where students have access to AI tools that are increasingly powerful, building those relationships and being able to observe the process is increasingly important in being able to assess student learning and the student learning process. CATHY: Yeah, I absolutely agree with that, and that has been one of the really fortunate things about introducing this method of assessment, just as AI became a thing, as with any of these things around being AI resilient, it's
actually good pedagogy in any case. You would want to do these things anyway. But in a world saturated by AI, it is particularly important. REBECCA: You've talked a little bit about that final self-assessment and the form to go through and to put a number in that magic box. I know that in some of our conversations with others talking about ungrading, that there's been concern about students who are too hard on themselves rather than too easy on themselves. Have you experienced this in the self-assessments that
you've had in your classes? CATHY: So the thing about it is, we talk a lot before they put the number in the magic box about how they're doing. So I never ever have a discussion about numbers with them. And if they try and say, "But what number would you give it?" I'm just like, "No, what number would you give it?" So I'm not focusing on the number, but I think they do have a good sense of how much effort they've put in and of how they're doing as they go along, so it's unusual to get a mark which is kind of totally outside the ballpark. And if somebody gives themselves a mark that's on the low side or the
high side, I usually let that pass because I want to give them the message that I am not quibbling about the number because the number is arbitrary anyway, and I have contempt for the number, but I have to do it. [LAUGHTER] And so, I don't focus on that, but we do have conversations around how important it is in the world to advocate for yourself, and I say that particularly to those students that are perhaps a little bit quieter or might have been anchored to expecting a lower mark, precisely because of all these social currents that create awarding gaps, and so we talk a lot
about how when I have my job appraisal every year, I don't go in and just wait for my line manager to praise me. I go in and I advocate for myself, and I say what I've done, and I explain why it was good, and I list about areas for self-development because actually epistemic humility is a really big part of being a responsible member of the learning community and doing good academic work. So there was a really moving moment and I mentioned this in the article where a student from I think a relatively disadvantaged background and she grew up in London, she was from an ethnic minority background,
and even having grown up in London, she'd never been to Kew Gardens before. And she wrote in her self-assessment, "I've always been taught that I don't matter very much. I'm a brown disabled woman, but in this module, I have worked really hard. I've had feedback. I've paid attention to the assessment criteria. I have changed my work in response to feedback, and I have worked really, really hard. And so, I am giving myself an excellent mark, or I'm proposing for myself an excellent mark because I think I deserve it,” and she did deserve it.
Her work was superb. And the first step wasn't, and then we gave her feedback, and she got better, and she knocked it out of the park, and she gave herself an excellent mark, I think confident in knowing that she did matter in our module, and that her work was really, really good. REBECCA: A beautiful story. JOHN: It is, and it sounds like that the outliers are pretty much adjusted through the continuous feedback that you're providing, rather than overriding the students' voice, which I think gives students a little bit more agency or
a feeling that they have more agency. And it sounds like a really good approach. CATHY: Thank you. Yeah, that is the idea. So the idea is not that I call them in at the end of term and say, "I'm sorry, where did this mark come from?" [LAUGHTER] The idea is that everybody's being responsible all the way through, and so we kind of know where we are. I mean, it has happened from time to time that a student has come in with a mark which is quite a lot higher than the amount of effort that is on show, and where that has happened, and particularly where students haven't engaged. So the point where that can happen is if a student hasn't engaged at all, so
they haven't had feedback, and then they come in and they think it's a free ride. And one student really openly said to me, "You know, I'm an economics student, right? I'm a free rider in this situation." And we both laughed about it. [LAUGHTER] I was like, "Unfortunately,”… actually, this is another nice story. So we did collaboratively negotiated it, and he'd kind of given himself 85, which, like, in the UK system, if you turn water into wine, 82. So 85 is like really [LAUGHTER] beyond what we are expecting. And he hadn't even learned how to use the portfolio software. I mean, there had been no attempt [LAUGHTER] to do any work at all.
And so we had a lovely chat. We decided on a more appropriate mark that would reflect the amount of effort and engagement that there had been, and that was fine. And then he went away. But then later in the term, [LAUGHTER] he came to me, and I happened to be the student's personal tutor, and so he came to me later in the term in a panic because he'd handed in a piece of work late. And in my department, we have quite draconian lateness penalties, and so I said to him, "Right, the only thing you can do in this situation is tell me you've had diarrhea, because if you've had diarrhea, they can't ask
you for medical evidence, and they will let you off." [LAUGHTER] And he looked me in the eye and he said, "Cathy, I am not going to do that. I'm going to take the lateness penalty, and I'll tell you why. You have taught me that there are some things such as honesty that matter more than the grade. [LAUGHTER] REBECCA: It’s a great story. [LAUGHTER] CATHY: So, like even with these students that maybe sometimes slightly overestimate their own ability, I think actually there's always a really sort of pleasant conversation around it. I mean, some exceptions, but very very few exceptions. REBECCA: You talked about your course being about 40 students, do you see
the possibility of a process like this being scaled to even larger classes? And if so, how might that be manageable? [LAUGHTER] CATHY: So the whole point of this marking process is about relationships, and actually, there is a limit to how many people one human being can have a relationship with, that is just a fact. So if I had a class and I was teaching them on my own of 160 students, then probably not. Probably I couldn't even learn their names, to be honest with you. And there comes a point where I think that probably isn't even
teaching, that's broadcasting. But most classes that are that big, students do have a relationship with somebody. It just might not be the module convener, and that somebody is probably a teaching assistant, or it might be some kind of associate lecturer or somebody in that kind of role. And if you've got groups anywhere in the class where the groups are small enough that a relationship can happen, then it's scalable. But it means that the relationship is not between you and the student. The relationship is between you and their teacher, and then
their teacher and the student. And then what you're doing there is a piece of teacher education. You are supporting your PGTAs to be the amazing teachers of the future, the relational markers of the future, by supporting them to learn how to do this. And you really would have to support them. And I have colleagues in my department that are doing this on a big scale. And the support you have to give is quite intensive. But I would say: “Yes, I know your PGTA is too busy to learn everybody's name and negotiate their grade with them. But also, their lives will be so much more pleasant and enjoyable if they're doing
this kind of teaching, and not just now, but for the rest of their careers, they are going to enjoy their teaching more if you give them this opportunity now. It really is that good of a technique.” So, it is scalable if you're prepared to do that work of teacher education, and I really, really recommend that you do. JOHN: Would this approach work as well in STEM classes as it does in your environment? And if so, are there some things that would have to change in the process? CATHY: Yeah, so I don't know personally from my own class, of course, because my subject's very qualitative. But I know that Clarissa Sorenson-Unroh, in her, I think, chemistry
classroom, does amazing work on this kind of negotiated marking. And so the idea there is that if you've got the kind of assessment where there's a kind of right or wrong answer, then you're supporting the students to understand did they get the right answer or the wrong answer, and you're negotiating the mark on that basis. And in any class, there's always a little bit, or there's always some kind of interpretation or working or something like that that students are able to show as part of a process, and I haven't done any STEM subjects since I was 16, but
I do remember when I was doing my GCSE maths at 16 that actually showing your working was a really big part of the exam. And having the right answer wouldn't necessarily get you the top marks. You had to show the working, so that's marking process rather than product. And so all of this, I think, translates into being able to do relational grading, where you have some kind of ongoing, continuous assessment, and students are able to judge how well they're doing and negotiate the mark through part of that process. So I don't see any reason why not.
But of course, there would have to be sort of you have to think pedagogically about any class you're doing, don't you? There's no quick fix. You have to think about what do I want my students to be able to demonstrate, and how do I want them to be able to show me that they know what good work looks like in this particular discipline. REBECCA: Through the stories that you've shared, you kind of indicated the strong positive response that students seem to have had related to relational marking. How have your colleagues responded? CATHY: It's quite divisive among colleagues. So there is a quiet revolution happening.
I think, particularly at UCL and in my discipline as well more widely that I know of, where there are lots of people that are just quietly, maybe they wouldn't tell anybody, maybe they're doing it in some slightly more vanilla ways than me, are engaging in this stuff and excited about it, and so there are some spaces that I go into which are very pedagogically focused, where you've got people that really care about education, really care about students, and they are more or less carrying me round on their shoulders, singing "She's a Jolly Good Fellow." There are some people that absolutely love it, and then there are colleagues
that really, really dislike it, and they think that students will take a free ride. They'll try and get away with something. It's not rigorous. It's not robust. How can this possibly be fair? And I think this is just a clash of values. It's a clash of understanding about what we mean by fair. Like, is fair about treating everybody exactly the same, no matter where they're starting out from? Is rigor in our disciplines around punishment and failout classes and whittling out the ones that somehow don't belong, is that what fairness and rigor mean? Or can we interpret fairness and rigor in our disciplines in a kind
of more expansive, generous way? So there are some real disagreements with people, and we have had examples where it has been quite tricky, but so far I've kept my job and I've been allowed to continue doing this work. REBECCA: Well, that's good because that's really great work. [LAUGHTER] CATHY: Well, thank you. JOHN: And those conversations, I think, are taking place pretty much everywhere along very similar lines. And I think your side is starting to make some progress in terms of having a broader impact on teaching, because the old techniques just don't seem to
be working quite as well, especially in a world where there's generative AI and when students are coming in from very diverse backgrounds. We always end with the question: What's next? CATHY: So I have revamped my assessment for the module this year a little bit. So in previous years, I said to students, "I don't use AI. I suggest that you don't use AI. This is a class about nature. I'm an environmental researcher. It's not great. All those data centers, all that water, all that electricity. I just suggest we don't do it. But if you do want to, I'm not the cops.
I'm not going to try and police you and surveil you, but you do need to reflect on how you've used it so that we can have a conversation about that as part of the negotiation. So I've always done that. Increasingly, I just feel this isn't tenable anymore. So, in the UK, according to a recent HEPI report, 95% of our students are using AI. Can't pretend this isn't happening. And as a social scientist, I've been thinking increasingly about well, given that we live in this world, what's the social science framing?
How are we going to support students to think about AI as social scientists? So, in a sort of critical, ethical way. So, I'm starting to think: How do I support students to understand where AI is embedded in social, political, and economic systems? How do I help them understand that it's a probabilistic system, and therefore it is regurgitating assumptions that you might be making when you ask it questions, or that are embedded in the knowledge that already exists? And given that it's drawing on resources that are sexist, racist, all these things,
how do you engage with it in that way? How do you understand it as a device that reproduces worldviews? These are political scientists that we're talking to. So, is AI reproducing a liberal worldview, for example? Is it reproducing a view of the world in which humans are considered more important than the non-human world? And in a politics of nature class, do we agree with that or don't we? We might do, but we might not. And so, I am increasingly thinking, I can no longer say to students, I suggest you don't use it. So, coming out of a jokey conversation I had with them last year. So last year it became apparent to me that they were writing formative
assessments, and then they were asking AI what questions should I ask my teacher about this to make it better. And then those AI questions were coming to me, and then I was giving feedback, and then they were giving my feedback back to the AI to make the work better. I was like, "Whoa, I'm not sure this is what we want." So I asked them if this is what was happening, and we were in the art gallery. And I think sometimes in the art gallery, because of the way you move around, you can have a sort of better quality, honest conversation. And so I asked them, like, "Am I going mad? Is this what you're doing?" And we have this relationship. We all trust each other and I know their names.
They know that I am going to take them seriously when it comes to grading. And yet, suddenly, they're all shuffling their feet. They won't catch my eye. They are giggling a little bit, and I'm like, "What's going on?" And I said to them, "Listen, don't worry. I'm not the cops." And this kind of ripple of laughter went through the room. And I think there's something about this juxtaposition of like, are you a cop or are you a teacher? Which is so ridiculous because you can't be both. Clearly, they're very different jobs. That it just made them laugh.
But it also brought to the surface then a really interesting conversation where they talked about the fact that they know how their teachers talk about them. They know that faculty are going around saying these students they're cheating and they're using AI and none of our exams are secure anymore and they're trying to get away with things and they don't do the work and they don't turn up to class and they don't do the reading and they know we say this about them they're also on social media they overhear, they get those messages from their teachers and they don't particularly like it but they're also confused because they know that AI is coming to murder us all in our beds, [LAUGHTER] and they can't
get a job unless they interact with AI. And AI will be interviewing them, and they will need to prove that they can use AI really well in order to enter the job market. But also, they're told that if you interact with it in any way at all, then you're cheating and you're a bad person. What are they supposed to think? And so I was like very moved by that conversation. So I've changed my AI policy now, and my AI policy now basically says AI is bad for nature. Data centers are bad for nature. Lots of things are bad for nature. Building houses in London, where we've got housing crisis, is bad for nature.
And paying a carbon levy on your flight to plants for spruce is bad for nature. Lots of things we do are bad for nature. And actually, if you want to use AI in this class, I am not going to stop you because I think you're using it anyway. There's no point having a kind of a whole set of lies in between us. So you can use it if you want to, but you will not be disadvantaged if you don't want to because that's the way the assessment is designed, to value process over product. So there's no disadvantage if you don't want to use it for reasons of nature or otherwise.
But if you do want to use it, the only condition is you write about how you're using it every week in your learning journal, and then we have conversations about using it critically, ethically, and well. In which we're all going to have to learn together because I'm not an AI expert, but we're all going to figure out how is this technology working, and how could you do better, either by using it or by not using it at different key moments? And so, in order to do that well, I've also then rejigged the portfolio, so they're no longer just writing anything they ever want each week. They're having to write a learning journal alongside their own essays, which
maybe might add to their workload a bit. But I'm saying to them, just like an hour every week. It doesn't have to be major, and so we can then hopefully have those conversations in class. So that's my exciting new challenge for the term ahead. JOHN: It sounds like a great policy. REBECCA: Yeah, and full of metacognition. [LAUGHTER] CATHY: Yeah, yeah. So supporting them to do that is also difficult, isn't it, right? It's not very easy to reflect on your learning journey. So that's all part of it. JOHN: But in a world where students will continually learn to develop new skills in response to evolving technology, improving their ability to
reflect on their learning to learn more efficiently is one of the most valuable things I think we can teach students. CATHY: Yeah, I hope so. Yeah. JOHN: Well, thank you for joining us. This was a great conversation. It touched on a lot of things we haven't addressed recently. One other thing I want to mention is we've only had two podcasts now involving people from UCL. One of them was recorded a few weeks before, and in class right now we're discussing the concept of utility. And on the opening module page I have in Brightspace, there's a picture of
Jeremy Bentham in his current environment as the Auto-icon, [LAUGHTER] so a lot of things came together recently. [LAUGHTER] CATHY: Fantastic, yeah, yeah. [LAUGHTER] It's very strange to walk past Jeremy Bentham's mummified corpse every day, [LAUGHTER] but somehow we've got used to it. [LAUGHTER] JOHN: He was an interesting person. CATHY: He really was. [LAUGHTER] REBECCA: Well, thank you so much. CATHY: Thank you so much for the great questions. Thank you for having me. [MUSIC]
JOHN: If you've enjoyed this podcast, please subscribe and leave a review on Apple Podcasts or your favorite podcast service. To continue the conversation, join us on our Tea for Teaching Facebook page. REBECCA: You can find show notes, transcripts and other materials on teaforteaching.com. Music by Michael Gary Brewer. JOHN: Editing Assistance provided by Madison Lee. [MUSIC]
Most students who enter community colleges leave without completing a degree or other educational credential within 6 years. In this episode, Kelli Bird joins us to discuss her study examining the effects of embedding...
Institutional assessment of general education often involves faculty and institutional research offices, but rarely involves direct student input. In this episode, Jessamyn Neuhaus, Amanda Johnson Sanguiliano, and...
The transition from high school to college can be challenging for students, especially for first-generation students and students with weaker preparation for this transition. In this episode, Timothy McKay joins us to...
Discussion boards in learning management systems, particularly in the age of AI, rarely capture the authentic voice of students. In this episode, Greg Campbell joins us to discuss how the use of a chat application can...
Many past guests have emphasized the role curiosity can play in stimulating student learning. In this episode, Yaniv Abir discuss his recent study examining the impact of satisfying initial curiosity on subsequent...