Good scores still need the comments

Student Voice Weekly, episode 38, with Dr Stuart Grey. Published 13 November 2026.

About this episode

Students can value their course and still suggest useful changes. Dr Stuart Grey explores research linking survey ratings with written comments, practical institution-wide analysis, and the latest Office for Students examples of AI-assisted assessment feedback. He also follows up last week's discussion of Excel reports with a look at how generative summaries help colleagues understand the wider picture while keeping the original comments available.

In this episode

What students with higher estimated satisfaction still wanted to improve.
Bringing together relevant comments across the institution, with survey questions, periods and local context retained.
Comparing school and programme views while keeping group size, coverage and response rates in mind.
Keeping assessment feedback accurate, relevant to the work and led by the lecturer when AI is involved.
Summarising comments already grouped by category and sentiment, alongside the Excel reports.
Classifying each sentence into one or more categories, then rolling the results up to the comment without losing mixed experiences.
Returning from a summary to the filtered source comments to check what students actually said.

Chapters

00:00 - What happy students still want changed
00:38 - Research: reading ratings alongside comments
01:27 - What satisfied students wanted to improve
02:11 - Institution-wide analysis and local context
03:33 - OfS examples and AI-assisted assessment feedback
04:34 - How generative summaries complement Excel
05:41 - Sentence-level categories and mixed sentiment
06:21 - Deterministic classification and transparent summaries
06:53 - Worked examples and closing

Research and sector sources

The Italian study's overall survey had 5,280 respondents; the university-experience text analysis used 1,355 respondents with the required satisfaction estimates and comments. It is observational evidence from one institution, with self-selected participation.

The OfS examples were published on 11 November 2026 under existing quality conditions in England. Stuart's suggestion of using AI to improve the presentation of lecturer-written or dictated feedback is his practical example.

Explore the approach

Transcript

Hi, and welcome to Student Voice Weekly, episode 38. I'm Dr Stuart Grey, founder of Student Voice AI, and today's question is: what do happy students still want changed? So, good scores in a survey can end a good discussion about issues at universities too early. So students can value their course and still see room for improvement.

So this week, we're looking at some research on this topic, an OfS update, and some tips on practical institutional analysis. We're also going to pick up last week's conversation about use of Microsoft Excel and bring in our generative summaries.

So first of all, the research. So Annalina Sarra and colleagues at an Italian university studied a large survey where they took the ratings alongside students' written comments. There were separate questions asking for the strengths of the course and also for things to improve.

There were over five thousand respondents in the overall survey and just over thirteen hundred comments from students who were remaining after they were mapped against satisfaction. So this smaller group had to have the satisfaction scores, estimates, and the full text responses for both questions.

So we have about thirteen hundred samples from the survey, which is good. And then from this, the researchers estimated satisfaction from the questionnaire answers and they themed and categorised all the comments and compared them across different groups.

So what did the students say? So in the more positive accounts, they talked about relationships, friendships, and personal growth. In the suggestions for improvement accounts, they talked about exams, administration, classrooms, and organisation.

So what it shows is that the higher satisfaction group still have suggestions for change. They were things like spaces, like study spaces, schedules materials, learning resources, et cetera, among those suggestions from people who are very satisfied. So there's still practical improvements we can make for these students.

So at a university-wide meeting, we can't just focus on what the students are complaining about, but also on what we can improve for all students.

So as institutional research and surveys teams, we bring together lots of relevant survey comments. So it's normally from across the institution. We want to look at the broadest range. Again, we can focus too much on the problems in the problem programmes when there may be opportunities we can really big lift across all students.

So to do this, we need to have really accurate data on the survey questions used, the period, the faculty, school or department or programme context. We need the timing and if we have all this, then we can dig into it and look, okay, what are the cross-cutting issues across everything, across students who are both happy and unhappy, and what can we do to fix them? Because then we can really see the full picture.

If we focus too much on just the negative comments, we can lose the sight of the wood for the trees.

This wide-scale analysis also lets us compare different relevant school and programme views. We can see where issues are concentrated in a particular context. We can think... But we've got to keep in mind sort of group size and coverage, response rates.

And ultimately, as I say every week, we've got to go in there and read the actual comments and see what the students are actually talking about. So the categorisation sentiment analysis is vital, especially with large-scale analyses to let us know what we're doing, but it's just a means to an end to get us to reading the right comments to better understand what our students' issues are.

So, also this week, there was a OfS update giving examples of institutional AI practice. This was mapped to the existing quality conditions in England, and there are lots of examples under those conditions. It's not like a new AI-specific condition.

So in particular, it concerns AI-assisted assessment feedback given to students. So we've got to be really careful here. We need to make sure it's accurate and aligned with the assessment criteria, tailored to the student's actual work, and specific about what it can improve.

And fundamentally, I personally believe you can't have an AI just create this out of whole cloth. It needs to be guided and led by the lecturer in practice.

So for example one way to do this would be to, okay, quickly take lots of notes or dictate the comment and have an AI tool smarten it up, right? Make sure all the grammar is correct, correct things, right? Make sure you're hitting the right notes, but have it fundamentally be from still the lecturer's feedback on the actual work.

Then also last week, we talked about Excel reports that give colleagues all the data and the underlying comments, so access to everything.

And a part that we didn't have time to talk about last week was that we also generate summaries. So using generative AI again, but this time to create summaries of comments under specific themes that complement those Excel sheets really well.

Now, a generative summary isn't going to replace sort of the fundamental work of reading the comments and understanding what the students really talk about, but it gives you a really good idea of the wider context without having to read tens of thousands of comments.

So with our summaries, we take all the comments. They've already been categorised and grouped by sentiment, which gives us a massive advantage. So then we can just take the comments about a given topic, whether it be the given sentiment, valency, and then summarize those. And it lets us give really rich, accurate summaries that are quite precise, but at school, department, programme level, which we've found that people in our customer universities absolutely love them at those sort of levels of the institution.

So finding, maintaining that level of detail is really important. So to maintain that level of detail, we always classify all of our comments at the sentence level.

Then each sentence can have multiple categories and then those sentence results are brought together, rolled up onto the comment level. This way, we can preserve those different experiences within the same response. We can have different sentiments within the same comment, right, across sentences. And this allows us to really maintain that richness. We're not flattening anything.

So this is all available in the Excel sheets and allows us to create those really cool generative summaries that allow us to have a different, a higher level view.

Again, all our classification is deterministic, so that means if you give it the same sentence, it will give you exactly the same answer. This is really, really important, and that gives a really strong bedrock for our summaries.

And as ever, anyone who has access to the summaries also has access to the spreadsheet. So if they see something that catches their eye in a given category in a positive or negative section, they can dive into the spreadsheet, filter by that category, filter by positive sentiment, and immediately read all the comments that went into that summary. So it's fully transparent.

So thank you very much for listening today. If you'd like to find out more, please go to the Student Voice AI website where you can find more about the research and OfS updates.

You can find worked examples. We've got a new page that shows some worked examples using synthetic comments that shows the different ways we analyse sentences, roll them up, categorize, do sentiment, et cetera.

And please do get in touch if you are interested in discussing more about how to analyse your university's comments.

And please, if you did find this podcast useful, please do share with other colleagues in institutional research, or surveys, or quality teams. Thank you very much