Evaluations do not improve teaching, conversations do

Audio briefing based on Student Voice Weekly issue #7.

This Week

This week, the episode discusses evaluation dialogue, belonging data, and survey design. Feedback helps teaching only when staff have space to discuss it. The main topics are grouped below by student voice practice, research, sector developments, archive context, and practical application.

Main Topics Discussed

Student Voice Practice

It is Easter break for many teams this week, and I hope plenty of people are getting a well-earned pause before the final push of the academic year.

Research Spotlight

Sector Watch

From the Archive

Practical Application

Student complaints often include proposed fixes, not just frustration.

Subscribe to The Student Voice Weekly: https://www.studentvoice.ai/blog/newsletter/

Transcript

Hi, and welcome back to Student Voice Weekly. I'm Dr Stuart Grey, founder of Student Voice, and today's theme is student feedback that actually changes something: why evaluations do not improve teaching on their own.

Today I'd like to talk about a really common mistake in UK universities. We treat evaluation results as if they are self-explanatory. As if you can publish a dashboard, send a PDF to a course leader, and then teaching quality will automatically improve.

The key thing is this. Student feedback is not the intervention. The conversation you have after the feedback is the intervention. If you do not create the conditions for that conversation, all you have done is collect data and move it around.

Most staff are not ignoring feedback because they do not care. They are dealing with time pressure, marking, admin, and competing priorities. If the institutional process is, here is your report, now write an action plan, you get a rushed and defensive response. That is a process design problem, not a motivation problem.

So what does the research say.

The lead study this week looks at how teachers actually use evaluation feedback. And what comes through is that reports become useful through dialogue. Dialogue with students, with peers, with course teams, sometimes with a line manager. Teachers do not treat comments as instructions. They interpret them.

That interpretation work is practical and quite skilled. Staff look for patterns, they check context, and they separate three things: what is a one-off, what is a signal, and what is a misunderstanding that can be fixed with clearer communication. A report does not do that by itself. People do.

This is also where a lot of UK practice breaks down. We time evaluation collection so the data arrives when everyone is least able to think. And then we act surprised when nothing changes.

So if you want evaluations to improve teaching, ask an operational question. Where is the 45 minute slot, with the right people present, where someone can say, what are students actually saying here.

Without that, the failure modes are predictable. A team overreacts to one angry comment. Or they dismiss negative feedback as noise. Or a genuine pattern is spread across modules and nobody joins the dots. Or the only output is a comms line to students, we have reminded staff, and the same issue returns next year.

My judgement is that most universities do not have a feedback collection problem. They have a feedback sense-making problem.

So here is the practical implication.

If you work in quality, academic development, or student experience, stop treating evaluation as a one-stage pipeline: collect, analyse, publish. Build it as a two-stage process: collect, then interpret. And interpretation should not mean one person alone at a laptop. It means a short, structured conversation with a clear output.

A lightweight model is a 30 to 60 minute meeting soon after results land, focused on three prompts.

First, what are the recurring issues. Not the loudest issues, the recurring ones.

Second, what is within our control before the next run. Two or three changes you can actually deliver.

Third, what are we not changing, but we need to explain better, because students cannot see the rationale unless we make it visible.

Then you close the loop properly. Tell students: here is what we heard, here is what we changed, and here is what we are keeping and why.

Now, a second research signal this week is about belonging.

A systematic review looked across a large body of studies and found that belonging is being measured in a very fragmented way. Lots of different measures, often used once, and many not meeting the standards you would want for benchmarking or for making confident subgroup comparisons.

This is not an argument against measuring belonging. It is an argument against over-interpreting a single belonging score. If you put it on a dashboard and treat it like a stable KPI, you risk false precision.

Because in practice, belonging can mean different things for different students. It can be about social connection, academic confidence, being recognised by staff, representation, safety, or access to support. Those can move differently, and a single number can hide what is actually breaking.

So what should you do. Audit the items you use. Check where they came from and what validation evidence exists for your context. Then pair the scale with open text that tells you which part of belonging is not working. The actionable part is not the score. It is what students describe.

Now, sector watch.

Universities UK published five quality principles for maintaining and enhancing quality in challenging times. One point worth pulling out is the emphasis on using data on engagement, attainment, and feedback to evaluate the impact of changes and respond quickly.

This matters because institutions are making efficiency decisions right now. Portfolios are being reviewed, services redesigned, assessments standardised. The risk is that student feedback is only used afterwards, as reassurance. Did students complain, did the scores drop, can we defend the decision.

A better use is earlier, when options are being shaped. Student voice evidence should be visible at the point trade-offs are made, not just when decisions are justified. So if you are doing redesign work, make sure someone in the room can say: here is what students consistently report in this area, and here is what tends to break when we change X.

And one more practical sector item. Jisc Online Surveys changed some question types and issued a fix affecting drop-out data in Insights.

This is easy to ignore, but it matters. Data quality can be undermined before analysis even starts. If templates behave differently, or question types do not work well on mobile, you get messy responses and inflated drop-out. So if your institution uses Jisc templates, review them, especially where departments build their own forms. It is unglamorous work, but it protects the credibility of your evaluation system.

Finally, a practical way to handle free text comments.

When you read comments, separate them by what they are doing.

Type one is diagnosis: what happened. Feedback was late. The brief was unclear. Seminars felt unstructured.

Type two is interpretation: what students think caused it. Sometimes that is right, sometimes it is wrong, but it tells you how students are making sense of their experience.

Type three is proposed fixes: what would have helped. This is often the most useful part for a team conversation.

If you just theme everything into a bucket like assessment and feedback, you do not get to action. If you separate diagnosis, interpretation, and proposed fixes, you can run a better conversation. You can say: here is what students experienced, here is how they interpreted it, and here is what they are asking for. Then staff can decide what to accept, what to redesign, and what to explain.

Right. One thing to try this week.

Build a 30 minute post-survey review ritual for each programme or module team, with a simple structure and a tangible output.

Bring one page.

Section one: the top three recurring issues, with a couple of anonymised quotes.

Section two: one change you will make before next delivery, and who owns it.

Section three: one thing you are not changing, but the explanation you will give students next time.

Then use that page for your loop-closure message to students, and as a handover note for the next person teaching. The meeting is not the win. The follow-through is the win.

That is it for this week. The full set of links and summaries is in Student Voice Weekly. If you work with student feedback and want the research, regulation and sector signals in one place each week, you can subscribe at studentvoice.ai.