Response rates are not a shortening problem

Audio briefing based on Student Voice Weekly issue #2.

This Week

This week, the episode discusses evals, OfS pulse, QAA toolkit. Incentives beat shortening surveys for evaluations, plus OfS pulse results The main topics are grouped below by student voice practice, research, sector developments, archive context, and practical application.

Main Topics Discussed

Student Voice Practice

I think March is when student voice collection collides with operational reality: module evaluations, NSS messaging, and committees all want usable evidence at the same time.

Research Spotlight

Sector Watch

From the Archive

Subscribe to The Student Voice Weekly: https://www.studentvoice.ai/blog/newsletter/

Transcript

Hi, and welcome to Student Voice Weekly. I'm Dr Stuart Grey, founder of Student Voice, and this week I'm talking about response rates and trust: why students do not fill in evaluations, and what actually shifts the dial.

Today I'd like to talk about the default plan I see across UK universities at this time of year. Module evaluations are coming. NSS messaging starts to appear. People need evidence for committees. And the fix, quite often, is to shorten the questionnaire, tidy the email, send another reminder, and hope.

The key thing is this. That treats response rate like a friction problem, as if students are basically willing and just need less hassle.

I do not think that is what is going on. This week's research and the sector signals point to something more uncomfortable, but also more actionable. Response is mostly about value exchange and credibility.

So, the main research finding.

There is an experiment with 1,061 undergraduates looking at what increases willingness to complete teaching evaluations. The headline is simple.

Incentives increased willingness much more than shortening the survey. Messaging about impact helped a bit. Shortening did not meaningfully move the needle.

Now, it matters to be clear about what this is and is not. This is a hypothetical scenario experiment, not a live institutional trial measuring actual completion. So it does not let you forecast your response rate next semester.

But it is still a useful reality check on what students are optimising for. When they decide whether to respond, they are not primarily thinking, how many questions is this. They are thinking, is this worth my time, and do I believe anything will happen if I do it.

That is the value exchange.

In UK higher education we often imply the value exchange is moral. We suggest students should do it because it helps quality, or because it is part of being a good member of the academic community.

Some students buy that. Plenty do not. And the ones who do not are not necessarily disengaged. They might just be unconvinced, or they might have learned from experience that nothing visible changes.

This is where universities get stuck. We keep trying to engineer response rates without earning them. We focus on the instrument, not the relationship. We focus on the number, not the credibility.

So if you are sitting in a meeting thinking, we need response rates up, my first question is not, how short is the survey. It is, what are we offering students that is credible.

Not a slogan. Not a general promise. A credible exchange.

A practical way to do that is straightforward. In the invite, show one concrete "you said, we did" change from that programme or module. Make sure it is specific and recent. Then tell students, in plain language, what will happen with the results and when.

This is what students are actually saying, especially in free text. They are not only judging the module. They are judging whether the system listens. If they think it does not, they opt out.

The research also tested messaging that evaluations affect lecturers' careers. That increased willingness, but less than incentives. In UK terms, I would be cautious with that message. It can sound like emotional pressure, and it can make students more strategic. Neither helps you get better data.

If you are going to talk about impact, lead with student-facing change. The boring operational stuff that students actually notice.

Now, the second research piece in the issue helps explain what sits underneath this.

It is a study of taught postgraduates looking at when students experience themselves as learners versus customers. The interesting point is not that customer language exists. It is what triggers it.

In that study, a minority, around one in seven, primarily identified as customers. And the value for money framing was often triggered by negative experiences or social cues, not fees alone.

In practice, it is not simply, they pay therefore they think like customers. It is, when something goes wrong, the interpretation shifts. They stop interpreting the experience through learning, and start interpreting it through transaction and trust.

There is also a feedback-system warning that matters. When dissatisfied, only a small proportion used formal teaching evaluation surveys. Much more dissatisfaction circulated privately, with friends, or indirectly with staff.

So if you are relying on end-of-module surveys as your early warning system, you are going to be late. By the time the survey tells you there is a problem, the trust shift has already happened.

That brings us to sector watch.

We have the Office for Students student pulse survey results for Autumn term 2025. The OfS is careful to say it is not used for regulatory decisions. But it still shapes the national narrative, and it will be quoted.

Two signals stood out.

First, a large majority said their experience matched what they were promised when they enrolled.

Second, a significant minority reported barriers to progression linked to personal characteristics.

Do not over-interpret one pulse survey. But the practical implication is clear. Term-time perceptions around promises and fairness are becoming part of the landscape alongside the big annual surveys.

So make sure you have a light, credible internal pulse layer with clear ownership. Something that can pick up early whether students feel mis-sold, or whether particular groups are hitting barriers that others are not.

The other sector signal is the QAA assessment literacy toolkit.

I like this because it points to a practical truth. A lot of assessment complaints are not really about standards. They are about expectations and shared language.

If students do not understand what good looks like, or how criteria translate into marks, you will get comments that sound like unfairness even when marking is technically sound.

A result worth noting is that many students were unfamiliar with the term assessment literacy, but most felt they could apply the toolkit. So the term is jargon, but the underlying need is real.

And this links back to what students are actually saying in evaluation comments. Often it is, I did not know what you wanted, the feedback did not help me improve, different tutors said different things. Those are legibility problems. You do not fix them by polishing a survey.

So what do you do with all this in the next few weeks, when evaluations go live.

I would separate three types of comment that often get mixed together.

First, effort and time. Students saying they are overloaded and deadlines are clustered. Real issue, but it needs programme-level coordination.

Second, trust and credibility. Students saying, nobody listens, it never changes, why bother. That is a feedback loop problem.

Third, fairness and legibility. Students saying expectations were unclear and feedback did not help them improve. That is often an assessment literacy and communication problem.

The reason to separate them is that the action owners are different. And if you treat all of them as a survey design problem, you will not fix any of them.

One more thing to listen for, especially with taught postgraduates, is the switch into customer framing. Words like value, service, paying, not worth it. When that appears, it often signals that something has broken in reliability or communication.

You do not respond by turning teaching into a corporate service script. You respond by being reliable, communicating clearly, and fixing avoidable failures quickly.

So, one thing to try this week.

Take the next evaluation invite, or reminder, and add two lines.

Line one: one specific "you said, we did" example from that programme or module from the last cycle.

Line two: an honest completion time, plus what happens next and when. For example, results go to the module team within two weeks, and we will post back two actions we are taking.

Then make sure you track not just response rate, but respondent mix. Are you hearing from the same students as last time, or are you broadening who responds.

Because the goal is not a bigger number. The goal is a dataset you can trust.

That is it for this week. The full set of links and summaries is in Student Voice Weekly.