What students actually mean when they talk about AI

This week, Dr Stuart Grey discusses why universities need sharper AI-related student feedback questions, especially as guidance moves from broad policy statements into modules, assessments, and local teaching practice.

The episode covers AI attitude survey design, Jisc's latest AI guidance and governance work, student mini-publics, and how universities can separate AI comments by usefulness, clarity, confidence, fairness, and support need.

In This Episode

Why broad "do you use AI" or "are you concerned about AI" questions no longer give universities enough evidence to act on.
Why student AI attitudes need to be separated into usefulness, identity and confidence, and concern.
How module-level and assessment-level AI guidance changes what universities should ask students.
Why human-in-the-loop governance matters when AI is used to summarise or interpret student comments.
How to code AI-related comments so teams can distinguish unclear rules, low trust, fairness concerns, and support needs.
Why feedback loops matter when students are asked for views on fast-moving institutional decisions.

Student Voice Practice

AI feedback should not be treated as one broad theme. A useful analysis should distinguish whether students are talking about practical usefulness, unclear rules, confidence using tools appropriately, fairness between students, detection and false positives, staff inconsistency, or the need for better support.

Research Spotlight

Across the Sector

From the Archive

Practical Takeaway

Dig into what students mean when they talk about AI. Separate usefulness from confidence, confidence from clarity, and clarity from concern, then tag comments in a way that shows whether the problem is unclear rules, low trust, or support needs.

Subscribe to The Student Voice Weekly: https://www.studentvoice.ai/blog/newsletter/

Transcript

Hi, and welcome to Student Voice Weekly. I'm Dr Stuart Grey, founder of Student Voice, and today's theme is a fairly simple one: one AI question is not enough.

Today I'd like to talk about why that matters, especially as we get close to NSS results day and start thinking about how universities will read this year's feedback alongside everything that is happening around AI.

The thing I keep coming back to is that a lot of AI questions look sensible when they are written down, but fall apart as soon as you imagine a real conversation with students. You ask, do students use AI. Or, are students worried about AI. Or, do students think AI is useful.

Those are not terrible questions. They are just too flat.

If you sat with a group of students after a seminar and asked them what they think about AI, you would not get one attitude. You would get a mixture. Some students would say it helps them get started. Some would say it helps them understand difficult material. Some would say they are using it because they assume everyone else is. Some would say they are worried about accidentally crossing a line. Some would say the rules change from one module to another. Some would say they trust one lecturer's guidance and are much less sure about another's.

So the key thing is that "AI" is not the object students are really responding to. They are responding to usefulness, confidence, fairness, risk, identity, and the judgement of staff.

That is why broad AI survey questions can be a bit misleading. They give you a number, but they do not tell you what conversation needs to happen next.

This matters most in assessment. I still teach part-time at the University of Glasgow, and in teaching settings you can see how quickly AI stops being an abstract policy topic. Students are trying to work out what good practice looks like. Staff are trying to give guidance that is fair, realistic, and academically defensible. Everyone is doing that in a situation where the tools keep changing.

So when students talk about AI, they are often talking about uncertainty. They are asking, will this help me learn, or will it get me into trouble. They are asking, is this allowed here, or only in another module. They are asking, if I use it honestly, will I be treated fairly. And they are asking whether there is still a person making a careful academic judgement at the end of the process.

The research worth using this week is helpful because it gives us a cleaner way to think about those mixed responses.

The first paper in the newsletter is by Li and Wang, and their work on student attitudes to AI suggests that we should not treat AI attitudes as one thing. Their scale separates utility and knowledge, value expression, and ego defence.

Now, those terms sound a little formal, but the underlying point is quite intuitive.

Utility and knowledge is the practical side. Does AI help me do something. Does it help me understand the topic. Do I know enough to use it sensibly. Can I use it without making my work worse.

Value expression is about how the student sees themselves. Am I the kind of student who uses these tools. Does using AI make me feel capable, up to date, resourceful, or perhaps uncomfortable because it does not fit my idea of what studying should be.

Ego defence is about protection. Am I worried about being judged. Am I worried about being accused. Am I worried the rules are unfair. Am I worried that my own work will somehow become less mine.

That distinction is useful because it explains something universities are going to see more and more: usage can go up while unease stays high.

A student can use AI every week and still feel anxious about it. They can find it useful and still feel uncomfortable about authorship. They can feel confident with the tool and still distrust the process around detection, marking, or academic integrity.

So if a survey asks, "are you concerned about AI?", the answer might be yes. But the follow-up is the part that matters. Concern about what. Concern about fairness. Concern about unclear rules. Concern about competence. Concern about the loss of human judgement. Those are different academic and institutional problems.

Across the sector, the same question is showing up in guidance. Jisc's June HE AI meetup is a useful signal here. The discussion is moving away from the big, general question of whether AI should exist in higher education, and towards the smaller but much more difficult question of what guidance looks like in a particular assessment, in a particular module, with a particular group of students.

That feels right to me. Students do not experience policy as a PDF on a website. They experience it through the essay brief, the Moodle page, the throwaway comment in class, the marking criteria, and the conversation they have with a tutor when they are unsure.

So if we want to know whether AI guidance is working, we have to ask students at that level. Did you understand what was allowed in this assessment. Did the examples make sense. Did different staff say the same thing. Did you know where to ask. Did the guidance make you more confident, or just more cautious.

The other Jisc item this week, on human in the loop governance, links to the same question from the institutional side. If universities are going to use AI in student feedback analysis, then there needs to be some clear human judgement in the process. Who checked the summary. What comments were included. Were small groups of students lost in the averaging. Was the interpretation challenged by someone who knows the context.

That is not just a technical question. It is an academic question about interpretation. If students are expected to be transparent and careful in how they use AI, universities should be transparent and careful in how they use AI to interpret what students say.

This is where free text comments really matter.

If I were looking at AI-related comments, I would be very wary of one big code called "AI concern". It might be convenient, but it hides too much.

Some comments will be about clarity. Students saying, I do not know what is allowed, or different lecturers say different things, or the assessment brief is vague.

Some comments will be about fairness. Students worrying that others are getting an advantage, or that detection tools will be wrong, or that honest use will be treated suspiciously.

Some comments will be about learning. Students saying AI helped them understand a difficult idea, or that it made their writing more fluent, or that it helped them prepare but did not replace the work.

Some comments will be about identity and confidence. Students feeling they ought to know how to use AI, but are embarrassed to ask. Or students feeling that using it somehow makes the work less authentic.

And some comments will be about staff judgement. Students wanting to know that there is still a person reading, thinking, and making a fair decision.

Those are all AI comments, but they are not the same kind of comment. The response is different in each case. Sometimes the answer is better examples. Sometimes it is more consistent module guidance. Sometimes it is a conversation about assessment design. Sometimes it is reassurance about human review. Sometimes it is simply making space for students to say they are unsure without feeling foolish.

The second paper in the newsletter is by Simon Pek and Jeffrey Kennedy, and it makes a related point about student mini-publics. Participation is not the same as impact. A student mini-public can be a very thoughtful, serious process, but the wider student body still wants to know what changed afterwards. Who listened. Who acted. How were the recommendations explained.

The same is true of AI feedback. If students tell you they are confused, or worried, or making quiet use of tools without knowing whether they should, then the important thing is not just that the university has collected the data. The important thing is whether the conversation changes.

So the takeaway is this: ask AI questions in a way that respects the complexity of the answer. Separate usefulness from confidence. Separate confidence from clarity. Separate clarity from fairness and trust. And when you read the comments, keep enough of the texture that you can hear what students are actually saying.

That is it for this week. The full set of links and summaries is in Student Voice Weekly. If you want the research, regulation, and sector signals in one place each week, you can subscribe at studentvoice.ai. And if this was useful, please share it with someone who works on student experience, teaching, or assessment, or leave a review in your podcast app.