NSS open-text analysis methodology for UK HE

Updated Sep 07, 2026

Student Voice's NSS open-text analysis groups comments into topics and measures sentiment within each topic. This page explains the source population, classification coverage, denominators, reporting thresholds and comparison limits behind the public research briefs.

Use it to check what a figure means before quoting it or comparing your institution with the sector. The public aggregate research and a private institutional analysis have different scopes. For help choosing an institutional workflow, see the NSS comment analysis buyer’s guide and Student Voice Analytics.

The 2026 public research briefs

Our NSS open-text insights hub publishes authorised aggregate analysis for 2018–2026. These research briefs are a Student Voice analysis of open-text comments, not the official Office for Students quantitative NSS results. They contain no raw comment text and should be read as sector evidence that helps a provider decide where to investigate locally, not as a diagnosis of an individual institution.

The public release uses deterministic supervised learning to identify topics and sentiment at sentence level. The approach is fixed across the release, so the same input produces the same output and results can be reproduced consistently.

• The source population is Office for Students NSS national undergraduate open-text comments. In 2026, 40,822 of 43,870 source comments were classified (93.1%); annual classification coverage across 2018–2026 ranges from 93.1% to 96.7%.
• A comment counts once in each topic mentioned by any of its sentences. Because one comment can cover several topics, topic shares do not add to 100%.
• Mention rate is the percentage of classified comments in the same year and population that mention the topic. Unclassified comments are not included in this denominator.
• The sentiment index is 100 × (positive probability − negative probability). We first average matching sentences within each comment and topic, then average those comment-level values. This prevents a long comment from dominating simply because it contains more sentences.
• A page is eligible for a full public brief only when the entity has at least 100 comments across 2018–2026 and at least three reportable topics or comparison cuts.
• A displayed topic or comparison cut needs at least 20 comments. A 2026-versus-2025 statement needs at least 30 comments in both years.
• The 2023 NSS questionnaire redesign is treated as a comparability break. Current-questionnaire results are shown for 2023–2026; 2018–2022 appears separately as historical context.
• Valid CAH3 codes outside the reviewed 107-page public catalogue remain in sector and topic-page calculations but do not create new subject-page URLs.

Pages that do not pass the publication tests retain their stable URL and explain why a full brief is unavailable. They are excluded from search indexing and from the XML sitemap until the evidence is sufficient. This avoids turning small samples into claims while preserving references and future continuity.

What these results cannot establish

• Classification coverage is not a measure of classification accuracy. A repeatable model can make repeatable errors; ambiguous language and unfamiliar expressions still need review.
• Analysing the supplied corpus does not make respondents representative of every student. Non-response, who chooses to write a comment and which comments remain unclassified can affect the patterns.
• A topic mention rate measures the share of classified comments mentioning that topic. It is not the percentage of all students experiencing a problem, nor an official NSS satisfaction score.
• A sentiment change does not establish that a university intervention caused it. Question wording, population mix and other contextual changes may contribute.
• Reporting thresholds reduce the risk of weak or disclosive outputs; they do not by themselves establish statistical significance. Interpret differences with their sample sizes and comparison context.

For citation, name Student Voice AI as the author of the analysis, identify the topic or subject brief, link to its stable URL and this methodology, and record the displayed data version and access date. Do not attribute these derived measures to the Office for Students as official quantitative results.

What "open-text analysis" means (in practice)

Open-text analysis turns NSS free-text comments into evidence teams can act on, not just quotes they can repeat. Done well, it shows what students are actually saying, where experience is breaking down, and which issues deserve attention first. In practice, that usually means:

• A taxonomy of themes and categories, so teams can see recurring issues consistently
• Topic-aware sentiment analysis, so positive and negative patterns are separated with appropriate caveats
• Priorities, so teams know which issues are both frequent and negative
• Evidence packs, so boards, TEF panels, and programme teams can trace claims back to comments

The payoff is simple: teams stop relying on isolated quotes and start working from patterns they can prioritise, explain, and track over time.

A defensible workflow (step-by-step)

1) Define scope and inclusion rules

Set the rules before you look at the outputs. That keeps later conversations focused on action instead of arguments about what was counted, and it makes year-on-year comparisons easier to defend.

• Which survey(s): NSS only, or NSS + module evaluations + PTES/PRES/UKES?
• Which populations: UG only, or include PGT/PGR where relevant?
• What counts as "in scope": duplicates, empty strings, sarcasm/jokes, multi-issue comments.

2) Prepare data (minimal spec)

If the input table is inconsistent, every downstream chart becomes harder to trust. A clean base table makes later cuts by subject, cohort, and unit usable, and it stops teams from rebuilding the same filters later.

At minimum, your table should include:

• comment_id, comment_text, survey, survey_year
• organisation/unit fields (school/faculty/department) where permitted
• discipline fields (CAH/HECoS) where available
• cohort fields (level, mode, domicile group, etc.) where policy allows

3) Apply redaction and privacy controls

These controls let you share findings safely, not just analyse them internally. They also make it easier to brief leaders with confidence without creating unnecessary risk. For a practical control list, use the student comment analysis governance checklist.

• Decide what personal data is in scope to remove (names, emails, phone numbers, identifiers).
• Define small-cohort handling rules (roll-ups, multi-year aggregation).
• Document retention and access policies (least privilege).

4) Categorise comments (repeatably)

Repeatability is what separates governed reporting from a one-off interpretation. It lets you rerun the analysis, compare years fairly, and explain changes without relying on memory or informal judgement.

• Prefer stable, documented categories (with examples).
• Track coverage: the percentage of comments assigned to a meaningful theme.
• Track drift: if your categories change year to year, keep a mapping and change log.

5) QA and traceability

Even strong theme labels are hard to use if nobody can verify them later. QA and traceability turn a plausible result into evidence teams can rely on, especially when findings are challenged in formal settings.

• Human QA: sample checks, edge cases, disagreement review.
• Traceability: every headline claim should link back to supporting comments (anonymised).
• Versioning: record model/prompt/version so results are reproducible.

Reporting: what good outputs look like

Good reporting should help teams decide what to fix next, not just describe what students said. The best outputs shorten the distance between comments, decisions, and action, so insight turns into an improvement plan.

• A small set of headline themes (highest-volume and most negative)
• "What changed vs last year", to separate real shifts from cohort-mix artefacts
• Benchmarked views where possible (by discipline and cohort), so teams can tell whether a pattern is local or sector-wide
• A short actions section, so teams know what to change next term and what needs longer-term work

Where tools usually fail (and what to validate)

A platform can look impressive in a demo and still fail when teams need defensible reporting. Validate these points before you commit to a workflow, especially if the output needs to stand up in QA, enhancement, or TEF settings.

If you are comparing platforms rather than building a workflow in-house, our guide to text analysis software for education sets out where desktop, cloud, and HE-specific tools fit.

• Low coverage ("too many uncategorised"), which leaves teams guessing about what was missed
• Generic categories that don’t map to HE reality
• No benchmarking, or benchmarking with unclear methodology
• Weak governance (no audit trail, unclear data pathways)

If you’re considering generic LLM workflows, compare them against the governance standard you will need later, not just the speed of a first draft. Start with Student Voice Analytics vs generic LLMs. Then see how Student Voice Analytics helps teams move from raw comments to reproducible, benchmark-ready reporting without weakening methodology.