University-wide retention work needs more than one metric
Updated Aug 25, 2026
Retention initiatives are easy to launch and hard to evaluate. A higher continuation rate might reflect the intervention, stronger attainment, a different cohort, or several changes happening at once. Sara de Freitas, Guy J. Curtis and Stephen M. Ritchie's Journal of Further and Higher Education paper, "A multi-layered evaluation of university-wide education interventions to improve student engagement and retention", shows what a more credible evaluation can look like. For universities using student voice to guide retention work, its message is practical: connect what students report with what they do and whether they continue.
Context and research question
Universities often evaluate a retention initiative through the measure closest to hand: attendance, survey satisfaction, pass rates, or re-enrolment. Each can reveal part of the story, but none explains the whole chain from experience to outcome. This matters because institutional conditions can predict engagement more strongly than student background, yet teams still need a defensible way to test which changes help.
The study evaluated two concurrent programmes at a small-to-medium Australian university. Supplemental Instruction used peer-assisted study sessions. A broader School-Led Teaching Effectiveness Programme included welcome videos, mentoring, gamification, online laboratory work, learning-management-system support, writing software, forums, and assessment-planning tools. The programmes reached 2,990 students, or 65% of the first-year cohort, across all 24 discipline areas.
The authors combined three layers of evidence. First, students completed an 18-item Need Satisfaction and Frustration Scale early and late in the semester, producing 3,439 usable survey responses. The measure covered autonomy, relatedness, and competence. Second, the team linked survey evidence with attainment, demographics, attendance, campus activity, learning-platform use, and subsequent enrolment. Third, it modelled the financial value of improved progression. The research question was not simply whether students liked an intervention, but whether changes in experience aligned with engagement, continuation, and institutional benefit.
Key findings
Students' reported experience improved over the semester, including outside the intervention groups. Relatedness and competence satisfaction were higher later in the term, while several forms of need frustration were lower. That general movement is important because it prevents a simple before-and-after claim. Some improvement may have reflected the passage of time or wider first-year adjustment rather than a specific programme.
Supplemental Instruction produced the clearest pattern of benefit. Students in these units reported greater relatedness and competence satisfaction at the later survey point than students in comparison units. Students in the wider teaching-effectiveness programme also showed signs of improvement, including lower relatedness frustration. The authors report an average 8% progression gain for Supplemental Instruction, although the design does not establish that peer-assisted study caused the whole difference.
Academic performance remained the strongest predictor of re-enrolment. Grade point average came first in the classification analysis. Beyond attainment and demographic factors, relatedness satisfaction at the later survey point, attendance at Supplemental Instruction, the number of sessions attended, and enrolment in an intervention unit were also associated with retention. This puts belonging and support in context: they mattered, but they did not replace the central importance of students passing their studies.
The financial model made retention evidence legible to senior leaders. The university invested about $200,000 in the programmes. The authors estimated a first-year return of $486,320 from Supplemental Instruction and just under $1 million from the other interventions, using progression and domestic fee data. These figures belong to one Australian institutional context and depend on attribution assumptions, so UK teams should borrow the modelling approach rather than the amounts.
The paper is unusually clear about that limitation:
"causal inferences concerning the impact of interventions on engagement and progression must be made cautiously"
Students were not randomly allocated to units, survey response rates were about 15% early in the semester and 25% later, and the study evaluated a bundle of changes rather than estimating a clean effect for each one. The results support continued testing, not a universal prescription.
Practical implications
First, UK universities should design the evaluation before launching the intervention. Define the intended chain of change, such as peer support improving relatedness, relatedness supporting engagement, and engagement contributing to progression. Set baselines, comparison groups, survey points, and continuation measures in advance. That gives Student Experience and Planning teams evidence they can interpret, not a collection of favourable indicators assembled afterwards.
Second, institutions should combine behavioural outcomes with students' explanations. Progression, attainment, attendance, and platform activity show where patterns change. Open comments can show whether students experienced peer support, clearer expectations, stronger confidence, or new friction. This is where retention work needs belonging evidence rather than a single score, and where Student Voice Analytics can help teams compare recurring themes across cohorts, disciplines, and intervention groups. The benefit is a clearer account of why an initiative appears to work and for whom.
Third, teams should separate monitoring from attribution. A dashboard can signal that relatedness, participation, or continuation improved, but stronger claims require a credible comparison and careful treatment of other differences between students and units. The same discipline should apply to qualitative evidence: use a documented open-text analysis methodology and preserve links back to source comments. That makes the evaluation more useful for quality enhancement and more defensible in governance discussions.
Finally, universities should compare reach as well as average impact. A staff-intensive peer-support model may produce the clearest result but reach fewer students, while a welcome video may be inexpensive and widely available but deliver a smaller effect. Examine take-up, experience, progression, cost, and differences between student groups together. The practical gain is a portfolio of support that matches intensity to need rather than searching for one retention fix.
FAQ
Q: How can a university apply this evaluation approach to a new retention initiative?
A: Start with a short theory of change and choose one measure for each link in it. For example, measure whether students use peer support, whether their comments and survey responses indicate stronger relatedness or competence, and whether progression changes against a suitable comparison. Add cost and reach data so leaders can judge both effectiveness and scale.
Q: Does the study prove that Supplemental Instruction caused an 8% progression gain?
A: No. The study was quasi-experimental, students were not randomly assigned, and intervention units differed from comparison units. The reported gain is promising, but it should be read alongside the authors' warning about causal attribution, the relatively low survey response rates, and the single-university context. Replication with stronger comparisons would increase confidence.
Q: What does this mean for student voice in retention work?
A: Student voice should explain progression evidence, not sit in a separate survey report. Comments about belonging, confidence, academic support, assessment, and workload can reveal the mechanisms behind a continuation pattern. Teams should join those sources with clear consent, access, and review rules, following a student comment analysis governance checklist when sensitive feedback is involved.
References
[Paper Source]: Sara de Freitas, Guy J. Curtis and Stephen M. Ritchie "A multi-layered evaluation of university-wide education interventions to improve student engagement and retention" DOI: 10.1080/0309877X.2025.2560999
Request a walkthrough
Book a free Student Voice Analytics demo
See all-comment coverage, sector benchmarks, and reporting designed for OfS quality and NSS requirements.
- All-comment coverage with HE-tuned taxonomy and sentiment.
- Versioned outputs with TEF-ready reporting.
- Benchmarks and BI-ready exports for boards and Senate.
UK-hosted · No public LLM APIs · Same-day turnaround