Engagement scores can mask student change
Updated Aug 26, 2026
An engagement score can describe a cohort and still mislead you about what changed for an individual student. Chun Cao, Takacs Tamas and Ronghua Liang's new Studies in Higher Education paper, "Engaged and then committed, or vice versa? Longitudinal links between academic engagement and student commitment during the first college year", shows why that distinction matters. For UK universities using student voice to understand first-year participation and retention, the message is practical: stable differences between students are not the same as change within students over time.
Context and research question
Engagement and commitment are often treated as parts of one simple sequence. A student participates, becomes more connected to academic goals and the institution, then becomes more likely to persist. The reverse could also be true: students who already feel committed may invest more attention, effort, and emotion in their studies. Universities need to understand the direction because it affects whether they focus on participation, institutional attachment, or both.
The paper tests those relationships across three dimensions of academic engagement, behavioural, cognitive, and emotional, and two forms of commitment, commitment to academic goals and commitment to the institution. It uses three waves of survey data from 1,986 first-year students, whose mean age was 18.63 years. The authors compare a conventional cross-lagged panel model with a random-intercept cross-lagged panel model. The first estimates patterns between students over time; the second separates stable differences between students from change within the same student.
That methodological distinction is important for UK teams. Evidence that students with stronger engagement also show stronger commitment does not prove that raising one student's engagement will later raise that student's commitment. It is the same measurement problem universities face when institutional conditions and student characteristics both shape engagement: a pattern across a population can be real without describing how every person changes.
Key findings
At the between-student level, different forms of engagement had different relationships with goal commitment. Behavioural and emotional engagement predicted later goal commitment in one direction, while cognitive engagement and goal commitment predicted each other. This suggests that participation, emotional investment, and mental effort should not be collapsed into one engagement score if teams want to understand what the score means.
Institutional commitment appeared to precede some forms of engagement. In the conventional models, institutional commitment predicted later behavioural and cognitive engagement. Its relationship with emotional engagement ran both ways. A student who feels attached to the university may be more willing to participate and invest effort, while positive emotional engagement may reinforce that attachment.
The most important result is that these relationships did not hold at the within-student level. Once the random-intercept models separated stable differences between students from fluctuations within each student, the earlier cross-lagged effects were not reproduced.
"stable inter-individual effects were not replicable at the within-person level"
This does not mean engagement and commitment are unrelated. It means the paper found stronger evidence for persistent differences between students than for a reliable sequence in which a rise in one measure preceded a rise in the other within the same person. That is a much more cautious finding than a simple claim that engagement causes commitment, or commitment causes engagement.
The study therefore changes how longitudinal survey results should be read. If an engaged group remains more committed across three waves, a standard analysis may suggest a developmental pathway. The within-person analysis asks a harder question: when a particular student becomes more engaged than usual, does their commitment rise later? In this study, the answer was not reliably yes. Similar caution is needed when survey comparisons across time appear to show changes in belonging.
Practical implications
For UK universities, the first implication is to avoid turning group differences into individual stories. A cohort with high engagement and high commitment may differ in prior confidence, course fit, support, expectations, or other stable characteristics. Describe the association, but do not assume an engagement intervention caused later commitment unless the design can test that claim. This gives decision-makers a more defensible basis for retention work.
Second, institutions should measure engagement as more than one construct. Behavioural participation, cognitive investment, and emotional connection showed different relationships with goal and institutional commitment. A single headline item can conceal whether students are attending without thinking deeply, working hard without feeling connected, or emotionally positive without participating consistently. Separating these dimensions gives Student Experience teams clearer levers for action.
Third, teams should keep repeated measures stable and add qualitative explanation. Use the same core questions at several points in first year, document any wording changes, and add a concise open prompt such as, "What has most affected your willingness to take part in your course this month?" Student Voice Analytics can then group recurring explanations across waves while a clear student comment analysis governance process protects consistency, traceability, and appropriate use. This helps teams distinguish a movement in scores from the experiences driving it.
Fourth, universities should connect survey evidence with support and engagement data without treating either as proof of cause. Attendance, VLE activity, assessment submission, pulse-survey scores, and open comments each capture a different part of the first-year experience. Reviewing them together can reveal where a pattern is concentrated and where follow-up is needed, while preserving the difference between early warning and causal explanation. The benefit is earlier, better-targeted support without overstating what the data can establish.
FAQ
Q: How can a university apply these findings to first-year surveys?
A: Use two or three short, planned survey waves with stable measures of behavioural, cognitive, and emotional engagement, plus goal and institutional commitment where relevant. Add one open-text question at each wave, then compare both cohort patterns and changes among students who responded more than once. This produces a more useful view than a single induction snapshot or year-end average.
Q: Does the absence of within-person effects mean engagement initiatives do not work?
A: No. The study shows that the between-student relationships in its conventional models did not reappear as lagged within-student relationships in the random-intercept models. It does not test every intervention, mechanism, or timescale, and its survey design cannot establish simple causation. Universities should treat the result as a warning against overinterpretation, then evaluate local initiatives directly.
Q: What does this change about student voice practice more broadly?
A: It shifts the question from "Which students are engaged?" to "What is changing, for whom, and why?" Scores can identify patterns, but comments can explain whether movement reflects course fit, relationships, workload, confidence, support, or institutional attachment. Combining stable measures with structured qualitative analysis makes student voice more diagnostic and more useful for action.
References
[Paper Source]: Chun Cao, Takacs Tamas and Ronghua Liang "Engaged and then committed, or vice versa? Longitudinal links between academic engagement and student commitment during the first college year" DOI: 10.1080/03075079.2026.2626789
Request a walkthrough
Book a free Student Voice Analytics demo
See all-comment coverage, sector benchmarks, and reporting designed for OfS quality and NSS requirements.
- All-comment coverage with HE-tuned taxonomy and sentiment.
- Versioned outputs with TEF-ready reporting.
- Benchmarks and BI-ready exports for boards and Senate.
UK-hosted · No public LLM APIs · Same-day turnaround