The disconnect on what makes good feedback
By Andrew Carlin
Feedback is one of the most critically reviewed aspects of modern higher education courses in recent times, with studies showing that students frequently do not engage with feedback [1, 2]. In parallel, academic literature on the topic shows the basic definition is not agreed upon, and that feedback as a 2-way communication channel, ironically, is not enacted in many educational institutes, as students are simply given information, with little to no time to act upon it to improve their performance [1]. Yet, despite this, comprehensive studies have found that students do value feedback and that what students look for in feedback, is disconnected from what the literature and designers of feedback (educators), think they want.
Price, Handley and Millar say the feedback in its current form in higher education is simply a product, which is listened to, but not reflected upon, invoking at best a behavioural response rather than a cognitive one [3]. Meanwhile, McFadden and Munns assert that ‘student engagement is a process rather than a product' [3, 4]. Hughes has an interesting take on this lack of opportunity to action feedback, stating that trying to learn without reviewing is "like trying to fill the bath without putting the plug in!".
In a comprehensive study, Dawson et al studied the responses of 406 staff and 400 students from two Australian universities, from a large scale survey on feedback [5]. Overall, there was a distinct difference in what educators think makes good feedback, and what students do. The former felt that timing, communication channel and connected tasks were the key features of good feedback. They, the educators, who design feedback, focused on the design-heavy aspects of feedback. In contrast, the students strongly felt high-quality and actionable comments, which were detailed yet personalised, was what makes feedback useful. They, the students, who receive the feedback, were essentially focused on the quality of information given, not necessarily the structure, timing and framework around it. Critically, this is at odds with much of what is said in the literature and those who have proposed new methods for feedback in recent decades [1-3, 6]. Contradictory too is that both staff and students mostly had positive things to say about feedback.
Focusing on student responses specifically, they felt that good feedback had some of the following qualities/features:
- Rubrics that are accurately filled out/detailed
- Digital recordings that were easy to understand
- Face-to-face feedback, which makes the process more personal and is perceived as being more thorough
In line with recent literature, Carless [2] defines feedback as having the following features:
- Information about performance or understanding from different sources e.g. teachers, peers or self
- Students responses to actions
- A process in which learners make sense of comments about the quality of their work to inform the development of future performance or learning strategies
The latter point re-iterates the point that designers of feedback, feel the timing and structure over feedback in both the short and long term, is of importance. Additionally, Carless defines 3 conditions for effective feedback:
- Learners need to possess a concept of the standard being aimed for
- Compare the current level of performance with the standard
- Engage in an appropriate action which leads to some closure of the gap between the two
The final point places an implicit condition on feedback which is commonly stated in the literature - it must be given in a timely manner, for the student to understand the feedback, question the feedback, and turn the feedback into actionable steps. Ironically, timeliness was not identified frequently by students in the mass survey, as being important for good feedback. However, the authors assessing the data, Dawson et al, interpreted this to mean that students were not focused on the timing between handing an assessment in and receiving feedback, so long as ample time was available to act upon the feedback. This re-iterates the findings that students are focused on the quality of feedback and not the structure or methods around it. The disconnect between literature and educators was also noted, as staff, in addition to students, did not seem to notice, have experienced or prioritised evaluative judgement, peer feedback, exemplars or feedback moderation - techniques commonly featured in modern feedback literature [5].
Overall, on the topic of feedback, it is clear there is a disconnect between: what learners’ value in feedback; what educators think makes good feedback; how educators design and structure feedback; and what the literature says makes goods feedback. With such complexity and contradictions, educators may feel puzzled about how best to improve feedback. It is therefore suggested that educators and institutions:
- Conduct regular surveys on students’ thoughts and values on feedback, to inform feedback frameworks
- Ensure students are engaged and involved in any changes to feedback, from the task to curriculum level
- Engage students in feedback design, using common tasks from the business/manufacturing world such as quality groups
- Educate students on feedback design and self-regulation on their transition into higher education
- Allow students to question feedback and engage with their markers, with ample time given between feedback and the subsequent task
FAQ
Q: How can student voice be more effectively integrated into the feedback process to ensure it aligns with their needs and expectations?
A: Integrating student voice into the feedback process requires a more interactive and participatory approach to designing and implementing feedback mechanisms. Educators can start by actively seeking out students' opinions on what constitutes helpful feedback through surveys, focus groups, and one-on-one discussions. This direct engagement not only ensures that feedback aligns with students' actual needs and expectations but also empowers students by valuing their input in the educational process. Furthermore, institutions could implement platforms or forums where students can share their feedback experiences and suggestions for improvement. Incorporating text analysis tools can help educators and institutions to systematically analyse qualitative feedback from students, identifying common themes and areas for enhancement. By prioritising student voice in this way, the feedback process becomes more relevant and actionable for students, fostering a culture of continuous improvement and mutual respect between students and educators.
Q: What role does text analysis play in improving the quality of feedback provided to students?
A: Text analysis can play a crucial role in improving the quality of feedback by enabling educators and institutions to analyse and understand the vast amounts of textual feedback data collected through student surveys, feedback forms, and online forums. By employing text analysis techniques, educators can identify common issues, patterns, and student preferences in feedback across a large dataset, which might be missed through manual analysis. This insight allows for a more evidence-based approach to redesigning feedback mechanisms, ensuring they are tailored to address students' specific concerns and preferences. Moreover, text analysis can help in assessing the effectiveness of feedback by evaluating changes in student satisfaction and engagement over time, providing a data-driven foundation for continuous improvement. By leveraging text analysis, educators can ensure that feedback is not only more personalised and actionable but also aligned with students' evolving needs.
Q: How can feedback be designed to be more actionable for students, according to their expressed preferences?
A: Designing feedback to be more actionable for students involves a few key strategies that directly incorporate their expressed preferences. Firstly, feedback should be specific, detailed, and personalised, addressing the unique work of each student and providing clear guidance on how to improve. This could involve highlighting strengths, identifying areas for improvement, and offering concrete suggestions for action. Secondly, incorporating student voice through regular consultations can ensure that feedback methods evolve in line with student preferences, such as the format (written, oral, digital) and the medium (e.g., face-to-face, video feedback). Educators should also consider the timing of feedback to allow students ample opportunity to act upon it before subsequent assessments. Implementing peer feedback and self-assessment as part of the feedback process can further encourage students to engage critically with feedback and apply it effectively. These approaches, informed by ongoing dialogue with students and supported by text analysis to identify trends and preferences, can make feedback more actionable and relevant to students' learning journeys.
References
[1] Boud, D. and E. Molloy, Rethinking models of feedback for learning: the challenge of design. Assessment and evaluation in higher education, 2013. 38(6): p. 698--712.
DOI: 10.1080/02602938.2012.691462
[2] Carless, D., Feedback loops and the longer-term: towards feedback spirals. Assessment and evaluation in higher education, 2019. 44(5): p. 705--714.
DOI: 10.1080/02602938.2018.1531108
[3] Price, M., K. Handley, and J. Millar, Feedback: focusing attention on engagement. Studies in higher education (Dorchester-on-Thames), 2011. 36(8): p. 879--896.
DOI: 10.1080/03075079.2010.483513
[4] McFadden, M. and G. Munns, Student Engagement and the Social Relations of Pedagogy. British journal of sociology of education, 2002. 23(3): p. 357--366.
DOI: 10.1080/0142569022000015409
[5] Dawson, P., et al., What makes for effective feedback: staff and student perspectives. Assessment and evaluation in higher education, 2019. 44(1): p. 25--36.
DOI: 10.1080/02602938.2018.1467877
[6] Quinton, S. and T. Smallbone, Feeding forward: using feedback to promote student reflection and learning - a teaching model. Innovations in education and teaching international, 2010. 47(1): p. 125--135.
DOI: 10.1080/14703290903525911