Research on Education Text Analysis and Teaching Best Practice
Stuart Grey - Feb 07, 2022
This study provides preliminary evidence that suggests the relationship between numerical student evaluations and instructor likability is causal in nature. That is, student evaluations may be an indicator of a student-perceived construct similar to likability. This gives yet more support to the idea that when trying to capture the student voice to improve teaching and the student experience, free-text comments are far more valuable than numerical scale answers.
This study looked at the influence of likability on the student evaluation of teaching. They found that if you know nothing about an instructor or how a class was taught, students' perception of likability accounted for two-thirds of the total variance in evaluations.
module evaluations are designed assuming students are rating their evaluation of teaching based on effectiveness. It has been found however that students will frequently ignore any given question and answer in a manner consistent with some overriding issue or concern
A key point is that the use of module evaluation does not appear to improve teaching as measured by the same module evaluation. The goals of evaluation are not always clear to the target. This is a problem because it means that students may be biased in their responses, which will affect the results of the evaluation.
The statistical analyses show that the survey instruments do not have a relationship with measurable learning.
The results of this study show that the likability of the professor is a strong predictor for self-reported learning and grades. There are many possible explanations for the correlation between instructor evaluations and student perceptions of likability. The research has not been able to show with any certainty what this relationship is due to.
Researchers found that the students' perception of their instructor's likability, personality, and responses on a module evaluation instrument are highly related. This led some researchers to suggest that the evaluations create what could be called a likability scale.
This study has demonstrated that the likability of an instructor is not a simple construct. It is composed of many different components which are related to one another through complex relationships, but are not all directly correlated with the likability of the instructor.
The paradigm shift of understanding that the student evaluation of teaching is a measure of what students like and dislike brings to mind the idea of customer satisfaction surveys.
Module evaluation instruments are measuring what students like and dislike, not necessarily the intended teaching and its effectiveness. The only way to both have more nuanced responses and to be able to interrogate the student voice for actionable insight is to better promote and analyse free-text comments.
[Paper Source]: Dennis Clayson (2022) The student evaluation of teaching and likability: what the evaluations actually measure, Assessment & Evaluation in Higher Education, 47:2, 313-326,