Insights and resources to support better data analysis in education
By David Griffin
In all types of lectures, students are susceptible to ‘mind-wandering’, where one’s focused attention moves away from external stimulation and towards unrelated, self-generated thoughts (Smallwood & Schooler, 2006). Poorer outcomes in learning are associated with this behaviour (Risko et al., 2012). At any given lecture, between a third and a half of students may experience mind-wandering, with the frequency of this phenomenon increasing with time spent in the lecture (Bunce et al., 2010).
The problem of mind-wandering is exacerbated by online learning. Online classes are often undertaken in settings full of distractions (Hollis & Was, 2016) and without the presence of an instructor. Given the growing popularity of online learning, accelerated by the current Covid-19 global pandemic, this is a concern for both learners and educators.
Various methods have been trialled to reduce or avoid the negative impacts of mind-wandering in lectures. These include methods of detection and intervention (Bixler & D’Mello, 2015), and methods of prevention. The authors of this paper investigated the use of pretesting, or testing students on information before it is presented to them, as a means of reducing mind-wandering in online video lectures (Pan et al., 2020).
Intuitively, pretesting students on knowledge they have yet to acquire often leads to incorrect answers (Pan et al., 2019). However, when pretesting is succeeded by presentation of the correct answers or by study of the material, improved long term retention of the material is seen (Richland et al., 2009). This pretesting effect has been shown across a wide range of educational settings and circumstances, including retention of word pairs and trivia facts (Kornell et al., 2009) and educational videos (Toftness et al., 2018). The success of the pretesting effect has been attributed to several reasons. It has been suggested that it may encourage the student to acknowledge gaps in their knowledge and attempt to then fill those gaps during study (Carpenter & Toftness, 2017). It has also been suggested that a learner’s curiosity is piqued by pretesting and their attention is improved as a result (Geller et al., 2017). Work by St. Hilaire & Carpenter (2019) suggested that the pretesting effect causes students to focus on retaining the material pretested. They indicated this focused effort may contribute to reduced mind-wandering.
The authors of this paper performed two experiments, with a focus on learning from video lectures (Pan et al., 2020).
The first experiment investigated interpolated pretesting. Interpolated pretesting pretests learners at several intervals during the lecture. Undergraduate psychology students were split into two groups: the pretest group and the control group. Both groups were presented with a 26-minute video lecture split into four sections, in a controlled laboratory environment. The control group were given algebra problems unrelated to the lecture to solve before each video section; the pretest group questions relating to the information from the subsequent section of video lecture. After the final video section, all students were required to complete the same test. Half of this test was made up of questions taken from the pretests and half consisted of new questions. All questions were multiple choice.
The second experiment replicated the work of the first experiment while also investigating conventional pretesting. Conventional pretesting presents students with all questions before the lecture commences. Students were divided into three different groups: control, interpolated pretest (as before) and conventional pretest. For the conventional pretest group, all questions were together presented prior to commencement of the four video sections, which were played successively. Unlike the first experiment, this was performed completely online, enabling the students to view the lectures remotely in an uncontrolled environment.
No significant difference was observed between the final test results of students in the interpolated and conventional pretest groups. This suggests they are both equally effective in the improvement of learning. However, both pretest groups (interpolated and conventional) outperformed the control group.
In keeping with patterns seen in other similar mind-wandering studies (Bunce et al., 2010; Thomson et al., 2014), all students’ reported attention levels waned with time spent viewing the video lecture. However, the reported attention levels of those in both pretest groups (interpolated and conventional) were on average 8-21% higher across the entire lecture than those of the control group.
In conclusion, both interpolated and conventional pretesting resulted in improved learning and less mind-wandering when compared with the control groups. This effect was seen in both experiments, suggesting pretesting works for both controlled and uncontrolled learning environments. The authors of this work concluded that their results further strengthen the argument that pretesting is a ‘desirable difficulty’; while pretesting contributes to the initial challenge of learning, outcomes are improved in the long term.
Bixler, R., & D’Mello, S. (2015). Automatic gaze-based detection of mind wandering with metacognitive awareness. International Conference on User Modelling, Adaptation, and Personalization, 31–43.
Bunce, D. M., Flens, E. A., & Neiles, K. Y. (2010). How long can students pay attention in class? A study of student attention decline using clickers. Journal of Chemical Education, 87, 1438–1443.
Carpenter, S. K., & Toftness, A. R. (2017). The effect of pre-questions on learning from video presentations. Journal of Applied Research in Memory and Cognition, 6, 104–109.
Geller, J., Carpenter, S. K., Lamm, M. H., Rahman, S., Armstrong, P. I., & Coffman, C. R. (2017). Prequestions do not enhance the benefits of retrieval in a STEM classroom. Cognitive Research: Principles and Implications, 2, 42.
Hollis, R. B., & Was, C. A. (2016). Mind wandering, control failures, and social media distractions in online learning. Learning and Instruction, 42, 104–112.
Kornell, N., Hays, M. J., & Bjork, R. A. (2009). Unsuccessful retrieval attempts enhance subsequent learning. Journal of Experimental Psychology: Learning, Memory, and Cognition, 35, 989–998.
Pan, S. C., Lovelett, J., Stoeckenius, D., & Rickard, T. C. (2019). Conditions of highly specific learning through cued recall. Psychonomic Bulletin & Review, 26, 634–640.
Pan, S. C., Schmitt, A.G., Bjork, E. L., Sana F. (2020). Pretesting reduces mind wandering and enhances learning during online lectures. Journal of Applied Research in Memory and Cognition, 9, 542-554.
Richland, L. E., Kornell, N., & Kao, L. S. (2009). The pretesting effect: Do unsuccessful retrieval attempts enhance learning? Journal of Experimental Psychology: Applied, 15, 243–257.
Risko, E. F., Anderson, N., Sarwal, A., Engelhardt, M., & Kingstone, A. (2012). Everyday attention: Variation in mind-wandering and memory in a lecture. Applied Cognitive Psychology, 26, 234–242.
Smallwood, J. (2010). Why the global availability of mind wandering necessitates resource competition: Reply to McVay and Kane (2010). Psychological Bulletin, 136, 202–207.
St. Hilaire, K. J., Carpenter, S. K., & Jennings, J. M. (2019). Using prequestions to enhance learning from reading passages: The roles of question type and structure building ability. Memory, 27,1204–1213.
Thomson, D. R., Seli, P., Besner, D., & Smilek, D. (2014). On the link between mind wandering and task performance over time. Consciousness and Cognition, 27, 14–26.
Toftness, A. R., Carpenter, S. K., Lauber, S., & Mickes, L. (2018). The limited effects of prequestions on learning from authentic lecture videos. Journal of Applied Research in Memory and Cognition, 7,370–378.