Designing AI-transformed Student Feedback for Instructor Engagement

Abstract

Many instructors minimally engage with or avoid student evaluations of teaching (SETs) due to the significant time, cognitive, and emotional cost associated with effective usage. Nevertheless, SETs can contain feedback about students' learning experiences that instructors can use to improve instructional and educational delivery. In this work, we explore how to redesign SET reports to increase instructor engagement with this feedback. We explore the use of language models (LMs) to process and filter students' feedback to highlight recurring or important ideas, to identify actionable changes for instructors, and to de-emphasize demotivating aspects of this feedback. We explored a $4 imes 4$ strategy-presentation design space, generating six representative mock-ups that combine different strategies with various presentation formats. Through interviews with 16 post-secondary instructors, we learned how and when they engage with current SETs, and how they would perceive and use the LM-powered redesigned SET mock-ups. We found that instructors valued different kinds of presentation strategies depending on their needs, be it to actually improve their teaching, to get a one-time gestalt impression of their teaching performance, or to provide summative reports about their teaching performance. These findings shed light on new opportunities for designers to design dynamic SET reports, customized to instructors needs.