In spring 2019, a group of instructors at K took part in a pilot study of an online format for student course evaluations. The survey questions were identical to the traditional pencil-and-paper format, but the students responded online using their choice of devices: computers, tablets, phones, etc.
The results of the pilot study show:
- Identical student response rate compared with traditional paper and pencil format when students were given time in class to complete the online evaluation form. These results contrast starkly with a pilot study conducted at K a decade ago.
- Accuracy of student responses: legible written responses, elimination of bubbling errors, accurate numerical counts due to elimination the of scanning errors frequently encountered in the current format
- Evaluation responses are available immediately at the end of the term. The current process takes weeks to complete and consumes many hours of staff time that could be better spent ensuring smooth operation of other areas of our academic mission.
The pilot study included student evaluation data from the 17 courses and 15 instructors — 8 women and 7 men. The sample included tenured faculty volunteers diverse with respect to division, discipline, course level and course size. The sample intentionally didn’t include non-tenured faculty to safeguard against any possible negative repercussions in the tenure review process.
Results:
*The pilot showed that in order to maximize student response rate, class time should be provided for students to complete the evaluation form. The overall student response rate for the 17 courses in the pilot study was acceptably high: 82%. What is more, for the 11 classes in the pilot in which the instructors provided class time for students to complete the online evaluation forms, the response rate was 91% — the same as the historical response rate using class time for paper forms.
*Numerical averages didn’t change: In comparing historical averages for each course/instructor pair with the online results, the data from the 17 courses in the trial gave no evidence that overall numbers for “course” and “instructor” are in any way different when comparing the online evaluation format to the traditional paper format.
*Students liked the online format: In a follow-up post-evaluation survey, 87% of the responding students from the pilot study courses responded favorably, reporting that they found their experience with the online format to be the same (7%), slightly better (30%) or much better (50%) than their previous experiences with the paper format.
TLC understands that change always involves startup costs in terms of time for familiarization with a new process, and that a change of format will require that we each spend a little time becoming familiar with a new process for course evaluations.
Faculty feedback is highly sought and most valued and will help determine how we go forward in this — any member of TLC is available for your comments and concerns. We anticipate bringing a plan for adopting an online format for student course evalutions based on those discussions for a faculty vote at the 6th week fall faculty meeting.
The soonest a switch to online format could occur is winter term 2020.