Online Course Observation
Faculty Evaluation Form and Guidelines

Introduction

The College of Arts & Letters (CAL) is interested in developing resources for class observations of online courses. The purpose is to provide a fair and transparent course observation process for online instructors.

This form may be adapted by departments or units if they would find it useful, and then given to committees who are reviewing online instructors. Units are free to make changes to adapt this resource, including both changes in content and changes in form (such as putting the questions in a table, delivering the content digitally or in hard copy, etc.).

A class observation is required by the College’s Fixed Term Designation B Review Guidelines for the promotion of fixed-term system faculty and for reappointment of academic specialists in the continuing system, and is recommended for any departmental evaluation of faculty or academic staff.

Versions

This version was created based on research on online teaching and revised based on feedback on Version 1.0 from CAL faculty, and Version 1.1 based on feedback from CAL and other faculty and administrators from across the University. Some concerns from instructors and administrators that this form is working to address include:

• Transparency and fairness in the process
• The multitude of ways of organizing and administering online courses
• Making effective teaching and innovation visible to reviewers of varying experience with online pedagogy
• Instructors should have the autonomy and opportunity to present their online teaching in a clear and representative way
• The process should be roughly equivalent to a face-to-face observation; online instructors should not be disadvantaged by the process

Scope and Purpose

This resource is intended for the course observation portion of a teaching evaluation. It aims to be a fair process that offers online instructors the same autonomy and opportunities that a face-to-face course observation would. It is intended to be suitable for use in high-stakes teaching evaluations (such as reviews for promotion); the primary purpose is a fair and transparent class observation process for online instructors, and an important secondary purpose is creating a valuable professional development experience for those involved.

This set of forms is intended ONLY to be an equivalent of the class observation, and NOT the entirety of the teaching evaluation.

There are several other issues that faculty and administrators have brought up that are beyond the scope of these forms but that might merit further conversations within units:

• Online instructors should have access to appropriate professional development opportunities around online pedagogy; this resource is not intended to be the only form of support or professional development for online instructors.
• Departments/units might have conversations about how to create a culture that normalizes the reviewing and observing the teaching of other faculty, for both face-to-face and online courses
• Departments/units might consider how serving on review committees/performing course observations is valued. For the experience to be fair, transparent, and constructive, the observer will need to invest time, whether the observation is for an online or face-to-face course. According to some faculty feedback, it might be useful to further discuss faculty incentives (or disincentives) for collegial generosity in the review and observation process.
• Whether for face-to-face or online courses, the observation process might benefit from conversations or guidelines about how to offer and receive constructive feedback.
• While it is possible to evaluate an online course (as opposed to evaluating an instructor), this resource is intended to contribute to the evaluation of an instructor. It may be adapted for this other purpose if desired, but the unit might wish to make substantial changes.
• Departments/units might set their own policies for hybrid courses. One solution is to allow hybrid instructors to determine which method they prefer for course observation: an online observation, a face-to-face observation, or a combination of both.
• Departments/units might particularly wish to specify the expectations for the third principle, “Respects Diversity and Diverse Learners.” In this resource, there are multiple possible ways that instructors can demonstrate this principle, but if the unit has particular priorities around this principle (e.g, inclusion of course materials from globally diverse cultures, accessibility, et al.), they might wish to adapt the form to reflect these priorities.

Overview of the Online Course Observation Process

A class observation is required by the College’s Fixed Term Designation B Review Guidelines for the promotion of fixed-term system faculty and for reappointment of academic specialists in the continuing system, and is recommended for any departmental evaluation of faculty or academic staff.

Please consider the following throughout the evaluation process:

• The class observation is only one part of the process of evaluating teaching, and these materials and guidelines are intended ONLY to be an equivalent to the class observation, NOT the entirety of the teaching evaluation.
• The purpose of the evaluation is to provide fair and transparent documentation of teaching effectiveness, as well as to provide constructive feedback that contributes to the professional development of the instructor.
• While all course observations should involve multiple communications and clarifying questions and answers between the reviewee and reviewers, an online course observation will often require more frequent and in-depth communications. Reviewers should ask questions about anything they have difficulty finding, or anything for which they have trouble understanding the purpose.
• Selecting the reviewers: The best practice is for all reviewers to have experience teaching online. Where it is impossible to do so, it is preferable to have as many reviewers with online teaching experience as possible; a minimum of one reviewer must have online teaching experience. Additionally, if a reviewer without online teaching experience is doing the observation, they should consider doing so with someone who does have a great deal of knowledge about online teaching. In any case, it is appropriate for the instructor to have some say over who performs the course observation.

Process

• The instructor should be given all forms at least four weeks before the semester in which the observation will occur
• Instructors may choose to inform students in the syllabus or elsewhere that the course will be observed
• In general, it is appropriate for the instructor to have some say over who performs the course observation
• Reviewers and the instructor should read over the Guidelines and all forms before the review begins
• The instructor should fill out the Pre-Observation form and bring it to the pre-observation meeting with the reviewers
  The instructor should meet (online or in person, separately or as a group) with all reviewers before the observation, and follow the instructions on the Pre-Observation form
• The instructor will add the reviewers to the course, and the reviewers will observe only those parts of the online course that were agreed upon (to parallel the face-to-face instructor’s ability to choose the date of a classroom observation)
• Reviewers should ask the instructor if they are unsure of how to find any of the materials in the course, or if they have questions or concerns about anything else.
• Reviewers should read the Online Faculty Evaluation form before observing the course and use the form’s principles, examples, and questions to take sufficient notes (though not yet completing the final draft of the form)
  • The reviewer is only evaluating the aspects of the course that the instructor being evaluated is responsible for (see Pre-Observation form)
• The reviewer should spend 1-2 hours in total (i.e., comparable time to a face-to-face observation) in the online classroom space, and during that time take notes that can be used to complete the online course observation form. In other words, the reviewer should take 1-2 hours in total to examine and explore the course elements specified by the instructor in the Pre-Observation form.
• The completed draft of the Course Observation form should be sent to the instructor at least 72 hours before the Post-Observation Meeting
• After receiving the Course Observation form draft, the instructor should fill out the Post-Observation Form
• The reviewers (separately or together) meet with the instructor, using the instructions on the Post-Observation Form
• After this meeting, the reviewers may revise the Course Observation form and send it to the instructor, who has an additional chance to clarify, ask questions, correct misunderstandings, etc., before the final version of the Course Observation form is submitted to the instructor’s records. The unit will determine if the Pre-Observation form and the Post-Observation form should also be part of the instructor’s records.

General Guidelines on Evaluating Online Teaching

• Keep in mind that learning may look different in the online classroom; online courses may utilize activities that may not look like traditional “in-class” activities but do promote learning. There are a multitude of possible ways for online courses to work, and even if you have extensive experience teaching online, the reviewee’s class may not be organized or run like other classes you have seen.
• Instructors should not be held responsible for course elements that the instructor does not have the autonomy to change (see pre-observation form)
• The look and feel of the online classroom should not be given excessive emphasis. The look and feel of the online space is perhaps parallel to an appealing public speaking style; it can help, but it is not the primary factor in pedagogical effectiveness.
• Some online course activities require students to complete activities in the “real world” and not only to do work in the online environment; these activities should be included when evaluating the rigor and effectiveness of the course.
• For some online courses that include a great deal of online discussion or have many students, it may not be expected that the instructor provide feedback to each individual student for every post.
  • Some instructors, for example, might grade discussion “pass/fail” or with a point system, or offer the class group feedback on the conversation on a weekly basis, or provide individual feedback primarily on larger assignments, or provide feedback for some but not all online discussion posts, or use peer ratings to grade discussion posts, etc.
  • Some activities might not be worth points, or have points based only on completion; the instructor may consider performing the activity to be a valuable step in learning, reflection, scaffolding, etc., even without graded evaluation
  • Students may say incorrect things or show somewhat flawed reasoning as part of their online discussion. It is not always beneficial to online discussion to correct every single thing that it is possible to correct, particularly if one of the goals of the discussion is to explore new and challenging concepts.
• Some choices made in the online classroom that do not appear to make sense at first glance are shaped by the combination of the learning objectives and the technological affordances/limitations of the online environment; again, ask the instructor questions frequently.
• Asynchronous elements are those in which students engage in the course at different times; synchronous elements are those in which students and instructors are interacting in the online environment at the same time (online chat, e.g.). Online courses may include only asynchronous elements or both types (or occasionally, only synchronous). Keep in mind that asynchronous discussions and other activities may have different dynamics than face-to-face discussions and should be evaluated accordingly.
• Office hours might be virtual, in person, or some combination.
• Although there is research on effective online pedagogy, online teaching is still a relatively new field with many unanswered questions. A characteristic of good online teachers is often that they thoughtfully analyze their own choices in creating and maintaining an online learning environment.
  • Instructors who constantly strive to improve may be very forthcoming about their own flaws or plans to improve; clearly, this openness should not be penalized.
  • Experimental teaching methods may still be evaluated according to principles of sound pedagogical practice (e.g., active learning)
• Be aware that student evaluations (SIRS) have been shown in research to be biased in some respects; in addition to the more well-known biases related to instructor gender and race, etc., some researchers debate about whether online courses may sometimes have lower SIRS ratings than face-to-face courses; additionally, student evaluations of online courses have been shown to have more diversity of opinion than for face-to-face.
• Be cognizant of class size; what is feasible in a small class may not be in a larger class. It is a common, but incorrect, myth that the size of an online course does not affect the teaching methods.

Contact

Any questions, comments, concerns, feedback should be directed to Ellen Moll (mollelle@msu.edu) and Kate Sonka (ksonka@msu.edu) in the College of Arts & Letters.

This work is licensed under the Creative Commons Attribution 4.0 International License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/ or send a letter to Creative Commons, PO Box 1866, Mountain View, CA 94042, USA.