The Ideal in Evaluating and Improving Teaching Effectiveness

Quality teaching yields quality learning, and important step in improving teaching is evaluating teaching success and changing teaching strategies based on that evaluation. But what is the best way to evaluate a teacher's effectiveness? And what data sources can illuminate the path forward? the I recently had a conversation with my good friend John Louviere at USU, and through our mutual brainstorming, we generated the following data sources for effective teaching evaluation. The idea here is to utilize the principle of triangulation - the more data points you can use in your analysis, the more realistic picture you are able to paint of the situation. This in turn gives one a better process and method for moving forward.

The ideal data sources for effective teaching evaluation, particularly in higher education.

Teaching Evaluation Data Points
To make a truly holistic evaluation of a faculty member's teaching effectiveness, the following data points should be analyzed:

• Teacher Observations
• Student Teaching Evaluations 
• Formative Evaluation Tools
• Course Design Principles Rubric
• Course Design Quality Rubric
• LMS Data Analytics

Creating and Implementing Teaching Evaluation Tools

To create and implement effective data analytics tools, the following general phases should be followed. These phases should be used for each of the six data points described above. 

1. Identify and Synthesize Standards of Excellence - Determine based on the literature what the ideal is for each tool. For example, what should an observer find in the ideal teacher? This phase should include a thorough review of literature to find the research-based best-practices and should also include a clear synthesis of those practices into a coherent whole.
2. Develop Reports, Tools, and Rubrics Based on Standard - Create methods for gathering data on how well the standards are being used by the teacher. These can be rubrics, checklists, data analytics reports, and other tools. The goal is to have a way to efficiently and effectively gather data for each data point. For example, a teaching observation form would be developed to guide the observation of a teacher.
3. Gather Formative Feedback and Perform Validity Testing - Get sufficient expert feedback on the drafted tool or report and refine it based on the feedback. For example, have an expert or a scholar of teaching review the teaching observation form and give you feedback on how to refine it.
4. Pilot Testing - Conduct a pilot test of the tool and gather formative feedback on how well it functions and on the quality of the data being generated. For example, have someone use the teaching observation rubric to observe teaching and gather data on how well it worked and on whether the data is useful.
5. Implementation - After the tool has been refined and improved, implement it. This should inherently include the gathering of data on how it is being used and on the outcomes of its use.
6. Summative Evaluation - After the tool has been implemented, gather data on its effectiveness and draw conclusions from its use. Publish the results of the findings and create plans to move forward based on the results of the tool.  

Steps for establishing, implementing and evaluation tools, rubrics, and reports for improving teaching effectiveness.

Interactions of Data 

There will likely be a strong correlation between the many of the measures in each of these tools. For example, the use of effective design principles could correlate strongly to positive reports in other measures. The key is to create the tools, gather data on their use and results, and look for ways to further analyze and improve teaching effectiveness. As data is gathered, we will likely be able to determine what measure(s) are the strongest predictors of successful student learning, and this in turn gives us increased power to develop and improve future teaching evaluation and effectiveness. And in the increasingly competitive higher education environment, the universities who use data to help students succeed will both survive and thrive.

So, what do you think? Am I missing something? What else could be included in this? I believe that the effective establishment and use of these strategies will lay the foundation for increased and continued excellence in teaching at any university.

Podcast: 6 Qualities of a Good Online Instructor

My Brother J. Clark and I recently recorded a podcast discussion about what it takes to be a good online instructor. We created a list of 6 qualities that we believe good instructors embody. A good online instructor: 

1. is comfortable with technology.
2.  clearly articulates expectations for students.
3.  guides learners through student-led activities.
4.  responds to students’ needs in a timely way.
5.  manages time efficiently.
6.  is willing to do it all over again.

Here is a link to the Podcast. 

Planning Teaching Activities in Online and Hybrid Courses

It is a growing trend that instruction and learning in higher education is taking place increasingly online. Traditional face-to-face courses are now looking more like hybrids. And online courses are becoming the norm. But what if you have never taught online before? How can you begin to bridge your current skills to the new environment? In this post I share my advice:

1. Learn the tools of the online environment
2. Understand the weekly flow of your class
3. Plan your online teaching and course management activities

Learn the Tools of the Online Environment
It is important to become aware of the new tools that you can use to accomplish the activities you do in a face-to-face teaching environment. The table below identifies some of the general tools that are found in most Learning Management Systems. You can employ these tools to accomplish many of the activities you normally accomplish in a traditional environment.

It is crucial that you become familiar with these tools and master how to use them effectively in your work. Doing so will empower you to use the strengths of the online environment and will enable you to help your students move forward effectively. The table below outlines the tools that you might use to accomplish the activities you normally do as an instructor.

Traditional Teaching Activity	Online Tools
Grading	Grade Book
Feedback	Announcement (general) Email (general or specific) Assignment feedback in Grade Book (specific)
Communication to students	Announcement Email
Giving instructions	Announcement Email
Giving additional materials	Discussion Build Content Video Conference
Discussions	Discussion Boards Video Conference
Meeting with students	Video Conference Lecturing
Additional Instruction	Video Conference both live and prerecorded
Sharing Examples	Video Conference Discussion Board Announcement Build Content

The Weekly Flow of Your Class
Below is one visual display of how I plan out my week for courses that I teach in the IDPT Program at Franklin University. While this may not apply directly to your own teaching context, it should give you an idea of the kinds of activities you might schedule and perform in your own courses. You will notice that this image also displays the kinds of activities the students are performing during the same time period. You can see how the faculty member's activities are designed to support the work and the learning of the students.

Implementing Hybrid Technologies into Your Daily and Weekly Schedule
Once you have obtained the skills of using these tools, it is important that you plan out how you will begin to use these tools in your own teaching. This means that you must plan out which tasks you will perform throughout the week. I have personally found that scheduling time to do course management and teaching activities throughout the week really helps me to facilitate learning in an online course. As is implied in the image above, these activities should support your students' learning. I have found it helpful to schedule the following into my calendar:

• grading and feedback - I try to complete grading the day or two after the assignments are submitted so that students can read my feedback and use it in future assignments.
• planning and preparing my class meetings and slides to ensure an effective online meeting.
• participating in student discussion boards to help guide the students' thinking and to increase my presence in the class.
• responding to student emails (usually every morning and often throughout the day) to give students the support they need in their coursework.
• proactively scheduling online and phone office hours to meet with students in need.

Planning Your Own Online Teaching and Management Activities
If you are new to online teaching, or want to improve your effectiveness and efficiency as an instructor, take some time to plan your own weekly activities. In this case, I recommend identifying what activities you will perform on which days and at what times. This will help you have a clear idea of what you must do to maintain your presence in your online course and will help you more effectively support your students' learning .