The Ideal in Evaluating and Improving Teaching Effectiveness

Quality teaching yields quality learning, and important step in improving teaching is evaluating teaching success and changing teaching strategies based on that evaluation. But what is the best way to evaluate a teacher's effectiveness? And what data sources can illuminate the path forward? the I recently had a conversation with my good friend John Louviere at USU, and through our mutual brainstorming, we generated the following data sources for effective teaching evaluation. The idea here is to utilize the principle of triangulation - the more data points you can use in your analysis, the more realistic picture you are able to paint of the situation. This in turn gives one a better process and method for moving forward.

The ideal data sources for effective teaching evaluation, particularly in higher education.

Teaching Evaluation Data Points
To make a truly holistic evaluation of a faculty member's teaching effectiveness, the following data points should be analyzed:

• Teacher Observations
• Student Teaching Evaluations 
• Formative Evaluation Tools
• Course Design Principles Rubric
• Course Design Quality Rubric
• LMS Data Analytics

Creating and Implementing Teaching Evaluation Tools

To create and implement effective data analytics tools, the following general phases should be followed. These phases should be used for each of the six data points described above. 

1. Identify and Synthesize Standards of Excellence - Determine based on the literature what the ideal is for each tool. For example, what should an observer find in the ideal teacher? This phase should include a thorough review of literature to find the research-based best-practices and should also include a clear synthesis of those practices into a coherent whole.
2. Develop Reports, Tools, and Rubrics Based on Standard - Create methods for gathering data on how well the standards are being used by the teacher. These can be rubrics, checklists, data analytics reports, and other tools. The goal is to have a way to efficiently and effectively gather data for each data point. For example, a teaching observation form would be developed to guide the observation of a teacher.
3. Gather Formative Feedback and Perform Validity Testing - Get sufficient expert feedback on the drafted tool or report and refine it based on the feedback. For example, have an expert or a scholar of teaching review the teaching observation form and give you feedback on how to refine it.
4. Pilot Testing - Conduct a pilot test of the tool and gather formative feedback on how well it functions and on the quality of the data being generated. For example, have someone use the teaching observation rubric to observe teaching and gather data on how well it worked and on whether the data is useful.
5. Implementation - After the tool has been refined and improved, implement it. This should inherently include the gathering of data on how it is being used and on the outcomes of its use.
6. Summative Evaluation - After the tool has been implemented, gather data on its effectiveness and draw conclusions from its use. Publish the results of the findings and create plans to move forward based on the results of the tool.  

Steps for establishing, implementing and evaluation tools, rubrics, and reports for improving teaching effectiveness.

Interactions of Data 

There will likely be a strong correlation between the many of the measures in each of these tools. For example, the use of effective design principles could correlate strongly to positive reports in other measures. The key is to create the tools, gather data on their use and results, and look for ways to further analyze and improve teaching effectiveness. As data is gathered, we will likely be able to determine what measure(s) are the strongest predictors of successful student learning, and this in turn gives us increased power to develop and improve future teaching evaluation and effectiveness. And in the increasingly competitive higher education environment, the universities who use data to help students succeed will both survive and thrive.

So, what do you think? Am I missing something? What else could be included in this? I believe that the effective establishment and use of these strategies will lay the foundation for increased and continued excellence in teaching at any university.