Investigating Instructional Design Management and Leadership Competencies - a Delphi Study

Joel Gardner

Franklin University

Joel.l.gardner@gmail.com
Lewis Chongwony
Franklin University

Lewis.chongwony@franklin.edu

Tawana Washington

The Ohio State University

Washington.181@osu.edu

Abstract
Research in instructional design and educational technology journals typically focuses on the theories, technologies, and processes related to practice of instructional design. There is little research emphasis, however, on leadership and management of instructional design in higher education. Investigating the competencies associated with effective leadership and management of instructional design is critical as it would provide the field with guidance on how to more effectively prepare and train instructional design leaders and managers in higher education. In this study, we explored the competencies required for an instructional design manager to be effective in higher education settings. We used a Delphi study surveying managers and leaders of instructional design through an anonymous consensus-building process consisting of two rounds of surveys. Results identified eight major categories with 64 competencies as relevant for leading and managing instructional design in higher education. Managers and leaders surveyed identified communication, project management, and visioning and strategic alignment as integral competencies to be successful in leading and managing instructional design. We discuss the implications of this research and provide recommendations for research, practice, and education of future instructional design professionals.
Introduction

Instructional designers are playing an increasingly important role in improving teaching quality in higher education (Rubley, 2016). There are now roughly 13,000 instructional designers working in higher education (Intentional Futures, 2016). Some scholars assert that instructional designers are particularly well equipped to lead in higher education, noting that quality online instruction will be a key feature of the future of higher education (Ashbaugh & Piña, 2014; Brigance, 2011; Shaw, 2012). In higher education, as in most environments, a director or manager typically leads instructional designers. Many university-trained instructional designers will likely become managers who lead and direct the work of instructional design (Ashbaugh & Piña, 2014; Merrill, 2007), and might therefore benefit from management and leadership courses within instructional design university curricula.

Numerous research studies show the impact--both positive and negative--that a manager’s or leader’s behavior and competency have on an employee’s engagement at work (e.g. Madlock & Kennedy-Lightsey, 2010; Wang & Hsieh, 2013; Xu & Cooper Thomas, 2011). This impact likely applies to employees serving in an instructional design role. However, the research on instructional design leadership and management is sparse. Da Silva, Diana, and Catapan (2015) conducted a search for recent articles related to instructional design and management and found few articles that discussed both topics, though project management--a component of management--was identified as a common theme in the instructional design literature. Recognizing a need to investigate instructional designer leadership competencies, Ashbaugh (2011) and Ashbaugh and Piña (2014) gathered the expertise of instructional design practitioners in a Delphi study. This study resulted in the 7 Ps of leadership for instructional design (7PL4ID), which includes the following characteristics: (1) Prescience – envision and promote a vision of the future; (2) Preventive or proactive thinking – strategize to anticipate future problems and opportunities; (3) Provision for unexpected and unknown – have backup plans and resources in reserve; (4) Personality – collaborate, communicate effectively, and show care for others; (5) Productivity – work hard and expect excellence from others; (6) Psychological/emotional toughness – make difficult decisions based on sound reasoning; and (7) Personal convictions—exhibit consistent, moral behavior. However, this study appears to be focused on leadership by instructional design professionals broadly, and not specifically on positional leaders and managers of instructional designers in higher education. Researchers note a need to clarify and validate the specific competencies associated with effective leadership and management of instructional design (Ashbaugh & Piña, 2014).

In the related field of distance education, it is clear that leadership and management have not been adequately researched. A Delphi study involving international distance education experts identified research on management and change as important to moving the field forward (Zawacki-Richter, 2009). Beaudoin (2003) noted “it seems we have not yet paid adequate attention to new roles required of leaders” (p.1) in distance education. While some authors identify leadership theories that might apply to higher education (e.g., Dashtahi, Ekrami, Navehebrahim, and Sarmad, 2016; Nworie, 2012), the majority of the literature focused on management and organization in distance education have been interpretive rather than empirical in nature (Dashtahi et al., 2016), suggesting a need for more empirical research in this area.

Given that a manager likely impacts the success and engagement of instructional designers and that research on effective management and leadership of instructional designers is lacking, we conducted an exploratory study on what these competencies are. In this study, we used a Delphi study surveying managers and leaders. The research questions for this study include the following:

What competencies do managers of instructional designers believe are necessary for effectively leading and managing instructional design in higher education?
Which of these competencies do managers of instructional designers believe are most important for effectively leading and managing instructional design in higher education?

Method
We used a Delphi methodology for this study (Brill, Bishop, and Walker, 2006; Hasson, Keeney, & McKenna, 2000). The Delphi method is an approach to building a consensus of experts’ opinions through multiple rounds of surveys (Hasson et al., 2000). A Delphi study typically includes (a) an initial survey that collects qualitative comments, (b) a second questionnaire that seeks to quantify and statistically analyze participants’ responses, and (c) sometimes a third or even a fourth survey that seeks to further quantify and analyze those responses until consensus is obtained (Hasson et al., 2000). Brill et al. (2006) have noted that a Delphi study “is a particularly good research method for deriving consensus among a group of individuals having expertise on a particular topic when information sought is subjective and where participants are separated by physical distance” ( p. 8).
Participant recruitment
To gather participants for our study, we went to the Department of Education’s Database for Post-secondary accredited institutions and programs (U.S. Department of Education, n.d.). We used a recent list of accredited organizations within the last quarter, which included 2,591 total instances of organizations seeking institutional accreditation during that period. We then cleaned the data to focus on four-year institutions, remove repetitions, and remove those with no website listed in the spreadsheet.
We then conducted a random sampling procedure to select 800 of these schools, assuming that only a percentage of the universities employ instructional designers. We collected the names and email addresses of potential participants by reviewing the institutional website to find whether instructional designers and their leaders existed. To focus the population, we included any school that was a bachelor-level or higher degree granting institution and that had an instructional design manager on staff. This process yielded 148 potential participants. We recruited these participants through an email soliciting their participation in the survey. As an incentive, we offered participants the opportunity to be entered into a drawing to receive one of two $25 gift cards.

Instruments
We employed two instruments in this study. Our first survey (Round 1) gathered the expertise of the participants regarding their perception of the key competencies for leading and managing instructional designers based on questions posed. We designed the second survey (Round 2) to quantify and further analyze the importance of the identified competencies that emerged from qualitative analysis of round 1 survey.

Round 1
The purpose of the round 1 survey in a Delphi study is to gather the expertise of the participating experts (Brill et al., 2006; Hasson et al., 2000). Our goal was to identify the competencies that managers of instructional designers believe are necessary for effectively leading and managing instructional design in higher education. We designed the round 1 survey to collect qualitative comments in response to the questions posed.

When conducting a Delphi study, it is important to direct participants’ responses toward the stated goal of the study, which in this case was the research topics identified for the study. We did not want to focus the attention of the participants too specifically, so we attempted to use definitions of leadership and management that were somewhat general in nature and that reflected the definitions commonly seen in management and leadership literature. We used the following definitions. Leadership is influencing others to accomplish mutually agreed upon purposes for the organization. It entails possessing the ability to: (a) rally others to accomplish a common purpose, (b) forge a path for self and others to follow, and (c) inspire others to productive action. Management is the act/process of coordinating people and/or resources efficiently to achieve or to reach organizational goals. It involves five basic functions of planning, organizing, directing, staffing, and controlling. We made the definitions as open-ended as possible to enable the participants to share their expertise.

We solicited candidates by email using the email address identified in the candidate selection process. The email summarized the study and solicited the candidate’s participation in the study. The linked round 1 survey included the informed consent form and the survey items. After participants completed the informed consent form, we provided the definitions to give general guidance to participants on what the researchers meant by leadership and management in the context of this study and then posed the following questions:

What management knowledge do you believe a manager of instructional designers in higher education must possess in order to be successful?
What management skills do you believe a manager of instructional designers in higher education must possess in order to be successful?
What leadership knowledge do you believe a manager of instructional designers in higher education must possess in order to be successful?
What leadership skills do you believe a manager of instructional designers in higher education must possess in order to be successful?
What additional knowledge and skills must an instructional design leader know and possess in order to be successful?

We kept the round 1 survey open for a two-week period, emailing participants after a week to remind them of our request for participation.

Round 2
Our goal for the round 2 survey was to identify which competencies managers of instructional designers believe are most important for effectively leading and managing instructional design in higher education. We developed the round 2 survey to validate the importance of the themes identified in the round 1survey. We included the informed consent form on the initial page and created several items to gather more detailed demographic information about each of the participants. The round 2 survey then asked participants to rate the importance of each of the 64 individual competencies identified using the following scale: not important, somewhat important, moderately important, very important, and extremely important.

To solicit participation for the round 2 survey, we again emailed the 148 potential participants and opened the survey for a total of two weeks with an initial email inviting their participation and a follow-up email after one week.

The round 2 survey began with several demographic items. The other categories as determined by the first survey were teaching, learning, design, and technology expertise competencies (10 items); project management competencies (13 items); communication competencies (7 items); interpersonal and people skills competencies (6 items); inspiring, motivating, and empowering others (8 items); environmental and organizational awareness (5 items); visioning and strategic alignment (7 items) and organizational politics and relationships (8 items). In total, there were 70 items on the questionnaire.

Reliability and internal consistency of the scale used in our questionnaire representing various categories of competencies was assessed using Cronbach’s alpha (Cronbach, 1971). The items in the categories had high internal consistency as follows: teaching, learning, design, and technology expertise consisted of 10 items (α = .748). Project management consisted of 13 items (α = .840), while communication consisted of 7 items (α = .888). Interpersonal and people skills consisted of 6 items (α = .611); inspiring, motivating, and empowering others consisted of 8 items (α = .861); environmental and organizational awareness consisted of 5 items (α = .730); visioning and strategic alignment consisted of 7 items (α = .768) and organizational politics and relationships consisted of 8 items (α = .885). The overall internal consistency and reliability among all 64 items was (α = .942), greater than the recommended 0.70 (Nunually, 1978).

Results

Round 1 results
Round 1 survey had a total of 22 participants out of 148, for a response rate of 15%. Eighteen percent of these participants reported 0-2 years of managing and leading instructional design, 32% had 3-5 years, 23% had 6-8 years, 9% had 9-11 years, and another 18% had more than 12 years.

We conducted a thematic analysis of the round 1 survey results using constant comparative analysis to create the themes by iteratively (a) reviewing the gathered data, (b) identifying themes within the data, (c) developing categories based on these themes, and (d) coding and sorting the data to categorize specific comments into those themes. Following the advice of Brill et al. (2006), all three research team members completed individual analyses and then met as a research team to compare our findings and consolidate those themes into a comprehensive set, being careful to review the raw data to remain true to the comments of our experts.

For example, as we reviewed the data, we found that respondents included comments such as these relating to project management: “Project management cycle,” “backwards planning ability to manage timing of projects,” and “Knowing how to manage large projects, assigning tasks as necessary to meet deadlines & goals; knowing how to coordinate and combine multiple small parts in order to make a cohesive whole.” As a research team, we grouped these comments into a project management theme and category. We then placed all responses relevant to that theme within that category. See Appendix A for a larger sample of the raw data and emerging categories.

This thematic analysis resulted in eight themes or categories: (1) teaching, learning, design, and technology expertise; (2) project management; (3) communication; (4) interpersonal and people skills; (5) inspiring, motivating, and empowering others; (6) environmental and organizational awareness; (7) visioning and strategic alignment; and (8) organizational politics and relationships. Within these categories, we identified 64 statements of competence drawn from the comments of the survey participants. Appendix B includes the resulting categories and the competencies associated with each category.

Round 2 results
Twenty-three participants responded to the round 2 survey, a response rate of 16%. Of this, 52% of respondents had 0-5 years of experience leading or managing instructional designers, and 17% had 6 - 8 years. Additionally, 31% reported over 9 years of experience leading instructional designers. Further, 61% of participants reported that they lead between 1 - 3 designers, another 17% lead 4 - 6 designers, and furthermore 22% of participants lead 7 - 9 designers. In terms of type of institution, 61 % work in public institutions and 39% work in private non-profit institution. Also, 44% of our respondents were female, 56% male. Regarding the highest level of education attained, 35% of respondents had doctorates, 52% had masters, 4% had bachelors, and another 9% reported other as their highest level of education.

Importance of competencies
We used descriptive statistics to determine the relative importance of each of the 64 competencies identified in the first survey. The results ranged from the highest mean (M = 4.81) to lowest mean (M = 2.95) out of five possible points. Table 2 summarizes the top ranked competencies, with communicate effectively in writing and in speaking across all levels of management; and accept change and help to implement it at the institution rated highest at M=4.81.

Table 2. The 15 top ranked leadership and management competencies.

The lowest rated competency was solve technical problems (M = 2.95). This was followed by understand personality styles; and create a budget with M = 3.43 and 3.52 respectively. Draw on experience teaching in higher education; and know classroom design principles followed with M = 3.57 and 3.62 respectively. Table 3 summarizes the lowest ranked competencies in this study.

Table 3. The 5 lowest ranked leadership and management competencies.

The competencies in the communication category rated highest with M = 4.68. This was followed by project management, visioning and strategic alignment competencies at M = 4.33 and 4.29 respectively. Organizational politics and relationships; environmental and organizational awareness; and inspiring, motivating, and empowering others followed. See Table 4 below. Appendix Bincludes all competencies rated in this study and their relative ranking by survey participants.

Table 4. Overall Rating of Competency Categories

Interpretation and discussion
Our first goal was to determine which competencies managers of instructional designers believe are necessary for effectively leading and managing instructional design in higher education. While our results are preliminary, they have identified eight categories of competency and several individual competencies.

When comparing the instructional designer competencies espoused by the International Board of Standards for Training, Performance, and Instruction (Koszalka, Russ-Eft, & Reiser, 2012) with the competencies in this study, there are some clear similarities. For example, there are clear similarities in the categories of Professional Foundations, Design and Development, Evaluation and Implementation, and Management competencies. However, the way in which these competencies are applied is likely different due to the clearly different context in which the skills are applied. Two categories identified in this study – (1) environmental and organizational awareness, and (2) visioning and strategic alignment – are not reflected in the IBSTPI competencies. If many of the graduates of instructional design programs become leaders and managers, then educators might consider including these competencies in their associated graduate program goals. Appendix C provides a comparison of IBSTPI instructional designer competencies and those identified in this study.

Our second goal was to identify the competencies that managers of instructional designers believe are most important for effectively leading and managing instructional design in higher education. We identified Communication, Project Management, and Visioning and Strategic Alignment as the top three competency categories for the study. We were surprised by the relatively low ranking of the Teaching, Learning, Design, and Technology Expertise categories, as compared to the ranking of the other competencies. Still, participants gave it an overall ranking of 3.9, just below the threshold for very important, indicating that these lower-ranking competencies are still important in instructional design leadership and management.

In reflecting on the results of this study, we believe it is important to compare what we have found with what has been written previously regarding instructional design leadership, specifically comparing these results with the 7PL4ID model (Ashbaugh & Piña, 2014). Table 5 provides a comparison of the 7PL4ID model (Ashbaugh & Piña, 2014) and the results of this study. The term Prescience has similar meaning to the Visioning and Strategic Alignment category, and Preventive or Proactive Thinking in the 7PL4ID is related to our Environmental and Organizational Awareness category. Personality, as described by Ashbaugh and Piña (2014) aligns with the Communication, Interpersonal Skills and People Skills, and Politics and Relationships categories. Also, 7PL4ID’s Productivity appears to relate to Inspiring, Motivating, and Empowering Others, as well as Project Management. There did not immediately appear to be a relationship with the Provision for Unexpected and Unknown, Psychological/Emotional Toughness, or Personal Convictions categories described in 7PL4ID. The category of Teaching, Learning, Design, and Technology Expertise, our lowest-ranked category, did not immediately appear to align with 7PL4IDs.

Table 5. Comparing the 7PL4ID and the results of this study.

Implications for practice
Universities and employers of instructional designers may benefit from considering these competencies when hiring and developing instructional design leaders and managers. Although the instructional design-specific competencies were rated lower than other competencies by those surveyed in this study, the competencies were still rated as very important by the participants. This implies that design, technology, and learning-specific competencies should be considered when hiring managers and leaders of instructional design.

Implications for educators of instructional designers
In instructional design programs, the focus of educators is typically to prepare graduates to design instructional interventions (Koszalka, Russ-Eft, & Reiser, 2012). However, instructional designers often assume a director or manager role in higher education and might benefit from formal leadership and management training in their studies. If we do not concern ourselves with the development of leadership and management capacities, we acquiesce leadership to others who may have less expertise in learning, instruction, processes, and systems for supporting and improving learning. We as a field should assume a greater role in leadership and decision-making in higher education.

Limitations
This study includes several limitations. We only studied the perspective of managers of instructional designers and not the views of others, including their managers or the people they lead. Another limitation is the low response rate of the first survey. The sample size of participants also limited our ability to draw conclusions with certainty based on the results. This survey only provides a snapshot of the overall competencies deemed important to effectively lead and manage design by instructional design leaders and managers. How they are applied may change from person to person and potentially change over time. These results are limited to instructional design leadership in higher education settings, and the results may not be generalizable to other environments such as business or government. Further research would clarify the importance and impact of these competencies in other settings.

Future work
Our goal was to identify the competencies that instructional design leaders and managers believe are important for leading and managing instructional designers in higher education. More research is needed to further clarify these competencies, including the situations, contexts, and strategies in which these competencies are employed. Analyzing the job descriptions of leaders of instructional designers would yield greater understanding of these competencies. It would be meaningful to investigate the competencies identified by those who manage and are managed by leaders of instructional designers. In addition, further research related to the importance of design, teaching, and learning competencies in leading instructional designers is warranted. Research of this kind would help clarify this and other studies on leadership and provide more evidence related to training in instructional design leadership and management competencies instructional design programs.

Conclusion
Leading and managing instructional design in higher education requires different skills than traditional instructional design skills. In this study, we have investigated these leadership and management skills. More research on the competencies associated with leading and managing instructional designers is warranted. In instructional design and distance education in higher education, we must be aware of these competences so that we can effectively prepare our students and employees to lead.

References
Ashbaugh, M. L. (2011). Online pedagogical quality questioned: Probing instructional designers’ perceptions of leadership competencies critical to practice (Doctoral dissertation). Retrieved from ProQuest (UMI No. 3460621)

Ashbaugh, M. L. & Piña, A. (2014). Improving instructional design processes through leadership-thinking and modeling. In B. Hokanson & A. Gibbons (Eds.), Design in educational technology: Design thinking, design process and the design studio (pp. 223-248). New York, NY: Springer.

Beaudoin, M. F. (2003). Distance education leadership for the new century. Online Journal of Distance Learning Administration, 6(2), 2003.

Brigance, S. K. (2011). Leadership in online learning in higher education: Why instructional designers for online learning should lead the way. Performance Improvement, 50(10), 43-48.

Brill, J. M., Bishop, M. J., & Walker, A. E. (2006). The competencies and characteristics required of an effective project manager: A web-based Delphi study. Educational Technology Research and Development, 54(2), 115-140.

Cronbach, L. J. (1971). Test validation. American Council on Education, Washington, DC, pp. 443-507.

Dashtahi, A. K., Ekrami, M., Navehebrahim, A., & Sarmad, M. R. (2016). Application of leadership theories in higher distance education leadership. International Journal of Humanities and Cultural Studies, 1(1), 1087-1097.

da Silva, A. R. L., Diana, J. B., & Catapan, A. H. (2015). Management and instructional design: Building intersections. US-China Education Review, 5(2), 133-138.

Intentional Futures. (2016). Instructional design in higher education: A report on the role, workflow, and experience of instructional designers. Retrieved from https://intentionalfutures.com/wp-content/uploads/2017/08/Instructional-Design-in-Higher-Education-Report.pdf

Hasson, F., Keeney, S., & McKenna, H. (2000). Research guidelines for the Delphi survey technique. Journal of Advanced Nursing, 32(4), 1008-1015.

Koszalka, T. A., Russ-Eft, D. F., Reiser, R. (2013). Instructional design competencies: The standards (4th ed.). Charlotte, NC: Information Age Publishing.

Madlock, P. E., & Kennedy-Lightsey, C. (2010). The effects of supervisors’ verbal aggressiveness and mentoring on their subordinates. Journal of Business Communication, 47(1), 42-62. doi:10.1177/0021943609353511

Merrill, M. (2007). The proper study of instructional design. In J. V. Dempsey & R. Reiser (Eds.), Trends and issues in instructional design and technology (2nd ed., pp. 336-341). Upper Saddle, NJ: Prentice Hall.

Nworie, J. (2012). Applying leadership theories to distance education leadership. Online Journal of Distance Learning Administration, 15(4), n4.

Rubley, J. N. (2016). Instructional designers in higher ed: Changing the course of next-generation learning. Washington, D.C.: Chronicle of Higher Education.

Rivard, S. & Huff, S. L. (1988). Factors of success for end-user computing. Communications of the ACM, 31(5), 552-561.

Shaw, K. (2012). Leadership through instructional design in higher education. Online Journal of Distance Learning Administration, 15(3), n3.

U.S. Department of Education, Office of Postsecondary Education. (n.d.) The database of accredited postsecondary institutions and programs. Retrieved from https://ope.ed.gov/accreditation/

Xu, J., & Cooper Thomas, H. (2011). How can leaders achieve high employee engagement?. Leadership & Organization Development Journal, 32(4), 399-416.

Wang, D. S., & Hsieh, C. C. (2013). The effect of authentic leadership on employee trust and employee engagement. Social Behavior and Personality: An International Journal, 41(4), 613-624.

West, R. E. & Borup, J. (2014). An analysis of a decade of research in 10 instructional design and technology journals. British Journal of Educational Technology, 45(4), 545-556.
Zawacki-Richter, O. (2009). Research areas in distance education: A Delphi study. International Review of Research in Open and Distance Learning, 10(3), 1.

Appendix A: Raw Data Coding Sheet – Delphi StudySurvey 1 with Coding (Questions 1 and 3).

Question 1: What management knowledge do you believe a manager of instructional designers in higher education must possess in order to be successful?

Project management cycle

How to budget, how to set priorities, how to create workflow processes, how to track workflow processes

people skills; operational knowledge; understand how academic technology can be used to educate someone

I don't believe that the management knowledge required is distinct to IDs in higher education. Managing people requires the manager to be proficient with institutional structures, policies, culture, mission/vision, and the vertical in which they operate (higher ed here). It also requires the manager to know the direct reports they are managing: what their goals are, how they fit within the model, where they want to grow and develop, how they function with the team, and what their strengths are. A manager should understand motivational techniques, basics of human resource management, and a variety of performance improvement strategies.

backwards planning ability to manage timing of projects

Deep knowledge of ID practice (design skills and theory), project management strategies,

Understanding the principles of sound pedagogy

skills needed by staff, workflow solution for instructional design process, monitor progress, understanding of workload

collaborative leadership styles; servant leadership styles; knowledge of how resources are located at the central and sub-unity levels

(So much depends on the definition of "instructional design" you're employing. In higher education, there are different ways of understanding this work. The term is not used synonymously in all contexts.) In addition to the usual things all managers need? How to motivate people to orient themselves to instructional design work in ways that align with the mission of my center; how to identify the right instructional designer for the task at hand; how to identify appropriate professional development opportunities for instructional designers (not always a simple matter)

how to organize work teams, how to match skills with needs

Knowing how to manage large projects, assigning tasks as necessary to meet deadlines & goals; knowing how to coordinate and combine multiple small parts in order to make a cohesive whole.

The manager must know how to manage technical employees, who tend to be more independent workers than in some other fields. The manager should have an equal or superior knowledge of instructional design to also be an educator.

Question 3: What leadership knowledge do you believe a manager of instructional designers in higher education must possess in order to be successful?

Strategic thinking, visioning, leading others, leading change and goal/objective attainment

motivational theory, leadership styles, personality styles

Visionary; delegation; persuasion; strategy; communicator; change agent

To be successful leaders, managers must have knowledge of ID theory and practice, knowledge of scholarship in teaching and learning, awareness of industry trends and technologies, knowledge of where instructional deficits lay in the industry, and knowledge of their own biases in decision making.

motivational techniques

Importance of short- vs long-term planning, not much you can teach here: intuitive observation of good leaders and awareness of strategies that work or not

Knowing team's strengths and weaknesses

how to get buy in, what motivates people,

collaborative leadership; servant leadership; motivation;

theories of motivation and how they align with instructional development work

organizational awareness

Understanding motivational principles and team dynamics. I think that the manager of instructional designers needs only know the basics of leadership.

I think that the manager of instructional designers needs only know the basics of leadership.