Problem

Leadership is widely recognized as an essential competency for clinicians in order to provide accessible, high-quality health care, and promote collaborative change within the healthcare system. Health care has lagged behind many other industries in terms of leadership training despite this consensus [Reference Rotenstein, Sadun and Jena1,Reference Lerman and Jameson2]. Since 2012, peer-reviewed literature shows a marked increase in papers discussing clinician leaders. Studies have also demonstrated that training clinician leaders improves professional satisfaction [Reference Shanafelt, Gorringe and Menaker3]. Courses to train clinician leaders have expanded in response to increasing demand. This is an encouraging development given that health care tends to select leaders based on demonstrated clinical excellence or scientific innovation, but these forms of success do not necessarily translate into strong leadership proficiency for clinicians [Reference Lobas4].

The barriers to adding leadership development to already packed professional school curricula are numerous and well documented [Reference Cadieux, Lingard, Kwiatkowski, Van Deven, Bryant and Tithecott5,Reference Barry, Grunberg and Kleber6]. For example, at medical schools, leadership training tends to fall into three categories: (1) dual degree programs (MD/MBA); (2) longitudinal tracks with a leadership focus; and (3) classes that range from a one-off session often consisting of autobiographical soliloquies to multiple, months-long components using case-based learning. The Association of American Medical Colleges (AAMC) published explicit entrustable professional activities, or competencies for medical school graduates regarding the ability to collaborate on an interprofessional team [7]. This led to a medical education curriculum redesign, which emphasizes team-based care, including teamwork, team leadership, and change in leadership skills [Reference Beck8].

There are a growing number of continuing education leadership training opportunities for clinicians at varied career stages. In most of these examples, trainees opt in to the leadership development track or course, which increases the likelihood that participants are actively seeking a management or leadership-oriented career. It does not account for those who might become “accidental” or “volunTOLD” leaders at some point in their career. For example, after a medical director left the practice, a participant found himself taking on the role, as there were no other available or willing physicians to do it. Those who find themselves in new, and sometimes unexpected roles, often look for just-in-time leadership training to support them [Reference Sardell9].

As noted, there are significant barriers and issues with leadership training across all levels of training. A major barrier is an access to resources for training, where resources are defined as money and time [Reference Sardell9,Reference Bharwani, Kline, Patterson and Craighead10]. In this paper, we aim to describe a novel community-academic online clinical leadership training program, which aims to make leadership training content more relevant and accessible.

Approach

The Harvard/Clinical Directors Network (CDN) e-Leadership Academy was developed as a community–academic partnership between the Harvard Medical School Center for Primary Care and CDN in 2018, after a successful three-webinar pilot series in 2017–2018.

CDN is a not-for-profit clinician membership organization, practice-based research network (PBRN), and clinician training organization. CDN provides peer-based activities for clinicians practicing in Federally Qualified Health Centers (FQHCs) and other safety net primary care practices serving low-income, minority, and other under-resourced communities. CDN’s goal is to translate clinical research into clinical practice for the enhancement of health equity and improvement of public health. Founded as a clinical leadership training organization and PBRN based on the principle of peer-to-peer learning and support, CDN originally presented in-person clinical leadership training through conferences and workshops in the 1990s [Reference Sardell9], and then migrated to online CME-accredited webcasts in 2000. CDN has designed and conducted over 1000 online CME-accredited programs in collaboration with other institutions designed to engage and encourage clinical leaders and team members to critically examine their delivery of care. Most of these programs carry enduring CME credits and are available on demand from the CDN Webcast Library (https://www.CDNetwork.org/Library).

The Harvard Medical School Center for Primary Care (https://primarycare.hms.harvard.edu/) works to strengthen health care through the transformation of systems, teams, and leaders. Developing leaders within primary care has been a core part of the Center’s mission since its inception in 2011. The Center’s leadership development portfolio includes programs that train leaders at various career stages, from medical students through health system executives [Reference Dwiel, Weilnau, Hunt, Azzara, Phillips and Sullivan11,Reference Duong, Sullivan, Minter-Jordan, Giesen and Ellner12]. The partnership with CDN presented an opportunity for the Center to repurpose and digitize its basic leadership development content for an online and national audience of safety net, health center, and health department clinicians and staff.

This community–academic partnership aligned with many of CDN’s founding principles, which include supporting clinical networks to create professionally satisfying work experiences for clinicians working in underserved areas. CDN has also provided managerial training in response to needs articulated by clinicians in their network since 1986 [Reference Sardell9]. Furthermore, professional satisfaction and improved clinician retention have also been linked to leadership training [Reference Shanafelt, Gorringe and Menaker3,Reference Shanafelt and Swensen13].

As this was a community–academic partnership, CDN and Harvard’s Center for Primary Care conceptualized the online program with input from community-based clinicians and other stakeholders, actively soliciting feedback about content while the program was under development and being delivered. CDN and the Center for Primary Care ensured a sustainability and dissemination plan to make the content accessible beyond the end of the “live” version of the program in an asynchronous on-demand format [Reference Drahota, Meza and Brikho14].

The result of the collaboration was a 100% virtual leadership academy, offered over a 10-month period that covered the fundamental concepts and skills for leading within a clinical practice. We aimed to reach a broader audience with limited time, ability to travel, and financial resources to pay for training. The audience for this program was mainly FQHCs, other community health centers, PBRNs, health departments, and clinical and translational science award grantees across the USA. It is unlikely that these typically resource-constrained organizations would be able to afford sending an entire team to an in-person leadership development program, but the lower cost of distance learning makes it more feasible to support teams in attending a monthly webinar.

Given the interprofessional nature of healthcare teams, we aimed to be as inclusive as possible in participant recruitment. The course was promoted as appropriate for current or aspiring clinical leaders from a variety of backgrounds: physicians, nurse practitioners, physician’s assistants, nurses, pharmacists, administrators, and social workers. Training is often ineffective if only one individual is trained [Reference Eisenstat, Spector and Beer15], and data supporting team training’s efficacy have been increasing over the past 7 years [Reference Miller, Kim, Silverman and Bauer16–Reference Hughes, Gregory and Joseph18]. While the evidence base on the efficacy of team training in non-acute settings lags behind the evidence base for acute settings, the overall evidence on team training’s impact is growing; in particular, the evidence demonstrates that team training can improve team processes, patient safety, and organizational outcomes. Further, team training is effective at any career stage [Reference Hughes, Gregory and Joseph18], thus, we encouraged entire teams to register and participate together in the e-Leadership Academy and a multi-registrant discounted pricing structure was designed to encourage team participation. As the course was accredited for 15 CME credits, many participants used funds from their CME budgets to support team participation.

Studies on clinician leadership development currently lack a consensus on the core skills and competencies clinicians need to lead, such as problem-solving skills and communication skills, high degrees of emotional intelligence, and a deep understanding of topics like quality improvement, payment, and health systems [Reference Frich, Brewster, Cherlin and Bradley19]. We opted to design the curriculum for the e-Leadership Academy based on the Harvard Center for Primary Care’s in-person Leadership Academies that were held in the greater Boston area. In building the e-Leadership Academy, we repurposed existing content from the Center’s in-person physician leadership trainings. We presented the existing in-person curriculum to a representative group of clinicians from FQHCs who are part of CDN’s Board of Directors, in order to receive feedback on course content during the development phase. We also provided an initial course session in which we were able to provide an overview of the program and get feedback from potential participants.

The overarching goal was for participants to learn the fundamental concepts and skills for leading oneself, a team, and change within clinical practice. We created a 10-month series comprised of a monthly 90-minute webinar (see Table 1). The 10 sessions were divided into 3 specific modules, each with their own key objectives:

1. 1. Leadership Concepts (sessions 1–3): Participants will understand fundamental concepts related to leadership and identify their own specific leadership style.

2. 2. Leading Teams (sessions 4–7): Participants will learn how to build and maintain an effective team; develop strategies for hiring and performance evaluation; and optimize communication and have difficult conversations.

3. 3. Leading Change (sessions 8–10): Participants will evaluate frameworks for leading change; assess challenges in responding to change due to the external environment or internal initiatives; and build strategies for resilience and joy in work in a constantly changing healthcare landscape.

Table 1. Harvard/Clinical Directors Network, Inc. (CDN) clinical leadership virtual academy course curriculum

Faculty affiliated with the Harvard Center for Primary Care were recruited based on their willingness to migrate their in-person leadership-related course content to an online format. Faculty were supported in preparing their online materials and planning interactive session activities by the course director and project coordinator. Harvard faculty were responsible for session content, while CDN maintained responsibility for all technical aspects of the course, including the online training infrastructure, CME accreditation, and marketing.

Each session was structured with 50–60 min of content delivery by faculty presenters, drawing heavily on case studies, and included audience engagement activities via polls and chat boxes; chat box responses were summarized in real time and shared with participants using frequency-based word clouds. A 30-minute question and answer session followed didactic content delivery. Participants entered questions into a chat box during the content presentation, and faculty answered questions presented by the CDN moderator. Webinars were held at 12 pm ET on the same day of each month (i.e., second Wednesday) to facilitate participant attendance and faculty participation. Webinars were also available on-demand to all participants so those unable to attend the live sessions were able to attend at their convenience, and participants could review parts or entire sessions again. The complete online course was accredited for up to 15 CME/CNE credits (1.5 credits awarded per session), and carries enduring CME/CNE accreditation so new participants can register and view the full course asynchronously on-demand (www.CDNetwork.org/HarvardLeadership).

Outcomes

Assessment of the e-Leadership Academy included both formative and summative evaluations. In order to receive CME credit for each session, participants were required to complete a five-question quiz after attending, or viewing on-demand, each monthly session. Additionally, for each session, we included three–four questions related to participant satisfaction, feedback on course content, and feedback on the faculty presenter. For this article, we primarily analyzed the participant responses to the final course survey. The survey was conducted at the conclusion of the course to assess course quality, participant satisfaction, and provide data about future offerings that would be useful to our target audience.

Participation and Engagement

Two hundred seventy-six people from 36 states/territories participated, the top 3 states being California (16.7%), New York (16.3%), and Maryland (9.1%) (see Fig. 1). The majority (72%) of the cohort registered as part of teams, with 198 teams represented in the program; teams ranged in size from 2 to 9 participants (45%), 10 to 20 participants (19%), or 21+ participants (8%). A total of 78 registrants (28%) enrolled individually, rather than as part of a team. Anecdotally, we know some groups chose to watch and discuss each webinar together as a team, while other groups had their members watch the webinars individually and came together subsequently as a team, either monthly or quarterly, to discuss how to apply the content to their practice. A total of 15 different occupations were represented, with physicians (46%) and administrators (18%) being the largest categories. A total of 97 unique organizations were represented in the program; those with 10 or more registrants included medical schools, health systems, and hospitals.

Fig. 1. Geographic distribution of registrants.

In addition to survey responses, a more objective method of measuring participant engagement was through the use of Zoom “attention scores.” Attention scores were captured passively and defined as the percent of time a participant had the webinar window active and in the foreground on their device. Conversely, participants were considered “inactive” if they clicked away from the live webinar window to use a different program. Participants displayed a cumulative average attention score of 88% during the course sessions, with a total of 85,364 active minutes of attention to the course programming (see Table 2). On average, 107 (71.3%) participants viewed the webinars live each month, and 43 (28.7%) viewed the on-demand recordings (yielding an average of 150 viewers per month overall).

Table 2. Participant attendance and engagement

* Attention scores are calculated by Zoom and it is defined as the percent of time the Zoom webinar window was the active screen on the participant’s device.

Satisfaction Data

Quantitative data from participant evaluations were collected on a 5-point Likert scale ranging from 1 = Poor to 5 = Excellent and analyzed both during, and at the conclusion of the course. In the final evaluation, 96% of participants reported that they were either “Very Satisfied” or “Satisfied” with the course; 92% said that they were likely to recommend it to a colleague/friend, and 88% indicated that the course content will help their daily practice. Qualitative data provided in response to open-ended questions in the course evaluations demonstrated that participants found the entire course valuable, many reporting sentiments such as “As a new medical director of a primary care clinic, the entire course was valuable,” or “Participating as a team has been exceptionally beneficial to the organization.” Participants also suggested that they would use course content to improve their current work environment, “Hearing a variety of examples and suggestions for ways to improve the work environment was helpful,” or “This course helped me to learn how to listen and to implement changes in a way that will be comfortable to my staff.”

The following aspects of the course were rated as the most valuable: leadership concepts and theories, content related to the work environment (team building, burnout, culture, workforce), and managing difficult situations. Respondents rated the subject-matter organization and delivery on a 5-point Likert scale ranging from 1 = Poor to 5 = Excellent and 99% rated the subject-matter organization and delivery of the course as “Excellent” or “Good”; and 96% of respondents rated the quality of the course as “Excellent” or “Good.”

In the final survey, we asked participants about modalities that best support their professional development goals. Over 50% of respondents reported that they preferred an entirely virtual course (see Fig. 2). We also learned that during the live virtual format, our audience felt connected to each other via the “chat” feature and the Q&A segment of each session, where they reported being able to share insights or learn from other organizations. Participants also found the course accessible, commenting that “once a month is easier to keep up with for someone with a busy schedule” and “having access to the recordings afterward was a great feature for the meetings.”

Fig. 2. Preferred modality for continuing education.

Limitations

One limitation is related to our monthly participation rates, which averaged 150 viewers per month (out of 276, 54%). These results may underestimate the true attendance numbers for two specific reasons. First, we know, anecdotally, that members of a team watched the program together at a single computer or via a conference room screen. In those instances, that would only count as one participant view. Second, while all viewers were asked to complete the session evaluation each month, that evaluation was connected to CME credit. If a participant did not need the CME credit, it is possible they did not fill out the evaluation. Finally, another consideration for participation rates relates to the possibility that groups of participants were enrolled by a senior leader, but some of those participants had varied interests or availability to attend the program.

In this preliminary collaboration, we did not collect any independent measures of implementation or impact of the new content in clinical practice. Future training programs could measure the changes in the behavior of teams (process) and evaluate the clinical impact (outcomes) using electronic health records (EHR) data. Nevertheless, the course was designed to provide learners with an opportunity to assimilate what they are learning, try new ways of leading and managing, and reflect on those actions or experiments.

Next Steps

We plan to sustain and disseminate our current e-Leadership Academy through an online portal, which contains the course content, supplemental reading materials, and tools and templates for participants. While we were able to leverage the existing CME budgets of many FQHCs to offset registration fees, we continue to seek new sources of funding (i.e., novel trainings, philanthropic support) to support additional participant access and expansion of our offerings.

The Harvard/CDN e-Leadership Academy focuses on delivering relevant content so that our clinical audience can acquire new knowledge and team leadership skills. We acknowledge that acquiring new knowledge and skills does not necessarily translate into measurable action. To that end, learners in Year 2 of the e-Leadership Academy will view recorded webinars from Year 1, and complete an activity that allows them to directly apply the learning principles into their practice settings during the webinar. Then, the cohort will attend bimonthly office hours, which are structured to increase participant interaction and engagement with faculty and other learners, and create a space to reflect and seek feedback from faculty and peers. This increases the potential audience and maximizes participant–faculty interaction using “flipped classroom” principles [Reference Huang, Chou, Leu, You, Tiao and Chen20].

Future plans for expansion of the e-Leadership Academy include developing additional modules, or short courses, which build on the initial curriculum and provide more advanced, in-depth content tailored to training gaps identified by our audience. Prior to the COVID-19 pandemic, Harvard/CDN was working to identify a way to position the current program as a blended, longitudinal program with an in-person component, such as at the beginning and/or at the end of the series, perhaps conducted during a national conference with high levels of attendance by participants. An in-person component would allow the curriculum to include more interactive activities, such as role plays and simulations, in which participants can practice leading and receive real-time feedback from peers. Given the pandemic and concurrent rapid increase of distance learning options, immediate future plans for expanding the e-Leadership Academy include more individualized options for participants, such as virtual coaching and mentoring to provide real-time support to frontline clinical leaders.

We presented here a reproducible, scalable model for developing, disseminating, and evaluating a community-academic learning collaborative that is enduring and far-reaching, and was well received by a wide range of clinicians and other healthcare workers. The key components of creating the e-Leadership Academy included an emphasis on team-based learning, partnerships, content curation, faculty support and recruitment, marketing and participant recruitment, easy-to-use web-based technology, and ongoing evaluation. We identified several innovations, extensive use of chat to provide a communications platform for participants, and real-time qualitative analysis and feedback to participants using word clouds, in addition to using attention scores. We believe the model is viable, timely, and critical within the context of public health emergencies, such as COVID-19, in providing on-demand content as well as relevant additional short courses that provide just-in-time information for managing and leading during a crisis. The virtual model also allows participants to strengthen their connections to each other and leverage CDN’s long-standing peer support network that remains accessible virtually and beyond the conclusion of any one specific program.