Introduction
Analyses of federal, state, and local data in the USA confirm the burden that the COVID-19 pandemic has placed on Black, Indigenous, and People of Color (BIPOC) [Reference Mackey, Ayers and Kondo1–Reference Dorn, Cooney and Sabin4]. This has exacerbated existing health disparities for BIPOC populations who have experienced disproportionately higher rates of infection, hospitalization, and death [Reference Yan, Hwang, Ng, Chu, Tsoh and Nguyen5–Reference Arrazola, Masiello and Joshi8].
To address these disparities and the challenges of COVID-19, research, policy, clinical, and patient stakeholder groups are strongly advocating for a more patient-centered and health equity focused research agenda [Reference Quinn and Kauh9]. To achieve this, active engagement and partnerships between patients and communities most impacted by COVID-19 and researchers are required to guide research study development and implementation, as well as the dissemination of research findings.
However, the COVID-19 pandemic has presented unique challenges that have had a significant adverse impact on patient and community engagement in research. These challenges include shelter in-place orders, the rapid transition to virtual platforms for communication, public mistrust of science, as well as the magnification of healthcare disparities. Members of the British Medical Journal’s patient and public advisory panel noted that the “nothing about us without us” principle of patient engagement in research has often been ignored during the COVID-19 pandemic [Reference Richards and Scowcroft10]. This was confirmed by a report, published by the National Health Service (NHS) in the United Kingdom, during the early phases of the pandemic that found that the number of studies sponsored by the NHS in which patients were engaged in designing, managing or disseminating research fell from 80% to 22%. This was despite an expectation from the NHS sponsor for involvement of community and patient stakeholders in research [Reference Denegri and Starling11, 12]. Researchers self-reported that the need to rapidly set up studies prevented or removed the usual need for patient engagement in design, conduct or review [Reference Denegri and Starling11]. While this perspective may have been justifiable during the very early phases of the pandemic, many current funding calls for COVID-19 research still do not mention the need to include patients or community members as partners [Reference Cukier13]. Even as COVID-19 related trials proliferated, many research teams were ill-equipped to recruit BIPOC participants most impacted by COVID-19 [Reference Chastain, Osae, Henao-Martínez, Franco-Paredes, Chastain and Young14]. This resulted in researchers mid-study seeking assistance from community engagement programs at academic institutions [Reference Grumbach, Cottler and Brown15]. This created understandable frustrations from community members at being asked to inform studies midstream instead of being involved as partners from the beginning [Reference Grumbach, Cottler and Brown15].
Outside of COVID-19 research, there has been a necessity to adapt clinical and translational research on other topics to meet restrictions placed by the COVID-19 public health response. Even prior to COVID-19, patient and community engagement in research was not universal with many researchers not considering engagement an essential part of the research process and often lacked the skills to create and sustain academic-community partnerships [Reference Shea, Young and Powell16].
To ensure that research conducted throughout the pandemic remains patient-centered, systematic engagement of patient and community partners must be operationalized. This includes developing strategies where patients and community members can meaningfully engage with researchers to provide input on emerging COVID-19 research, as well as to adapt existing research to a COVID-19 landscape. Therefore, the objective of this article is to describe the rapid implementation and evaluation of a COVID-19 Research Patient and Community Advisory Board (PCAB) at the University of California San Francisco (UCSF). The COVID-19 Research PCAB was created to provide dynamic, multi-layered patient, and community input into clinical and translational research studies.
Materials and Methods
Below, we describe the process of creation and governance of the COVID-19 Research PCAB, the implementation of the research study consultation process, and the design of the evaluation of the PCAB.
Setting and Context
UCSF is a large urban academic medical center and university dedicated exclusively to health sciences. The Accelerating Systematic Stakeholder, Patient, and Institution Research Engagement (ASPIRE) [17] project within the UCSF Clinical and Translational Science Institute (CTSI) [18] seeks to create institutional mechanisms to provide patient and community stakeholder input into clinical and translation research projects. ASPIRE was funded by the Patient Centered Outcomes Research Institute (PCORI) and facilitated several efforts to understand, and then actively address, structural and systemic institutional barriers related to patient and community engagement in research. At the beginning of the COVID-19 pandemic, ASPIRE leaders noted the rapid emergence of COVID-19 focused research, and, using funds from a PCORI supplement, pivoted to create institutional infrastructure and processes that facilitated patient and community consultation on COVID-19 research studies and institutional policies. Creation of an institutional PCAB was necessary, as while individual Patient & Family Advisory Councils (PFACs) and Community Advisory Boards (CABs) existed prior to the pandemic for specific health system or research projects, few if any were centrally coordinated by the CTSI or were designed for access by researchers across the entire institution. The UCSF Institutional Review Board (IRB) approved activities related to the evaluation of the PCAB.
Creation of the COVID-19 Research PCAB
Recruitment of patient and community advisors to the PCAB
We conducted two forms of outreach to identify and invite potential patient and community members to join the PCAB. Primary outreach involved contacting participants who were currently, or had previously, participated in stakeholder engaged activities at UCSF (e.g., community advisory boards, patient, and family advisory councils) and were known to the UCSF team [Reference Palinkas, Horwitz, Green, Wisdom, Duan and Hoagwood19]. Secondary outreach involved contacting the leaders of existing patient and community advisory boards at UCSF and the San Francisco Bay Area to publicize the PCAB opportunity [Reference Parker, Scott, Geddes, Atkinson, Delamont, Cernat, Sakshaug and Williams20]. These leaders were then asked to connect the UCSF team with any interested individuals to follow up with them about joining the PCAB. Ensuring diverse representation was a guiding principle from the outset of recruitment, including diversity related to age, gender, race, ethnicity, sexual orientation, occupation, and life experiences. Given the urgent need to create the PCAB during COVID-19, it was preferable if potential members had previous experience working on an advisory board and/or research projects, though this was not mandatory. Completion of an application form to join the PCAB was not required.
Orientation of patient and community advisors to the PCAB
Potential PCAB members were emailed an invitation and overview of PCAB working procedures. Individuals interested in participating were invited to attend a virtual orientation session where they met the UCSF team, were orientated to PCAB operations, and had an opportunity to have their questions answered. Those who were unable to attend were asked to review a Zoom video recording of the session and if necessary, attend a one-on-one follow-up meeting with a member of the UCSF team. Following orientation, potential members were asked to confirm their willingness to participate on the PCAB.
Ongoing engagement with PCAB members
Communication and engagement with the PCAB members included a biweekly request for availability for future consultation sessions. PCAB members were also asked to participate in a quarterly all-PCAB meeting, which provided opportunities for relationship building between PCAB members and the UCSF team and for the dissemination of program information and updates.
Staffing
The PCAB was managed and coordinated by a team of staff, faculty, and a community investigator. A 0.20 FTE project coordinator and 0.05 FTE program manager was responsible for managing the day-to-day operations including tracking consultation requests, coordinating research consultation sessions, and communicating with PCAB members and researchers. In addition, four UCSF faculty researchers and a community investigator were included as part of the PCAB staffing team.
COVID-19 Research PCAB Proposal Consultation Process
Activities related to coordinating and conducting the research proposal consultation process can be categorized into five distinct domains that are summarized below and in Fig. 1.
Fig. 1. Overview of COVID-19 research PCAB consultation process. COVID-19 = SAR-CoV-2 coronavirus disease 19; PCAB = Patient and community advisory board; UCSF = University of California San Francisco, CTSI = clinical and translational science institute.
Research team engagement and submission of consultation request
The COVID-19 Research PCAB was advertised as a resource to UCSF investigators with COVID-19 research funding via CTSI newsletters and announcements [21]. A dedicated CTSI research consultation services webpage was also created [22]. In addition, the UCSF IRB shared information with the UCSF team about active COVID-19 research studies, including principal investigator name, and the UCSF team sent individual emails to those investigators encouraging them to submit their study for PCAB consultation. Researchers interested in a PCAB consultation were asked to submit an online consultation request to initiate the process.
Request for information from the study team
Once researchers had completed an online consultation request, they were sent an email asking them to submit 1 to 2 pages of a plain language summary describing their study, up to three focused questions for the PCAB’s specific input, and information regarding the stage of project (ideation, protocol development, implementation), funding status, details of any previous or current community input into their study, and their availability to participate in a consultation (see Supplemental Material 1). The plain language summary statement was reviewed by members of the UCSF team, who then worked with the researcher to edit the plain language summary to ensure suitability of language and content for the PCAB, and refine the questions proposed to the PCAB.
Creation of PCAB consultation session panel
Plain language summaries were sent by email to all members of the PCAB by the PCAB coordinator. PCAB members were asked to respond and provide information about their interest in the topic, previous experience/knowledge of the topic, and availability for participating in a consultation session. These responses were reviewed by the UCSF team to select 5 to 7 PCAB members for each consultation session. Other considerations in selecting PCAB members related to ensuring a diverse panel and fair distribution of opportunity to participate. At this same time, a consultation session chair, facilitator, and note-taker were selected from the UCSF team to participate. Selected PCAB and UCSF team members were asked to review the plain language summary and questions submitted by the research team prior to the consultation session.
Convening of 1-hour PCAB consultation session
A 1-hour virtual PCAB consultation was held via Zoom within 7–10 days of the study team requesting a consultation. There was an expectation that a faculty researcher in addition to study coordinators and staff attend. At the beginning of each consultation, one of the UCSF team members facilitated a welcome, introductions, and provided an overview of the flow of the meeting and the roles of the UCSF team. The facilitator guide is provided in Supplemental Material 2.
Researchers on the study team were then asked to provide a brief 5-minute description of their project and accompanying questions. The facilitator then moderated a 35–40-min discussion allowing PCAB members to provide their input and responses to the questions posed. Researchers were asked to only respond to clarifying questions from the PCAB and to not respond directly to each piece of feedback until the end of the facilitated discussion, thereby maximizing the amount of time for hearing from PCAB members. Researchers were then allocated 10 min to provide their responses and reflections. In the last 5 minutes of the consultation, the meeting chair summarized the main discussion points and adjourned the meeting. Consultations were digitally recorded, and a note-taker recorded the main discussion points. Technical support for PCAB members using Zoom was available if needed.
Post consultation session follow-up
Immediately after each consultation, the UCSF team met to debrief and provide reflections. Within 7 days of the consultation, the research team would receive the minutes of their consultation, and a transcript of messages written in Zoom chat. PCAB members would receive an email thanking them for their participation and were sent $125 honoraria that acknowledged their expertise and time spent preparing for and participating in each consultation.
Evaluation of the COVID-19 Research PCAB
We created an evaluation working group comprised of the UCSF team and volunteers from the PCAB. The working group identified the following four areas of focus for the evaluation: (1) PCAB processes and operations; (2) PCAB member experience and satisfaction; (3) Researcher experience and satisfaction; and (4) Overall lessons and learnings from COVID-19 Research PCAB.
PCAB processes and operations
To describe PCAB process and operations, we captured the following information: number of PCAB members recruited and their sociodemographic characteristics, number of consultations completed, number of PCAB and researcher participants at each consultation, research topic, and design of studies reviewed.
PCAB member experience and satisfaction
We developed a survey for PCAB members to capture their experiences, perspectives, and suggestions for improvement of the consultation process. The content of the survey was informed by questions from previously developed instruments that evaluated patient engagement in research from the UCSF CTSI and PCORI [23]. An initial draft survey comprised of 50 questions was sent to academic and community members of the ASPIRE Advisory Board for feedback. The final surveys included 24 fixed and open-ended response questions that assessed PCAB member perspectives of community member-research partnerships, preparedness for consultations, satisfaction with consultations, perceptions of being respected and listened to, and PCAB operational processes (e.g., access to staff, diversity of the PCAB) and sociodemographic characteristics (Supplemental Material 3). Surveys were sent to PCAB members to complete anonymously up to 3 months after PCAB members had participated in a review session. Survey distribution was managed using Qualtrics (Provo, UT) and non-responders were sent two reminders [Reference Van Mol24].
Researcher experience and satisfaction
We used the same process to develop the researcher survey. The only difference was that the content of surveys was informed by questions from the UCSF CTSI consultation services and community engagement evaluations. The resulting 24 fixed and open-ended response questions assessed researchers’ overall experience and satisfaction with the consultation process, interactions with PCAB staff, faculty and members, and the impact of the consultation on their research (Supplemental Material 4). Surveys were sent to researchers immediately after their review session using Qualtrics (Provo, UT) and non-responders were sent two reminders [Reference Van Mol24].
Overall lessons and learnings from the COVID-19 research PCAB
To understand and describe the lessons and learnings from the PCAB, we accessed the video recordings of review sessions, video recordings of all-PCAB meetings, and minutes from the debriefing sessions conducted by UCSF team members after each review session. We used thematic analysis to systematically identify, analyze and report patterns within the data [Reference Vaismoradi, Turunen and Bondas25]. Two members of the research team independently performed open coding using a data-driven (inductive) approach. To ensure methodological rigor, reviewers met at regular intervals during analysis to develop a code book and to resolve any coding discrepancies using negotiated consensus [Reference Bradley, Curry and Devers26]. Codes were then grouped into higher order themes.
Results
COVID-19 Research PCAB Processes and Operations
We recruited 32 patients and community members to participate in the PCAB. Twenty were recruited by primary outreach (i.e., currently or previously engaged in stakeholder activities at UCSF) and 12 were recruited by secondary outreach (i.e., through leaders of existing patient and community advisory boards at UCSF and the San Francisco Bay Area). Between June 2020 and June 2021, 23 review sessions were completed, each with 5–7 PCAB members participating. Twenty-seven members participated in at least one review session (84% participation rate). The number of members participating in one review session was 1, 2 sessions were 5, 3 sessions were 4, 4 sessions were 4, 5 sessions were 2, 6 sessions were 4, 7 sessions were 1, 8 sessions were 2, 9 sessions were 1, 11 sessions were 1, and 12 sessions were 2. Twenty-two members participated in ≥3 review sessions. Reasons for nonparticipation by the five PCAB members who did not take part in any review sessions were due to scheduling conflicts and/or prioritization by these individuals of other COVID-19 related community engagement activities. Research studies reviewed by the PCAB were broad including vaccine trials, observational studies, and disparities research. Three review sessions focused on informing COVID-19 related institutional research policies and practices. Details of the studies and policies reviewed by the PCAB can be found in Supplemental Material 5.
PCAB Member Experience and Satisfaction
Twenty-two of the 27 PCAB members who participated in at least one session completed an evaluation survey (81% response rate). Sociodemographic characteristics of PCAB members who responded to the survey are shown in Table 1. No PCAB members reported the need to access Zoom-related technical support to access any review sessions. The preponderance of PCAB members agreed or strongly agreed it was a productive and valuable use of their time (86%), they were satisfied with their role in the PCAB (86%), their expectations were met (77%), and they felt they were listened to and valued (82%) (Table 2). When asked to provide additional information about their experiences, PCAB members shared a range of perspectives. Some members noted the value of “shared learnings among PCAB members, researchers and the UCSF team” during consultations, while others felt sessions highlighted the “apparent disconnect between researchers and those people being impacted by their research.” Many PCAB members were hopeful that consultations would lead to “greater exposure to community views of what is important for COVID-19 research.”
Table 1. Sociodemographic characteristics of COVID-19 research PCAB and researcher evaluation survey respondents
COVID-19 = SAR-CoV-2 coronavirus disease 19; PCAB = patient and community advisory board; GED = General education development.
Table 2. PCAB member evaluation of participation and experiences (n = 22) *
COVID = SAR-CoV-2 coronavirus disease 19; PCAB = patient and community advisory board.
* Where responses are missing from individual questions, percentages do not equal 100%.
Researcher Experience and Satisfaction Engaging the PCAB
Nineteen of the 23 researchers who received a consultation completed an evaluation survey (83% response rate). Researchers typically requested a PCAB consultation to ensure community input on their proposal and to determine how the research study could better serve community needs. Other reasons for PCAB engagement were to obtain advice on recruitment strategies and study data collection approaches or materials. Researchers noted the importance of the PCAB and reported very positive experiences related to: (1) their overall experience and value of the PCAB consultation; (2) how it met their expectations for feedback; (3) how well the PCAB consultation services met their needs; and (4) the efficiency of the PCAB review process (Table 3). Researchers also provided examples of the impact of PCAB members feedback and how it influenced their study. This included changes to recruitment strategies, dissemination activities, and community engagement efforts. A summary of the impact of PCAB member feedback is shown in Table 4.
Table 3. Researcher evaluation of PCAB participation and experiences (n = 19) *
COVID-19 = SAR-CoV-2 coronavirus disease 19; PCAB = patient and community advisory board.
* Where responses are missing from individual questions, percentages do not equal 100%.
Table 4. The impact of PCAB member feedback on COVID-19 research studies as reported by researchers
COVID-19 = SAR-CoV-2 coronavirus disease 19; PCAB = patient and community advisory board; CBOs = community-based organizations.
Overall Lessons and Learnings
Lessons and learnings from the COVID-19 Research PCAB are summarized in Table 5. We noted the importance of creating equal partnerships between community members and researchers during the development of COVID-19 research and policies. However, despite this need, there was a significant lack of allocated funding in studies for community engagement. We also recognized skill deficiencies in researchers related to community engagement, such as providing plain language summaries of research and practicing active listening skills. We found that many multi-center COVID-19 research studies were inflexible at meeting the local needs of research participants and that standardized study recruitment, study activity, and retention strategies were often not patient-centered or culturally appropriate. We also found a lack of cultural humility reflected in proposed study activities that often did not consider the implications of the burden and trauma of COVID-19 experienced by BIPOC communities. Examples include proposed high-touch or high-burden study activities in shared household environments or study activities that could cause potential loss of income or privacy. The lack of cultural understanding often resulted in feedback from the PCAB that challenged the feasibility of many proposed study activities.
Table 5. Summary of lessons learned from PCAB process organized by themes and definition
COVID-19 = SAR-CoV-2 coronavirus disease 19; PCAB = patient and community advisory board.
Discussion
COVID-19 has reinforced the urgent need to develop a more patient-centered and health equity focused research agenda to address the health inequities exacerbated by the pandemic [Reference Quinn and Kauh9]. Active engagement between researchers and patients and community members most impacted by COVID-19 is one approach to meeting this need. By leveraging existing institutional capacity and community relationships, we were able to rapidly implement a PCAB that has been accessible to researchers during the development and implementation of COVID-19 research studies and policies. Twenty-seven PCAB members participated in 23 research and policy consultations. Evaluation of the PCAB highlighted high levels of satisfaction and perceived value from both PCAB members and researchers.
The success of the PCAB was possible, despite the persistent and evolving challenges of COVID-19, because we were able to build on existing institutional capacity and relationships established as part of the UCSF CTSI Community Engagement and Health Policy (CE&HP) Program [27]. Engagement awards from PCORI and an award from the NIH Community Engagement Alliance Against COVID-19 program allowed us to leverage the existing relationships and infrastructure to rapidly create a PCAB as a resource for multiple COVID research studies. The PCAB was operational within 6 weeks of a PCORI award, allowing timely and efficient engagement of community members and researchers to meet the urgency of the rapidly evolving pandemic. The fact the PCAB was organized at an institutional level, through the CTSI CE&HP Program, further facilitated a more rapid response rather than expecting individual investigators to conduct their own patient and community engagement. The implementation and sustainability of the PCAB add to a growing list of collaborative efforts between CTSA sites and community partners that have responded to the challenges of COVID-19 [Reference Grumbach, Cottler and Brown15, Reference Eder, Millay and Cottler28, Reference Michener, Aguilar-Gaxiola and Alberti29]. This has included dedicated programs that have listened and responded to community concerns, collected data to describe how COVID-19 has impacted local communities, communicated science, addressed misinformation and strategically engaged public health departments and BIPOC populations in COVID-19 research.
We have been particularly encouraged with the breadth of COVID-related research topics and study designs presented to the PCAB. PCAB members were able to directly interact with researchers and impart their knowledge and expertise. The PCAB structure provides a model that can address the barriers to early involvement of patient and community stakeholders in research that has been reported during the COVID-19 pandemic. The PCAB remains just as important now as it did at the start of the pandemic as we seek to develop research agendas that address the ongoing challenges and health inequities caused by COVID-19 and engage patients and communities as collaborators in this work. Importantly, PCAB consultations also extended to sessions where members provided recommendations to enhance the patient-centeredness of institutional COVID-19 research policies. The opportunities for the PCAB to engage with institutional policy makers, as well as individual research teams, is an important strategy to promote the adoption and sustainability of institution-wide patient and community-engaged research policies and practices. PCAB members reported high levels of satisfaction with the review process, future evaluations of the PCAB may consider if, and how, the nature of feedback changed over the course of the pandemic, or if the participation frequency of PCAB members produced differential feedback. Exploring factors such as these will likely improve our understanding of this specific type of patient and community engagement in research.
As we developed the PCAB, we intentionally invited potential members with diverse backgrounds, and we defined diversity as broadly as possible to capture a range of perspectives. Diverse PCAB membership also provided an important lesson about this model of academic-community engagement. During consultations, PCAB members often described the health, social and economic burden experienced by them and their communities, which resulted in triggers that evoked their individual and collective experiences of cultural, racial, ethnic, and historical trauma. Such reactions by PCAB members are not surprising given that COVID-19 has exacerbated healthcare inequities. However, this observation points to areas where the academic community may need to consider employing a trauma-informed approach to research engagement when partnering with BIPOC communities impacted by COVID-19 [Reference Jumarali, Nnawulezi, Royson, Lippy, Rivera and Toopet30]. We acknowledge that our sampling and recruitment approach of individuals currently or previously involved in stakeholder-engaged activities limited PCAB membership to those with the franchise to participate, however this was necessary due to the urgent need presented by the COVID-19 pandemic to create a PCAB that could inform and guide research.
The PCAB process revealed that researchers struggled preparing lay language summaries of their projects for PCAB members to review in advance of consultations. This required the UCSF team to work with researchers to create intelligible and succinct information. Similarly, within consultation sessions, we purposely had to hold researcher reactions and responses to the end of each review session to maximize the time for input from PCAB members. We found that if we did not, many researchers launched into lengthy justifications or responses to feedback. Based on these experiences, we recommend better training of both trainees, early career researchers, and established researchers on best practices for conversations with patient and community consultants. This could include setting and evaluating investigator competencies for community-engaged research [Reference Matthews, Castillo and Anderson31] and having researchers watch a short video module on ground rules and best practices prior to attending the consultation session.
In summary, by leveraging existing institutional capacity and community relationships, we were able to rapidly implement a COVID-19 Research PCAB. The creation of a systematic consultation process allowed researchers to hear directly from patient and community members who provided valuable insights and recommendations to maximize patient-centered and equity-focused approaches to the conduct of science. Our description of the process of developing, implementing, and evaluating our COVID-19 Research PCAB may be helpful for others wishing to adopt and adapt this model of engagement at their institution.
Supplementary Material
To view supplementary material for this article, please visit https://doi.org/10.1017/cts.2022.413.
Acknowledgements
We would like to thank all COVID-19 Research PCAB members for their passion, commitment, expertise and time: Jim Banta, TaNefer Camara, Lowell Caulder, Bilal Chaney, Diane Dooley, Juno Duenas, Ammo Eisu, Justin Flores, David Goldstein, Steve Heilig, Artanesha Jackson, Loren Jones, Stutee Khandelwal, Monique LeSarre, Patricia Levenberg, Devon Low, Leticia Márquez-Magaña, Sabra Matovsky, Ting Pun, Ramon Ramirez, Andrew Reynolds, Teri Rose, Zanna de Sant’Anna, Angela Sun, Adam Sussman, Gina Symczak, Lisa Tealer, Patsy Tito, Renee Underwood, Arianna Wassmann, Hiroshi Yamaguchi, and Jorge Zepeda. We are also grateful to the researchers who received consultations and completed evaluations.
This work was supported by the Patient-Centered Outcomes Research Institute (PCORI) Eugene Washington PCORI Engagement Award (10566-UCSF), the UCSF Clinical and Translational Science Institute funded by the National Center for Advancing Translational Sciences at the National Institutes of Health (NIH UL1TR001872), and an award from the National Institutes of Health, Community Engagement Alliance Against COVID-19 program (Grant 21-312-0217571-66106L). James Harrison was supported in part by the Agency for Healthcare Research and Quality (K12HS026383). Nynikka Palmer was supported in part by the National Cancer Institute/NIH (K01CA211965). The content is solely the responsibility of the authors and does not necessarily represent the official views of PCORI or the NIH. The funding bodies played no role in the design of the study and collection, analysis, and interpretation of data and in writing the manuscript.
Disclosures
The authors have no conflicts of interest to report.
Open access
