Introduction
Mental health challenges among college students have grown over the past decade, with the COVID-19 pandemic only making matters worse with respect to growing rates of depression, anxiety, suicidality, and other mental health challenges (Cuijpers, Reference Cuijpers2021; Duffy et al., Reference Duffy, Twenge and Joiner2019). Mental health difficulties are also strongly associated with poor academic performance and attrition (Mandracchia and Pendleton, Reference Mandracchia and Pendleton2015). The extent of mental health challenges in college students supports the need for increased focus on prevention and treatment strategies. Even small effect sizes, when considering the population-wide burden of college student mental health issues, may make a meaningful difference (Cuijpers, Reference Cuijpers2021).
Recent calls for thoughtful and nuanced interventions have come about in the context of worsening mental health. According to the World Health Organization’s 2022 report on mental health, and their recent social development goals, investing in promotion, prevention, and treatment of mental health can improve public health in a cost-effective way by addressing risks and building systems of resilience and support (World Health Organization, 2022; World Health Organization, n.d.). Similarly, the White House recently issued multiple mental health research priorities. One such priority is strengthening implementation of a variety of interventions that are sensitive to specific university contexts, are culturally informed, and focus on resilience and protective factors (White House, Reference House2023).
Students also face multiple barriers to mental health care access related to mental health stigma, attitudes towards mental health care, consequences, and financial, cultural and time barriers (Conley et al., Reference Conley, Travers and Bryant2013b; Ebert et al., Reference Ebert, Mortier, Kaehlke, Bruffaerts, Baumeister and Auerbach2019; Kazdin, Reference Kazdin2019; Vidourek et al., Reference Vidourek, King, Nabors and Merianos2014; Worsley et al., Reference Worsley, Pennington and Corcoran2022). Interventions that target novel settings for intervention dissemination, and offer access to those that might not be willing or able to present for traditional psychological services, are useful strategies for focusing on improving overall wellbeing and life functioning (Cuijpers, Reference Cuijpers2021; Kazdin, Reference Kazdin2022). As Worsley et al. (Reference Worsley, Pennington and Corcoran2022) summarize, universities need to move beyond reactive interventions, and to provide alternative and accessible resources to help students before their mental health deteriorates.
Course-based interventions
Setting-based interventions are one potential solution to move beyond reactive interventions, with accessible resources to support student mental health. Within a socio-ecological model, setting-based interventions assume healthy structures are necessary for healthy processes, and both are needed for healthy outcomes (Dooris et al., Reference Dooris, Wills and Newton2014). Fernandez et al. (Reference Fernandez, Howse, Rubio-Valera, Thorncraft, Noone, Luu and Salvador-Carulla2016) reviewed settings-based interventions for university students, reporting that academic interventions that alter curriculum to provide mental health related skills and education are a promising way forward to support student mental health. However, they discuss how more research is needed to establish effectiveness of certain amounts of content, structure of the interventions such as in separate elective or required courses or infused in existing curriculum, and effectiveness of teaching strategies such as mindfulness for students in various degree programs outside of the health professions. Relatedly, in a comprehensive review of review-level evidence for university student mental health interventions, Worsley et al. (Reference Worsley, Pennington and Corcoran2022) emphasized the strong evidence base of support for CBT and mindfulness-based interventions. They also discussed that there is support for both settings-based interventions, and acceptance and commitment therapy (ACT) skills training, but limited review level evidence (Worsley et al., Reference Worsley, Pennington and Corcoran2022). As there was not a strong enough base of reviews to determine which interventions work best for which students and in what contexts, more research is needed examining settings-based interventions and ACT for university students. It follows that an ACT framework could be examined as a settings-based intervention as well, specifically as a framework for embedding mental health skills within coursework.
The educational setting is an ideal target for mental health promotion, because the years in university are an important developmental time for students, where they are open to ideas and desire solutions, and their entire social context is in one spot (DeArmond and Marsh, Reference DeArmond and Marsh1984). Interventions embedded in college courses are an ideal next step to potentially eliminate some of these barriers to care, given the relevance of mental health symptoms on school related outcomes, the ecological validity of the school setting, and the potential for both a treatment and public health approach (Chugani et al., Reference Chugani, Fuhrman, Abebe, Tallis, Miller and Coulter2020; Conley et al., Reference Conley, Durlak and Dickson2013a; Petersen et al., Reference Petersen, Davis, Renshaw, Levin and Twohig2022; Renshaw et al., Reference Renshaw, Weeks, Roberson and Vinal2022).
Preventive, portable, and small-group based mental health treatment programs (such as those incorporated into courses) can meet a broad range of student mental health needs and levels of severity, and can be customized according to context, setting and population (Conley et al., Reference Conley, Travers and Bryant2013b; Schiraldi et al., Reference Schiraldi, Spalding and Hofford1998). These programs can involve group participation, skills practice, and values clarification and should be creatively embedded (DeArmond and Marsh, Reference DeArmond and Marsh1984). This is important so that students are able to receive the content of the interventions in a non-judgemental and non-stigmatized setting where they do not have to actively make a choice to receive treatment given many stigma related barriers for students to accessing mental health care (Zancan et al., Reference Zancan, Constantinopolos, Pankowski, Bellini and da Silva Oliveira2023).
Important considerations in course-based interventions include skills practice, participant’s engagement levels, student year in school, and the training of facilitators given that there is support for lay providers as well as licensed providers (Abelson et al., Reference Abelson, Lipson, Eisenberg and Bowman2022; Arnold et al., Reference Arnold, Haubrick, Klasko-Foster, Rogers, Barnett, Ramirez-Sanchez and Gaudiano2022; Conley et al., Reference Conley, Durlak and Dickson2013a; Conley et al., Reference Conley, Travers and Bryant2013b; Kazdin, Reference Kazdin2019; Schiraldi and Brown, Reference Schiraldi and Brown2001). For example, multiple studies have found positive effects of cognitive behavioural interventions embedded in courses that included skills practice and homework, compared with general stress management courses or general topic discussions (Brown and Schiraldi, Reference Brown and Schiraldi2004; Conley et al., Reference Conley, Travers and Bryant2013b).
Acceptance and commitment therapy
ACT is a skill-focused intervention. It targets core processes of change within psychological flexibility, specifically (mindfulness, acceptance, flexible thinking, and awareness of personal values) in order to shift relationships with thoughts and feelings to be more flexible, and to help people take actions that align with their values (Hayes et al., Reference Hayes, Levin, Plumb-Vilardaga, Villatte and Pistorello2013a). It emphasizes teaching participants to take a non-judgemental stance towards thoughts and feelings and to use certain skills flexibly in response, and not to focus on changing distress specifically (Hayes et al., Reference Hayes, Luoma, Bond, Masuda and Lillis2006). For example, a student who in the past struggled with mathematics courses, but needs to pass one more to satisfy requirements for their major, may use mindfulness and defusion skills to provide some distance from stressful thoughts that they are not good at those subjects, and then would reflect on how their degree goals align with their values, and they would frame actions towards studying as aligning with those values.
ACT has been successful for many psychological difficulties (Gloster et al., Reference Gloster, Walder, Levin, Twohig and Karekla2020; Regehr et al., Reference Regehr, Glancy and Pitts2012). It is also a viable intervention that can be efficiently adapted and delivered across cultures and identities. ACT and its foundational science, contextual behavioural science, are ideal for expanding the reach and scope of interventions across cultures and contexts because of their focus on functional contextualism. More specifically, they target predicting and changing behaviour with precession, scope and depth, through better understanding the current function of behaviour, and how certain values can be a longer term and more sustainable reinforcer for behaviour change (White et al., Reference White, Gregg, Batten, Hayes and Kasujja2017). ACT can be thus adapted to local cultural knowledge and context given its focus on processes instead of topographical behaviours.
Furthermore, it focuses on values, which are of course still influenced by cultural context, but can be freely chosen by an individual (White et al., Reference White, Gregg, Batten, Hayes and Kasujja2017). In their discussion on their Model of Sustainable Mental Health, Bohlmeijer and Westerhoff (Reference Bohlmeijer and Westerhof2020) refer to White et al.’s (Reference White, Gregg, Batten, Hayes and Kasujja2017) article, stating: ‘The emphasis in ACT on clarification of personal values boosts its cross-cultural applicability’. For example, its focus on valued-living allows for specific cultural values to guide treatment such as spirituality or interdependence, making it potentially more acceptable than CBT in its focus on rationality and change for those of diverse backgrounds (Woidneck et al., Reference Woidneck, Pratt, Gundy, Nelson and Twohig2012). Cultural norms and values inform the behavioural processes that will be targeted in treatment, and the focus is on adapting those target processes instead of on current distress (Masuda et al., Reference Masuda, Morgan, Spencer, Qinaʻau and Jo2021).
In their recent chapter, Masuda et al. (Reference Masuda, Morgan, Spencer, Qinaʻau and Jo2021) reflect on the cultural adaptation of ACT, and how it relates to behavioural health concepts of cultural adaptation, competence and humility. They echo the same ideas that when ACT is truly practised functionally and contextually, this means it is being adapted to an individual within their own cultural context, and thus it is ideal for application across culture and identities. In practice, this is difficult to do, although given ACT’s fundamental focus on processes rather than content, this moves progress in the right direction (Masuda et al., Reference Masuda, Morgan, Spencer, Qinaʻau and Jo2021). Notably, they state the importance of further examination of the health and wellbeing of people with minoritized identities such as those related to ethnicity, gender and sexuality, and that this work can then inform knowledge gaps and service delivery (Masuda et al., Reference Masuda, Morgan, Spencer, Qinaʻau and Jo2021). This makes it important to continue adapting ACT within future work with ACT that reports on demographics of samples and occurs in various contexts.
ACT embedded in the university curriculum
While ACT has a strong base of empirical support broadly, more work is needed exploring the use of ACT and other forms of cognitive behavioural therapies in brief, easily disseminated forms that afford access to a broader audience (Petersen et al., Reference Petersen, Davis, Renshaw, Levin and Twohig2022; Williams et al., Reference Williams, Botanov, Giovanetti, Perko, Sutherland, Youngren and Sakaluk2022).
ACT is especially well-suited for college students given its transdiagnostic nature, focus on developing agency over one’s mental health, and potential for cost-effective delivery (Boone and Canicci, Reference Boone, Canicci and Pistorello2013; Downs and Eisenberg, Reference Downs and Eisenberg2012; Hayes et al., Reference Hayes, Pistorello, Levin and Pistorello2013b; Renshaw et al., Reference Renshaw, Weeks, Roberson and Vinal2022; Viskovich et al., Reference Viskovich, Pakenham and Fowler2021).
ACT has been successfully delivered to university students embedded in workshops, seminars and online, and resulted in improved mental health, psychological flexibility, and attitudes towards school and mental health treatment (Barrasso-Catanzaro, Reference Barrasso-Catanzaro2015; Browning et al., Reference Browning, Morena, Gould and Lloyd-Richardson2022; Danitz and Orsillo, Reference Danitz and Orsillo2014; Danitz et al., Reference Danitz, Suvak and Orsillo2016; Eustis et al., Reference Eustis, Williston, Morgan, Graham, Hayes-Skelton and Roemer2017; Mullen et al., Reference Mullen, Tracy, Block-Lerner, Marks, Sandoz and Ricardo2021; Pistorello et al., Reference Pistorello, Hayes, Seeley, Biglan, Long, Levin, Kosty, Lillis, Villatte, MacLane, Vilardaga, Daflos, Hammonds, Locklear, Hanna, Block-Lerner and Cardaciotto2016; Viskovich and Pakenham, Reference Viskovich and Pakenham2020). These studies varied in format including course-based interventions (Browning et al., Reference Browning, Morena, Gould and Lloyd-Richardson2022; Danitz et al., Reference Danitz, Suvak and Orsillo2016; Mullen et al., Reference Mullen, Tracy, Block-Lerner, Marks, Sandoz and Ricardo2021; Pistorello et al., Reference Pistorello, Hayes, Seeley, Biglan, Long, Levin, Kosty, Lillis, Villatte, MacLane, Vilardaga, Daflos, Hammonds, Locklear, Hanna, Block-Lerner and Cardaciotto2016) and workshop formats (Danitz and Orsillo, Reference Danitz and Orsillo2014; Eustis et al., Reference Eustis, Williston, Morgan, Graham, Hayes-Skelton and Roemer2017; Viskovich and Pakenham, Reference Viskovich and Pakenham2020). They also vary in lengths of time, and support to engage in skills out of class, which is an important future direction that Pistorello et al. (Reference Pistorello, Hayes, Lillis, Long, Christodoulou, Lejeune, Villatte, Seeley, Villatte, Jeffcoat, Plumb-Vilardaga, Yadavaia and Pistorello2013) discuss, in that engagement in skills may wane over time.
As Danitz et al. (Reference Danitz, Suvak and Orsillo2016) reflect, attrition in studies, and all of the competing resources and priorities for college students, are both influential in designing ACT-based studies for college students. On the one hand, embedding ACT in a required course reaches students who may not have otherwise sought help, but on the other hand, there may be more motivation present in students self-selecting into an elective course or workshop (Pistorello et al., Reference Pistorello, Hayes, Lillis, Long, Christodoulou, Lejeune, Villatte, Seeley, Villatte, Jeffcoat, Plumb-Vilardaga, Yadavaia and Pistorello2013).
In terms of outcome measures, each study assesses some combination of mental health constructs and ACT processes, and outcomes related to the experience of participants such as receptivity or satisfaction. Similarly, Ong et al. (Reference Ong, Sheehan and Haaga2023) describe measurement in ACT studies. They describe how it is important for ACT studies to assess psychological inflexibility and flexibility, given that they are related but distinct constructs that are both influential in progresses of change but do not exist on two ends of the same spectrum. Furthermore, inherent in targeting psychological flexibility and inflexibility is helping someone to build a meaningful life, therefore Ong et al. (Reference Ong, Sheehan and Haaga2023) state that quality of life should be assessed in ACT studies as well. Finally, they agree that measuring psychological distress symptoms is against theoretical tenets of ACT. They also state that one part of building a meaningful life involves reducing suffering. This aligns with various public health and consumer priorities to understand if a treatment can reduce suffering. Therefore, measuring depression and anxiety symptoms still has a useful function within ACT studies.
Christodoulou et al. (Reference Christodoulou, Flaxman and Lloyd2021) implemented a one-day ACT skills training for university students that reduced psychological distress and negative emotions with changes influenced by psychological flexibility and mindfulness. They recommend the importance of continuing training, outreach, and skills-based interventions, which can provide students with a skillset that might be sufficient to prevent onset of psychopathology, and affords students access to mental health support outside of the therapy room when they might not have otherwise had access. They also recommend that interventions be embedded in traditional college activities to keep costs down, such as organizing one-day trainings, embedding them within societies, clubs or teams, or providing asynchronous online and electronic support. It is important that future research assess various delivery formats (Christodoulou et al., Reference Christodoulou, Flaxman and Lloyd2021).
Pistorello et al. (Reference Pistorello, Hayes, Lillis, Long, Christodoulou, Lejeune, Villatte, Seeley, Villatte, Jeffcoat, Plumb-Vilardaga, Yadavaia and Pistorello2013) had discussed the importance of further examination of effectiveness of delivery of ACT in college courses by people who are not licensed mental health providers. This is an example of task-shifting which can help scale mental health care (Kazdin and Rabbit, Reference Kazdin and Rabbitt2013). As Christodoulou et al. (Reference Christodoulou, Flaxman and Lloyd2021) also stated: ‘Specific academic training in ACT is not a prerequisite to practice’. The point of the potential of lay providers was further confirmed when Arnold et al. (Reference Arnold, Haubrick, Klasko-Foster, Rogers, Barnett, Ramirez-Sanchez and Gaudiano2022) found in their review that ACT can be delivered successfully by lay providers who are not licensed clinicians. Thus, this study will implement an intervention with trained graduate students who are unlicensed.
The current study aims
When testing psychosocial interventions, in order to achieve maximum public health impact, they should be tested in the contexts and settings they are developed for where possible, and studies should blend tests of effectiveness and implementation (Beidas et al., Reference Beidas, Saldana and Shelton2023). This pilot intervention study aimed to test feasibility, acceptability and preliminary effectiveness of a brief ACT intervention. The intervention was delivered by trained but unlicensed graduate students under the supervision of a licensed psychologist during five consecutive weekly class meetings. The control condition was a second class session of the seminar taught by the same instructor on the same day. This class received their usual curriculum for this seminar which involves a standardized set of topics with some flexibility within each instructor. This course received health education content (e.g. sleep and healthy eating) from their professor on the days that the intervention group received ACT content.
Aim 1 was to examine the limited efficacy of the intervention according to Bowen et al.’s (Reference Bowen, Kreuter, Spring, Cofta-Woerpel, Linnan, Weiner, Bakken, Kaplan, Squiers, Fabrizio and Fernandez2009) feasibility study definition. We examined pre to post changes in the intervention with non-parametric tests given the small sample size. We also report effect sizes in the context of prior research. Specific outcomes tested include psychological flexibility, quality of life, and psychological distress (combined stress, anxiety and depression). We expected the intervention group would increase in positive psychological flexibility and quality of life, and would decrease in psychological inflexibility and psychological distress. Given the natural course of a semester at university and potential increases in stress, we expected that potentially the control condition course as usual group may also increase in psychological distress.
For Aim 2, combining Bowen et al.’s (Reference Bowen, Kreuter, Spring, Cofta-Woerpel, Linnan, Weiner, Bakken, Kaplan, Squiers, Fabrizio and Fernandez2009) feasibility framework and the results of Klaic et al.’s (Reference Klaic, Kapp, Hudson, Chapman, Denehy, Story and Francis2022) review of reviews, we defined feasibility as the degree of practicality of an intervention, demand, ease of delivery and possibility to undertake. Specifically, we measured this through recruitment/participation rate, attrition rate, adverse events, adherence (number of lessons attended, skills practice on average in the last week, and text-based examples of skills applied), reports of feeling that students lived a valued life, and anecdotal descriptions of perceptions/acceptability/training/time/knowledge on the part of the instructors. We expected no adverse events, and low attrition and moderate to high adherence and participation based on past research (Mullen et al., Reference Mullen, Tracy, Block-Lerner, Marks, Sandoz and Ricardo2021; Pistorello et al., Reference Pistorello, Hayes, Lillis, Long, Christodoulou, Lejeune, Villatte, Seeley, Villatte, Jeffcoat, Plumb-Vilardaga, Yadavaia and Pistorello2013; Pistorello et al., Reference Pistorello, Hayes, Seeley, Biglan, Long, Levin, Kosty, Lillis, Villatte, MacLane, Vilardaga, Daflos, Hammonds, Locklear, Hanna, Block-Lerner and Cardaciotto2016).
For Aim 3, we examined acceptability of the intervention. Again using Klaic et al. (Reference Klaic, Kapp, Hudson, Chapman, Denehy, Story and Francis2022) and Bowen et al. (Reference Bowen, Kreuter, Spring, Cofta-Woerpel, Linnan, Weiner, Bakken, Kaplan, Squiers, Fabrizio and Fernandez2009), we defined acceptability as whether or not the intervention was deemed satisfactory, appropriate, fair, and reasonable and helpful to the students. Based on past research, we expected more than half of students to report the intervention to be helpful, to desire to learn more, and to recommend it to others (Browning et al., Reference Browning, Morena, Gould and Lloyd-Richardson2022; Eustis et al., Reference Eustis, Williston, Morgan, Graham, Hayes-Skelton and Roemer2017; Mullen et al., Reference Mullen, Tracy, Block-Lerner, Marks, Sandoz and Ricardo2021; Pistorello et al., Reference Pistorello, Hayes, Lillis, Long, Christodoulou, Lejeune, Villatte, Seeley, Villatte, Jeffcoat, Plumb-Vilardaga, Yadavaia and Pistorello2013; Pistorello et al., Reference Pistorello, Hayes, Seeley, Biglan, Long, Levin, Kosty, Lillis, Villatte, MacLane, Vilardaga, Daflos, Hammonds, Locklear, Hanna, Block-Lerner and Cardaciotto2016). In terms of specific skill components, we hypothesized a priori that students would find mindfulness to be helpful and that they would enjoy it based on the results of Browning et al. (Reference Browning, Morena, Gould and Lloyd-Richardson2022).
Method
Participants
The study took place within two class sessions of a freshmen seminar in a liberal arts college at a medium size northeastern public university in the United States. The university has large percentages of students who are first-generation college students and utilize financial aid. The study included a quasi-experimental non-equivalent groups design. Both sessions were taught by the same instructor who was a psychology professor. They elected the course based upon the professor and their schedule, without knowing about the potential of the study to participate in. All liberal arts students are required to take this course in their first semester. The curriculum is standardized across all professors, with some sessions that can be tailored to the individual professor’s interests and goals. This professor who agreed to participate in the study was a colleague and contact of the study team. Both of her sessions ran on the same day. The study team randomly assigned one session to receive the ACT content, and one to receive their course as usual, with health education content on the days the ACT class received ACT content. All students received lesson content as part of their class time and were asked to reflect on activities they practised outside of class in the same weekly discussion post format of their course. Students could choose to participate in the other research portion of the study with further assessments outside of class.
An a priori power analysis on G*Power (Faul et al., Reference Faul, Erdfelder, Lang and Buchner2007) necessitated a sample size of 34 for a .8 power, .05 alpha level, medium effect size, two measurements, two groups and a within between interaction. The control condition class had 19 students enrolled, and seven students participated in the study, and the intervention class had 21 students enrolled, and 14 students consented to participate in the study. This was an overall participation rate of 52.5%. In total, participants in the study were mostly female (n=11; 52%) and White (n=14; 67%). The make-up of the sample actually mirrors and somewhat exceeds institutional enrollment data which reflects around 30–45% people of colour, around 15% Black students and around 7% Asian students among other identity groups (University of Massachusetts Dartmouth, n.d.). Over half of the sample (52.4%) had experience with psychotherapy and almost half (47.6%) had received at least one psychiatric diagnosis. See Table 1 for more information.
Table 1. Participant demographics
Procedure
Safety screening
Questionnaire responses at each time point were screened to identify those with concerning responses. Other data collected in the context of this project involved an assessment of risk behaviours and suicidality (ranging from impulsive behaviours such as shopping and fighting, to substance use, to suicidality and self injury). Responses were flagged as concerning based on severe levels of symptoms on the DASS-21, recent suicidality in the past month, recent self-injury or other substance use or risk behaviour, in congruence with standard clinical suicide risk assessment procedures. This process was done with both graduate students and author E.L.-R., who is a licensed clinical psychologist. Participants whose responses were concerning were not excluded from research participation, but were encouraged to seek support through pathways available at the university. Regardless, all study participants received a list of counseling resources.
Graduate student training
Christodoulou et al. (Reference Christodoulou, Flaxman and Lloyd2021) emphasized the importance of some clinical and ACT training for delivering basic interventions, even though in-depth training is not necessary. Similarly, Arnold et al. (Reference Arnold, Haubrick, Klasko-Foster, Rogers, Barnett, Ramirez-Sanchez and Gaudiano2022) showed in their review that lay providers can effectively deliver ACT. Thoughtful and coordinated training and supervision is important within task shifting structures where lay providers help deliver an intervention (Kazdin and Rabbitt, Reference Kazdin and Rabbitt2013). While this study was not specifically designed to assess the outcomes of the training component, in the spirit of transparency we report what we did with the training and metrics recorded so that future work could build upon it.
Three psychology graduate students worked to deliver and facilitate the intervention. The students were all students in an applied psychology two-year master’s degree program, working with trained clinical psychologists as research mentors, and taking psychopathology-related elective courses with training in differential diagnosis and basic interviewing skills. The lead graduate student (M.B.) was in her second year of the program and had prior training and experience working as a bachelor’s degree/entry level counsellor in which she worked supporting and coaching patients in practising and applying skills for addressing anxiety, OCD and emotion dysregulation using ACT, cognitive behavioural therapy, and dialectical behavioural therapy concepts while under licensed supervision. Additionally, her experience included training in crisis/risk assessment.
The other two graduate students had no prior clinical experience or training aside from academic coursework about psychopathology and their research. One was in their first year of the two’year master’s program, and one was in her second year. Both students’ research was aligned with clinical topics where they had some brief introduction to CBT topics.
A tiered model of supervision was utilized, similar to Balkhi et al.’s (Reference Balkhi, Reid, Guzick, Geffken and McNamara2016) progressive cascading model of supervision, where there was some baseline didactic training developed by M.B. and E.L.-R., and much of the training occurred experientially in the context of the intervention. The lead graduate student participated in supervision and training to competency in supervision skills, and then oversaw supervision of the two more novice graduate students in their skill acquisition. This was all in coordination and supervised by the study PI, who is a licensed psychologist (E.L.-R.).
As part of the baseline training, M.B. provided the other two students with materials about ACT concepts that the intervention was building from, for them to read and watch. The students met for about 2 hours to discuss concepts, and for the novice students to practise describing psychoeducation about ACT generally and while role-playing responding to a potential student comment. Additionally, this meeting focused on basic counselling skills such as validation, active listening, and a need for no self-disclosure about personal mental health topics on the part of the graduate students unless it was done in a way that demonstrated a conclusion to the challenge, and was not graphic or vulnerable beyond what might come up in discussing common life stressors. It was reiterated to the novice students that a licensed clinician spends a lot of time training to use disclosure effectively and safely, and that it was important to balance the level of genuineness and vulnerability needed for experiential treatments like ACT, with knowing that none of the graduate students were licensed, and there was not adequate time in the course-based setting for processing with students if any challenges arose. Although likelihood was low given that the lead graduate student would always be present in class, the other two graduate students were also instructed if a student disclosed things related to self-harm or suicidality to remain as neutral as they could, to validate, and to as discreetly and quickly as they could, pass the information to the lead graduate student so that she could assess and address further.
That main baseline didactic and experiential training occurred the week before the study started, and the lead graduate student also met with the study PI before and after the meeting. During the 5 weeks of the study, the lead graduate student was in touch with the other two students to provide them with the lesson materials ahead of time, answer questions, and delegate roles. The role delegation was done in consultation with E.L.-R. in order to ensure that the graduate students were learning and the goals of the intervention were being met. For example, for the first lesson, one of the new students had prior teaching experience, so she was instructed to shadow the lead student a couple instances and then to walk around and help facilitate small group discussion, but to not engage in much clinical coaching until the lead student had observed her a few times. The other student was completely new to clinical settings and teaching, so he strictly shadowed the lead student the first week. They both progressed in the following two weeks to eventually initiating skills coaching and discussion facilitation with the lead graduate student observing, and then working independently the third week. These opportunities came throughout each lesson as there were times for students to discuss readings in small groups, and then to work on the skill being practised.
Each week, the lead graduate student met with the study PI individually to be in communication about any concerns, such as one student who had seemed visibly distressed, and to discuss progress of lessons and the other graduate students. Additionally, the study PI and all three students met weekly. Those meetings were reserved for problem-solving anything that occurred in the prior lesson, the novice graduate students reflecting on what they were learning and observing, and for further psychoeducation about ACT concepts being covered. These meetings were led by the lead graduate student with support from the PI, and the PI at times directed conversation. Lastly, the lead graduate student was always actively in communication with the PI if anything came up remotely beyond the expected delivery of course content to ensure safety for students and the PI’s licence as a provider.
Of note, the lead graduate student was out sick for the fourth lesson, and everyone felt comfortable with the two new graduate students leading the lesson based on the skills they had gained to that point. The communication and supervision infrastructure ensured that the lesson was delivered and the two new graduate students were supported in the process.
Intervention content
All intervention sessions were delivered by at least two facilitators for five consecutive weekly class periods. The control condition class received the seminar’s usual curriculum, with the exception of brief educational, health-focused wellness content, in order to ensure the conditions were sufficiently distinct. The entire intervention class received the content regardless of study participation status. They also received ‘Stop Avoiding Stuff: 25 Microskills to Face your Fears & Do It Anyway’ (Boone et al., Reference Boone, Gregg and Coyne2020) before the intervention. Students received weekly readings and skills practice assignments. Additionally, brief writing reflections asked students to reflect on these experiences for their graded homework as part of their seminar, aligning with the usual curriculum. Importantly, for the entire seminar, most homework assignments consisted of a one paragraph discussion post where students would be asked to reflect on a topic related to their lecture material or something else within their experience as a first-year student. They received full credit if they met the word count requirement (250) and were on topic generally, thus students who did not participate in the study could still fairly and reliably meet their homework requirements, considering that the lecture instructor was present during most sessions so she understood generally what was covered. Lesson topics included an introduction to ACT, discussion of values, mindfulness, willingness and acceptance, committed action, and self-compassion (Hayes et al., Reference Hayes, Levin, Plumb-Vilardaga, Villatte and Pistorello2013a). Additionally, phrases such as being open, aware, accepting, and taking action which are touched on within Strosahl et al.’s (Reference Strosahl, Robinson and Gustavsson2012) approach to brief assessment and intervention, ‘Focused Acceptance and Commitment Therapy’. Lesson content outline and references are detailed in Table 2.
Table 2. Wellbeing Wednesdays lesson block plan
Note: Italics reflect an in-class read, pair, share activity.
References for intervention:
Boone, M., Gregg, J., & Coyne, L. (Reference Boone, Gregg and Coyne2020).
Boone, M. S., & Myler, C. (Reference Boone and Myler2012).
Browning, M. E., Morena, A., Gould, E. R., & Lloyd-Richardson, E. E. (Reference Browning, Morena, Gould and Lloyd-Richardson2022).
Ciarrochi, J., Atkins, P., Hayes, L., Sahdra, B., & Parker, P. (Reference Ciarrochi, Atkins, Hayes, Sahdra and Parker2016).
Clarke, S., Kingston, J., James, K., Bolderston, H., & Remington, B. (Reference Clarke, Kingston, James, Bolderston and Remington2014).
Clarke, S., Kingston, J., Wilson, K. G., Bolderston, H., & Remington, B. (Reference Clarke, Kingston, Wilson, Bolderston and Remington2012).
Danitz, S., & Orsillo, S. (Reference Danitz and Orsillo2014).
Danitz, S., Suvak, M., & Orsillo, S. (Reference Danitz, Suvak and Orsillo2016).
Eustis, E. H., Williston, S. K., Morgan, L. P., Graham, J. R., Hayes-Skelton, S. A., & Roemer, L. (Reference Eustis, Williston, Morgan, Graham, Hayes-Skelton and Roemer2017).
Harris, R. (Reference Harris2009).
Harris, R. (Reference Harris2008).
Harris, R. (Reference Harris2019).
Hayes, L., Ciarrochi, J., & Hayes, S. (Reference Hayes, Ciarrochi and Hayes2015).
Hayes, S. C., & Smith, S. (Reference Hayes and Smith2005).
Hayes, S., Strosahl, K., & Wilson, K. (Reference Hayes, Strosahl and Wilson2011).
Hayes, S. C., Levin, M. E., Plumb-Vilardaga, J., Villatte, J. L., & Pistorello, J. (Reference Hayes, Levin, Plumb-Vilardaga, Villatte and Pistorello2013a).
Oliver, J. (Reference Oliver2011).
Räsänen, P., Lappalainen, P., Muotka, J., Tolvanen, A., & Lappalainen, R. (Reference Räsänen, Lappalainen, Muotka, Tolvanen and Lappalainen2016).
Stafford-Brown, J., & Pakenham, K. (Reference Stafford-Brown and Pakenham2012).
Strosahl, K. D., Robinson, P. J., & Gustavsson, T. (Reference Strosahl, Robinson and Gustavsson2012).
University College Dublin (2021).
Measures
At baseline, students answered questions about demographics. Additionally, they completed validated surveys of psychological processes and outcomes at baseline (one week before lesson 1), post-intervention (the week following lesson 5), and follow-up (6 weeks after the post assessment). At post-intervention and follow-up, students answered questions about skill retention and implementation.
Assessments included across all time points for both groups
All groups completed the following measures. For both groups at baseline combined with the sample included in analyses, all Cronbach’s alphas ranged between .73 and .93.
Depression, Anxiety and Stress Scale-21 (DASS-21; Lovibond and Lovibond, Reference Lovibond and Lovibond1995)
The DASS–21 is a 21-item validated measure that assesses symptoms of depression, stress and anxiety (Henry and Crawford, Reference Henry and Crawford2005; Osman et al., Reference Osman, Wong, Bagge, Freedenthal, Gutierrez and Lozano2012). It is a shortened form of the DASS-42 (Lovibond and Lovibond, Reference Lovibond and Lovibond1995), and has been validated for non-clinical and university student populations, and maps onto three subscales of depression, stress and anxiety, as well as a general factor. The depression subscale consists of questions which assess different aspects of anhedonia and hopelessness; the anxiety subscale consists of questions which assess aspects of anxious thoughts as well as physiological hyperarousal; and the stress subscale consists of questions which assess chronic non-specific arousal.
Quality of Life Enjoyment and Satisfaction Questionnaire Short Form (Q-LES-Q-SF; Endicott et al., Reference Endicott, Nee, Harrison and Blumenthal1993; Riendeau et al., Reference Riendeau, Sullivan, Meterko, Stolzmann, Williamson, Miller, Kim and Bauer2018)
The Q-LES-Q-SF is a valid and reliable single factor self-report measure of satisfaction and enjoyment within various domains of daily living. The life satisfaction and enjoyment score, in the validation study, was correlated with but distinct from mental illness measures. This scale is calculated as a percentage of the total possible score such that scores can be interpreted as a percentile of quality of life.
Unified Flexibility and Mindfulness Scale (UFM; Rogge & Daks, Reference Rogge and Daks2021)
The UFM is a 28-item scale that assesses both psychological flexibility and inflexibility according to components of the ACT hexaflex in a process-based and context-sensitive way that is relatively new and more comprehensive by using questions from the Multidimensional Psychological Flexibility Inventory (MPFI; Rolffs et al., Reference Rolffs, Rogge and Wilson2016), as well as the Five Facet Mindfulness Questionnaire (FFMQ; Baer et al., Reference Baer, Smith, Hopkins, Krietemeyer and Toney2006). The UFM assesses flexible mindfulness and inflexible mindlessness as two different subscales. The scale of flexible mindfulness can further be separated into eight subscales which each measure a separate facet of flexible mindfulness (describing thoughts/feelings, observing sensations, acceptance, present moment awareness, self as context, defusion, values, and committed action). Similarly, the scale of inflexible mindlessness can be separated into six other subscales (experiential avoidance, lack of contact with the present moment, self as content, fusion, lack of contact with values, and inaction). This study utilized the inflexible mindlessness (psychological inflexibility) and flexible mindfulness (psychological flexibility) scales.
Post and follow-up questions for ACT group only
At post-intervention, participants in the ACT group were asked to rate whether or not they found the intervention helpful, whether or not they desired to learn more, and whether or not they would recommend it to others. They were also asked to describe what parts of the intervention they found most and least helpful, and liked and disliked (Browning et al., Reference Browning, Morena, Gould and Lloyd-Richardson2022; Firestone et al., Reference Firestone, Cardaciotto, Levin, Goldbacher, Vernig and Gambrel2019). These analyses align also with Mullen et al.’s (Reference Mullen, Tracy, Block-Lerner, Marks, Sandoz and Ricardo2021) assessment of ‘receptivity and helpfulness’ in their ACT intervention. They align with the importance of assessing how acceptable an intervention is to participants which encompasses components related to their perceived feelings and experiences in the intervention, and their perceived associated burden and opportunity costs; acceptability can affect adherence in an intervention (Sekhon et al., Reference Sekhon, Cartwright and Francis2017).
At post- and follow-up intervention participants reported what ACT skills they practised most frequently, and provided examples with quantitative ratings about the past week, and text-based responses (Browning et al., Reference Browning, Morena, Gould and Lloyd-Richardson2022).
Analysis plan
All analyses were calculated with IBM SPSS version 28 (IBM Corporation, 2021).
Missing data
For Aim 1, with listwise deletion, there were six participants in the intervention condition and three participants in the control condition who completed baseline and post assessments. Missing data was 57% for the intervention and 50% for the control. Follow-up data were not included in final assessments because this would have lowered the listwise n numbers to 5 and 1 for the intervention and control.
For Aims 2 and 3, for all quantitative rating questions, at post-assessment, 10 students completed the surveys out of 14, and for the specific questions for feasibility and for acceptability, seven students answered, leaving missing data at 50%. At follow-up, for rating on skills practised and a valued life, eight students completed surveys but six answered the specific questions for these aims, leaving missing data at 57%.
For text responses for Aim 3 (acceptability), five and six students answered each question about helpful aspects and unhelpful aspects, and likes and dislikes, leaving missing data at 57–64%. For the text responses for Aim 2 (feasibility), seven students provided responses at post assessment, leaving missing data at 50%, and two at follow-up, leaving missing data at 86%.
Aim 1: To examine changes in mental health and wellbeing across the intervention
Aim 1 was to examine the limited efficacy of the intervention, and we did so with non-parametric tests given the small sample size. This aligns with Bowen et al.’s (Reference Bowen, Kreuter, Spring, Cofta-Woerpel, Linnan, Weiner, Bakken, Kaplan, Squiers, Fabrizio and Fernandez2009) guidelines which reflect that a limited efficacy test in a feasibility study may utilize convenience samples, short follow-ups, and may have limited statistical power. This study is an example of testing expansion, or the extent to which a previously tested program (ACT) can be tested in a new setting (the classroom first year seminar setting at our university). We examine changes from pre to post intervention in each outcome (quality of life and psychological distress) as well as process (psychological flexibility) with non-parametric tests and we also report effect sizes in the context of prior research. Of note, we had originally planned mixed design ANOVAs or paired samples t-tests across pre, post and follow-up, but abandoned the parametric tests and inclusion of follow-up means with the low assessment completion.
Aim 2: Feasibility
For Aim 2, combining Bowen et al.’s (Reference Bowen, Kreuter, Spring, Cofta-Woerpel, Linnan, Weiner, Bakken, Kaplan, Squiers, Fabrizio and Fernandez2009) feasibility framework and the results of Klaic et al.’s (Reference Klaic, Kapp, Hudson, Chapman, Denehy, Story and Francis2022) review of reviews, we defined feasibility as the degree of practicality of an intervention, demand, ease of delivery and possibility to undertake. We measured feasibility through the percentage of students who participated in the study initially out of the potential total (recruitment rate), the percentage who completed all three assessments (participation rate), the percentage of sessions attended (participation rate), reports of adverse events or drop-out (attrition), the average number of homeworks completed (adherence), skills practised on average in the last week (adherence), reports of whether students agreed they were living a life that aligned with their values, and anecdotal descriptions of perceptions/acceptability/training/time/knowledge on the part of the instructors.
We also measured adherence through descriptively reporting skills practised over the course of the intervention.
Specifically, qualitative responses were coded according to categories of mindfulness, acceptance, committed action, considering values, self-compassion, or other, with cognitive defusion considered part of mindfulness. Participants received one point for each instance of a skill reported; this coding scheme is based on responses and coding from Browning et al. (Reference Browning, Morena, Gould and Lloyd-Richardson2022). M.B., and A.T., who is an trained graduate research assistant and co-author, coded responses within specific questions for helpfulness, and likes/dislikes. Initial 80% inter-rater reliability was achieved such that A.T. and M.B. aligned by at least 80% for scoring within each code for each question aside from the ‘other’ category which was treated as exploratory. Any discrepancies were agreed on by M.B., A.T. and E.L.-R. until full agreement was reached. Frequencies were reported.
Aim 3: Acceptability
Specific components referenced in both papers include satisfaction, intent to continue to use, perceived positive and negative effects and perceived appropriateness and recommendation to others. We measured this quantitatively through asking participants whether or not they felt the intervention was helpful, and whether or not they would recommend it to someone else and desired to learn more. Additional qualitative questions asked participants to report aspects of the intervention that they found helpful and unhelpful, and that they liked and did not like. Responses were coded in the same manner described in Aim 2.
Results
Aim 1: Limited efficacy
Bakker et al. (Reference Bakker, Cai, English, Kaiser, Mesa and Van Dooren2019) recommend that when reporting effect sizes, they should be contextualized with study goals and past effect sizes in the literature. A recent meta-analysis of ACT interventions for university students that focused on a goal of wellbeing found a pooled effect size of d=.29, which they said is in line with other well-being focused positive psychology interventions (Howell and Passmore, Reference Howell and Passmore2019). For past studies of ACT for college students, and in a review of overall studies of ACT, there were effect sizes Cohen’s d<.15 and .3 < Hedge s’ g<.49 for quality of life, psychological flexibility, and depression.
Additionally, we report Hedges’ g effect size as provided by SPSS to somewhat correct the bias of Cohen’s d in small sample sizes (Goulet-Pelletier and Cousineau, Reference Goulet-Pelletier and Cousineau2018). We mostly compare effect sizes to past related studies, but also to Cohen’s (Reference Cohen1988) and (Reference Cohen1992) guidelines which state that for Hedge s’ g, .2 is a small effect size, .5 is medium, and .8 is large.
We assessed changes from baseline to post-intervention in quality of life, depression, stress, anxiety, psychological inflexibility and psychological flexibility. We did so separately for the intervention condition and the control condition.
For the intervention condition, with Wilcoxon signed rank tests, there were not significant changes in psychological flexibility (Z=–.18; p=.85), quality of life (Z=–.23; p=.22), depression (Z=–.82; p=.41), stress (Z=–.14; p=.89) and anxiety (Z=–.54; p=.59). The p-value was under .1 but still non-significant for changes in psychological inflexibility (Z=–1.69; p=.09). Effect sizes and confidence intervals for change were as follows for psychological flexibility [Hedges’ g=–.17; 95% CI (–.91, .58)], psychological inflexibility [Hedges’ g=.82; 95% CI (–.09, 1.68)], quality of life [Hedges’ g=.51; 95% CI (–.31, 1.23)], depression [Hedges’ g=.37; 95% CI (–.42, 1.12)], stress [Hedges’ g=0; 95% CI (–.74, .74)], anxiety [Hedges’ g=.19; 95% CI (–.57, .93)]. For everything except stress, changes were in expected directions (lowered distress and psychological inflexibility, and improved psychological flexibility and quality of life). Effect sizes for psychological inflexibility were higher than past studies, effect sizes for anxiety were lower, and effect sizes for quality of life and depression mirrored past studies. These effect sizes were all small to medium according to Cohen’s (Reference Cohen1988) and (Reference Cohen1992) directions as well, except for psychological inflexibility which was large, and stress which was 0.
For the control condition, with Wilcoxon signed rank tests, there were not significant changes in psychological flexibility (Z=–1.60; p=.11), psychological inflexibility (Z=–.82; p=.41), quality of life (Z=–.27; p=.79), depression (Z=0; p=1.00), stress (Z=–1.34; p=.18) and anxiety (Z=–1.34; p = .18). Effect sizes and confidence intervals for change were as follows for psychological flexibility [Hedges’ g=.89; 95% CI (–.36, 2.07)], psychological inflexibility [Hedges’ g=–.46; 95% CI (–1.41, .58)], quality of life [Hedges’ g=.21; 95% CI (–.74, 1.11)], depression [Hedges’ g=–.29; 95% CI (–1.21, .68)], stress [Hedges’ g=–.87; 95% CI (–2.02, .37)], and anxiety [Hedges’ g=–.76; 95% CI (–1.85 .43)]. For each variable in the control condition, scores worsened (higher distress and psychological inflexibility, lower quality of life and psychological flexibility). For everything except stress, changes were in expected directions (lowered distress and psychological inflexibility, and improved psychological flexibility and quality of life). Effect sizes ranged from small to large according to Cohen’s (Reference Cohen1988) and (Reference Cohen1992) guidelines. See Table 3 for details on results.
Table 3. Changes from baseline to post in mental health, wellbeing and psychological flexibility
Aim 2: Feasibility
The control condition class had 19 students enrolled, and seven students participated in the study, and the intervention class had 21 students enrolled, and 14 students consented to participate in the study. This was an overall recruitment rate of 52.5%. As described in the missing data paragraph, missing assessment data from post and follow-up ranged from 50 to 86% so there was high attrition on assessments. When considering adherence, the average number of lessons attended was 4 (range 2–5; SD=1.3). There were five possible homework reflections, and research participants on average completed three homeworks (range 0–5; SD=1.47).
Of students who completed the post assessment, most students reported practising mindfulness at least 2 days in the previous week (n=5; 71%), as well as the microskills taught in the intervention (n=5; 71%), and valued behaviours (n=4; 57%). Most students agreed that they ‘live a life in line with the characteristics, ideas, goals, and activities that matter to me’ (n=5; 71%) and the other two students selected neutral, not disagree. At follow-up, 50% (n=3) of students reported engaging in the microskills at least 1 day in the previous week, 50% (n=3) reported engaging in mindfulness at least 1 day in the previous week, and 33% (n=2) reported engaging in valued behaviours at least 2 days in the previous week. Most students agreed that they lived a life in line with their values (n=5; 71%) and the other student who did not agree, selected neutral and not disagree.
For analysis of text-based responses of skills practised, at post assessment, mindfulness (n=5; 71%), noticing avoidance and addressing it (n=4; 57%), and committed action (n=3; 43%) were most often reportedly used outside of intervention. At follow-up, few students reported using ACT skills, with the exception of mindfulness (n=2; 29%). See Table 4 for information on qualitative results from Aim 2.
Table 4. Themes of participants skill engagement
Note. Overall, 7 of 14 intervention participants answered the skills question at post, and 2 out of 14 answered it at follow-up. Percentages are out of the total completed for that aim. Themes coded by two raters are based on Browning et al. (Reference Browning, Morena, Gould and Lloyd-Richardson2022).
There were no adverse events reported for research participants.
With respect to the feasibility of the clinical training of the graduate students, there were no formal assessments of knowledge given. Anecdotally, students and ELR all enjoyed the experience and were able to fall into a routine of communication between sessions and weekly supervision. Of note, the lead graduate student (M.B.) was sick in week 4, and the other two students both felt comfortable and confident delivering the lesson. The participants expressed to M.B. the following week how much they enjoyed learning from the other graduate students.
Aim 3: Acceptability
Of students that completed assessments at post intervention, 100% (n=6) felt the intervention was helpful, 67% (n=4) desired to learn more, and 67% (n=4) would recommend it to someone else.
Within text responses, the most favorably endorsed skills were mindfulness (n=3, 50%), and committed action (n=2; 33%). Few students disliked any of the intervention skills, with only one (16.7%) disliking mindfulness and one (16.7%) disliking committed action. For skills found helpful, mindfulness was most frequently endorsed (n=6, 100%) and 16.7% of students (n=1) found committed action unhelpful. See Table 5 for information on qualitative results from Aim 2.
Table 5. Participant reports of skills liked and disliked, and skills found helpful and unhelpful
Discussion
Summary of results
This study describes a pilot implementation of a brief ACT-based classroom intervention in a freshman required course. The sample sizes were very small, so results should be interpreted mostly as descriptive and preliminary. That being said, interestingly the study was only moderately acceptable according to students within the various metrics that were assessed.
Aim 1: Changes in mental health
In terms of outcomes and processes, the intervention increased and decreased in expected directions, and the control condition course either worsened or did not change in each variable, but there were no significant differences within each group, and the sample size was very small. Effect sizes had wide variation, but for depression and quality of life in the intervention condition, they were similar to past research. Psychological inflexibility also had a large effect size for change in the intervention condition and a p-value that neared significance, so while small sample sizes can inflate effect sizes, there may have been some evident improvement in psychological inflexibility.
It is important to note that ACT interventions in school-related settings and those that target wellbeing often find small to medium effect sizes, and studies do not always find between-group differences on wellbeing outcomes (Howell and Passmore, Reference Howell and Passmore2019; Petersen et al., Reference Petersen, Davis, Renshaw, Levin and Twohig2022). Furthermore, other studies such as Clarke et al. (Reference Clarke, Kingston, Wilson, Bolderston and Remington2012), have found an incubation effect, where the processes in ACT and skills learned show strongest effects at a longer follow-up such as around 6 months, where people have had more opportunities to engage with daily life in the present and valued way they practicsd in the intervention.
Thus, potentially these results suggest that this intervention can serve as a buffer against the natural declines in mental health over the course of a college semester. It may be that further change would have been found with longer follow-up, and more robust assessment participation.
The non-significant results could also be due to lack of power and the small sample size. There are potential cohort effects related to the attrition with assessments. Participation was very low, even though students were compensated for each assessment and given extra credit if they completed all assessments in the intervention class. This lower participation might be because students were freshmen coming out of years of school impacted by COVID, and did not have the foundational knowledge of the importance of research participation that they might later have impressed upon them in their liberal arts curriculum. It is also possible that there were specific differences with this cohort of students as many of them did not complete the freshman seminar along the same curricular timeline that most students do.
The non-significant results could also reflect that this intervention was not robust enough to influence changes in target processes or outcomes. Potentially additional homework support and engagement outside of class, or further work with motivational interviewing concepts could support initial buy-in and engagement. Alternatively, perhaps this reflects why other projects have utilized a design where students can sign up for the course, so they have initial motivation to participate.
Aim 2: Feasibility
In terms of feasibility, of students that completed assessments, there was only moderate participation in the research assessments outside of class, and there was high attrition in assessments at post-intervention and follow-up. That being said, on average, most participants attended most class sessions during the study, and of the students who completed assessments, they reported frequent skills practice in the past week, and their skills mapped onto expected domains after ACT treatment.
Most students agreed at post-assessment and follow-up that they lived a life that aligned with their values and priorities. This is not a validated measure, but represents an example of an idiographic assessment of a specific psychological flexibility component. Other measures such as the Personalized Psychological Flexibility Index (PPFI; Kashdan et al., Reference Kashdan, Disabato, Goodman, Doorley and McKnight2020) are being created to better assess individual change within valued actions and working towards goals, and they could be used in future course-based ACT interventions.
As expected, mindfulness was the most frequently endorsed skill applied at post and follow up after coding of open response answers. Close behind, about one-third of participants reported engaging in committed action. Importantly, over half (57%) of students endorsed noticing avoidance and addressing it, in relation to mindfulness. This shows students used mindfulness as a tool to accept circumstances and engage in valued actions, instead of using it to control thoughts and feelings (Tifft et al., Reference Tifft, Underwood, Roberts and Forsyth2022). All of these feasibility reports should be interpreted with caution, given that they represent a sub-sample of research participants who completed assessments.
Aim 3: Acceptability
Over half of participants endorsed the intervention as helpful, aligning with past research that students enjoy brief ACT interventions (Browning et al., Reference Browning, Morena, Gould and Lloyd-Richardson2022; Mullen et al., Reference Mullen, Tracy, Block-Lerner, Marks, Sandoz and Ricardo2021).
Less than half of the sample indicated they would recommend ACT or would want to learn more, not aligning with hypotheses. This might reflect response bias, or that participants found the intervention helpful, but were not motivated to continue further. Considering the Theory of Planned Behaviour, students appeared to have neutral attitudes towards the intervention, and perhaps there were moderate influences of social norms of the class and perceived behavioural control (Ajzen, Reference Ajzen1991). Further work could integrate contact with others’ lived experiences of mental health and treatment in college, or even of that intervention, in order to better support intentions to engage in the treatment.
We expected students to enjoy mindfulness and find it most helpful. Results echo this, but committed action was close behind for a skill most enjoyed. Results mostly align with past research Browning et al. (Reference Browning, Morena, Gould and Lloyd-Richardson2022) found and summarized or that of Mullen et al. (Reference Mullen, Tracy, Block-Lerner, Marks, Sandoz and Ricardo2021). Qualitative results reflect natural variation, and also some perceived challenges in engaging with difficult internal and external experiences as part of ACT. Overall, results echo past support that ACT interventions embedded in classes and college settings are useful and helpful, although these results reflected overall moderate acceptability (Browning et al., Reference Browning, Morena, Gould and Lloyd-Richardson2022; Christodoulou et al., Reference Christodoulou, Flaxman and Lloyd2021; Mullen et al., Reference Mullen, Tracy, Block-Lerner, Marks, Sandoz and Ricardo2021; Petersen et al., Reference Petersen, Davis, Renshaw, Levin and Twohig2022; Pistorello et al., Reference Pistorello, Hayes, Lillis, Long, Christodoulou, Lejeune, Villatte, Seeley, Villatte, Jeffcoat, Plumb-Vilardaga, Yadavaia and Pistorello2013; Pistorello et al., Reference Pistorello, Hayes, Seeley, Biglan, Long, Levin, Kosty, Lillis, Villatte, MacLane, Vilardaga, Daflos, Hammonds, Locklear, Hanna, Block-Lerner and Cardaciotto2016).
Limitations
Next, we reflect on some study limitations. Analyses in this manuscript from the intervention are based on a very small sample size of participants who consented to participate in the research portion of the project and completed all assessments. Therefore results should be interpreted with caution, and as mostly descriptive. Moreover, while effect sizes are somewhat promising and in expected directions, Cohen’s d and Hedges’ g effect sizes have a higher tendency to over-estimate population effect sizes with a smaller sample size (Goulet-Pelletier and Cousineau, Reference Goulet-Pelletier and Cousineau2018). Participation may have been impacted with assessments due to them being outside of class, or students not properly understanding the importance and context of research assessments.
The sampling for this study occurred with an instructor who taught two sessions of the same course. While students were not aware of the option of study participation at the time of course enrolment, they did choose to enrol in this class based on the instructor’s background as a psychology professor and on the students’ schedule. Therefore there may be limits to generalizability. The study also occurred during COVID-19, providing important context to the mental health of university students currently but those cohort effects will always follow results.
The intervention curriculum was designed using evidence-based sources from an ACT lens, but was delivered and organized according to the training background of the study team. While a recent review article found that ACT can be delivered successfully by lay providers and improve distress and health behaviours (Arnold et al., Reference Arnold, Haubrick, Klasko-Foster, Rogers, Barnett, Ramirez-Sanchez and Gaudiano2022), there may be limitations with the training of the student providers and supervision structure that could be further addressed through targeted changes in the training program.
There were no checks aside from self-report to oversee how students practise skills outside of class, so it is unknown for sure, if at all, how many students practised skills and to what extent. The intervention was given universally to students in the class, with suggestions to students to differentiate according to their own needs, but this means students needing more intensive support might not get the full extent of treatment that they would benefit from.
Future directions
Future projects should focus on how to build assessments and other research components into class time to ensure participation in assessments, and focus on skills practice outside of the intervention. Scheduled and structured support outside of a direct intervention might be helpful (Shim et al., Reference Shim, Mahaffey, Bleidistel and Gonzalez2017), such as peer-support coaching which facilitated adherence in Klimczak et al.’s (Reference Klimczak, Twohig, Peacock and Levin2023) study.
Christodoulou et al. (Reference Christodoulou, Flaxman and Lloyd2021) echo the importance of creatively embedding skills-based supports in college settings. More work should continue in course-based settings like this study did, given the theoretical support but need for further testing and implication on a larger scale. As Browning et al. (Reference Browning, Morena, Gould and Lloyd-Richardson2022) discuss, further studies could explore other course settings such as existing courses that students are required to take for a certain major, or those that are more general such as the freshman seminar this study was implemented in, or other elective courses. As Danitz et al. (Reference Danitz, Suvak and Orsillo2016) also reflect on after their projects implementing acceptance-based treatments in university settings, these treatments are ideally implemented in a required part of the college setting such as a course or orientation module, but there are difficulties with limited time, money, and competing priorities, so online options are another potential direction. They discuss the importance of also gaining a better understanding of the benefits of universally provided supports versus those in settings students self-select into, such as when they sign up for an elective course. This is the structure that Chugani et al. (Reference Chugani, Fuhrman, Abebe, Tallis, Miller and Coulter2020) are taking in their project.
Along these lines, the degree of group support should be considered. As many people have discussed, such as Zancan et al. (Reference Zancan, Constantinopolos, Pankowski, Bellini and da Silva Oliveira2023) and Conley et al. (Reference Conley, Travers and Bryant2013b), often stigma is a barrier to students even deciding to access care. However, group and curriculum-based ACT seems to alleviate some of these challenges where students feel empathy towards fellow students while they work together to learn skills and problem-solve (Browning et al., Reference Browning, Morena, Gould and Lloyd-Richardson2022; Zancan et al., Reference Zancan, Constantinopolos, Pankowski, Bellini and da Silva Oliveira2023). The benefits of group support with the necessary time, resources, and institutional support should be weighed and assessed.
Intervention programs should integrate trainees wherever possible in delivering skills, and should continue to streamline and assess the utility of brief trainings that target embodied cognition of trainees (Kosmas and Zaphiris, Reference Kosmas and Zaphiris2018). This can further vary available mental health treatments and democratize clinical training opportunities to decrease the public health burden of mental illness and expand the number of providers who are trained in evidence-based practice (Ernestus et al., Reference Ernestus, Fleming, Wenze and Blomquist2022; Forcino et al., Reference Forcino, Grimes, Capriotti and Woodhead2022; Kazdin and Blase, Reference Kazdin and Blase2011; McCarty et al., Reference McCarty, Cooke, Lazaroe, Guzick, Guastello, Budd, Downing, Ordway, Mathews and McNamara2022).
This could be scaled by utilizing it as a practicum opportunity for clinical programs at various degree levels (Forcino et al., Reference Forcino, Grimes, Capriotti and Woodhead2022), or a way for peers who completed the intervention to be involved. Petersen et al. (Reference Petersen, Davis, Renshaw, Levin and Twohig2022) echo the importance of examining other types of deliverers of ACT school-based treatments. Assessing outcomes related to the training could explore if students involved have a more positive and enriching college experience, or if their future choices of evidence-based practice and health service careers are influenced (McCarty et al., Reference McCarty, Cooke, Lazaroe, Guzick, Guastello, Budd, Downing, Ordway, Mathews and McNamara2022). Additionally, treatment outcomes could be compared based on who delivered the treatment and the training they had received, as well as other variables related to how the participants interpreted the treatment, such as assessing treatment acceptability (Sekhon et al., Reference Sekhon, Cartwright and Francis2017).
Another path towards scaling related interventions would be considering how to effectively train university professionals and faculty on how to deliver intervention components in a way that is safe and promotes their self-efficacy. A recent report by Lipson et al. (Reference Lipson, Talaski, Cesare, Malpiede and Humphrey2021) found that faculty currently recognize that they often respond to student mental health concerns, and they feel they currently do not have enough training to feel comfortable doing so. While this study was delivered by students within an existing course, a future study could train instructors to deliver the intervention content.
Students enjoyed mindfulness and found it helpful, and future work should explore how messaging, terms, and psychoeducation about mindfulness can influence students’ mental health and engagement in ACT interventions (Tifft et al., Reference Tifft, Underwood, Roberts and Forsyth2022). Health behaviours are important for college students, so future interventions should involve direct conversations and skills practice surrounding things like healthy sleep (Gusman et al., Reference Gusman, Grimm, Cohen and Doane2021). Committed action skills are similarly important to explore given the importance of feeling skilled enough for things such as communicating with professors, engaging in health behaviours, navigating challenging social situations, or taking steps to reach professional goals in college. Work like that of Viskovich et al. (Reference Viskovich, Pakenham and Fowler2021), who detail students’ values and experiences in ACT-based values exercises could be used to further tailor values and experiential exercises towards college students and university contexts.
Given many different mindfulness-based interventions can support university students including ACT, mindfulness-based stress reduction (MBSR), and mindfulness-based CBT (MBCBT) (Ma et al., Reference Ma, Wang, Pan, Cui and Schluter2022), relevant processes and components could be examined to tailor interventions. This aligns with Stenhoff et al.’s (Reference Stenhoff, Steadman, Nevitt, Benson and White2020) recommendation that future research examine how components of ACT like cognitive defusion influence subjective well-being. Large enough sample sizes will be needed in order to differentiate amongst multiple processes of change unlike we were able to in the current project.
Additionally, there was only moderate acceptability and participation in the current study, so intentions and engagement/participation need to be strengthened. More change-oriented skills from cognitive behavioural therapy that focus on challenging irrational thought patterns and unhelpful behaviours compared with acceptance could be helpful, as well as motivational interviewing focused coaching interventions (Abelson et al., Reference Abelson, Lipson, Eisenberg and Bowman2022). Gonçalves (Reference Gonçalves2023) discussed processes of change and activities involved in ACT, and solution focused therapy approaches and described their conceptual overlap. He emphasized the importance of honest accounting of shared processes across therapies, and continued work to personalize treatments within specific contexts. Solution focused therapy concepts could help inform brief interventions for university students as well given their focus on hope, strengths, and goals, such as within the single session format delivered by Schleider et al. (Reference Schleider, Sung, Bianco, Gonzalez, Vivian and Mullarkey2021).
Throughout any future work, assessments for feasibility and acceptability should be further considered, standardized, and should be in line with current understanding of measurement in ACT treatment. For example, as idiographic measurements develop, such as the PPFI, these can be included (Kashdan et al., Reference Kashdan, Disabato, Goodman, Doorley and McKnight2020). Assessments of treatment fidelity (Ong et al., Reference Ong, Sheehan and Haaga2023; O’Neill et al., Reference O’Neill, Latchford, McCracken and Graham2019) and therapist knowledge of ACT concepts (Luoma and Plumb, Reference Luoma and Plumb2013) can be included. Assessments of acceptability should align with theoretically coherent frameworks and concepts and should move beyond pure ratings of satisfaction. As an example, Sekhon et al.’s (Reference Sekhon, Cartwright and Francis2022) measure guidelines allow for a structured assessment of perceptions of an intervention. This work will allow for more thoughtful and comprehensive understanding of participants’ experiences in ACT interventions.
The current study demonstrates an application of a brief randomized ACT intervention embedded in a freshman seminar for university students, delivered by trained but unlicensed graduate students, that students enjoyed and found helpful. The sample size was very small so results should be interpreted mostly as descriptive, and the literature reviewed could help generate future studies. Interventions such as the one described in this study are important to pursue as our society seeks to support our young people as they struggle with broad mental health challenges and transition into adulthood. Research should be blended with tests of effectiveness and implementation to be blended for maximum public health impact (Beidas et al., Reference Beidas, Saldana and Shelton2023).
Key practice points
-
(1) Acceptance and commitment therapy can be integrated into an existing freshman seminar curriculum and delivered by unlicensed psychology students with supervision.
-
(2) Acceptance and commitment therapy can be targeted to both wellbeing and psychological distress outcomes.
-
(3) Students will moderately engage with acceptance and commitment therapy skills outside of class and show a moderate desire to learn more.
Data availability statement
The data that support the findings of this study are available from the corresponding author, M.B., upon reasonable request.
Acknowledgements
We would like to acknowledge Dr Joshua Masse and Dr Judith Sims Knight for their support.
Author contribution
Morgan Browning: Conceptualization (lead), Data curation (lead), Formal analysis (lead), Funding acquisition (lead), Investigation (lead), Methodology (lead), Project administration (lead), Resources (equal), Supervision (equal), Writing – original draft (lead), Writing – review & editing (lead); Elizabeth Lloyd-Richardson: Conceptualization (supporting), Formal analysis (supporting), Funding acquisition (supporting), Investigation (supporting), Methodology (supporting), Project administration (supporting), Resources (supporting), Supervision (lead), Writing – original draft (supporting), Writing – review & editing (equal); Akshay Trisal: Data curation (supporting), Formal analysis (supporting), Investigation (supporting), Project administration (supporting), Writing – review & editing (supporting); Victoria Kelleher: Project administration (supporting), Writing – original draft (supporting), Writing – review & editing (supporting); Mary Kayyal: Conceptualization (supporting), Formal analysis (supporting), Supervision (supporting), Writing – review & editing (supporting); Anna Schierberl Scherr: Conceptualization (supporting), Supervision (supporting), Writing – review & editing (supporting).
Financial support
This research received financial support from the Phi Kappa Phi Love of Learning Grant, and the University of Massachusetts Dartmouth College of Arts & Sciences Dean’s Office Graduate Thesis Support Grant.
Competing interests
The authors declare none.
Ethical standard
The authors have abided by the Ethical Principles of Psychologists and Code of Conduct as set out by the APA (http://www.apa.org/ethics/code/) which align with the Ethical Principles of Psychologists and Code of Conduct as set out by the BABCP and BPS. This study was approved by the university’s Institutional Review Board (approval number 21.056). All participants provided consent.
Comments
No Comments have been published for this article.