Introduction
In certain countries with similar developmental, demographic and societal structures to Ireland, anxiety related school avoidance can affect up to 5% of primary and secondary school students each year (Sewell Reference Sewell2008). With close to a million students enrolled in primary and secondary schools in Ireland currently (Department of Education 2021), this potentially amounts to a significant number of students (up to 50,000). In addition to the significant distress for families, poor attendance and engagement in school can have negative consequences for a young person’s social, emotional, and academic development (Heyne et al. Reference Heyne, King, Tonge and Cooper2001).
Exposure therapy is a well-established treatment for anxiety disorders. It involves graded and repeated exposure to the anxiety provoking stimulus. This in turn leads to an attenuation of the anxiety response. Exposure therapy is an effective treatment for many anxiety disorders including panic disorder, social phobia, post-traumatic stress disorder (PTSD) and other specific phobias (Shiban Reference Shiban2018; Carl et al. Reference Carl, Stein, Levihn-Coon, Pogue, Rothbaum, Emmelkamp, Asmundson, Carlbring and Powers2019).
Virtual reality exposure therapy (VRET) involves controlled exposure to a virtual representation of the anxiety provoking stimulus. VRET has been shown to be as effective as conventional exposure therapy for treating specific phobias, social anxiety disorder, performance anxiety and PTSD (Carl et al. Reference Carl, Stein, Levihn-Coon, Pogue, Rothbaum, Emmelkamp, Asmundson, Carlbring and Powers2019). VRET demonstrates potential benefits in terms of costs and time savings, greater control, and greater repeatability of the exposure (Pot-Kolder et al. Reference Pot-Kolder, Veling, Geraets, Lokkerbol, Smit, Jongeneel, Ising and van der Gaag2020; Geraets et al. Reference Geraets, van der Stouwe, Pot-Kolder and Veling2021). Despite the potential of VRET, and the likely appeal to young people, there is limited research into its use in Child and Adolescent Mental Health Services (CAMHS). This is largely because it is a relatively new technology whose use until recent times was limited by cost, lack of expertise and technical limitations (Bioulac et al. Reference Bioulac, de Sevin, Sagaspe, Claret, Philip, Micoulaud-Franchi and Bouvard2018).
Working closely with young people attending a CAMHS service, we co-designed and developed a VRET prototype tool aimed at reducing school related anxiety and its impact on school attendance. The adolescents participated in 8 workshops where we introduced the concept of exposure therapy and through design thinking principles they developed the storyboard, setting, visualisations etc. This was an iterative process completed in parallel with the virtual reality simulation creation thus ensuring that any development issues could be fed back to the participating adolescents to ideate and resolve. We then completed a study to assess the feasibility of this tool as a VRET. The aim of this study was to assess the feasibility of using VRET as a treatment for school anxiety. The study involved the recruitment of eighteen teenagers to trial the newly developed VRET over one session each. They then completed a structured feedback questionnaire on their experience.
Method
The aim of this research is to co-design and evaluate a VRET intervention for students experiencing anxiety resulting in school avoidance. This was achieved via two distinct phases.
-
Phase one involved recruiting CAMHS attendees to co-design and develop the VR world story board, script, and prototype in an iterative process with a VR development company.
-
Phase two involved recruiting both CAMHS and non-CAMHS adolescents to engage with the completed virtual reality scenarios and provide both quantitative and qualitative feedback on their experience.
Phase 1: VR experience development
Despite the promise and proliferation of eMental Health interventions, they have failed to engage young people (Achilles et al. Reference Achilles, Anderson, Li, Subotic-Kerry, Parker and O’Dea2020). Research suggests that end-user involvement in design and delivery may be a key component for successful engagement (de Beurs et al. Reference de Beurs, van Bruinessen, Noordman, Friele and van Dulmen2017). An essential and innovative aspect of this project was the co-creation of the prototype with young people attending a CAMHS service. In direct response to the criticism that treatments are most often developed without the input from the service users who they hope to help (de Beurs et al. Reference de Beurs, van Bruinessen, Noordman, Friele and van Dulmen2017), we deliberately set out to develop this therapy in collaboration with patients. We involved young people who were attending CAMHS in all stages of the project, from defining the problem, designing the solutions to building and testing the prototype. Extensive collaboration also took place with schools, primary care, youth services and young people not attending CAMHS.
Eighteen teenage CAMHS attendees were recruited for the co-design workshops. The group consisted of 10 females and 8 males aged between 13 and 17 years. They participated over a six-month period where they explored their own experiences of anxiety in school, created an avatar they named ‘Dala’, ideated novel solutions which were developed into a script and story board. In order to recruit these participants, CAMHS staff were approached to suggest suitable patients. These potential participants were then approached directly and given information about the workshops and invited to participate in the workshops. Informed consent was obtained from both participants and their parents. The workshops used design thinking principles where methods that support open-ended, collaborative, and visual ways of expressing experiences, meanings, and ideas are used to understand the problem and create novel solutions.
The groups were facilitated by two senior Occupational Therapists and a consultant psychiatrist. Online meetings were held between all facilitators before and after each session.
Three groups of six adolescents were formed. Two groups met in person in two separate locations and one group was online. The workshop groups completed preparations in the first two weeks regarding their own journey with anxiety and then progressed to process mapping of the main areas of challenge before finally employing design thinking to develop the VR scenarios (Appendix B). Table 1 presents a summary of the workshops.
Table 1. Workshop overview
All groups engaged via a video conferencing system (Webex) and the design thinking format was completed with use of Miro Board (miro.com, San Francisco, U.S.A.), a visual collaboration platform and paper documents.
They created a VR avatar named ‘Dala’ whose initial problem statement was ‘How to get anxious Dala to their first class in school’. They engaged in each stage of the design thinking process, learning from each other’s experiences, identifying the problems, ideating, building, and testing their solutions. They then pitched their ideas to an objective third party in the form of a facilitated workshop.
A process map (Appendix A) was created to incorporate all of the participants ideas. This map was used by the VR developers to inform their build.
VR consultants and facilitators engaged in weekly meetings in an iterative process to review the feasibility of ideas based on budget requirements, software design, and accessibility. Samples of the programme were reviewed continuously and compared to the original ideas from participants via the process map.
Once completed all participants of the original design thinking groups were invited back for testing of the initial videos. It was important to include original members of the design team in testing. This was in keeping with the ethos of design thinking, an iterative process that is non-linear.
With funding from the Public Services Innovation Fund, Virtual Reality (VR) consultants were engaged and a VR production company was hired to build three bespoke virtual reality videos which are described below.
Video scenario descriptions
Using Design Thinking methodology, the team identified a metaphor for anxiety so that it could be effectively translated to and portrayed in a virtual environment. The metaphor likens anxiety to having an Anxious App on your phone that is draining your battery and will eventually cause the phone to shut down. This metaphor resonated powerfully with the team as many had experienced feeling unable to think, move or talk when experiencing feelings of extreme anxiety.
The three virtual reality scenarios developed are called Anxious App, Pathfinder and Good to Go. Prior to using the videos there is a video introduction to the technology which explains how the videos work and advises users to simply remove the headset if they feel discomfort at any time. In the VR scenarios the user embodies Dala (the anxious student) who is sitting in a school toilet cubicle. The following is a description of the VR scenarios.
Anxious app
A voiceover of Dala’s thoughts along with visual and sound effects convey the emotional state of Dala and work to influence the anxiety levels of the user. Their worries include an argument with their Dad that morning, concerns about homework, fears about social pressure and exclusion and being late for class. The actor conveys Dala’s emotional state and their feelings of increasing panic, while also narrating their physical symptoms (increased heart rate, shallow breathing) and mental distress (what is wrong with me?). Dala’s thoughts are animated by images of an ‘angry Dad’ and ‘angry teacher’ and cuts to a classroom scene where Dala is being berated by the teacher. Sound and visual effects are used to fully immerse the user and provoke an anxiety response. Throughout the scene, Dala’s phone has been notifying them to close Anxious App and it now registers a warning beeps and a flashing red low battery symbol representing how Dala’s anxiety has escalated.
Pathfinder
This scenario opens with the same anxiety provoking stimuli as in scenario 1 (Anxious App) but Dala chooses to close Anxious App and try Pathfinder App. They meet ‘Moxie’ (our virtual peer mentor) who helps Dala to understand their stress response, practise simple breathing techniques and notice the thoughts that are fuelling their reaction. Moxie’s interaction with Dala is compassionate and warm, representative of an older sister/cousin, who shares their own experience of what helped them in similar situations.
Good to go
The teenagers specifically requested a scenario where Dala has learned what they need from Moxie and can now manage by themselves. In the final scenario Dala is experiencing anxious thoughts but as these increase they remember what Moxie taught them about breathing, the stress response and how believable anxious thoughts can feel (I don’t fit in, nobody likes me). Dala is able to practise their breathing, reduce their stress response and remind themselves that their anxious thoughts are not helpful, all without the help of their peer mentor Moxie.
Before the scenario ends the voiceover talks you through some grounding exercises to help the user transition from the virtual world back into the real world.
Once the VR videos were produced, phase two of the project involved feasibility testing.
Phase 2: Testing
We tested the VRET prototype to understand the feasibility of use with CAMHS service users and non-users. CAMHS staff were approached to suggest suitable CAMHS participants to engage in this testing phase. Non-CAMHS students were recruited from a local youth centre with ties to the local CAMHS service following suggestions from youth centre staff. Both the CAMHS and non-CAMHS potential participants were then approached and given information about the study and invited to participate. To aid recruiting, information leaflets and a video explaining the project and study were distributed to patients, staff, and parents. An information session was held to further discuss the project for all interested. This resulted in 18 participants being recruited for this study. Written, informed consent from the teenager and parent was obtained prior to participation in the study. Ethical approval was obtained from the Clinical Research Ethics Committee of Galway University Hospital (Ref Ca. 2580) on the 3rd of March 2021.
Testing procedure
The testing was conducted over two full days in March and April of 2021 in Galway and Roscommon CAMHS offices. Each participant attended on one day. The participant was in the company of a therapist at all times. Each participant trialled each of the three virtual reality simulations. After each simulation trial, participants completed a questionnaire to capture their experience of the simulation. To the best of our knowledge, a tool for assessing the feasibility of a VRET for school related anxiety does not exist in the current literature, hence it was not possible to use a validated instrument. Instead a questionnaire was developed for this process. Table 2 shows the questions and Likert answers used in the Questionnaire.
Table 2. Questions from testing questionnaire
The participants answered the above questions after they completed each of the three VR scenarios. This enables a comparison of anxiety levels after each. It was important, as in all exposure therapy that the simulation elicits an anxious response but not so severe that it leads to excessive distress or flooding. Descriptive statistical analysis was used to analyse the data.
Qualitative data were collected on a one-to-one basis with a facilitator after the three VR scenarios. The following are the five questions that were posed to the teenagers after completing the evaluation, see Table 3.
Table 3. Qualitative questionnaire
Participants were offered the opportunity to comment freely on the virtual reality simulation. Thematic analysis was then conducted iteratively to highlight the main themes in this data using the Braun and Clarke method (Braun and Clarke Reference Braun and Clarke2006). The results of the semi-structured interviews were analysed manually by the study authors in a five step process which involved familiarisation, generating initial codes, searching for themes, reviewing themes and finally defining and naming themes.
Side effects were also monitored. VRET in general has minimal to no side effects with the main potential concern being motion sickness. The participants were made aware of this potential prior to using the VR equipment and advised how to resolve this side effect should it occur (by simply removing the VR headset).
Results
Anxiety response
Each scenario is designed to elicit an anxiety response about school and then guide the participant to new ways of responding. Figures 1 and 2 demonstrate the self-reported level of anxiety experienced by the participants after trialling each of the three scenarios listed above. These figures demonstrate that the scenario designed to elicit the greatest anxiety response (Anxious App) did in fact do this. It also shows that no scenario caused extreme levels of anxiety in any participant. Comparison between these two charts also shows that the evoked anxiety response was much greater for CAMHS attendees compared with non-CAMHS users.
Figure 1. Anxiety level post test (Child and Adolescent Mental Health Services attendees).
Figure 2. Anxiety level post test (non-Child and Adolescent Mental Health Services attendees).
Prototype quality
Immersion, where the user forgets they are in a virtual world, is crucial for the efficacy of VR. It was important to evaluate this alongside believability and the quality of sound and graphics, which can increase or reduce immersivity. We asked ‘How long did you feel you were inside (immersed in) the world (school)’. Answers were given in terms of the percentage of time they felt they were completely immersed in the simulation.
As can be seen from Fig. 3, participants rate the immersivity highly with most teenagers reporting that they were immersed in the simulation for most of the time. Questionnaire results for all participants demonstrated that 93% of participants reported feeling like they were inside the world at least 50% of the time.
Figure 3. Combined measure of immersion.
The other metrics rated as part of the study were the quality of the sound, graphics, and the believability of the scenario.
Ninety four percent of all participants found the scenarios believable, very believable or totally believable. One hundred percent of participants rated the sound effects as good, very good or excellent. Ninety four percent of all participants rated the graphics as good, very good or excellent.
Eighty three percent of study participants could relate somewhat, a lot or very much to Dala. When broken down, figures reveal that 89% of CAMHS attendees but only 72% of non-CAMHS attendees found the main character relatable.
100% of participants believed that VRET would help with school anxiety and would try other VRET in CAMHS.
Results of post testing qualitative analysis
Qualitative analysis was undertaken using thematic analysis (Braun and Clarke Reference Braun and Clarke2006). The initial codes identified included believability, relatability, sound effects, visual effects, efficacy and co-design. Thematic analysis highlighted the themes of immersion, virtual reality effects, effectiveness and collaboration that were further explored and fed back into the iterative design process.
-
Immersion. Participants highlighted that the feeling of immersion is critically important to the simulation. They emphasised the importance of the location (particularly the school toilet) and relatability of the storyline and main character in creating a believable and immersive experience. The perceived accuracy of the portrayal of the experience of school anxiety was a prominent theme here.
-
Virtual reality effects. Participants highlighted the importance of high-quality virtual reality effects (namely visual and sound) in creating the immersive experience and how believable effects are critical to this being an effective VRET.
-
Effectiveness. Participants raised what they felt were the most effective aspects of the simulation, particularly the visual representation of what before had been their own internal experience. They also highlighted that providing relaxation guidance directly after experiencing the anxiety stimulus raised their expectation of how helpful this tool could be.
-
Collaboration. The importance of collaboration and how it can build a better VR tool was emphasised by the participants. The teenagers felt peer input into the making of the VRET greatly enhanced the end result. The participants also reported that the VR design process itself helped with social, team working and communication skills.
-
Outside of these core themes the engagement process around feedback and thematic analysis raised numerous insightful suggestions for improvement that have now been incorporated into the build. These suggestions included additional visuals in the toilet cubicle such as graffiti, more animations of Dala’s thoughts and greater use of light and sounds to evoke Dala’s feelings.
Adverse effects
The testing demonstrated a favourable side effect profile. No motion sickness was reported. 92% of patients reported no physical discomfort whatsoever with mild and transient dizziness being the only reported adverse effect.
Discussion
Despite VRET’s proven effectiveness (Carl et al. Reference Carl, Stein, Levihn-Coon, Pogue, Rothbaum, Emmelkamp, Asmundson, Carlbring and Powers2019) and potential cost and time savings (Pot-Kolder et al. Reference Pot-Kolder, Veling, Geraets, Lokkerbol, Smit, Jongeneel, Ising and van der Gaag2020; Geraets et al. Reference Geraets, van der Stouwe, Pot-Kolder and Veling2021), there is limited research into its use in Child and Adolescent Mental Health Services (Bioulac et al. Reference Bioulac, de Sevin, Sagaspe, Claret, Philip, Micoulaud-Franchi and Bouvard2018). A recent systematic review found there was insufficient research attention given to VR interventions in children and young people and there is a clear need for further co-design, development and evaluation of VR interventions (Halldorsson et al. Reference Halldorsson, Hill, Waite, Partridge, Freeman and Creswell2021). Another recent systematic review in 2021 on this topic noted specifically the lack of research on the efficacy of VRET for children and could find only four studies that met inclusion criteria with a total of only 100 participants (Kothgassner & Felnhofer, Reference Kothgassner and Felnhofer2021). This can be compared to a 2017 systematic review of virtual reality and the treatment of anxiety across all ages that included 49 relevant studies (Oing & Prescott, Reference Oing and Prescott2018). Anxiety around school refusal was chosen as the target of this VRET because of its prevalence and because it is potentially easier to identify by non-expert staff than specific anxiety disorders. In this approach, efficacy may be sacrificed to facilitate practical roll out and increased usability of the tool and this may need to be revisited depending on testing results.
This is a proof-of-concept study designed to test the face validity of this VRET prototype’s ability to elicit an anxiety response. This is the first step in developing an exposure therapy that could help to treat school anxiety earlier. Further testing is required to ascertain if repeated exposure to these virtual reality anxiety stimuli will lead to a significant reduction or even extinction of the users school anxiety. Consideration should be given to the fact that this intervention may increase CAMHS referrals as by highlighting this problem it is possible an increased number of cases could be uncovered and referred on.
This virtual reality tool not only exposes the student to an anxiety provoking stimulus but provides peer inspired supportive mentoring. The ultimate goal is to provide an early intervention in homes and schools before symptoms and associated avoidance become entrenched and a cycle of increasing absenteeism makes returning to school ever more difficult. This VRET was developed to empower teenagers who experienced school anxiety to help other students, using technology in a simple accessible way. Providing an early intervention, that is acceptable to young people, when the problem first arises, benefits not only families but education and primary/secondary care services too. This tool was developed to address school anxiety problems agnostic of the specifics of the individual however this approach has the potential more effective if tailored towards other more specific high prevalence anxiety groups including those with ADHD, autism and learning difficulties.
This automated VRET enables anxious students to learn from what other teenagers have found helpful. It can also provide parents, teachers and therapists with a unique perspective of what an anxious teenager is experiencing. For these reasons this tool has integrated the role of the therapist into the virtual reality environment. The scenarios start with evoking anxiety but then transition to guiding the person through relaxation and grounding techniques designed to alleviate this anxiety. It is envisioned that this VR tool will be rolled out across primary care, schools and homes and may if used early reduce referrals to CAMHS teams.
An integral part of this project was end-user involvement from problem identification to prototype testing. There are definite challenges to this approach. Most notably extensive collaboration with all the stakeholders required significant staff resources and time. However, we believe the believability, relatability, authenticity and immersivity of the VR scenarios could not have been achieved without this level of involvement. Many novel solutions emerged directly from the groups lived experience. This highlights a limitation of the study in that this tool was developed with students attending CAMHS but is envisaged to be used on students who do not yet meet the criteria for immediate CAMHS referral. Further testing will be required to ensure efficacy on this specific student cohort.
To date, VRET has concentrated on replicating the external triggers like heights, spiders and flying but through collaboration it became apparent that the most powerful trigger for the teenagers was not the school environment itself but their thoughts about being judged, excluded, less able or embarrassing themselves. For some members of the project group these thoughts had prevented them from getting into school while others managed by escaping to the school toilets which is the location for the videos. It is seen as their only safe, private space in school. It is also for this reason the place where students are most likely able to get the alone time to gather their thoughts and implement their relaxation and grounding techniques. Another novel approach that arose from collaborating with the students in the design is how the complex and abstract concept of anxiety is represented in the simulation. The analogy of an open phone app represents anxiety in a visual, tangible way that empowers students to pause and remember that when Anxious App is open this escalates anxious feelings creates urgency and drains the phone battery. Transition from the exposure part of the therapy to the relaxation technique part is represented visually in the VRET by the closing of one app and the opening of another app called ‘Pathfinder’.
Anxiety response
Each VRET scenario is designed to elicit an anxiety response around school attendance and then guide the participant on how to overcome this anxiety. One of the most difficult tasks of this build was to design the prototype to elicit an anxiety response that was sufficient to allow for practice of anxiety management techniques without it being overwhelming. The anxiety response was the most important parameter measured during testing.
The scenarios appear to have been pitched at the correct level. The scenario designed to elicit the greatest anxiety response (Anxious App) did in fact do this. It also shows that no scenario caused extreme levels of anxiety in any participant.
An important concept to assess is the relatability of the main character ‘Dala’. As can be seen from Fig. 4, most of the participants rated the relatability of the character highly. This probably reflects the fact that the scenarios and scripts inhabited by Dala were developed largely by teenagers who have difficulties with school anxiety and so accurately reflect the thoughts, beliefs, and situations that they can find themselves in.
Figure 4. Combined measure of main character relatability.
An advantage of using an exposure therapy at home is that it allows for increased frequency of exposure to the stimulus and thus has the potential for quicker habituation. This therapy could be used as frequently as multiple times per day should further testing prove this to be beneficial.
Study limitations
Study facilitators and participants were involved in the development of the prototype so feedback from participants could be susceptible to bias. Thematic analysis was conducted by the study authors leaving it susceptible to significant bias.
Conclusion
As the number of VR studies continue to rise and show promise as an effective potential treatment for anxiety, it is also important for researchers to be aware of what VR interactions work best and what the potential pitfalls are. These results are promising. The build quality of the VR production has been approved by the teenagers. The scenarios were successful in eliciting anxiety in study participants. The level of believability of the scenes was highly rated as was the relatability of ‘Dala’ the main character/avatar. These positive results reflect the time and effort invested in the design phase and are an endorsement of the co-design process we embarked on from the start. These results should encourage other developers to eschew the current norm of end users being involved at the point of testing only and to engage them in co-design of therapies from the very beginning.
The questionnaire developed to assess the VRET demonstrated good face validity. This demonstrates the potential of this prototype as a standalone virtual reality exposure therapy that could reduce the impact of school anxiety on students, their families and the wider school community. Having established evidence of face validity and proof of concept, the next step in this research is to expand the number and range of virtual reality scenarios available. Further testing to assess the efficacy of this tool as a standalone treatment to reduce the impact of school related anxiety and school avoidance is required. There is potential to expand testing to both younger and older students. Should this testing yield positive results we envisage this tool being available in schools and homes to pre-emptively treat and prevent anxiety symptoms and associated behaviours, ultimately reducing the level of anxiety related school avoidance. This in turn could reduce the burden this disorder places on child and adolescent mental health services.
Financial support
We would like to thank the Public Service Innovation Fund from the Department of Public Expenditure and Reform in the Government of Ireland from whom we won a competitive award for this study (PSIF20-0347).
Competing interests
The authors declare none.
Ethical standards
Ethical approval was provided by the Clinical Research Ethics Committee of University Hospital Galway (Ref Ca. 2580) on the 3rd of March 2021. Written informed consent was received from all study participants. The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committee on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008.
Appendix A. Example process map from workshops
Appendix B. Design thinking workshop, schedule and activities
Design thinking phase 1
Design Thinking Phase 2
Design Thinking Phase 3
