Introduction
Bullying in childhood may be one of the most tractable risk factors for mental health problems in childhood and adolescence (Clarkson et al., Reference Clarkson, Bowes, Coulman, Broome, Cannings-John, Charles and Stand Together2022; Ford, Mitrofan, & Wolpert, Reference Ford, Mitrofan and Wolpert2014). Generally defined as a pattern of ‘unwanted, aggressive behaviour … that involves a real or perceived imbalance of power’ (Olweus, Reference Olweus1994), the psychiatric morbidity that may arise from childhood bullying is substantial. Population studies suggest that 25–40% of mental health problems including depression, anxiety and self-harm in young adults may be attributable to childhood bullying (Bowes, Joinson, Wolke, & Lewis, Reference Bowes, Joinson, Wolke and Lewis2015; Fisher et al., Reference Fisher, Moffitt, Houts, Belsky, Arseneault and Caspi2012). Bullied children access more school health, primary care and specialist child mental health services than their counterparts (Brimblecombe et al., Reference Brimblecombe, Evans-Lacko, Knapp, King, Takizawa, Maughan and Arseneault2018) and experience poorer mental and physical health into adulthood (Takizawa, Maughan, & Arseneault, Reference Takizawa, Maughan and Arseneault2014). Bullying is also associated with school absenteeism (Brown, Clery, & Ferguson, Reference Brown, Clery and Ferguson2011), which may impact on future educational attainment and employment prospects. Furthermore, children who bully are also at risk of harm and more likely to show later violent behavior and illicit drug use (Ttofi, Farrington, Losel, Crago, & Theodorakis, Reference Ttofi, Farrington, Losel, Crago and Theodorakis2016). Universal whole-school interventions, promoting school-wide change are the most effective at reducing bullying (Gaffney, Farrington, & Ttofi, Reference Gaffney, Farrington and Ttofi2019) and likely to provide a non-stigmatizing approach to prevention. One of the most widely used bullying prevention programs across Europe is the Finnish Kiusaamisen Vastaan or ‘KiVa’ program. The program was informed by a social architecture model of bullying, which highlights the significant roles of bystanders in supporting or standing against bullying (Salmivalli, Lagerspetz, Bjorkqvist, & Osterman, Reference Salmivalli, Lagerspetz, Bjorkqvist and Osterman1996). In a large-scale RCT conducted in Finland between 2007–2009 across 234 schools (8166 students), KiVa significantly reduced bullying and victimization among 7 to 11-year-old students (Karna et al., Reference Karna, Voeten, Little, Poskiparta, Kaljonen and Salmivalli2011b) and reducing anxiety and depression (Williford et al., Reference Williford, Boulton, Noland, Little, Karna and Salmivalli2012). Since 2009, KiVa has been implemented in over 90% of Finnish public schools (approximately 2700 schools) and demonstrates year-on-year positive effects (Karna et al., Reference Karna, Voeten, Little, Poskiparta, Alanen and Salmivalli2011a). Whilst KiVa has been found to be successful in reducing victimization in trials in the Netherlands (Huitsing et al., Reference Huitsing, Lodder, Browne, Oldenburg, Van der Ploeg and Veenstra2020) and Italy (Nocentini & Menesini, Reference Nocentini and Menesini2016), other trials, including in Wales, found no effect (Axford et al., Reference Axford, Bjornstad, Clarkson, Ukoumunne, Wrigley, Matthews and Hutchings2020). There are important differences between the education systems in the UK and Finland that may influence whether KiVa intervention effects will transfer. The route to becoming a teacher in the UK is more varied and children are tested more to assess their academic capabilities. Schools are typically more diverse and have greater social inequalities (Hadjar & Uusitalo, Reference Hadjar and Uusitalo2016). Whilst every school in the UK is required by law to have an anti-bullying policy, more guidance on effective, evidence-based anti-bullying programs is needed. In this two-arm pragmatic multicenter cluster randomized controlled trial (cRCT) with embedded economic evaluation, our primary aim was to assess the effectiveness and costs of the KiVa anti-bullying program in reducing student-reported bullying victimization. Additionally, a range of secondary outcomes were explored including student-reported bullying perpetration, participant roles in bullying and school related well-being; teacher-reported student emotional and behavioral difficulties; and teachers' self-efficacy in dealing with bullying, mental well-being, and burnout.
Methods
A full outline of the research methodology can be found in our protocol paper (Clarkson et al., Reference Clarkson, Bowes, Coulman, Broome, Cannings-John, Charles and Stand Together2022).
Study design
We conducted a two-arm pragmatic, multicenter, cluster RCT with an embedded economic evaluation. Schools and students were recruited across four areas (North Wales, Birmingham, Southeast England, and Southwest England).
Ethical approval was granted by Bangor University Psychology Research Ethics and Governance Committee (2019-16592) on 13th November 2019. The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008.
Schools and participants
We included mainstream UK state-maintained primary schools with at least two KS (Key Stage) 2 classes for children aged 7–11 years. We excluded schools that: primarily delivered education through a language other than English or Welsh; were already implementing a recognized anti-bullying intervention program or had previously implemented KiVa; catered solely for children with Special Educational Needs; and schools without a leadership that could guarantee project participation for the year of data collection/ implementation. All students at participating schools in school years 3–5 (KS 2) were eligible for the trial. Head teachers provided opt-in consent for school-level data collection and the implementation of KiVa; parents/carers provided opt-out consent for data collection relating to their child; and teaching staff provided opt-in consent for data about themselves. Parents, carers or teachers could not withhold consent for KiVa to be delivered as part of the school's statutory personal, social education/personal, social, health and economic education (PSE/PSHE) curriculum. Children provided their written assent via e-tablets for self-reported outcomes at each data collection point. Prior to the 12-month follow-up, all parents received a letter detailing the plans to link their child's trial data to data held by the Department for Education, National Pupil Database (NPD) and in Wales, the Welsh Secure, Anonymised Information Linkage (SAIL) database. Parents had the opportunity to opt-out of this activity without it impacting on their child's continued participation in the trial.
Randomization and masking
Schools were randomized to receive the KiVa intervention or continue with UP by the Centre for Trials Research at Cardiff University using random permuted blocks in a 1:1 ratio. Randomization was stratified by the following: study area (North Wales, Birmingham, Southeast and Southwest England); number of KS2 students (UK school years 3–6, ages 7–11) (below or above site median); and percentage of students eligible for Free School Meals (FSM) (below or above site median). Randomization of schools occurred after all schools had been recruited. Due to COVID-19 pandemic related school closures between January and March 2021 delaying the timing of child-reported baseline data collection and the need for intervention schools to be trained in KiVa before the end of academic year, 48% of schools were informed of their allocation before child-reported baseline data were collected. Most (94%) of teacher reported TSDQ data were collected before schools were informed of their allocation. Researchers collecting baseline school data were blind to the randomization process, as were students (who were the source of the primary outcome data). It was not possible to blind researchers and field workers at follow-up due to the requirement that KiVa schools make the intervention public (e.g. through posters and staff tabards). The statistician analyzing the primary and secondary outcome data and the health economists undertaking the economic analysis remained blinded.
Patient and public involvement
Two advisory groups were established in North Wales, one with teachers and head teachers, and one with six children from three different primary schools. Groups met twice at the start of the project and reviewed and advised on data collection methods, participant information and consent forms. The groups reconvened after the COVID-19 pandemic in 2023 to advise on the output and dissemination activities.
Procedures
Baseline survey data were collected from students in school years 3–5 between 20th April–30th June 2021. Child questionnaires were administered via e-tablets with researchers reading the questions aloud to the whole class. All questions included the option ‘I'd rather not say’. Teachers completed questionnaires either online or with paper and pen. The KiVa anti-bullying intervention was implemented in the academic year which ran from September 2021 to July 2022, and survey outcomes were measured from 26th April–15th July 2022.
Intervention
KiVa is based on research demonstrating that ‘bystanders’ – children who are present during bullying but not actively involved – can contribute to the maintenance of bullying by assisting or reinforcing the perpetrator's behavior, giving the bully a position of power (Salmivalli et al., Reference Salmivalli, Lagerspetz, Bjorkqvist and Osterman1996). Defending the victim, on the contrary, can help to make bullying an unsuccessful strategy for attaining high social status. By influencing the behavior and norms of all students, the social rewards gained by perpetrators are reduced and, consequently, reduces their motivation to bully. The KiVa program has two distinct components; universal actions that target the whole school and indicated actions for tackling specific cases of bullying. Universal actions at the class and school level help students to recognize bullying, providing them with safe ways of responding to incidents, and teaching them to empathize with, and support, victims. This includes structured lessons, delivered by teachers. The manualized curriculum targets students in school Years 3 and 4 (aged 7–9) (Unit 1), and Years 5 and 6 (aged 9–11) (Unit 2). Although the lesson plans target KS2 classes, the program is introduced to all staff, parents and students and is visible across the school. Each unit contains ten structured 90-minute lessons, typically delivered fortnightly as twenty 45-minute lessons throughout the school year by class teachers. The curriculum encourages student engagement via oral presentations, role-play, videos, group work and whole class activities. Online games, that support lessons can be played at home or in school. Other universal actions include posters for school corridors, high-visibility vests for break-time supervisors to highlight the presence of supervision, and a parent's guide. A trained KiVa team (2 to 3 staff members) address confirmed bullying incidents with children involved using the structured and scripted indicated actions, with individual follow-up meetings 1–2 weeks later to ensure that the situation has improved. Two members of the teaching/management team from each intervention school attended a local two-day training course delivered in June 2021 by accredited KiVa trainers at each site and then led school-wide implementation. Due to ongoing disruption from the COVID-19 pandemic, some training was delivered in an online format.
The intervention delivery was over one academic year. However, the program is designed to be embedded into ongoing school practice.
Comparison
The mandatory PSE/PSHE curricula in Wales and England aim to develop students' pro-social values and attitudes and empower participation in school and community life as responsible citizens. Comparison schools continued to use existing methods to cover this curriculum, and to address bullying. UP schools continued with their standard practice and were free to implement any other programs or strategies they wished to in their school. At baseline, all schools reported which existing policies and practices they used to prevent and deal with bullying. At the follow-up point all schools were asked if there had been any changes to their policies or practices during the trial period.
Outcomes
Primary outcome
The primary study outcome was self-reported bullying victimization, measured by student responses on the Olweus Bully/Victim Questionnaire (OBVQ) (Olweus, Reference Olweus1996). The OBVQ measures different forms of bullying, including verbal, physical, relational and cyber-bullying. The global item: ‘How often have you been bullied at school in the last couple of months?’ was used to measure victimization. Students respond on a five-point scale (0 = ”not at all”, 1 = ”once or twice”, 2 = ”2 or 3 times a month”, 3 = ”about once a week”, 4 = ”several times a week”). The response was dichotomized with those scoring 2 to 4 classified as victimized and those scoring 0 to 1 as not victimized (20)
Secondary outcomes
i) Student-reported outcomes:
1. The OBVQ global item: ‘How often have you bullied others at school in the last few months?’ measures bullying perpetration. Students' responses, using the same five-point scale reported above were analyzed continuously and dichotomized, with scores of 2 to 4 classified as perpetration (Solberg, Reference Solberg2003).
2. Subjective student wellbeing in school was measured using the ‘How I feel about my school’ (HIFAMS) (Allen et al., Reference Allen, Marlow, Edwards, Parker, Rodgers, Ukoumunne and Ford2018). The survey is comprised of seven items which asks students how they feel about various aspects of school life, for example how they feel when completing their schoolwork or when they are in the playground.
3. Student empathy was measured using the Empathy Toward Victim Scale (Pöyhönen, Juvonen, & Salmivalli, Reference Pöyhönen, Juvonen and Salmivalli2010). The seven-item measure asked respondents to rate their level of empathy towards victims on a Likert scale anchored from ‘never’ (0) to ‘always’ (3).
4. Roles in bullying situations was measured using the ‘Participant Role Questionnaire’ (PRQ) (Salmivalli & Voeten, Reference Salmivalli and Voeten2004), designed to identify the different roles that peers play in bullying situations. The questionnaire identifies five different bullying roles; bully, assistant, reinforcer, defender and outsider. We were particularly interested in the role of defender, given that KiVa is intended to raise awareness among children about the importance of everyone taking a stand against bullying. The respondent rates how often they behave in the ways described for each role on a three-point scale (Never, Sometimes, Often).
ii) Teacher-reported student outcomes (measured through teacher survey reports for each student):
5. Teacher-reported Strengths and Difficulties Questionnaire (TSDQ) (R., Reference R.2001) was reported at baseline and follow-up. The TSDQ measure provides a Total Difficulties score from 20 behavioral and emotional problems items as well as a pro-social behavior score from five items. Subscales include a hyperactivity score, emotional score, conduct problems score and peer problems score.
-
We intended to collect data from the Absence and Exclusion, and KS2 attainment datasets for English pupils from the National Student Database (NPD) and from the Welsh Government SAIL database for students in both trial arms. However, due to COVID 19 pandemic-related delays in obtaining data, these data are not currently available. For a full list of tertiary measures (not included in this paper), see study protocol (Clarkson et al., Reference Clarkson, Bowes, Coulman, Broome, Cannings-John, Charles and Stand Together2022).
iii) Staff-level outcomes.
6. Adapted version of the 5-item Challenging Behavior Self-Efficacy Scale, designed as a measure of teacher self-efficacy related to challenging behaviors and adapted to specifically
7. to bullying behaviors (Hastings & Brown, Reference Hastings and Brown2002).
8. Warwick-Edinburgh Mental Wellbeing Scale (Tennant et al., Reference Tennant, Hiller, Fishwick, Platt, Joseph, Weich and Stewart-Brown2007). This 14-item positively worded scale measures adult mental wellbeing.
9. Maslach Burnout Inventory-Educator
Survey (MBI-ES), a 22-item psychological inventory measuring emotional exhaustion, depersonalization, and personal accomplishment (Maslach, Jackson, & Schwab, Reference Maslach, Jackson, Schwab, Jackson, Maslach and Leiter1996).
Process evaluation
The methods for the process evaluation are reported in our protocol paper (Clarkson et al., Reference Clarkson, Bowes, Coulman, Broome, Cannings-John, Charles and Stand Together2022) and the results will be in a separate publication.
Economic evaluation
In line with guidance for evaluating complex public health interventions (Skivington et al., Reference Skivington, Matthews, Simpson, Craig, Baird, Blazeby and Moore2021), a broad economic evaluation using cost-consequence analysis (CCA), cost-effectiveness analysis (CEA), and cost-utility analysis (CUA) frameworks explored the cost-effectiveness of the KiVa intervention compared to UP. An education-sector perspective (Edwards & McIntosh, Reference Edwards and McIntosh2019) was taken to evaluate incremental effects at 12 months after randomization. These results are reported separately.
Cost analyses from an education-sector perspective focused on two out of three components of the KiVa program; (i) class-level delivery of KiVa lessons and, (ii) whole-school activities to promote KiVa to the school population. KiVa cost components were categorized into recurrent and non-recurrent costs and unit costs were derived using appropriate UK estimates for the trial year (2021/2022) (Appendix X, Table 1). Resource use and cost data for KiVa were collated from the Children's Early Intervention Trust (CEIT) charity that holds the license for KiVa dissemination in the UK and process evaluation school checklists. Resource use data for UP were collected via PSE/PSHE school checklists. Cost calculations for usual practice considered the school-reported overlap between the KiVa and PSE/PSHE curriculums. Where there were missing data at the school-level mean imputation was used unless there was evidence that these costs had not been incurred. Discounting was not applied.
Table 1. Baseline characteristics for students and teachers (Data sources: Pupil, TSD and teacher questionnaire)
UP, Usual practice; OBVQ, Olweus Bully/Victim Questionnaire; HIFAMS, How I Feel About My School; ETV, Empathy Toward Victim Scale; PRQ, Participant Role Questionnaire; TSDQ, Teacher Strength and Difficulties Questionnaire; MBI-GS, Maslach Burn-Out Inventory General Survey; WEMWBS, Warwick-Edinburgh Mental Well-being Scale; CBSES, Challenging Behavior Self-Efficacy Scale.
Data are n (%), mean (s.d.) or median (IQR).
a Five children self-reported being six years of age despite being in school years 3–5.
b Data were missing for 870 (KiVa) and 749 (UP) students. Bullying victimization defined as ‘How often have you been bullied at school in the past couple of months?’ = 2 or 3 times a month/About once a week/Several times a week.
c Data were missing for 723 (KiVa) and 604 (UP) students. Bullying perpetration defined as ‘How often have you taken part in bullying another pupil(s) at school in the past couple of months?’ = 2 or 3 times a month/About once a week/Several times a week.
d Data were missing for 269 (KiVa) and 211 (UP) students. HIFAMS scores range from 0 to 14 – higher scores reflect greater happiness.
e Data were missing for 338 (KiVa) and 310 (UP) students. ETV scores range from 0 to 21 – higher scores indicate greater empathy for the victim.
f Teachers reported TSDQs for a higher number of students compared to the pupils self-reported questionnaires.
g Data were missing for between 2 and 5 students. For all subscales, higher scores indicate more strength or difficulty.
h Total of all subscales excluding Prosocial behavior subscale.
i Data were missing for between 10 and 15 teachers. Scores range from 0 to 30.
j Data were missing for 14 (KiVa) and 11 (UP) teachers. Scores range from 0 to 36.
k Data were missing for 8 (KiVa) and 9 (UP) teachers. CBSES scores range from 5 to 35.
l Data were missing for 7 (KiVa) and 6 (UP) teachers. WEMWBS scores range from 14 to 70.
Safety and adverse events
There were no anticipated risks to participants or schools.
Statistical analyses
Our sample calculation was based on previous research (reduction from 18% to 14% in rates of victimization) (Skivington et al., Reference Skivington, Matthews, Simpson, Craig, Baird, Blazeby and Moore2021), and a similar baseline victimization rate of 18% from a small UK based pre-post study (Edwards & McIntosh, Reference Edwards and McIntosh2019). Assuming 111 students in Years 3 to 5, an intracluster correlation coefficient (ICC) of 0.02, and allowing for one school dropout per arm, 10% student dropout due to either opt-out or loss to follow-up, an 18% rate of victimization, and a relative reduction of 22%, a trial involving 116 schools (58 per arm) would provide 90% power at a 5% significance level (a total of 12 828 students).
All analyses were intention to treat without imputation (a complete case analysis restricting to pupils with responses at both baseline and follow-up), with outcomes compared between KiVa and UP groups using three-level regression models (allowing for clustering between students within school, and between schools within sites). Analyses controlled for school level stratification variables (school size, proportion of students eligible for FSM), key student characteristics (age, sex) as well as baseline outcome measures (where collected).
For binary outcomes a logistic model was used, and the result presented as adjusted odds ratios (ORs) comparing the odds of an event in KiVa schools compared with UP schools. For continuous outcomes, we fitted a linear-regression model and presented results as difference in adjusted means (KiVa minus UP). Multilevel ordinal logistic regression model was used to compare TSDQ scales. Due to skewness in the TSDQ scales, data were categorized according to the clinical cut-offs (normal, borderline, abnormal). Box-cox transformations were applied to skewed data when necessary and Glass's delta standardized effect size calculated as the difference in means (KiVa – UP) divided by the standard deviation of the UP group. Between-group comparisons are presented with two-sided 95% confidence intervals (CI). Pre-specified exploratory sub-group analyses investigated the effect of the intervention by student gender and school-level percentage of students eligible for FSM status. Three additional post-hoc subgroups were performed: Pupils self-reported age (6–7, 8, 9, 10/11 years of age), whether the school previously recruited before COVID-19 pandemic for the Stand Together trial, and the timing of baseline data collection. For all subgroup analyses, an interaction term was added to the main comparison model between the study arm and each variable and results presented using 95% CIs.
The impact of missing outcome data on the trial conclusions, missing mechanisms were explored, and appropriate imputation methods applied via sensitivity analyses.
The effect of KiVa lesson dosage on the primary outcome was explored in a sensitivity analysis and estimated in a way that preserves randomization using complier-average causal effect estimates, using a two-stage least squares instrumental variable regression model (Dunn, Maracy, & Tomenson, Reference Dunn, Maracy and Tomenson2005).
The secondary outcome of the OBVQ bullying victimization and perpetrator score was originally omitted from the SAP but is included as a post-hoc analysis (ordinal logistic regression).
In exploratory analyses, mediation analyses were used to determine whether the effectiveness of the intervention on bullying victimization occurred through change in affective empathy and secondly with change self-efficacy in defending. The mediation analysis was parameterized as a 2-1-1 model where the independent variable (intervention) is at the school level, while the mediator (affective empathy or self-efficacy) and the outcome (bullying victimization) are at the pupil level. Both models adjusted for pupil-level characteristics (age, gender), school level characteristics (FSM, KS2) along with corresponding baseline measures.
Analyses were done by two statisticians (MB and RCJ) using IBM SPSS Statistics version 20.0 (IBM Corporation, Armonk, NY, USA), and Stata® version 17 (StataCorp LP, College Station, TX, USA).
An independent steering committee oversaw the study. The trial is registered at the ISRCTN 12300853.
Results
See Fig. 1 for the CONSORT Flow Diagram. Between 8th November 2019 and 12th February 2021, 118 schools were recruited from 396 schools approached across four trial sites. From a potential of 11 922 students, 11 111 (93.2%) students completed a baseline questionnaire. Twelve month-follow up data was collected from 9981 (90.1%) students, 5321 (90.0%) students in the KiVa schools compared to 4660 (90.2%) in UP schools. In addition, a total of 263 (KiVa) and 202 (UP) questionnaires were completed by students who had not completed a baseline questionnaire. Baseline student and teacher characteristics were well balanced across trial arms (Table 1). Forty-nine percent of students were female and on average aged 8.7 years.
Figure 1. The CONSORT Flow Diagram. *Of the 118 schools recruited to the study, 47 had previously completed baseline data for the Stand Together Trial prior to the COVID 19 pandemic and were re-recruited when the study restarted.
The total number of students available for the complete case analysis (baseline and 12-month follow-up) was 9981 (Table 2). At baseline, 20.3% of students in KiVa schools and 21.6% of students in UP schools experienced bullying victimization. After 12-month follow-up, the bullying victimization rate was reduced to 17.7% in the KiVa arm compared to 20.7% in the UP arm. The adjusted odds ratio indicated that the students from schools randomized to receive KiVa had a 13% lower odds of reporting bullying victimization compared to students in schools randomized to continue with UP (Table 2) (odds ratio (OR): 0.87, 95% CI 0.78–0.97, p value = 0.009). Pre-specified sensitivity analyses were performed to investigate the effect of dosage (at least seven out of the 10 annual lessons delivered over a full school year to each class) on intervention effect, and imputation for missing data. The intervention effect was consistent across all sensitivity analyses (appendix, Table S1.1). No differential effects were found for any primary outcome in subgroup analyses (appendix, Table S2.1). The additional post-hoc analysis on the maximum OBVQ bullying victimization and perpetrator score, supported the primary analysis (Table 3).
Table 2. Primary and secondary outcomes for students (Data sources: Pupil and TSD questionnaire)
OBVQ, Olweus Bully/Victim Questionnaire; HIFAMS, How I Feel About My School; ETV, Empathy Toward Victim Scale; PRQ, Participant Role Questionnaire; TSDQ, Teacher Strength and Difficulties Questionnaire.
Data are n (%). Mean (s.d.) or median (IQR).
a Adjusted for free school meals, KS2 school size, corresponding baseline measure, age and gender of pupil, and clustered within 118 schools within 4 recruitment sites.
b Adjusted odds ratio for KiVa v. usual practice.
c Adjusted difference in means for KiVa minus usual practice.
Table 3. OBVQ maximum self-reported victimization and perpetration scores – post-hoc analysis
Data are n (%).OBVQ = Olweus Bully/Victim Questionnaire.
a Adjusted for free school meals, KS2 school size, corresponding baseline measure, age and gender of pupil, and clustered within 118 schools within 4 recruitment sites.
Some secondary outcomes suggested positive impacts of the KiVa intervention (Table 2). Students in schools randomized to KiVa had increased empathy towards victims compared to students in the UP arm (adjusted mean difference: 0.45, 95% CI 0.20–0.69), however they reported significantly less self-efficacy in defending other children from being bullied compared to students in the UP arm (OR: 0.76, 95% CI 0.59–0.97). We found no evidence of a difference in school-related happiness (HIFAMS) or self-reported bullying perpetration. Students in the KiVa arm had 15% lower odds of having teacher-reported TSDQ peer problems as compared to students in the UP arm (OR: 0.85, 95% CI 0.75–0.96). Follow-up exploratory analysis suggested that this effect was driven by a reduction in teacher-reported peer victimization (see Table S3.2). We found no evidence of trial arm differences in the other TSDQ subdomains (including emotional problems, conduct problems, hyperactivity and prosocial behavior). We found no evidence of intervention effects on teacher self-efficacy in dealing with bullying, wellbeing, or burnout (Table S3.1).
Table S4.1 presents the direct, indirect and total effect of intervention and corresponding mediator on bullying victimization. These models indicated that as compared to usual practice, 22% effect of KiVa is mediated through change in affective empathy whereas 12.5% of this effect mediated through change in self-efficacy in defending.
Economic evaluation
Full economic evaluation results will be reported in a separate paper. The main cost components of the KiVa intervention were training costs and staff-time spent delivering the KiVa curriculum (Appendix 5, Table S5.1). Mean total education sector-related costs of the KiVa intervention were £38.18 (s.d. £11.36) per pupil in the first year of KiVa set-up and implementation (i.e. non-recurrent and recurrent costs). Means costs were £19.05 (s.d. £3.94) per pupil for recurrent costs only. An average 40% overlap between the KiVa and PSE/PSHE curriculums was reported by intervention schools. When considering the mean cost of delivering 40% of the PSE/PSHE curriculum (£17.40 (s.d. £6.74) per pupil), the cost of KiVa compared to usual practice is estimated to be £20.78 more per pupil in the first year of KiVa set-up and implementation and £1.65 more per pupil when only recurrent costs are considered. Further details of resource use and costs are reported in the appendix.
Discussion
The results of the study showed that the KiVa intervention reduced bullying victimization among students from KiVa schools reporting a 13% reduction in the odds of victimization compared with students from schools continuing with usual practice. The reduction in odds of bullying victimization observed in our trial (13%) was lower than reported in other studies (Finland – 30% (Karna et al., Reference Karna, Voeten, Little, Poskiparta, Kaljonen and Salmivalli2011b); Netherlands 43.4% (Huitsing et al., Reference Huitsing, Lodder, Browne, Oldenburg, Van der Ploeg and Veenstra2020), Italy 50.9% (Nocentini & Menesini, Reference Nocentini and Menesini2016)). This may reflect the significant challenges in implementing a whole-school intervention in the context of the COVID-19 global pandemic, positive effects of UP, or both. It is unclear whether a 13% reduction in bullying could be considered meaningful at the school level. The concept of identifying ‘clinically meaningful’ reductions in symptoms is common in clinical psychology but is yet to be addressed in anti-bullying interventions. (Carey, Ridler, Ford, & Stringaris, Reference Carey, Ridler, Ford and Stringaris2023).
In addition to reducing bullying victimization, students in the intervention group demonstrated increased empathy. Indeed, 22% effect of the KiVa was found to be mediated through change in affective empathy. These positive findings further support the efficacy of the KiVa program in improving the overall social dynamics within schools. Students in the intervention group had fewer reported peer relationship problems, however this effect was no longer significant when adjusting for multiple testing. We also observed some small, unexpected effects on student reported self-efficacy in defending victims of bullying, which decreased in the intervention group. These findings should be interpreted with caution and require replication in independent samples. It is also important to note that no significant differences were observed in teacher-reported emotional or conduct problems, or child-reported well-being. Whilst the original evaluation of KiVa in Finland observed a significant reduction of symptoms of anxiety (but not depression) (Williford et al., Reference Williford, Boulton, Noland, Little, Karna and Salmivalli2012), both the Dutch and Italian KiVa evaluations found small, largely non-significant effects on internalizing symptoms (Carey et al., Reference Carey, Ridler, Ford and Stringaris2023; Huitsing et al., Reference Huitsing, Lodder, Browne, Oldenburg, Van der Ploeg and Veenstra2020; Nocentini & Menesini, Reference Nocentini and Menesini2016). These results suggest that while the KiVa program may reduce bullying, effects may not extend to mental health outcomes.
Our exploratory findings also indicate that the effectiveness of the KiVa program may be consistent across different demographic groups, including those from more socioeconomically deprived settings and by gender. This suggests that the intervention may have the potential to benefit students from diverse backgrounds and is not limited to specific subgroups – an important finding when attempting to ensure equitable access to effective anti-bullying interventions. Furthermore, the economic evaluation suggests that the KiVa program is a low-cost intervention compared to many other UK school-based interventions. This is a significant finding and suggest that implementing whole-school approaches like KiVa is a feasible and efficient strategy for reducing bullying in schools.
Our study had several strengths. This is the largest randomized controlled trial in the world to date of the KiVa anti-bullying programme and included a broad and diverse range of schools and students. We included an embedded economic evaluation that collected data on the wider costs of bullying to schools and families. School and student participation were high. All analyses were pre-registered and were performed blind by statisticians at the Centre for Trials Research, a UKCRC Registered Trials Unit.
Limitations of our trial include our reliance on self-reported bullying victimization and perpetration, which may inflate response bias. Whilst peer-nomination is often considered the gold-standard method of measuring bullying, this is not acceptable to many schools in the UK, nor does it typically allow for measurement of bullying outside of the classroom context. It also does not capture student perceived threat. Given that teachers also reported fewer peer relationship difficulties in the intervention arm, we can be confident that our findings support our student-reported outcomes. Secondly, some schools were informed of their assignment to the intervention before student reported baseline data were collected due to the impact of COVID-19 pandemic related school closures on our data collection timetable and the need for schools to train staff and plan for the next academic year. We mitigated the impact of this by asking teachers not to inform students or research staff of their assignment, and by performing sensitivity analyses to check that this did not impact on our findings (see Supplementary Table S2.1). Additionally, some schools were recruited to our trial in 2019, prior to the COVID-19 pandemic. For these schools, baseline data collection needed to be repeated. We mitigated any impact of this repeated data collection by performing additional sensitivity analyses to ensure that intervention effects were similar for these schools (see Supplementary Table S2.1). Finally, we collected follow-up data before the end of the school year whilst pupils remained in the same classes when schools had delivered the KiVa intervention for less than one academic year. Complex, whole-school interventions typically take time to bed in, which may have contributed to the lower than predicted intervention effect sizes observed. In support of this, we found that intervention schools that had delivered more of the KiVa lessons showed a greater reduction in bullying. It is plausible that the effects of the intervention may increase as the program gets embedded within the broader school system – as found in an RCT in the Netherlands which found greater effect sizes after two years of program implementation (Huitsing et al., Reference Huitsing, Lodder, Browne, Oldenburg, Van der Ploeg and Veenstra2020), as well as the broader rollout of the KiVa program in Finland (Karna et al., Reference Karna, Voeten, Little, Poskiparta, Alanen and Salmivalli2011a).
Our economic evaluation compared costs of KiVa training and delivery including whole-school activities against the mean cost of delivering 40% of the PSE/PSHE curriculum in UP schools. However, it is a mandatory requirement for all schools in the UK to have anti-bullying policy and practices in place, and we were unable to cost any additional school time focused specifically on bullying in UP schools, so these costs are likely to be an underestimate of the true costs of tackling bullying in UK schools. Thus, it is probable that the relative additional costs of delivering KiVa are even smaller than reported here.
Our study was conducted during the COVID-19 pandemic, when schools were experiencing increased pressure, closures, staff and student absences and other COVID-19 restrictions. This impacted all elements of implementation, from teacher training through to lesson delivery and data collection, primarily through staff and student absences. Given this unusual context, it is notable that only one school dropped out of the intervention (but remained in the data collection), and that our data return rates were high.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S0033291724002666.
Data availability statement
De-identified patient data from this study along with a data dictionary can be made available after a signed data access agreement upon reasonable request to the principal investigators of the trial (j.hutchings@bangor.ac.uk and lucy.bowes@psy.ox.ac.uk).
Acknowledgements
The research study has been funded by the UK National Institute for Health and Care Research (NIHR) Public Health Research (17-92-11). We obtained separate funding for the intervention costs from the Rayne Foundation (for participating schools in Southeast England, from GwE – School Effectiveness and Improvement Service for north Wales (for participating schools in Wales), from Children's Services, Devon County Council (for participating schools in Exeter) and from HSPC Global Services (UK) Ltd (for participating schools in Birmingham and Warwick). We would like to thank all schools that participated in the Stand Together Trial.
Overall, the results support the effectiveness and low cost of the KiVa anti-bullying program as a public health intervention. The findings add to evidence for the importance of bystander effects and underscore the importance of implementing evidence-based whole-school approaches to address bullying. The results provide a valuable foundation for further research on the longer-term integration and embedding of KiVa into school practices. Hayes is supported by the National Institute for Health Research Applied Research Collaboration South West Peninsula (PenARC). The views expressed in this publication are those of the authors and not necessarily those of the National Institute for Health Research or the Department of Health and Social Care.
Authors’ contributions
JH, RPH, LB, and SC conceived the study and led the funding applications. JH, LB, RPH, TF, RH, and SC participated in the design of the study. LB led the drafting of the manuscript. JS designed the process evaluation. RCJ contributed to the design of the study and RCJ and MB wrote the analysis plan. RTE and JC contributed to the design of the study and design of the cost-effectiveness evaluation. RTE took over the economic evaluation and RTE and HW wrote the health economics protocol and conducted the HE analyses, EC and EOJ led the trial management. MRB led the Birmingham site, RH the South West England site, LB the South East England site and JH the North Wales site. JB led data collection at two sites and contributed to study material design. FLW led on data linkage, and MS on data cleaning. RCW led the PPI work. MB, JB, MRB, RCJ, SC, EC, RTE, TF, RPH, RH, EOJ, PP, JS, JT, RCW, HW, MEW and JH inputted on all drafts of the paper, and our discussion of findings. All authors had full access to all the data in the study and LB had final responsibility for the decision to submit for publication.
Competing interests
JH manages the employees in the CEIT registered charity. SC is employed as the UK KiVa Coordinator. RW receives a salary from GwE. The funding bodies were not involved in the study design, in collection, analysis and interpretation of data or in the writing of the manuscript. All other authors declare no competing interests.
Research in context
Evidence before this study
Bullied children access more school health, primary care and specialist child mental health services than their counterparts (5) and experience poor mental health into adulthood (3). Systematic reviews indicate that whole-school, multi-component interventions can be effective at reducing bullying. The KiVa intervention was developed and taken to scale in Finland and requires rigorous evaluation in different contexts.
Added value of this study
We present evidence from the largest randomized controlled trial to date of the KiVa anti-bullying intervention across 118 primary schools in England and Wales. We found that KiVa reduced child-reported bullying victimization by 13%, reduced peer problems and promoted increased empathy among children.
Implications of all the available evidence
Results support the effectiveness and low cost of the KiVa anti-bullying program as a public health intervention.
Open access
