There were 1689 suicides among youth ages 12 to 17 years in the United States in 2021, which reflect a 50% increase in the adolescent suicide rate since 2001 (Centers for Disease Control and Prevention, 2023a). Moreover, approximately 10% of high school students in the U.S. report one or more suicide attempts (SAs) within a 12-month period (Centers for Disease Control and Prevention, 2023b), and adolescent SAs have been associated with psychosocial impairment, risk for subsequent suicidal behavior, and suicide (Bridge, Goldstein, & Brent, Reference Bridge, Goldstein and Brent2006). There is a pressing need to develop research-based knowledge to guide our recognition of the warning signs (WS) for adolescent SAs, which could facilitate improved risk recognition and timely intervention.
Many distal risk factors for adolescent SAs – those occurring in the weeks, months, and years prior to an attempt – have been identified (Brent, Baugher, Bridge, Chen, & Chiappetta, Reference Brent, Baugher, Bridge, Chen and Chiappetta1999; King et al., Reference King, Grupp-Phelan, Brent, Dean, Webb, Bridge and Casper2019). Such factors are critically important as they indicate who is more likely to attempt suicide at some unknown point in time. Nevertheless, they fail to indicate when an individual may be at greatest risk (Glenn & Nock, Reference Glenn and Nock2014; Rudd et al., Reference Rudd, Berman, Joiner, Nock, Silverman, Mandrusiak and Witte2006), which is one of the major challenges facing healthcare providers who must make rapid triage or disposition decisions as well as the parents and educators who care for adolescents at risk for suicide. The critical question is, ‘How do I recognize when my patient's suicide risk (or child's, student's) has escalated and may warrant immediate action to prevent a suicide attempt?’ Given research suggesting that two-thirds of suicide attempt planning steps take place within 12 h of a suicide attempt (Millner, Lee, & Nock, Reference Millner, Lee and Nock2017), an improved understanding of near-term risk and WS is particularly critical.
A warning sign for suicide can be defined as a ‘…detectable sign that indicates heightened risk for suicide in the near-term (i.e. within minutes, hours, or days) …’ (Rudd et al., Reference Rudd, Berman, Joiner, Nock, Silverman, Mandrusiak and Witte2006). Acknowledging the lack of controlled research on near-term suicide risk, in 2006 an expert panel generated a consensus list of WS, with items encompassing the overt expression of heightened risk (e.g. suicidal threats) and proximal increases in certain behaviors, affects, and cognitions (Rudd et al., Reference Rudd, Berman, Joiner, Nock, Silverman, Mandrusiak and Witte2006). Several years later, the Substance Abuse and Mental Health Services Administration convened a national panel to establish a consensus list of WS for adolescent suicide risk. Based on the limited scientific literature available and input from researchers, clinicians and parent survivors of adolescent suicides, the panel put forth four WS: (1) talking about or making plans for suicide; (2) expressing hopelessness; (3) severe/overwhelming emotional pain or distress; and (4) worrisome behavioral cues or marked changes in behavior, particularly in the presence of other WS (http://www.youthsuicidewarningsigns.org/).
Candidate warning signs for adolescent suicide attempts
Most studies of adolescent risk for SAs have used a long assessment window, confounded distal and near-term factors, and used a between-person methodology (Rudd, Reference Rudd2008; Tucker, Crowley, Davidson, & Gutierrez, Reference Tucker, Crowley, Davidson and Gutierrez2015). Due to the ecological fallacy of extending between person findings to conclusions about within person processes, the risk factors identified in these studies cannot be conceptualized as WS; however, they do provide possible clues to candidate WS.
Two strong WS candidates for adolescent suicide attempts are recent suicidal ideation and low perceived school or social connectedness (King et al., Reference King, Grupp-Phelan, Brent, Dean, Webb, Bridge and Casper2019). Aspects of recent suicidal ideation, such as presence of suicidal planning (Nock et al., Reference Nock, Green, Hwang, McLaughlin, Sampson, Zaslavsky and Kessler2013), warrant consideration, as do threats or disruptions to connectedness. Social withdrawal has been associated with suicide risk (Rutter & Behrendt, Reference Rutter and Behrendt2004); and interpersonal conflicts, losses, and legal/disciplinary problems are common acute stressors associated with suicide (Brent et al., Reference Brent, Perper, Moritz, Baugher, Roth, Balach and Schweers1993b). Bullying victimization and perpetration (Borowsky, Taliaferro, & McMorris, Reference Borowsky, Taliaferro and McMorris2013), as well as sexual and physical abuse (Castellví et al., Reference Castellví, Miranda-Mendizábal, Parés-Badell, Almenara, Alonso, Blasco and Alonso2017), are possible near-term triggers. Other candidate WS include notable increases in established risk factors for adolescent SAs, such as depression (Goldston et al., Reference Goldston, Daniel, Reboussin, Reboussin, Kelley and Frazier1998); hopelessness (Goldston et al., Reference Goldston, Daniel, Reboussin, Reboussin, Frazier and Harris2001); anxiety (Brent et al., Reference Brent, Perper, Moritz, Allman, Friend, Roth and Baugher1993a); agitation and behavioral dyscontrol (Nock et al., Reference Nock, Green, Hwang, McLaughlin, Sampson, Zaslavsky and Kessler2013); sleep disturbance (Liu et al., Reference Liu, Tu, Lai, Lee, Tsai, Chen and Chiu2019); and aggressive behavior, alcohol, and substance use (Brent et al., Reference Brent, Perper, Moritz, Allman, Friend, Roth and Baugher1993a; Pena, Matthieu, Zayas, Masyn, & Caine, Reference Pena, Matthieu, Zayas, Masyn and Caine2012).
Near-term suicide risk factors
The current study was designed to fill the gap in our understanding of adolescent WS for SAs due to the paucity of research examining near-term predictors of suicide attempts in adolescents and the absence of empirical WS data from parents. One study conducted more than 25 years ago compared adolescents who did and did not make SAs during periods of acute suicidality. The adolescents who attempted suicide reported greater hopelessness, suicidal ideation, and social isolation (Negron, Piacentini, Graae, Davies, & Shaffer, Reference Negron, Piacentini, Graae, Davies and Shaffer1997). Other studies of proximal risk in adolescents have focused on suicidal ideation (Czyz, Horwitz, Arango, & King, Reference Czyz, Horwitz, Arango and King2019) or non-suicidal self-injury (NSSI) (Nock, Prinstein, & Sterba, Reference Nock, Prinstein and Sterba2009) rather than SAs. In contrast, studies with adults have identified a range of WS, including alcohol/substance use (Bagge & Borges, Reference Bagge and Borges2017), negative life events [especially interpersonal (Bagge et al., Reference Bagge, Littlefield, Wiegand, Hawkins, Trim, Schumacher and Conner2022)], insomnia (Goldstein, Bridge, & Brent, Reference Goldstein, Bridge and Brent2008), and changes in affect and cognitions (Bagge et al., Reference Bagge, Littlefield, Wiegand, Hawkins, Trim, Schumacher and Conner2022)).
Purpose of the present study
Our study aim was to answer the question, Why did an adolescent attempt suicide today, compared to a previous day when they were also at high risk but did not attempt suicide? We obtained adolescent and parent input, and we incorporated a within-subjects, case-crossover design (Maclure, Reference Maclure1991), which can provide information about the unfolding of risk immediately preceding an event. Initially developed to detect triggers for myocardial infarction (Maclure, Reference Maclure1991), this design compares factors on the day of the target event (‘case period’) to the same factors on another day (‘control period’) for each individual, controlling for stable risk factors.
Method
Participants
Adolescents, ages 13 to 18 years, were enrolled in this study by telephone between January 2018 and February 2019, following their final assessment for the Emergency Department Screening for Teens at Risk for Suicide (ED-STARS) Study (King et al., Reference King, Grupp-Phelan, Brent, Dean, Webb, Bridge and Casper2019). ED-STARS recruited youth from 14 geographically diverse U.S. EDs affiliated with the Pediatric Emergency Care Applied Research Network (PECARN) (King et al., Reference King, Brent, Grupp-Phelan, Casper, Dean, Chernick and Gibbons2021). Inclusion criteria for the current study were one or more correlates of suicide risk, based on reasons for ED visit and adolescents’ responses to baseline and 3-month follow-up evaluations: (1) psychiatric chief complaint; (2) lifetime history of suicide attempt; (3) endorsement of any Ask Suicide-Screening Questions item (ASQ; (Horowitz et al., Reference Horowitz, Bridge, Teach, Ballard, Klima, Rosenstein and Pao2012); (4) history of NSSI; (5) sexual/gender minority; (6) depression score > 2 on PHQ-4 (Kroenke, Spitzer, Williams, & Lowe, Reference Kroenke, Spitzer, Williams and Lowe2009); (7) alcohol abuse score > 1 on AUDIT-C (Chung, Colby, Barnett, & Monti, Reference Chung, Colby, Barnett and Monti2002; Saunders, Aasland, Babor, De la Fuente, & Grant, Reference Saunders, Aasland, Babor, De la Fuente and Grant1993); (8) self-reported illicit drug use. Criteria 1, 3, and 5 were only collected at baseline.
Among the 1344 ED-STARS study participants who met the current study's inclusion criteria and were approached for participation, parent/guardian consent and adolescent assent were obtained for 1094 participants (81.4%). The study's analytic sample is comprised of the subset of 105 adolescents who reported a SA on a text message survey (sent bi-weekly across an 18-month period) and confirmed they had engaged in intentional self-harm when contacted for an interview.
The analytic sample of 105 adolescents had a mean age of 15.0 years (s.d. = 1.46). Parent- or adolescent- reported sex at birth, race and ethnicity were distributed as follows: female (81.9%, n = 86)), male (18.1%, n = 19), White (66.7%, n = 70), Black (19.0%, n = 20), Asian/Pacific Islander (2.9%, n = 3), multi-racial (4.8%, n = 5), Unknown Race (6.7%, n = 7); Hispanic/Latino (20.0%, n = 21), Not Hispanic/Latino (77.1%, n = 81); Unknown Ethnicity (2.9%, n = 3). Fourteen adolescents (13.3%) self-reported a gender identity of nonbinary, gender queer, or transgender. Adolescents’ parents reported educational levels for up to two parents; the higher of these levels was distributed as follows: high school graduate or less (n = 19, 18.1%), some college/technical training (n = 24, 22.9%), college graduate/professional (n = 62, 59.0%). According to parent reports, 37.1% of families (n = 39) received public assistance.
Measures
Baseline descriptive data
Baseline demographic (age, birth sex, gender identity, race/ethnicity, parent education, family public assistance) and clinical data (e.g. lifetime history of suicide attempt) were collected as part of ED-STARS at baseline and follow-up, respectively (11). Clinical measures included the Columbia-Suicide Severity Rating Scale (C-SSRS) (Posner et al., Reference Posner, Brown, Stanley, Brent, Yershova, Oquendo and Shen2011), the Alcohol Use Disorders Identification Test- Consumption (AUDIT) (Chung et al., Reference Chung, Colby, Barnett and Monti2002; Saunders et al., Reference Saunders, Aasland, Babor, De la Fuente and Grant1993), the Patient Health Questionnaire-4- (PHQ-4) (Kroenke et al., Reference Kroenke, Spitzer, Williams and Lowe2009), and a NSSI question from the Youth Risk Behavior Survey (Centers for Disease Control and Prevention (Cartographer), 2012).
Text message surveys
The text message surveys consisted of 8–10 questions, presented as a sequence of messages, assessing adolescents’ functioning ‘over the past two weeks.’ The primary purpose of this survey was to ascertain if adolescents had made a SA and were eligible to be contacted for a Warning Signs for Suicide Attempt (WSSA) interview. The SA item was as follows: ‘In the past 2 weeks, have you made a suicide attempt or tried to harm yourself because you were at least partly trying to end your life?’ The response options were 1 = Yes and 2 = No. If the adolescent's response was a ‘1,’ they received three additional texts, as follows: (1) a precautionary note with phone numbers for crisis services, if needed; (2) an open-ended question asking the number of days since their most recent attempt; and (3) a question regarding the time of day of their attempt (four response options; e.g. 6:00 pm–11:59 pm). The latter two items were obtained to guide follow-up interviews, described below.
The text message survey also assessed emotional distress with one item, adapted from the PANAS (Laurent et al., Reference Laurent, Catanzaro, Joiner, Rudolph, Potter, Lambert and Gathright1999), ‘In the past 2 weeks, how distressed or upset have you felt?’ In addition, it assessed hopelessness (1 item, Mood and Feelings Questionnaire, (Angold et al., Reference Angold, Costello, Messer, Pickles, Winder and Silver1995); sleep disturbance (1 item, PHQ-9 (Kroenke, Spitzer, & Williams, Reference Kroenke, Spitzer and Williams2001); alcohol/drug use (2 items), and family and friend connectedness (2 items, taken from the ED-STARS survey). The alcohol and drug use items were as follows: ‘In the past 2 weeks, on how many days have you used an illegal drug or used a prescription medication for nonmedical reasons?’ ‘In the past 2 weeks, on how many days did you have at least one drink of alcohol?’ For each of these items, the adolescent was instructed to enter ‘0’ if they did not do this in the past 2 weeks (14 days). The connectedness items were as follows: ‘How much do people in your family understand you?’ ‘I have friends I'm really close to and trust completely.’ The response options for these two questions were on a 5 point Likert scale ranging from ‘not at all’ to ‘very much’ and ‘very true,’ respectively.
Warning signs for suicide attempt-teen and -parent interviews (WSSA-T, WSSA-P)
The WSSA-T and WSSA-P interviews are modified versions of the WSSA-Adult interview (Bagge, Conner, & Littlefield, Reference Bagge, Conner and Littlefield2019), whereby participants are asked to think back to two 24-hour time periods to examine behaviors/events, affects, and cognitions during 24-hour case and control period. We modified items to include promising candidate WS for adolescent SAs, including signs and characteristics that have been put forth by expert consensus panels and empirically identified risk factors for suicide attempts, as reviewed in the introduction. We developed a parent interview to identify WS that may be observable by parents.
The WSSA-T interview is structured and computer-assisted; however, interviewers are able to respond to questions, query further if a response is unclear, and encourage respondents to take their time. The interviewer begins by identifying the exact date and time of the SA with the adolescent, after which the 24 h period prior to SA is defined. The 49-item interview assesses candidate WS, including behaviors and events (e.g. negative interpersonal events, alcohol use), cognitions (e.g. hopelessness), and feelings/emotions. Response options are ‘yes’ and ‘no’ for behaviors/events and on a 6-point scale ranging from ‘not at all’ to ‘extremely’ for cognitions and feelings/emotions. Following assessment of the 24 h prior to the SA, questions are repeated for a control window during which a SA did not occur (typically the 24 h before the case period). Three adolescents in the study (2.86%) had alternative control days due to a suicide attempt the day prior to their index suicide attempt.
We took several measures to minimize the bias associated with the retrospective assessment strategy inherent in our interview approach. First, because individuals struggle to report change in attitudes, emotions, and cognitive states (Nisbett & Wilson, Reference Nisbett and Wilson1977), we simply asked adolescents to report their experiences/states during each 24-hours over the 48-hour period rather than the change in experiences/states between the case- and control-periods. In addition, we used a relatively short recall period (approximately 2–3 weeks) and a modified timeline follow-back method to facilitate improved recall (Bagge et al., Reference Bagge, Conner and Littlefield2019). This method involved (1) beginning with an event history calendar for the 24-hour period (e.g. events with peers or family members, school attendance), (2) allowing the participant sufficient time to reflect on relevant experiences, and (3) allowing for a correction of earlier responses during the interview. Finally, our use of a case-crossover design controls for individual biases that are consistent across the two reporting periods. Although research on adolescent retrospective reports of emotions is limited, the Birk, Olino, Klein, and Seeley (Reference Birk, Olino, Klein and Seeley2020) study, which used longitudinal data to compare adolescents’ concurrently and retrospectively assessed reports of depressive symptoms during the same time period, supports the validity of adolescents’ retrospective reports of depressive symptoms. Similarly, Czyz et al.'s study, which incorporated both daily and 1-month follow-up assessments of suicidal thoughts, documented the validity of adolescent's 1-month retrospective reports of suicidal ideation, although it is important to note that adolescents’ daily reports captured more SI across this period (Czyz, King, & Nahum-Shani, Reference Czyz, King and Nahum-Shani2018).
To determine the reliability of WSSA interview data, 15 parents and 24 teens with interviews were randomly selected. A trained study member reviewed the audiotapes of the original interviews and provided independent reliability ratings. Kappa coefficients were calculated for binary variables; intra-class correlations (ICCs) were calculated for continuous variables. Kappa values for parent case days ranged from 0.81 to 1.00, with a modal response of 1.00; kappa values for parent control days all had a score of 1.00. Kappas for teens ranged from 0.65 to 1.00 on case days, with 1.00 as the modal response); and from 0.64 to 1.00 on control days, with 1.00 as the modal response. ICC values for the continuous teen variables ranged from 0.77–1.00 (across case and control days; M = 0.92).
To determine the fidelity of interview administration, two study two members (PG, CB) coded all interviews, using five ordinal items (i.e. ‘How well did the interviewer facilitate recall of time periods?’; ‘How well did the interviewer obtain the information needed to score all questions?’; ‘How was the interviewer's overall clarity of communication with participant?’). The response scale for each item was: 1 = very poor, 2 = poor, 3 = fair, 4 = good, 5 = excellent. For both parent and teen interviews, the average fidelity ratings fell in the fair-excellent range (M range = 3.80–4.87, M range = 3.96–4.75, respectively), with modal responses falling in the good – excellent range (i.e. 4.00–5.00). Detailed information about fidelity ratings for each item are provided in the online supplement.
Procedures
Text messages surveys were sent bi-weekly to adolescents across an 18-month period. When an adolescent indicated they had made a SA, a customized text messaging system automatically sent a message to the study team indicating that a ‘case’ had been identified and that the parent and adolescent should be contacted to schedule WSSA interviews. Among the 1094 enrolled adolescents, 1033 received at least one text message survey; 61 adolescents did not receive a survey due to out of operation phone numbers or network errors. Most enrolled adolescents (n = 931/1033, 90.1%) completed at least one survey; 38 adolescents sent ‘Stop’ texts (3.7%); 418/1033 youth (40.5%) completed 65% of the surveys; 316/1033 youth (30.6%) completed 80% of the surveys. Adolescents received a $4 online gift certificate for each completed survey and an additional $25 or $35 if they completed 65% or 80% of surveys, respectively.
Trained interviewers administered the computer-assisted WSSA-T and WSSA-P by telephone within approximately two weeks of the adolescent's survey response indicating a SA. The mean time between text message survey responses and WSSA interviews was 11.64 days (s.d. = 4.83) for adolescents (data for 104 adolescents) and 9.76 days (s.d. = 4.58) for parents (data for 54 of 55). The range of time was 0 days (same day as adolescent responded to text message survey indicating SA) to 20 days. The duration of adolescent and parent interviews was approximately 40–50 and 20–30 min, respectively.
WSSA-T interviews were conducted for 105 of 164 study-eligible adolescents (64.0%). Although 274 adolescents reported a SA on at least one text message survey during the 18-month follow-up, when contacted for interview, 95 adolescents (34.2%) reported they had not made a SA when contacted for a WSSA-T interview and 15 adolescents (5.4%) reported a SA outside the 2-week survey window. WSSA interviews were not conducted with these adolescents.
We used only one suicide attempt interview for each adolescent. For adolescents who reported more than one attempt during the 18-month follow-up period, we used the first complete interview, which was defined as all questions answered for case and control periods. Among the 105 participants in the analytic sample, 60 (57.1%) reported one suicide attempt, 36 (34.3%) reported two suicide attempts, 6 (5.7%) reported three suicide attempts, and 3 (2.9%) reported four to six suicide attempts on text message surveys.
Adolescents who participated in WSSA interviews received a $20 online gift certificate and were connected with Boys Town National Hotline for resources and any needed crisis services immediately following the interview (Busby et al., Reference Busby, King, Brent, Grupp-Phelan, Gould, Page and PECARN Network2020).
Retention analyses compared the 105 adolescents with WSSA interviews to the 59 study-eligible adolescents lost to WSSA interview and to the 110 study-ineligible adolescents (Table 1). There were no significant differences in demographic characteristics, lifetime history of SAs, or C-SSRS Suicide Ideation Severity Scores between study-eligible adolescents who did and did not participate in interviews.
Table 1. Demographic and clinical characteristics of case-crossover sample of adolescent suicide attempters, adolescents lost to interview, and adolescents without confirmed suicide attempts
1Chi-squared test; 2Fisher's exact test; 3Wilcoxon rank-sum test.
Each p-value column shows tests of differences with the case-crossover population. ‘MISSING’ observations excluded from tests; variables with an unknown option incorporate missing data and are included.
WSSA-P interviews were conducted with the parents/guardians of 72 of the 105 study adolescents (69.0%). Missing parent interviews were due to: no contact with/observation of adolescent during one or both 24-hour periods (n = 17, 16.2%); lack of time/interest or could not schedule within allocated time (n = 8, 7.6%); lost contact with parent (n = 16, 15.2%); parent medically unable to participate (n = 1, 1.0%).
This study was approved by the Institutional Review Board. Participating adolescents provided informed assent and their parents/guardians provided informed consent.
Data analyses
We implemented a case-crossover, within-subject analysis using conditional logistic regression with SA as the outcome. The model was conditioned on subject, with each subject having an observation corresponding to a SA (case) and a control window (crossover). The predictor variables (candidate warning signs [WS]) were divided into the domains of adolescent-reported cognitions, feelings/emotions, and behaviors/events; in addition to parent-reported behaviors/events. We computed domain-specific multivariable models for several reasons. First, these domains reflect how warning signs have been conceptualized and disseminated in lists that have been made available to the public, which organize warning signs by categories. As an example, the NIMH lists warning signs under three categories: ‘talking about’ thoughts, such as being a burden to others, ‘feeling,’ and ‘changing behavior’ https://www.nimh.nih.gov/health/publications/warning-signs-of-suicide. Second, there has been very limited research on warning signs for adolescents and this domain-specific approach has been used in a study with adults. As we adapted the WSSA-T and WSSA-P interviews from an adult interview, we were interested in maintaining consistency with this approach so that we could build our ‘database’ on warning signs across ages. Finally, our statistical power and EPV ratio are more favorable with the domain-specific approach.
Within each domain, WS variables were identified using a p-value <0.05 from the univariable regression. In addition to p-values, Benjamini–Hochberg false discovery rate ‘q-values’ were calculated. For each domain, a multivariable model was fit using the identified WS variables as predictors. For these analyses, we excluded (or collapsed) predictors if cells had <4 subjects or if the odds ratio's confidence interval couldn't be estimated (or was uninterpretable) due to small sample size.
Results
At study baseline, 62 of the 105 study adolescents (59.0%) reported a lifetime history of multiple SAs, 18 (17.1%) reported a lifetime history of one SA, and 21 (20.0%) reported 5 or more incidents of NSSI in the previous 3 months. The mean baseline C-SSRS Suicidal Ideation Severity score was 1.7 (s.d. = 1.8), which falls between scores for the items, ‘…wished you were dead or wished you could go to sleep and not wake up?’ and ‘…ever in your life had any thoughts of killing yourself.’ Mean PHQ-4 and AUDIT-C total scores for the past three months were 5.9 (s.d. = 3.27) and 0.4 (s.d. = 1.16), respectively. The mean PHQ-4 score falls between the mild and moderate range [mild = 3–5; moderate = 6–8; (Löwe et al., Reference Löwe, Wahl, Rose, Spitzer, Glaesmer, Wingenfeld and Brähler2010)]; the mean AUDIT-C score is below the threshold for alcohol problem use, for which the cut-off value is a score >3 (Liskola et al., Reference Liskola, Haravuori, Lindberg, Kiviruusu, Niemelä, Karlsson and Marttunen2021). Baseline clinical data are in Table 1.
Conditional logistic regression analyses
Adolescent behaviors/events
As shown in Table 2, seven adolescent-reported behaviors and events significantly differentiated case and control periods in univariable analyses, meeting study criteria for designation as WS. Three of these remained significant in the multivariable model: serious conflict with parent, withdrawal, and suicidal communication.
Table 2. Adolescent-reported behavioral and event predictors of suicide attempt
Note. Behaviors with a cell count of less than 4, or for which the odds ratio's confidence interval couldn't be estimated due to small sample size, were removed from tables and analyses.
aN = 105 subjects.
bOdds ratios, confidence intervals, and p-values are from a conditional logistic regression, conditioned on subject; Model predicts 24-hr. suicide attempt period (v. control period).
cUnivariable-significant (p-value <0.05) warning signs used as predictors in model.
dFalse discovery rate q-values use Benjamini–Hochberg method.
eExamples include ‘falling out or serious disruptive argument’ with another student, teacher, coach, or other.
fExamples include encounter with police, traffic accident, bad grade, caught cheating at school.
For parent-reported information, four of the six adolescent behaviors and events considered in univariable analyses, withdrawal, problem with sleep, risky behavior, and suicidal communications, significantly differentiated case and control periods, meeting criteria for designation as WS. As shown in Table 3, withdrawal and suicidal communications remained significant in the multivariable model.
Table 3. Parent-reported adolescent behavior and event predictors (warning signs) of suicide attempt
Note. Parent reported behaviors with a cell count <4, or for which the odds ratio (OR) confidence interval (CI) couldn't be estimated due to small sample size, were removed from tables and analyses.
aN = 72 adolescents with eligible parent interview, multivariable model used 64 subjects with complete-case data.
bOdds ratios, CIs, and p-values are from a conditional logistic regression, conditioned on subject; model predicts 24-hr. suicide attempt period (v. control period).
cUnivariable-significant (p-value <0.05) warning signs used as predictors in model.
dFalse discovery rate q-values use Benjamini–Hochberg method.
Adolescent cognitions
As shown in Table 4, all adolescent-reported cognitions considered were statistically significant as WS for suicide attempts in univariable analyses. In a multivariable model that included thoughts of killing self as a predictor, only thoughts of killing oneself remained significant. In a second multivariable model that excluded thoughts of killing self (a possible endpoint of other cognitive WS), suicidal rumination, perceived burdensomeness, and angry or hostile thoughts significantly differentiated case and control periods.
Table 4. Adolescent-reported cognitions as predictors (warning signs) of suicide attempt
Note. Youth reported behaviors with a cell count <4, or for which the odds ratio (OR) confidence interval (CI) couldn't be estimated due to small sample size, were removed from tables and analyses.
aN = 105 subjects.
bOdds ratios, confidence intervals, and p-values are from a conditional logistic regression, conditioned on subject.
cUnivariable-significant (p-value <0.05) warning signs used as predictors in model.
dFalse discovery rate q-values use Benjamini–Hochberg method.
eMultivariable model did not include thoughts of killing self.
Adolescent feelings/emotions
As shown in Table 5, 14 of 16 feelings/emotions were significant WS in univariable analyses. The final multi-variable model included four emotions that each significantly differentiated the 24-hour SA and control periods: rush of feelings, emotional pain, rage toward others (less), and self-hate.
Table 5. Adolescent-reported feelings/emotions as predictors (warning signs) of suicide attempt
Note. Behaviors with a cell count <4, or for which the OR's or CIs couldn't be estimated due to small n, were removed from analyses.
aN = 105 subjects.
bOdds ratios, confidence intervals, and p-values are from a conditional logistic regression, conditioned on subject; Model predicts 24-hr. suicide attempt period (v. control period).
cUnivariable-significant (p-value <0.05) warning signs used predictors in model.
dFalse discovery rate q-values use Benjamini–Hochberg method.
eItems were coded in the positive direction, such that a higher score represented more positive feelings.
Discussion
Clinical providers who care for adolescents at risk for suicide, as well as the parents and other adults in their lives, are faced with the often repeated challenge of recognizing whether the adolescent is at proximal risk for a suicide attempt. We have a dearth of empirical knowledge to guide us in recognizing WS for possible acute high risk in adolescents, despite the importance of recognition to seeking timely crisis services (e.g. parent recognition) and clinical disposition. In this study, we used a case-crossover research design to compare adolescents’ and parents’ reports of adolescents’ experiences during the 24 h prior to their SAs and during a 24-hour comparison period. Adolescent reports identified key differences in their behaviors and events, cognitions, and affect during the 24 h prior to their SAs, and the inclusion of parents as informants in this study enabled us to identify behavioral WS that parents and other gatekeepers (e.g. teachers) could potentially learn to recognize.
We chose to define a warning sign as a candidate variable that is significant in the case-crossover analysis at the univariable level. Our study aim was to identify warning signs for adolescent suicide attempts that could be shared with those who interface with youth, and a candidate warning sign that is significant when considered on its own, at the univariable level, is meaningful within this context. Although multivariable models also provide information regarding the significance of each candidate warning sign when considered in the context of each other, not all warning signs are possible to observe or will be observed by parents, other caregivers and gatekeepers, and the clinicians who interface with youth.
Parent WSSA interviews identified withdrawal, sleep problems, risky behavior, and suicidal communications as WS, with withdrawal and suicidal communications remaining significant as WS in a multivariable model. Whereas suicidal communications may serve as direct WS to clinicians and caregivers, adolescents may not communicate their suicidal thoughts and intentions to others. The behavioral WS identified by parents in this study, such as withdrawal from social and other activities, have the potential to be observed by others, such as parents, enabling a preventive response. Previous longitudinal research has associated social disconnectedness and sleep problems with suicide risk (King et al., Reference King, Grupp-Phelan, Brent, Dean, Webb, Bridge and Casper2019), and this study suggests that these problems may be particularly exacerbated during the 24 h prior to a SA.
Our results also emphasize the importance of negative interpersonal experiences as 24-hour WS for suicide risk. Adolescent interview data independently identified serious conflict with parent and withdrawal as WS, providing converging parent and adolescent reports regarding the importance of withdrawal. Although previous research examining precipitants of suicidal behavior is limited by lack of specific delineations of how proximally these precipitants occurred relative to the suicidal behavior, our findings are consistent with those from the limited previous research suggesting that social isolation (Negron et al., Reference Negron, Piacentini, Graae, Davies and Shaffer1997) as well as social exclusion, and parent-child relationship problems (Park et al., Reference Park, Kim, Kim, Bae, Shin, Yoo and Cho2015) may be near-term risk factors. Youth facing a range of predisposing and more static risk characteristics (e.g. history of depression, chronic stress) lack the emotional resources necessary to cope with interpersonal conflict, and the associated distress may prompt the transition from suicidal thoughts to behaviors.
The WS identified from adolescents’ reports are wide ranging and consistent with those that emerged from a national consensus panel ((http://www.youthsuicidewarningsigns.org/) and one prior case control study comparing attempters to non-attempters (Negron et al., Reference Negron, Piacentini, Graae, Davies and Shaffer1997). They extend upon these findings by indicating that the ‘worrisome behaviors’ highlighted by the national consensus panel include withdrawal from social and other activities. They also provide more specific information regarding cognitive and affective WS (suicidal rumination, perceived burdensomeness, angry/hostile). Perhaps not surprisingly, there are some striking consistencies across these WS and previously identified adult WS for suicide attempts. These include negative interpersonal life events (such as serious conflict with parents for adolescents), hostility, perceived burdensomeness and suicidal communications (Bagge et al., Reference Bagge, Littlefield, Wiegand, Hawkins, Trim, Schumacher and Conner2022), suggesting the potential for a core group of warning signs across the lifespan. Alcohol and drug use did not, however, emerge as a WS for adolescents. This may be partially due to the overall reduction in adolescent alcohol use in recent years (Vashishtha et al., Reference Vashishtha, Pennay, Dietze, Marzan, Room and Livingston2021).
Further research is recommended that identifies differing patterns or profiles of warning signs in identifiable subgroups of adolescents. Such profiles may be characterized by different combinations of warning signs, different baseline levels of these warning signs (e.g. expressed hopelessness, sleep disturbance) as well as by their chronicity and time varying course prior to a suicide attempt. Although we have documented significant increases in certain experiences (e.g. withdrawal) during the 24-hour period prior to the attempt, we realize that, depending upon the adolescents’ baseline levels of these behaviors, such changes or ‘warning signs’ may or may not be easily recognizable by parents, other gatekeepers, and clinicians. For instance, a youth who has chronic sleep disturbance that is exacerbated prior to a suicide attempt may be more difficult to identify than a youth who moves from no sleep disturbance to significant sleep disturbance.
Regarding the representativeness of this study's sample of adolescents, enrolled in EDs, it is notable that approximately 19% of U.S. adolescents visit the ED in a one-year period (Burstein, Agostino, & Greenfield, Reference Burstein, Agostino and Greenfield2019). Furthermore, in a recent study of pediatric patients (10–18 years old) evaluated for suicidal ideation or a suicide attempt in EDs, most patients (56.1%) had an encounter with the healthcare system in the year prior to their ED visit (Sarin et al., Reference Sarin, Conners, Sullivant, Giovanni, Sherman, Zanaboni and Randell2021) and, within this group, 24.7% were seen in the ED or urgent care setting. Similarly, in a study of health system contacts prior to death in a geographically diverse sample of U.S. suicide decedents, 77.4% of individuals ages 19 years or younger had made a healthcare visit and 49.1% had visited an ED in the 52 weeks prior to their suicide (Ahmedani et al., Reference Ahmedani, Simon, Stewart, Beck, Waitzfelder, Rossom and Whiteside2014).
Our study sample of adolescents differs in some ways from the general population of adolescents. First, although nearly 20% of U.S. adolescents visit the ED each year, those with higher utilization of ED services are more likely to be publicly insured and/or characterized by higher medical complexity (Cohen, Berry, Sanders, Schor, & Wise, Reference Cohen, Berry, Sanders, Schor and Wise2018; Weiss, D'Angelo, & Rucker, Reference Weiss, D'Angelo and Rucker2014). In addition to presence in the ED setting, the current study required one or more indicators of suicide risk for inclusion. This criterion was met by approximately two-thirds of adolescents enrolled in the parent ED-STARS study (n = 2724/4031; 67.6%), which did not require an indicator of suicide risk for inclusion. However, all adolescents in the ED-STARS study who had made a suicide attempt by follow-up had at least one indicator of suicide risk. As such, the current study sample is most representative of the approximately 20% of adolescents who visit EDs each year and have at least somewhat elevated suicide risk.
Study limitations
This study had substantial strengths, including its relatively large and diverse sample of adolescents, its prospective design, its inclusion of parents and adolescents as WS informants, and its use of a case-crossover design. Nevertheless, it is important to interpret findings within the context of study limitations. Our study sample was geographically diverse but not nationally representative. Moreover, we enrolled adolescents from pediatric EDs in academic health centers, which are not representative of all EDs in the community, and we were only able to retain 64% of eligible adolescents for WSSA interviews. These adolescents did not vary from those lost to follow-up on demographic or suicide-related variables (history of suicide attempts, C-SSRS severity scores); however, study findings cannot be generalized to the population of adolescents who report SAs in the 18 months following ED visits.
Our retrospective examination of WS could impact the accuracy of data, as we relied on post-attempt recollections of the 48 h prior to the SA. We used bi-weekly text message surveys that could feasibly be implemented across a relatively long, 18-month period to capture sufficient SA outcomes. Despite the steps we took to minimize the measurement bias of retrospective reports (described in Methods), measurement bias may still occur for several reasons. These include recall biases related to memory limitations, which could be partially associated with cognitive functioning at the time of the event; the intentional non-disclosure of details; respondent personality; and respondent mood state, coping strategies, or situation at time of assessment (e.g. Levine & Safer, Reference Levine and Safer2002; Naicker, Norris, & Richter, Reference Naicker, Norris and Richter2021). Some of these reasons (e.g. intentional non-disclosure, situation at time of assessment) are not unique to retrospective recall and may also impact real-time, ecological momentary assessments (EMAs). Moreover, although EMA has been used to examine the contexts and frequencies of suicidal thoughts and behaviors among adolescents (Nock et al., Reference Nock, Prinstein and Sterba2009), in addition to day-to-day changes in suicidal ideation (Coppersmith, Kleiman, Glenn, Millner, & Nock, Reference Coppersmith, Kleiman, Glenn, Millner and Nock2019; Czyz et al., Reference Czyz, Horwitz, Arango and King2019), it is only feasibly implemented across short time periods, which makes it extremely difficult to capture a sufficient number of suicide attempt outcomes. Moreover, repeated EMA self-monitoring may have a self-regulatory impact and adolescent's reports of precursors to SAs (e.g. suicidal intent) may necessitate intervention to manage indicators of acute suicide risk in youth – a vulnerable study population. Our strategy did not disrupt the naturalistic unfolding of precursors to a SA and captured a relatively large number of SAs due to the extended study period.
Conclusions
Using a within-participant case-crossover research design, we identified adolescent emotions, cognitions and behaviors/experiences that differentiated the 24-hour period prior to adolescent SAs from a control period. Adolescents and parents both identified suicidal communications and withdrawal from social and other activities as key warning signs. Adolescents also identified rush of feelings, emotional pain, self-hate, rage toward others (less), suicidal ruminations, perceived burdensomeness, angry/hostile, and serious conflict with parent, among others. This WS information warrants widespread dissemination as it offers the potential to recognize and intervene at a time of acute risk. It also has the potential to inform new strategies for the recognition of warning signs and targeted interventions at times of heightened near-term risk.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S0033291723003112.
Acknowledgements
The authors are grateful for the contributions of the Pediatric Emergency Care Applied Research Network (PECARN) Site PIs and research coordinators who contributed to the ED-STARS study, which provided baseline data for the current study. They also are grateful for the contributions of the Survey Research Center team at the University of Michigan for collecting warning signs interview data, the study research assistants, and the adolescents and caregivers who participated.
Funding statement
This work was supported by the National Institute of Mental Health (grant number: R01MH113582).
Competing interest
Dr King receives support from NIMH, receives royalties from Guilford Press, serves as a scientific advisor (with stock) for Oui Therapeutics, LLC and Vytyl Health Management, Inc., and is a member of the Scientific Council of the American Foundation for Suicide Prevention. Her intellectual property, developed with NIH support and with no current financial interest, includes the Computerized Adaptive Screen for Suicidal Youth (CASSY), a suicide risk screen algorithm, and the Check-In intervention platform. Dr Brent receives research support from NIMH, AFSP, the Once Upon a Time Foundation, and the Beckwith Institute, receives royalties from Guilford Press, from the electronic self-rated version of the C-SSRS from eRT, Inc., and from duties as an UptoDate Psychiatry Section Editor, receives consulting fees from Healthwise, receives Honoraria from the Klingenstein Third Generation Foundation for scientific board membership and grant reviews, and is a scientific board member for AFSP. Intellectual Property, currently with no financial interest: Funding from NIMH supported the development of intellectual property for BRITE, the As Safe as Possible intervention, the CASSY, and the Screening Wizard screening tool. Dr Ahamed receives support from NIMH. He is a shareholder of Ubitrix, which helped to build the software system. He did not receive any remuneration from Ubitrix for this project. The other authors declare none.
Ethical standards
‘The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008.’
Open access
