Hostname: page-component-7479d7b7d-c9gpj Total loading time: 0 Render date: 2024-07-14T19:52:31.392Z Has data issue: false hasContentIssue false

Predictive validity of psychosis risk models when applied to adolescent psychiatric patients

Published online by Cambridge University Press:  24 May 2021

Maija Lindgren*
Affiliation:
Mental Health, Finnish Institute for Health and Welfare, Helsinki, Finland
Heidi Kuvaja
Affiliation:
Department of Psychology and Logopedics, Faculty of Medicine, Helsinki University, Helsinki, Finland
Markus Jokela
Affiliation:
Department of Psychology and Logopedics, Faculty of Medicine, Helsinki University, Helsinki, Finland
Sebastian Therman
Affiliation:
Mental Health, Finnish Institute for Health and Welfare, Helsinki, Finland
*
Author for correspondence: Maija Lindgren, E-mail: maija.lindgren@thl.fi

Abstract

Background

Several multivariate algorithms have been developed for predicting psychosis, as attempts to obtain better prognosis prediction than with current clinical high-risk (CHR) criteria. The models have typically been based on samples from specialized clinics. We evaluated the generalizability of 19 prediction models to clinical practice in an unselected adolescent psychiatric sample.

Methods

In total, 153 adolescent psychiatric patients in the Helsinki Prodromal Study underwent an extensive baseline assessment including the SIPS interview and a neurocognitive battery, with 50 participants (33%) fulfilling CHR criteria. The adolescents were followed up for 7 years using comprehensive national registers. Assessed outcomes were (1) any psychotic disorder diagnosis (n = 18, 12%) and (2) first psychiatric hospitalization (n = 25, 16%) as an index of overall deterioration of functioning.

Results

Most models improved the overall prediction accuracy over standard CHR criteria (area under the curve estimates ranging between 0.51 and 0.82), although the accuracy was worse than that in the samples used to develop the models, also when applied only to the CHR subsample. The best models for transition to psychosis included the severity of positive symptoms, especially delusions, and negative symptoms. Exploratory models revealed baseline negative symptoms, low functioning, delusions, and sleep problems in combination to be the best predictor of psychiatric hospitalization in the upcoming years.

Conclusions

Including the severity levels of both positive and negative symptomatology proved beneficial in predicting psychosis. Despite these advances, the applicability of extended psychosis-risk models to general psychiatric practice appears limited.

Type
Original Article
Copyright
Copyright © The Author(s), 2021. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Addington, J., & Heinssen, R. (2012). Prediction and prevention of psychosis in youth at clinical high risk. Annual Review of Clinical Psychology, 8, 269289. https://doi.org/10.1146/annurev-clinpsy-032511-143146CrossRefGoogle ScholarPubMed
Auther, A. M., McLaughlin, D., Carrión, R. E., Nagachandran, P., Correll, C. U., & Cornblatt, B. A. (2012). Prospective study of cannabis use in adolescents at clinical high risk for psychosis: Impact on conversion to psychosis and functional outcome. Psychological Medicine, 42(12), 24852497. https://doi.org/10.1017/S0033291712000803CrossRefGoogle ScholarPubMed
Bora, E., Lin, A., Wood, S. J., Yung, A. R., McGorry, P. D., & Pantelis, C. (2014). Cognitive deficits in youth with familial and clinical high risk to psychosis: A systematic review and meta-analysis. Acta Psychiatrica Scandinavica, 130(1), 115. https://doi.org/10.1111/acps.12261 [doi]CrossRefGoogle ScholarPubMed
Brown, G. S., & White, K. G. (2005). The optimal correction for estimating extreme discriminability. Behavior Research Methods, 37(3), 436449. https://doi.org/10.3758/BF03192712CrossRefGoogle ScholarPubMed
Buchy, L., Perkins, D., Woods, S. W., Liu, L., & Addington, J. (2014). Impact of substance use on conversion to psychosis in youth at clinical high risk of psychosis. Schizophrenia Research, 156(2–3), 277280. https://doi.org/10.1016/j.schres.2014.04.021CrossRefGoogle ScholarPubMed
Cannon, T. D., Cadenhead, K., Cornblatt, B., Woods, S. W., Addington, J., Walker, E., … Heinssen, R. (2008). Prediction of psychosis in youth at high clinical risk: A multisite longitudinal study in North America. Archives of General Psychiatry, 65(1), 2837. https://doi.org/65/1/28[pii]10.1001/archgenpsychiatry.2007.3CrossRefGoogle ScholarPubMed
Carrión, R. E., McLaughlin, D., Goldberg, T. E., Auther, A. M., Olsen, R. H., Olvet, D. M., … Cornblatt, B. A. (2013). Prediction of functional outcome in individuals at clinical high risk for psychosis. JAMA Psychiatry, 70(11), 11331142.CrossRefGoogle ScholarPubMed
Cornblatt, B. A., Auther, A. M., Niendam, T., Smith, C. W., Zinberg, J., Bearden, C. E., & Cannon, T. D. (2007). Preliminary findings for two new measures of social and role functioning in the prodromal phase of schizophrenia. Schizophrenia Bulletin, 33(3), 688702. https://doi.org/sbm029[pii]10.1093/schbul/sbm029CrossRefGoogle ScholarPubMed
Cotter, J., Drake, R. J., Bucci, S., Firth, J., Edge, D., & Yung, A. R. (2014). What drives poor functioning in the at-risk mental state? A systematic review. Schizophrenia Research, 159(2–3), 267277. https://doi.org/http://dx.doi.org/10.1016/j.schres.2014.09.012CrossRefGoogle ScholarPubMed
Demjaha, A., Valmaggia, L., Stahl, D., Byrne, M., & McGuire, P. (2012). Disorganization/cognitive and negative symptom dimensions in the at-risk mental state predict subsequent transition to psychosis. Schizophrenia Bulletin, 38(2), 351359. https://doi.org/sbq088[pii]10.1093/schbul/sbq088CrossRefGoogle ScholarPubMed
DeVylder, J. E., Muchomba, F. M., Gill, K. E., Ben-David, S., Walder, D. J., Malaspina, D., & Corcoran, C. M. (2014). Symptom trajectories and psychosis onset in a clinical high-risk cohort: The relevance of subthreshold thought disorder. Schizophrenia Research, 159(2–3), 278283. https://doi.org/S0920-9964(14)00419-8[pii]CrossRefGoogle Scholar
Ewald, B. (2006). Post hoc choice of cut points introduced bias to diagnostic research. Journal of Clinical Epidemiology, 59(8), 798801. https://doi.org/10.1016/j.jclinepi.2005.11.025CrossRefGoogle ScholarPubMed
Friedman, J. H., Hastie, T., & Tibshirani, R. (2010). Regularization paths for generalized linear models via coordinate descent. Journal of Statistical Software, 33(1), 122. https://doi.org/10.18637/jss.v033.i01CrossRefGoogle ScholarPubMed
Fusar-Poli, P., Bonoldi, I., Yung, A. R., Borgwardt, S., Kempton, M. J., Valmaggia, L., … McGuire, P. (2012). Predicting psychosis: Meta-analysis of transition outcomes in individuals at high clinical risk. Archives of General Psychiatry, 69(3), 220229. https://doi.org/69/3/220[pii]10.1001/archgenpsychiatry.2011.1472CrossRefGoogle ScholarPubMed
Fusar-Poli, P., Cappucciati, M., Rutigliano, G., Lee, T. Y., Beverly, Q., Bonoldi, I., … McGuire, P. (2016a). Towards a standard psychometric diagnostic interview for subjects at ultra high risk of psychosis: CAARMS versus SIPS. Psychiatry Journal, 7146341. doi: 10.1155/2016/7146341. Epub 2016 May 30. https://doi.org/10.1155/2016/7146341CrossRefGoogle ScholarPubMed
Fusar-Poli, P., Cappucciati, M., Rutigliano, G., Schultze-Lutter, F., Bonoldi, I., Borgwardt, S., … McGuire, P. (2015). At risk or not at risk? A meta-analysis of the prognostic accuracy of psychometric interviews for psychosis prediction. World Psychiatry, 14(3), 322332. https://doi.org/10.1002/wps.20250CrossRefGoogle ScholarPubMed
Fusar-Poli, P., Rutigliano, G., Stahl, D., Davies, C., Bonoldi, I., Reilly, T., & McGuire, P. (2017). Development and validation of a clinically based risk calculator for the transdiagnostic prediction of psychosis. JAMA Psychiatry, 74(5), 493500. https://doi.org/10.1001/jamapsychiatry.2017.0284CrossRefGoogle ScholarPubMed
Fusar-Poli, P., Salazar De Pablo, G., Correll, C. U., Meyer-Lindenberg, A., Millan, M. J., Borgwardt, S., … Arango, C. (2020). Prevention of psychosis: Advances in detection, prognosis, and intervention. JAMA Psychiatry, 77(7), 755765. https://doi.org/10.1001/jamapsychiatry.2019.4779CrossRefGoogle ScholarPubMed
Fusar-Poli, P., Schultze-Lutter, F., Cappucciati, M., Rutigliano, G., Bonoldi, I., Stahl, D., … Mcguire, P. (2016b). The dark side of the moon: Meta-analytical impact of recruitment strategies on risk enrichment in the clinical high risk state for psychosis. Schizophrenia Bulletin, 42(3), 732743. https://doi.org/10.1093/schbul/sbv162CrossRefGoogle ScholarPubMed
Gerstenberg, M., Theodoridou, A., Traber-Walker, N., Franscini, M., Wotruba, D., Metzler, S., … Heekeren, K. (2016). Adolescents and adults at clinical high-risk for psychosis: Age-related differences in attenuated positive symptoms syndrome prevalence and entanglement with basic symptoms. Psychological Medicine, 46(5), 10691078. https://doi.org/10.1017/S0033291715002627CrossRefGoogle ScholarPubMed
Grau, J., Grosse, I., & Keilwagen, J. (2015). PRROC: Computing and visualizing precision-recall and receiver operating characteristic curves in R. Bioinformatics (Oxford, England), 31(15), 25952597. https://doi.org/10.1093/bioinformatics/btv153CrossRefGoogle ScholarPubMed
Guloksuz, S., Pries, L. K., ten Have, M., de Graaf, R., van Dorsselaer, S., Klingenberg, B., … van Os, J. (2020). Association of preceding psychosis risk states and non-psychotic mental disorders with incidence of clinical psychosis in the general population: A prospective study in the NEMESIS-2 cohort. World Psychiatry, 19(2), 199205. https://doi.org/10.1002/wps.20755CrossRefGoogle ScholarPubMed
Healy, C., Brannigan, R., Dooley, N., Coughlan, H., Clarke, M., Kelleher, I., & Cannon, M. (2019). Childhood and adolescent psychotic experiences and risk of mental disorder: A systematic review and meta-analysis. Psychological Medicine, 49(10), 15891599. https://doi.org/10.1017/S0033291719000485CrossRefGoogle ScholarPubMed
Heinze, G., & Ploner, M. (2018). logistf: Firth's bias-reduced logistic regression. R package version 1.23.Google Scholar
Jeppesen, P., Clemmensen, L., Munkholm, A., Rimvall, M. K., Rask, C. U., Jørgensen, T., … Skovgaard, A. M. (2015). Psychotic experiences co-occur with sleep problems, negative affect and mental disorders in preadolescence. Journal of Child Psychology and Psychiatry and Allied Disciplines, 56(5), 558565. https://doi.org/10.1111/jcpp.12319CrossRefGoogle ScholarPubMed
Lin, A., Wood, S. J., Nelson, B., Brewer, W. J., Spiliotacopoulos, D., Bruxner, A., … Yung, A. R. (2011). Neurocognitive predictors of functional outcome two to 13 years after identification as ultra-high risk for psychosis. Schizophrenia Research, 132(1), 17. https://doi.org/10.1016/j.schres.2011.06.014 [doi]CrossRefGoogle ScholarPubMed
Lin, A., Yung, A. R., Nelson, B., Brewer, W. J., Riley, R., Simmons, M., … Wood, S. J. (2013). Neurocognitive predictors of transition to psychosis: Medium- to long-term findings from a sample at ultra-high risk for psychosis. Psychological Medicine, 43(11), 23492360. https://doi.org/10.1017/S0033291713000123 [doi]CrossRefGoogle ScholarPubMed
Lindgren, M., Manninen, M., Kalska, H., Mustonen, U., Laajasalo, T., Moilanen, K., … Therman, S. (2014). Predicting psychosis in a general adolescent psychiatric sample. Schizophrenia Research, 158(1–3), 16. https://doi.org/10.1016/j.schres.2014.06.028CrossRefGoogle Scholar
Lindgren, M., Manninen, M., Laajasalo, T., Mustonen, U., Kalska, H., Suvisaari, J., … Therman, S. (2010). The relationship between psychotic-like symptoms and neurocognitive performance in a general adolescent psychiatric sample. Schizophrenia Research, 123(1), 7785. https://doi.org/S0920-9964(10)01441-6[pii]10.1016/j.schres.2010.07.025CrossRefGoogle Scholar
Loewy, R. L., Bearden, C. E., Johnson, J. K., Raine, A., & Cannon, T. D. (2005). The prodromal questionnaire (PQ): Preliminary validation of a self-report screening measure for prodromal and psychotic syndromes. Schizophrenia Research, 79(1), 117125.CrossRefGoogle ScholarPubMed
Mechelli, A., Lin, A., Wood, S., McGorry, P., Amminger, P., Tognin, S., … Yung, A. (2017). Using clinical information to make individualized prognostic predictions in people at ultra high risk for psychosis. Schizophrenia Research, 184, 3238. https://doi.org/10.1016/j.schres.2016.11.047CrossRefGoogle ScholarPubMed
Miller, T. J., McGlashan, T. H., Rosen, J. L., Cadenhead, K., Cannon, T., Ventura, J., … Woods, S. W. (2003). Prodromal assessment with the structured interview for prodromal syndromes and the scale of prodromal symptoms: Predictive validity, interrater reliability, and training to reliability. Schizophrenia Bulletin, 29(4), 703715.CrossRefGoogle ScholarPubMed
Millman, Z. B., Gold, J. M., Mittal, V. A., & Schiffman, J. (2019). The critical need for help-seeking controls in clinical high-risk research. Clinical Psychological Science, 7(5), 11711189. https://doi.org/10.1177/2167702619855660CrossRefGoogle ScholarPubMed
Montemagni, C., Bellino, S., Bracale, N., Bozzatello, P., & Rocca, P. (2020). Models predicting psychosis in patients with high clinical risk: A systematic review. Frontiers in Psychiatry, 11, 223. https://doi.org/10.3389/fpsyt.2020.00223CrossRefGoogle ScholarPubMed
Muthén, L. K., & Muthén, B. O. (2017). Mplus user's guide (8th ed.). Los Angeles, CA: Muthén & Muthén, http://statmodel.comGoogle Scholar
Nelson, B., Yuen, H., Wood, S. J., Lin, A., Spiliotacopoulos, D., Bruxner, A., … Yung, A.R. (2013). Long-term follow-up of a group at ultra high risk (‘prodromal’) for psychosis: The pace 400 study. JAMA Psychiatry, 7(8), 793802. doi: 10.1001/jamapsychiatry.2013.1270CrossRefGoogle Scholar
Oliver, D., Reilly, T. J., Baccaredda Boy, O., Petros, N., Davies, C., Borgwardt, S., … Fusar-Poli, P. (2020). What causes the onset of psychosis in individuals at clinical high risk? A meta-analysis of risk and protective factors. Schizophrenia Bulletin, 46(1), 110120. https://doi.org/10.1093/schbul/sbz039CrossRefGoogle Scholar
Perkins, D. O., Jeffries, C. D., Cornblatt, B. A., Woods, S. W., Addington, J., Bearden, C. E., … McGlashan, T. H. (2015). Severity of thought disorder predicts psychosis in persons at clinical high-risk. Schizophrenia Research, 169(1–3), 169177. https://doi.org/10.1016/j.schres.2015.09.008CrossRefGoogle ScholarPubMed
Piskulic, D., Addington, J., Cadenhead, K. S., Cannon, T. D., Cornblatt, B. A., Heinssen, R., … McGlashan, T. H. (2012). Negative symptoms in individuals at clinical high risk of psychosis. Psychiatry Research, 196(2–3), 220224. https://doi.org/S0165-1781(12)00100-X[pii]10.1016/j.psychres.2012.02.018CrossRefGoogle ScholarPubMed
Raballo, A., Nelson, B., Thompson, A., & Yung, A. (2011). The comprehensive assessment of at-risk mental states: From mapping the onset to mapping the structure. Schizophrenia Research, 127(1–3), 107114. https://doi.org/10.1016/j.schres.2010.12.021CrossRefGoogle ScholarPubMed
R Core Team. (2021). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing.Google Scholar
Revelle, W. (2021) psych: Procedures for Psychological, Psychometric, and Personality Research. Northwestern University, Evanston, Illinois. R package version 2.1.3, https://CRAN.R-project.org/package=psych.Google Scholar
Ruhrmann, S., Schultze-Lutter, F., Salokangas, R. K., Heinimaa, M., Linszen, D., Dingemans, P., … Klosterkötter, J. (2010). Prediction of psychosis in adolescents and young adults at high risk: Results from the prospective European prediction of psychosis study. Archives of General Psychiatry, 67(3), 241251. https://doi.org/67/3/241[pii]10.1001/archgenpsychiatry.2009.206CrossRefGoogle Scholar
Schultze-Lutter, F., Michel, C., Schmidt, S. J., Schimmelmann, B. G., Maric, N. P., Salokangas, R. K. R., … Klosterkötter, J. (2015). EPA guidance on the early detection of clinical high risk states of psychoses. European Psychiatry, 30(3), 405416. https://doi.org/http://dx.doi.org/10.1016/j.eurpsy.2015.01.010CrossRefGoogle ScholarPubMed
Schummers, L., Himes, K. P., Bodnar, L. M., & Hutcheon, J. A. (2016). Predictor characteristics necessary for building a clinically useful risk prediction model: A simulation study. BMC Medical Research Methodology, 16(1), 123. https://doi.org/10.1186/s12874-016-0223-2CrossRefGoogle Scholar
Spada, G., Molteni, S., Pistone, C., Chiappedi, M., McGuire, P., Fusar-Poli, P., & Balottin, U. (2016). Identifying children and adolescents at ultra high risk of psychosis in Italian neuropsychiatry services: A feasibility study. European Child and Adolescent Psychiatry, 25(1), 91106. https://doi.org/10.1007/s00787-015-0710-8CrossRefGoogle ScholarPubMed
Steyerberg, E. W., & Vergouwe, Y. (2014). Towards better clinical prediction models: Seven steps for development and an ABCD for validation. European Heart Journal, 35(29), 19251931. https://doi.org/10.1093/eurheartj/ehu207CrossRefGoogle Scholar
Studerus, E., Ramyead, A., & Riecher-Rössler, A. (2017). Prediction of transition to psychosis in patients with a clinical high risk for psychosis: A systematic review of methodology and reporting. Psychological Medicine, 47(7), 11631178. https://doi.org/10.1017/S0033291716003494CrossRefGoogle ScholarPubMed
Therman, S., Lindgren, M., Manninen, M., Loewy, R. L. L., Huttunen, M. O. O., Cannon, T. D. D., & Suvisaari, J. (2014). Predicting psychosis and psychiatric hospital care among adolescent psychiatric patients with the prodromal questionnaire. Schizophrenia Research, 158(1–3), 710. https://doi.org/S0920-9964(14)00323-5[pii]CrossRefGoogle ScholarPubMed
Thompson, A., Nelson, B., & Yung, A. (2011). Predictive validity of clinical variables in the ‘at risk’ for psychosis population: International comparison with results from the North American prodrome longitudinal study. Schizophrenia Research, 126(1–3), 5157. https://doi.org/S0920-9964(10)01553-7[pii]10.1016/j.schres.2010.09.024CrossRefGoogle ScholarPubMed
Trotta, A., Arseneault, L., Caspi, A., Moffitt, T. E., Danese, A., Pariante, C., & Fisher, H. L. (2020). Mental health and functional outcomes in young adulthood of children with psychotic symptoms: A longitudinal cohort study. Schizophrenia Bulletin, 46(2), 261271. https://doi.org/10.1093/schbul/sbz069Google ScholarPubMed
Ulhaq, S., Thevan, K., & Adams, R. (2017). Innovations in practice: Identifying young people at ultra-high risk of psychosis in a child and adolescent mental health service. Child and Adolescent Mental Health, 22(2), 9195. https://doi.org/10.1111/camh.12173CrossRefGoogle Scholar
van Os, J., & Guloksuz, S. (2017). A critique of the ‘ultra-high risk’ and ‘transition’ paradigm. World Psychiatry, 16(2), 200206. https://doi.org/10.1002/wps.20423CrossRefGoogle ScholarPubMed
Velthorst, E., Nelson, B., Wiltink, S., De Haan, L., Wood, S. J., Lin, A., & Yung, A. R. (2013). Transition to first episode psychosis in ultra high risk populations: Does baseline functioning hold the key? Schizophrenia Research, 143(1), 132137. https://doi.org/10.1016/j.schres.2012.10.025CrossRefGoogle ScholarPubMed
Vrieze, S. I. (2012). Model selection and psychological theory: A discussion of the differences between the Akaike information criterion (AIC) and the Bayesian information criterion (BIC). Psychological Methods, 17(2), 228243. https://doi.org/10.1037/a0027127CrossRefGoogle ScholarPubMed
Walder, D. J., Holtzman, C. W., Addington, J., Cadenhead, K., Tsuang, M., Cornblatt, B., … Walker, E. F. (2013). Sexual dimorphisms and prediction of conversion in the NAPLS psychosis prodrome. Schizophrenia Research, 144(1–3), 4350. https://doi.org/10.1016/j.schres.2012.11.039CrossRefGoogle ScholarPubMed
Wechsler, D. (1987). Wechsler memory scale – revised. New York, NY: The Psychological Corporation.Google Scholar
Wechsler, D. (1997). Wechsler adult intelligence scale (3rd ed.). San Antonio, Texas: The Psychological Corporation.Google Scholar
Welsh, P., & Tiffin, P. A. (2014). The ‘at-risk mental state’ for psychosis in adolescents: Clinical presentation, transition and remission. Child Psychiatry and Human Development, 45(1), 9098. https://doi.org/10.1007/s10578-013-0380-zCrossRefGoogle ScholarPubMed
Werbeloff, N., Drukker, M., Dohrenwend, B. P., Levav, I., Yoffe, R., van Os, J., … Weiser, M. (2012). Self-reported attenuated psychotic symptoms as forerunners of severe mental disorders later in life. Archives of General Psychiatry, 69(5), 467475. https://doi.org/archgenpsychiatry.2011.1580[pii]10.1001/archgenpsychiatry.2011.1580CrossRefGoogle ScholarPubMed
Wigman, J. T., van Nierop, M., Vollebergh, W. A., Lieb, R., Beesdo-Baum, K., Wittchen, H. U., & van Os, J. (2012). Evidence that psychotic symptoms are prevalent in disorders of anxiety and depression, impacting on illness onset, risk, and severity – implications for diagnosis and ultra-high risk research. Schizophrenia Bulletin, 38(2), 247257. https://doi.org/sbr196[pii]10.1093/schbul/sbr196CrossRefGoogle ScholarPubMed
World Medical Association (2000). World medical association declaration of Helsinki: Ethical principles for medical research involving human subjects. JAMA, 284(23), 30433045. doi: 10.1001/jama.284.23.3043CrossRefGoogle Scholar
Worthington, M. A., Cao, H., & Cannon, T. D. (2019). Discovery and validation of prediction algorithms for psychosis in youth at clinical high risk. Biological Psychiatry: Cognitive Neuroscience and Neuroimaging, 5(8), 738747. https://doi.org/10.1016/j.bpsc.2019.10.006Google ScholarPubMed
Yung, A. R., Phillips, L. J., Yuen, H. P., & McGorry, P. D. (2004). Risk factors for psychosis in an ultra high-risk group: Psychopathology and clinical features. Schizophrenia Research, 67(2-3), 131142.CrossRefGoogle Scholar
Yung, A. R., Stanford, C., Cosgrave, E., Killackey, E., Phillips, L., Nelson, B., & McGorry, P. D. (2006). Testing the ultra high risk (prodromal) criteria for the prediction of psychosis in a clinical sample of young people. Schizophrenia Research, 84(1), 5766. doi: https://doi.org/S0920-9964(06)00108-3[pii]10.1016/j.schres.2006.03.014CrossRefGoogle Scholar
Supplementary material: PDF

Lindgren et al. supplementary material

Lindgren et al. supplementary material 1
Download Lindgren et al. supplementary material(PDF)
PDF 842.3 KB
Supplementary material: File

Lindgren et al. supplementary material

Lindgren et al. supplementary material 2

Download Lindgren et al. supplementary material(File)
File 355.1 KB