Hostname: page-component-6d856f89d9-xkcpr Total loading time: 0 Render date: 2024-07-16T03:26:30.423Z Has data issue: false hasContentIssue false
  • English
  • Français

Statistical strategies for multiple testing in the safety evaluation of a genetically modified crop

Published online by Cambridge University Press:  22 November 2016

C. I. VAHL  and
Q. KANG
C. I. VAHL*
Affiliation:
Department of Statistics, Kansas State University, Manhattan, KS 66506, USA
Q. KANG
Affiliation:
Independent Statistical Consultant, Manhattan, KS 66503, USA
*
*To whom all correspondence should be addressed. Email: vahl@ksu.edu
Get access Rights & Permissions [Opens in a new window]

Summary

Hazard identification is the first step in assessing the risk of a genetically modified (GM) crop. It employs the concept of substantial equivalence to evaluate crop safety. The current process relies on subjective opinions to integrate various comparisons among the GM crop, the non-GM counterpart and an assortment of non-GM references over an array of key endpoints measured in field trials. The pre-eminent need to control the consumer's risk in hazard identification has been left unaddressed. The current paper develops statistical strategies to resolve this issue. Hypotheses of individual tests are explicitly defined to reflect the study objectives. They are then grouped into families and connected by logical operators according to decision rules commonly used in crop safety evaluation. This pre-specification of hypotheses arranged in an organized layout leads to a simple, transparent decision-making process where the consumer's risk can be managed directly. A two-stage multiplicity adjustment procedure is created by applying fundamental principles for multiple testing to the newly assembled families of hypotheses. The practical utility of the proposed procedure is shown in a real-world example. Besides being easy to implement and convey, the proposed statistical strategies accommodate the addition of supportive evidence for safety and allow the nature of the genetic modification to be taken into account.

Type
Crops and Soils Research Papers
Information
The Journal of Agricultural Science , Volume 155 , Issue 5 , July 2017 , pp. 812 - 831
Check for updates
Copyright
Copyright © Cambridge University Press 2016 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Anderson, S. & Hauck, W. W. (1983). A new procedure for testing equivalence in comparative bioavailability and other clinical trials. Communication in Statistics – Theory and Methods 12, 2663–2692.Google Scholar
Bauer, P. & Kieser, M. (1996). A unifying approach for confidence intervals and testing of equivalence and difference. Biometrika 83, 934–937.Google Scholar
Benjamini, Y. & Hochberg, Y. (1995). Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal Statistical Society, Series B (Methodological) 57, 289–300.Google Scholar
Berman, K. H., Harrigan, G. G., Riordan, S. G., Nemeth, M. A., Hanson, C., Smith, M., Sorbet, R., Zhu, E. & Ridley, W. P. (2010). Compositions of forage and seed from second-generation glyphosate-tolerant soybean MON 89788 and insect-protected soybean MON 87701 from Brazil are equivalent to those of conventional soybean (Glycine max). Journal of Agricultural and Food Chemistry 58, 6270–6276.Google Scholar
Berry, S. M. & Berry, D. A. (2004). Accounting for multiplicities in assessing drug safety: a three-level hierarchical mixture model. Biometrics 60, 418–426.Google Scholar
Brink, K., Chui, C. F., Cressman, R. F., Garcia, P., Henderson, N., Hong, B., Maxwell, C. A., Meyer, K., Mickelson, J., Stecca, K. L., Tyree, C. W., Weber, N., Zeng, W. Q. & Zhong, C. X. (2014). Molecular characterization, compositional analysis, and germination evaluation of a high-oleic soybean generated by the suppression of FAD2-1 expression. Crop Science 54, 2160–2174.CrossRefGoogle Scholar
Chen, X., Capizzi, T., Binkowitz, B., Quan, H., Wei, L. & Luo, X. H. (2005). Decision rule based multiplicity adjustment strategy. Clinical Trials 2, 394–399.Google Scholar
Chi, G. Y. H. (1998). Multiple testings: multiple comparisons and multiple endpoints. Drug Information Journal 32, 1347S–1362S.Google Scholar
Codex Alimentarius Commission (2009). Foods Derived from Modern Biotechnology. Rome, Italy: Joint FAO/WHO Food Standards Programme.Google Scholar
CPMP (2002). Points to Consider on Multiplicity Issues in Clinical Trials. CPMP/EWP/908/99. London, UK: The European Agency for the Evaluation of Medicinal Products.Google Scholar
Dmitrienko, A. & D'Agostino, R. (2013). Traditional multiplicity adjustment methods in clinical trials. Statistics in Medicine 32, 5172–5218.CrossRefGoogle ScholarPubMed
Dmitrienko, A., Offen, W. W. & Westfall, P. H. (2003). Gatekeeping strategies for clinical trials that do not require all primary effects to be significant. Statistics in Medicine 22, 2387–2400.Google Scholar
Dmitrienko, A., D'Agostino, R. B. & Huque, M. F. (2013). Key multiplicity issues in clinical drug development. Statistics in Medicine 32, 1079–1111.Google Scholar
Dunnett, C. W. & Tamhane, A. C. (1991). Step-down multiple tests for comparing treatments with a control in unbalanced one-way layouts. Statistics in Medicine 10, 939–947.Google Scholar
Dunnett, C. W. & Tamhane, A. C. (1992). A step-up multiple test procedure. Journal of the American Statistical Association 87, 162–170.Google Scholar
EFSA (2008). Safety and nutritional assessment of GM plants and derived food and feed: the role of animal feeding trials. Food and Chemical Toxicology 46, S2–S70.Google Scholar
EFSA (2010). Statistical considerations for the safety evaluation of GMOs. EFSA Journal 8(1), 1250. doi: 10.2903/j.efsa.2010.1250.Google Scholar
EFSA (2011). Guidance on selection of comparators for the risk assessment of genetically modified plants and derived food and feed. EFSA Journal 9(5), 2149. doi: 10.2903/j.efsa.2011.2149.Google Scholar
EFSA (2014). Explanatory statement for the applicability of the Guidance of the EFSA Scientific Committee on conducting repeated-dose 90-day oral toxicity study in rodents on whole food/feed for GMO risk assessment. EFSA Journal 12(10), 3871. doi: 10.2903/j.efsa.2014.3871.Google Scholar
EFSA (2015 a). Scientific advice to the European Commission on the internal review submitted under Regulation (EC) No 1367/2006 on the application of the provisions of the Aarhus Convention against the Commission Implementing Decision 2015/687 to authorise genetically modified oilseed rape MON88302. EFSA Supporting Publications 12(8), EN–864. DOI: 10.2903/sp.efsa.2015.EN-864.Google Scholar
EFSA (2015 b). Scientific opinion on an application (EFSA-GMO-BE-2011-98) for the placing on the market of herbicide-tolerant genetically modified soybean FG72 for food and feed uses, import and processing under Regulation (EC) No 1829/2003 from Bayer CropScience. EFSA Journal 13(7), 4167. doi: 10.2903/j.efsa.2015.4167.Google Scholar
EFSA (2015 c). Scientific opinion on an application (Reference EFSA-GMO-NL-2011-100) for the placing on the market of the herbicide-tolerant, increased oleic acid genetically modified soybean MON 87705 × MON 89788 for food and feed uses, import and processing under Regulation (EC) No 1829/2003 from Monsanto. EFSA Journal 13(7), 4178. doi: 10.2903/j.efsa.2015.4178.Google Scholar
EFSA (2016). Scientific Opinion on an application by Bayer CropScience and Monsanto (EFSA-GMO-NL-2009-75) for placing on the market of genetically modified glufosinate-ammonium- and glyphosate-tolerant oilseed rape MS8 × RF3 × GT73 and subcombinations, which have not been authorised previously (i.e. MS8 × GT73 and RF3 × GT73) independently of their origin, for food and feed uses, import and processing, with the exception of isolated seed protein for food, under Regulation (EC) No 1829/2003. EFSA Journal 14(5), 4466. DOI: 10.2903/j.efsa.2016.4466.Google Scholar
Finner, H. & Roters, M. (2001). On the false discovery rate and expected type I errors. Biometrical Journal 43, 985–1005.Google Scholar
Finner, H. & Strassburger, K. (2002). The partitioning principle: a powerful tool in multiple decision theory. Annals of Statistics 30, 1194–1213.CrossRefGoogle Scholar
FDA (2009). Guidance for Industry. Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims. MD, USA: FDA, Silver Spring. Available from: http://www.fda.gov/downloads/Drugs/.../Guidances/UCM193282.pdf (verified 24 August 2016).Google Scholar
Gabriel, K. R. (1969). Simultaneous test procedures – Some theory of multiple comparisons. Annals of Mathematical Statistics 40, 224–250.Google Scholar
Goeman, J. J. & Solari, A. (2010). The sequential rejection principle of familywise error control. Annals of Statistics 38, 3782–3810.Google Scholar
Gould, A. L. (2008). Detecting potential safety issues in clinical trials by Bayesian screening. Biometrical Journal 50, 837–851.Google Scholar
Grechanovsky, E. & Hochberg, Y. (1999). Closed procedures are better and often admit a shortcut. Journal of Statistical Planning and Inference 76, 79–91.Google Scholar
Guilbaud, O. (2008). Simultaneous confidence regions corresponding to Holm's step-down procedure and other closed-testing procedures. Biometrical Journal 50, 678–692.Google Scholar
Guilbaud, O. (2012). Simultaneous confidence regions for closed tests, including Holm-, Hochberg-, and Hommel-related procedures. Biometrical Journal 54, 317–342.Google Scholar
Hasler, M. & Hothorn, L. A. (2013). Simultaneous confidence intervals on multivariate non-inferiority. Statistics in Medicine 32, 1720–1729.Google Scholar
Hayter, A. J. & Hsu, J. C. (1994). On the relationship between stepwise decision procedures and confidence sets. Journal of the American Statistical Association 89, 128–136.Google Scholar
Herman, R. A., Fast, B. J., Johnson, T. Y., Sabbatini, J. & Rudgers, G. W. (2013). Compositional safety of herbicide-tolerant DAS-81910-7 cotton. Journal of Agricultural and Food Chemistry 61, 11683–11692.Google Scholar
Hochberg, Y. (1988). A sharper Bonferroni procedure for multiple tests of significance. Biometrika 75, 800–802.Google Scholar
Holm, S. (1979). A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics 6, 65–70.Google Scholar
Hothorn, L. A. & Hasler, M. (2008). Proof of hazard and proof of safety in toxicological studies using simultaneous confidence intervals for differences and ratios to control. Journal of Biopharmaceutical Statistics 18, 915–933.Google Scholar
Hothorn, L. A. & Oberdoerfer, R. (2006). Statistical analysis used in the nutritional assessment of novel food using the proof of safety. Regulatory Toxicology and Pharmacology 44, 125–135.Google Scholar
Hsu, J. C. (1996). Multiple Comparisons: Theory and Methods. Boca Raton, FL, USA: Chapman and Hall/CRC.Google Scholar
Hsu, J. C. (2010). Multiplicity adjustment big and small in clinical studies. Clinical Pharmacology and Therapeutics 88, 251–254.CrossRefGoogle ScholarPubMed
Hsu, J. C. & Berger, R. L. (1999). Stepwise confidence intervals without multiplicity adjustment for dose-response and toxicity studies. Journal of the American Statistical Association 94, 468–482.Google Scholar
Hsu, J. C., Hwang, J. T. G., Liu, H. K. & Ruberg, S. J. (1994). Confidence intervals associated with tests for bioequivalence. Biometrika 81, 103–114.CrossRefGoogle Scholar
Hua, S. Y., Xu, S. Y. & D'Agostino, R. B. (2015). Multiplicity adjustments in testing for bioequivalence. Statistics in Medicine 34, 215–231.Google Scholar
Huang, L., Zalkikar, J. & Tiwari, R. C. (2011). A likelihood ratio test based method for signal detection with application to FDA's drug safety data. Journal of the American Statistical Association 106, 1230–1241.Google Scholar
Hung, H. M. J. & Wang, S. J. (2009). Some controversial multiple testing problems in regulatory applications. Journal of Biopharmaceutical Statistics 19, 1–11.CrossRefGoogle ScholarPubMed
Hung, H. M. J. & Wang, S. J. (2010). Challenges to multiple testing in clinical trials. Biometrical Journal 52, 747–756.CrossRefGoogle Scholar
Kang, Q. & Vahl, C. I. (2014). Statistical analysis in the safety evaluation of genetically modified crops: equivalence tests. Crop Science 54, 2183–2200.CrossRefGoogle Scholar
Kang, Q. & Vahl, C. I. (2016). Statistical procedures for testing hypotheses of equivalence in the safety evaluation of a genetically modified crop. Journal of Agricultural Science, Cambridge 154, 1392–1412.Google Scholar
König, A., Cockburn, A., Crevel, R. W. R., Debruyne, E., Grafstroem, R., Hammerling, U., Kimber, I., Knudsen, I., Kuiper, H. A., Peijnenburg, A. A. C. M., Penninks, A. H., Poulsen, M., Schauzu, M. & Wal, J. M. (2004). Assessment of the safety of foods derived from genetically modified (GM) crops. Food and Chemical Toxicology 42, 1047–1088.Google Scholar
Lepping, M. D., Herman, R. A. & Potts, B. L. (2013). Compositional equivalence of DAS-444Ø6-6 (AAD-12+2mEPSPS + PAT) herbicide-tolerant soybean and nontransgenic soybean. Journal of Agricultural and Food Chemistry 61, 11180–11190.Google Scholar
Liu, Y. & Hsu, J. (2009). Testing for efficacy in primary and secondary endpoints by partitioning decision paths. Journal of the American Statistical Association 104, 1661–1670.Google Scholar
Logan, B. R. & Tamhane, A. C. (2008). Superiority inferences on individual endpoints following noninferiority testing in clinical trials. Biometrical Journal 50, 693–703.Google Scholar
Lundry, D. R., Burns, J. A., Nemeth, M. A. & Riordan, S. G. (2013). Composition of grain and forage from insect-protected and herbicide-tolerant corn, MON 89034 × TC1507 × MON 88017 × DAS-59122-7 (SmartStax), is equivalent to that of conventional corn (Zea mays L.). Journal of Agricultural and Food Chemistry 61, 1991–1998.Google Scholar
Marcus, R., Peritz, E. & Gabriel, K. R. (1976). On closed testing procedures with special reference to ordered analysis of variance. Biometrika 63, 655–660.Google Scholar
Mehrotra, D. & Heyse, J. F. (2004). Use of the false discovery rate for evaluating clinical safety data. Statistical Methods in Medical Research 13, 227–238.Google Scholar
Oberdoerfer, R. B., Shillito, R. D., Beuckeleer, M. D. & Mitten, D. H. (2005). Rice (Oryza sativa L.) containing the bar gene is compositionally equivalent to the nontransgenic counterpart. Journal of Agricultural and Food Chemistry 53, 1457–1465.Google Scholar
OECD (1993). Safety Evaluation of Foods Derived by Modern Biotechnology: Concepts and Principles. Paris, France: Organization for Economic Cooperation and Development.Google Scholar
Quan, H., Bolognese, J. & Yuan, W. Y. (2001). Assessment of equivalence on multiple endpoints. Statistics in Medicine 20, 3159–3173.Google Scholar
Röhmel, J. & Pigeot, I. (2010). A comparison of multiple testing procedures for the gold standard non-inferiority trial. Journal of Biopharmaceutical Statistics 20, 911–926.Google Scholar
Sarkar, S. K. (1998). Some probability inequalities for ordered MTP2 random variables: a proof of the Simes conjecture. Annals of Statistics 26, 494–504.Google Scholar
Sarkar, S. K. (2008). On the Simes inequality and its generalization. In Beyond Parametrics in Interdisciplinary Research: Festschrift in Honor of Professor Pranab K. Sen (Eds Balakrishnan, N., Peña, E. A. & Silvapulle, M. J.), pp. 231–242. Beachwood, OH, USA: Institute of Mathematical Statistics.Google Scholar
Sarkar, S. K. & Chang, C. K. (1997). The Simes method for multiple hypothesis testing with positively dependent test statistics. Journal of the American Statistical Association 92, 1601–1608.Google Scholar
SAS Institute Inc (2011). SAS/STAT® 9·3 User's Guide. Cary, NC, USA: SAS Inst. Inc.Google Scholar
Schuirmann, D. J. (1987). A comparison of the two one-sided tests procedure and the power approach for assessing the equivalence of average bioavailability. Journal of Pharmacokinetics and Biopharmaceutics 15, 657–680.Google Scholar
Simes, R. J. (1986). An improved Bonferroni procedure for multiple tests of significance. Biometrika 73, 751–754.Google Scholar
Sonnemann, E. (1982). Allgemeine Lösungen multipler Testprobleme. EDV in Medizin und Biologie 13, 120–128.Google Scholar
Sonnemann, E. & Finner, H. (1988). Vollständigkeitssätze für multiple testprobleme. In Multiple Hypotheses Testing (Eds Bauer, P., Hommel, G. & Sonnemann, E.), pp. 121–135. Medizinische Informatik und Statistik, Vol. 70. Berlin, Germany: Springer.Google Scholar
Strassburger, K. & Bretz, F. (2008). Compatible simultaneous lower confidence bounds for the Holm procedure and other Bonferroni-based closed tests. Statistics in Medicine 27, 4914–4927.Google Scholar
Tamhane, A. C. (1996). Multiple comparisons. In Handbook of Statistics, Volume 13: Design and Analysis of Experiments (Eds Ghosh, S. & Rao, C. R.), pp. 587–630. Amsterdam, the Netherlands: Elsevier Science.Google Scholar
Vahl, C. I. & Kang, Q. (2016). Equivalence criteria for the safety evaluation of a genetically modified crop: a statistical perspective. Journal of Agricultural Science, Cambridge 154, 383–406.Google Scholar
Van der Voet, H., Perry, J. N., Amzal, B. & Paoletti, C. (2011). A statistical assessment of differences and equivalences between genetically modified and reference plant varieties. BMC Biotechnology 11, 15. DOI: 10·1186/1472-6750-11-15.Google Scholar
Westfall, P. H. & Kristen, A. (2001). Optimally weighted, fixed sequence and gatekeeper multiple testing procedures. Journal of Statistical Planning and Inference 99, 25–40.Google Scholar
Westfall, P. H., Tobias, R. D., Rom, D., Wolfinger, R. D. & Hochberg, Y. (1999). Multiple Comparisons and Multiple Tests Using the SAS System. Cary, NC, USA: SAS Institute.Google Scholar