Hostname: page-component-7bb8b95d7b-lvwk9 Total loading time: 0 Render date: 2024-09-06T14:34:43.487Z Has data issue: false hasContentIssue false
  • English
  • Français

Optimal experimental design: Formulations and computations

Part of: Partial differential equations, initial value and time-dependent initial-boundary value problems Sufficiency and information Sequential methods Statistics Design of experiments Communication, information

Published online by Cambridge University Press:  04 September 2024

Xun Huan ,
Jayanth Jagalur  and
Youssef Marzouk Open the ORCID record for Youssef Marzouk [Opens in a new window]
Xun Huan
Affiliation:
University of Michigan, 1231 Beal Ave, Ann Arbor, MI 48109, USA Email: xhuan@umich.edu
Jayanth Jagalur
Affiliation:
Center for Applied Scientific Computing, Lawrence Livermore National Laboratory, 7000 East Ave, Livermore, CA 94550, USA Email: jagalur1@llnl.gov
Youssef Marzouk
Affiliation:
Massachusetts Institute of Technology, 77 Massachusetts Ave, Cambridge, MA 02139, USA Email: ymarz@mit.edu
Rights & Permissions [Opens in a new window]

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the ‘Save PDF’ action button.

Questions of ‘how best to acquire data’ are essential to modelling and prediction in the natural and social sciences, engineering applications, and beyond. Optimal experimental design (OED) formalizes these questions and creates computational methods to answer them. This article presents a systematic survey of modern OED, from its foundations in classical design theory to current research involving OED for complex models. We begin by reviewing criteria used to formulate an OED problem and thus to encode the goal of performing an experiment. We emphasize the flexibility of the Bayesian and decision-theoretic approach, which encompasses information-based criteria that are well-suited to nonlinear and non-Gaussian statistical models. We then discuss methods for estimating or bounding the values of these design criteria; this endeavour can be quite challenging due to strong nonlinearities, high parameter dimension, large per-sample costs, or settings where the model is implicit. A complementary set of computational issues involves optimization methods used to find a design; we discuss such methods in the discrete (combinatorial) setting of observation selection and in settings where an exact design can be continuously parametrized. Finally we present emerging methods for sequential OED that build non-myopic design policies, rather than explicit designs; these methods naturally adapt to the outcomes of past experiments in proposing new experiments, while seeking coordination among all experiments to be performed. Throughout, we highlight important open questions and challenges.

MSC classification

Primary: 62-02: Research exposition (monographs, survey articles) 62B15: Theory of statistical experiments 62K05: Optimal designs
Secondary: 62L05: Sequential design 94A17: Measures of information, entropy 65M32: Inverse problems
Type
Research Article
Information
Acta Numerica , Volume 33 , July 2024 , pp. 715 - 840
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution, and reproduction in any medium, provided the original work is properly cited.
Copyright
© The Author(s), 2024. Published by Cambridge University Press

References

Agarwal, P. K., Har-Peled, S. and Varadarajan, K. R. (2005), Geometric approximation via coresets, Combin. Comput. Geom. 52, 1–30.Google Scholar
Aggarwal, R., Demkowicz, M. J. and Marzouk, Y. M. (2016), Information-driven experimental design in materials science, in Information Science for Materials Discovery and Design (Lookman, T., Alexander, F. and Rajan, K., eds), Springer, pp. 13–44.CrossRefGoogle Scholar
Alexanderian, A. (2021), Optimal experimental design for infinite-dimensional Bayesian inverse problems governed by PDEs: A review, Inverse Problems 37, art. 043001.CrossRefGoogle Scholar
Alexanderian, A. and Saibaba, A. K. (2018), Efficient D-optimal design of experiments for infinite-dimensional Bayesian linear inverse problems, SIAM J. Sci. Comput. 40, A2956–A2985.CrossRefGoogle Scholar
Alexanderian, A., Gloor, P. J. and Ghattas, O. (2016a), On Bayesian A- and D-optimal experimental designs in infinite dimensions, Bayesian Anal. 11, 671–695.CrossRefGoogle Scholar
Alexanderian, A., Nicholson, R. and Petra, N. (2022), Optimal design of large-scale nonlinear Bayesian inverse problems under model uncertainty. Available at arXiv:2211.03952.Google Scholar
Alexanderian, A., Petra, N., Stadler, G. and Ghattas, O. (2014), A-optimal design of experiments for infinite-dimensional Bayesian linear inverse problems with regularized ${\mathrm{\ell}}_0$ -sparsification, SIAM J. Sci. Comput. 36, A2122–A2148.CrossRefGoogle Scholar
Alexanderian, A., Petra, N., Stadler, G. and Ghattas, O. (2016b), A fast and scalable method for A-optimal design of experiments for infinite-dimensional Bayesian nonlinear inverse problems, SIAM J. Sci. Comput. 38, A243–A272.CrossRefGoogle Scholar
Alexanderian, A., Petra, N., Stadler, G. and Sunseri, I. (2021), Optimal design of large-scale Bayesian linear inverse problems under reducible model uncertainty: Good to know what you don’t know, SIAM/ASA J. Uncertain. Quantif. 9, 163–184.CrossRefGoogle Scholar
Allen-Zhu, Z., Li, Y., Singh, A. and Wang, Y. (2017), Near-optimal design of experiments via regret minimization, in Proceedings of the 34th International Conference on Machine Learning (ICML 2017) , Vol. 70 of Proceedings of Machine Learning Research, PMLR, pp. 126–135.Google Scholar
Allen-Zhu, Z., Liao, Z. and Orecchia, L. (2015), Spectral sparsification and regret minimization beyond matrix multiplicative updates, in Proceedings of the 47th Annual ACM Symposium on Theory of Computing (STOC 2015) , ACM, pp. 237–245.Google Scholar
Ao, Z. and Li, J. (2020), An approximate KLD based experimental design for models with intractable likelihoods, in Proceedings of the 23rd International Conference on Artificial Intelligence and Statistics , Vol. 108 of Proceedings of Machine Learning Research, PMLR, pp. 3241–3251.Google Scholar
Ao, Z. and Li, J. (2024), On estimating the gradient of the expected information gain in Bayesian experimental design, in Proceedings of the 38th AAAI Conference on Artificial Intelligence (Wooldridge, M., Dy, J. and Natarajan, S., eds), AAAI Press, pp. 20311–20319.Google Scholar
Artzner, P., Delbaen, F., Eber, J. and Heath, D. (1999), Coherent measures of risk, Math. Finance 9, 203–228.CrossRefGoogle Scholar
Asmussen, S. and Glynn, P. W. (2007), Stochastic Simulation: Algorithms and Analysis , Springer.CrossRefGoogle Scholar
Atkinson, A. C., Donev, A. N. and Tobias, R. D. (2007), Optimum Experimental Designs, with SAS , Oxford University Press.CrossRefGoogle Scholar
Attia, A., Alexanderian, A. and Saibaba, A. K. (2018), Goal-oriented optimal design of experiments for large-scale Bayesian linear inverse problems, Inverse Problems 34, art. 095009.CrossRefGoogle Scholar
Attia, A., Leyffer, S. and Munson, T. (2023), Robust A-optimal experimental design for Bayesian inverse problems. Available at arXiv:2305.03855.Google Scholar
Atwood, C. L. (1969), Optimal and efficient designs of experiments, Ann. Math. Statist. 40, 1570–1602.CrossRefGoogle Scholar
Audet, C. (2004), Convergence results for generalized pattern search algorithms are tight, Optim. Engrg 5, 101–122.CrossRefGoogle Scholar
Audet, C. and Dennis, J. E. (2002), Analysis of generalized pattern searches, SIAM J. Optim. 13, 889–903.CrossRefGoogle Scholar
Bach, F. (2013), Learning with submodular functions: A convex optimization perspective, Found. Trends Mach. Learn. 6, 145–373.CrossRefGoogle Scholar
Bach, F. R. and Jordan, M. I. (2002), Kernel independent component analysis, J. Mach. Learn. Res. 3, 1–48.Google Scholar
Baptista, R., Cao, L., Chen, J., Ghattas, O., Li, F., Marzouk, Y. M. and Oden, J. T. (2024), Bayesian model calibration for block copolymer self-assembly: Likelihood-free inference and expected information gain computation via measure transport, J. Comput. Phys. 503, art. 112844.CrossRefGoogle Scholar
Baptista, R., Hosseini, B., Kovachki, N. B. and Marzouk, Y. (2023a), Conditional sampling with monotone GANs: From generative models to likelihood-free inference. Available at arXiv:2006.06755.CrossRefGoogle Scholar
Baptista, R., Marzouk, Y. and Zahm, O. (2022), Gradient-based data and parameter dimension reduction for Bayesian models: An information theoretic perspective. Available at arXiv:2207.08670.Google Scholar
Baptista, R., Marzouk, Y. and Zahm, O. (2023b), On the representation and learning of monotone triangular transport maps, Found. Comput. Mat h. Available at doi:10.1007/s10208-023-09630-x.CrossRefGoogle Scholar
Barber, D. and Agakov, F. (2003), The IM algorithm: A variational approach to information maximization, in Advances in Neural Information Processing Systems 16 , MIT Press, pp. 201–208.Google Scholar
Batson, J. D., Spielman, D. A. and Srivastava, N. (2009), Twice-Ramanujan sparsifiers, in Proceedings of the 41st Annual ACM Symposium on Theory of Computing (STOC 2009) , ACM, pp. 255–262.Google Scholar
Beck, J. and Guillas, S. (2016), Sequential design with mutual information for computer experiments (MICE): Emulation of a tsunami model, SIAM/ASA J. Uncertain. Quantif. 4, 739–766.CrossRefGoogle Scholar
Beck, J., Dia, B. M., Espath, L. and Tempone, R. (2020), Multilevel double loop Monte Carlo and stochastic collocation methods with importance sampling for Bayesian optimal experimental design, Int. J. Numer. Methods Engrg 121, 3482–3503.CrossRefGoogle Scholar
Beck, J., Dia, B. M., Espath, L. F., Long, Q. and Tempone, R. (2018), Fast Bayesian experimental design: Laplace-based importance sampling for the expected information gain, Comput. Methods Appl. Mech. Engrg 334, 523–553.CrossRefGoogle Scholar
Belghazi, M. I., Baratin, A., Rajeswar, S., Ozair, S., Bengio, Y., Courville, A. and Hjelm, R. D. (2018), Mutual information neural estimation, in Proceedings of the 35th International Conference on Machine Learning (ICML 2018) , Vol. 80 of Proceedings of Machine Learning Research, PMLR, pp. 531–540.Google Scholar
Ben-Tal, A. and Nemirovski, A. (2001), Lectures on Modern Convex Optimization , SIAM.CrossRefGoogle Scholar
Benner, P., Gugercin, S. and Willcox, K. (2015), A survey of projection-based model reduction methods for parametric dynamical systems, SIAM Rev. 57, 483–531.CrossRefGoogle Scholar
Berger, J. O. (1985), Statistical Decision Theory and Bayesian Analysis , Springer Series in Statistics, Springer.CrossRefGoogle Scholar
Berger, J. O. (1994), An overview of robust Bayesian analysis (with discussion), Test 3, 5–124.CrossRefGoogle Scholar
Berger, M. P. F. and Wong, W. K. (2009), An Introduction to Optimal Designs for Social and Biomedical Research , Wiley.CrossRefGoogle Scholar
Bernardo, J. M. (1979), Expected information as expected utility, Ann . Statist. 7, 686–690.CrossRefGoogle Scholar
Bernardo, J. M. and Smith, A. F. M. (2000), Bayesian Theory , Wiley.Google Scholar
Berry, S. M., Carlin, B. P., Lee, J. J. and Müller, P. (2010), Bayesian Adaptive Methods for Clinical Trials , Chapman & Hall/CRC.CrossRefGoogle Scholar
Bertsekas, D. P. (2005), Dynamic Programming and Optimal Control , Vol. 1, Athena Scientific.Google Scholar
Bhatnagar, S., Prasad, H. L. and Prashanth, L. A. (2013), Stochastic Recursive Algorithms for Optimization , Springer.CrossRefGoogle Scholar
Bian, A. A., Buhmann, J. M., Krause, A. and Tschiatschek, S. (2017), Guarantees for greedy maximization of non-submodular functions with applications, in Proceedings of the 34th International Conference on Machine Learning (ICML 2017) (Precup, D. and Teh, Y. W., eds), Vol. 70 of Proceedings of Machine Learning Research, PMLR, pp. 498–507.Google Scholar
Blackwell, D. (1951), Comparison of experiments, in Proceedings of the 2nd Berkeley Symposium on Mathematical Statistics and Probability , University of California Press, pp. 93–102.CrossRefGoogle Scholar
Blackwell, D. (1953), Equivalent comparisons of experiments, Ann . Math. Statist. 24, 265–272.CrossRefGoogle Scholar
Blanchard, A. and Sapsis, T. (2021), Output-weighted optimal sampling for Bayesian experimental design and uncertainty quantification, SIAM/ASA J. Uncertain. Quantif. 9, 564–592.CrossRefGoogle Scholar
Blau, T., Bonilla, E. V., Chades, I. and Dezfouli, A. (2022), Optimizing sequential experimental design with deep reinforcement learning, in Proceedings of the 39th International Conference on Machine Learning (ICML 2022) (Chaudhuri, K. et al., eds), Vol. 162 of Proceedings of Machine Learning Research, PMLR, pp. 2107–2128.Google Scholar
Blum, J. R. (1954), Multidimensional stochastic approximation methods, Ann. Math. Statist. 25, 737–744.CrossRefGoogle Scholar
Bochkina, N. (2019), Bernstein–von Mises theorem and misspecified models: A review, in Foundations of Modern Statistics (Belomestny, D. et al., eds), Vol. 425 of Springer Proceedings in Mathematics & Statistics, Springer, pp. 355–380.CrossRefGoogle Scholar
Bogachev, V. I., Kolesnikov, A. V. and Medvedev, K. V. (2005), Triangular transformations of measures, Sbornik Math. 196, art. 309.CrossRefGoogle Scholar
Bogunovic, I., Zhao, J. and Cevher, V. (2018), Robust maximization of non-submodular objectives, in Proceedings of the Twenty-First International Conference on Artificial Intelligence and Statistics (Storkey, A. and Perez-Cruz, F., eds), Vol. 84 of Proceedings of Machine Learning Research, PMLR, pp. 890–899.Google Scholar
Borges, C. and Biros, G. (2018), Reconstruction of a compactly supported sound profile in the presence of a random background medium, Inverse Problems 34, art. 115007.CrossRefGoogle Scholar
Bose, R. C. (1939), On the construction of balanced incomplete block designs, Ann. Eugen. 9, 353–399.CrossRefGoogle Scholar
Bose, R. C. and Nair, K. R. (1939), Partially balanced incomplete block designs, Sankhyā 4, 337–372.Google Scholar
Box, G. E. P. (1992), Sequential experimentation and sequential assembly of designs, Qual. Engrg 5, 321–330.CrossRefGoogle Scholar
Boyd, S. P. and Vandenberghe, L. (2004), Convex Optimization , Cambridge University Press.CrossRefGoogle Scholar
Brockwell, A. E. and Kadane, J. B. (2003), A gridding method for Bayesian sequential decision problems, J. Comput. Graph. Statist. 12, 566–584.CrossRefGoogle Scholar
Buchbinder, N., Feldman, M., Naor, J. and Schwartz, R. (2014), Submodular maximization with cardinality constraints, in Proceedings of the Twenty-Fifth Annual ACM-SIAM Symposium on Discrete Algorithms , SIAM, pp. 1433–1452.Google Scholar
Bui-Thanh, T., Ghattas, O., Martin, J. and Stadler, G. (2013), A computational framework for infinite-dimensional Bayesian inverse problems I: The linearized case, with application to global seismic inversion, SIAM J. Sci. Comput. 35, A2494–A2523.CrossRefGoogle Scholar
Caflisch, R. E. (1998), Monte Carlo and quasi-Monte Carlo methods, Acta Numer. 7, 1–49.CrossRefGoogle Scholar
Calinescu, G., Chekuri, C., Pál, M. and Vondrák, J. (2011), Maximizing a monotone submodular function subject to a matroid constraint, SIAM J. Comput. 40, 1740–1766.CrossRefGoogle Scholar
Campbell, T. and Beronov, B. (2019), Sparse variational inference: Bayesian coresets from scratch, in Advances in Neural Information Processing Systems 32 (Wallach, H. et al., eds), Curran Associates, pp. 11461–11472.Google Scholar
Campbell, T. and Broderick, T. (2018), Bayesian coreset construction via greedy iterative geodesic ascent, in Proceedings of the 35th International Conference on Machine Learning (ICML 2018) , Vol. 80 of Proceedings of Machine Learning Research, PMLR, pp. 698–706.Google Scholar
Campbell, T. and Broderick, T. (2019), Automated scalable Bayesian inference via Hilbert coresets, J. Mach. Learn. Res. 20, 551–588.Google Scholar
Carlier, G., Chernozhukov, V. and Galichon, A. (2016), Vector quantile regression: An optimal transport approach, Ann. Statist. 44, 1165–1192.CrossRefGoogle Scholar
Carlin, B. P., Kadane, J. B. and Gelfand, A. E. (1998), Approaches for optimal sequential decision analysis in clinical trials, Biometrics 54, 964–975.CrossRefGoogle ScholarPubMed
Carlon, A. G., Dia, B. M., Espath, L., Lopez, R. H. and Tempone, R. (2020), Nesterov-aided stochastic gradient methods using Laplace approximation for Bayesian design optimization, Comput. Methods Appl. Mech. Engrg 363, art. 112909.CrossRefGoogle Scholar
Carmona, C. U. and Nicholls, G. K. (2020), Semi-modular inference: Enhanced learning in multi-modular models by tempering the influence of components, in Proceedings of the 23rd International Conference on Artificial Intelligence and Statistics , Vol. 108 of Proceedings of Machine Learning Research, PMLR, pp. 4226–4235.Google Scholar
Caselton, W. F. and Zidek, J. V. (1984), Optimal monitoring network designs, Statist. Probab. Lett. 2, 223–227.CrossRefGoogle Scholar
Cavagnaro, D. R., Myung, J. I., Pitt, M. A. and Kujala, J. V. (2010), Adaptive design optimization: A mutual information-based approach to model discrimination in cognitive science, Neural Comput. 22, 887–905.CrossRefGoogle ScholarPubMed
Chaloner, K. (1984), Optimal Bayesian experimental design for linear models, Ann. Statist. 12, 283–300.CrossRefGoogle Scholar
Chaloner, K. and Larntz, K. (1989), Optimal Bayesian design applied to logistic regression experiments, J. Statist. Plann. Infer. 21, 191–208.CrossRefGoogle Scholar
Chaloner, K. and Verdinelli, I. (1995), Bayesian experimental design: A review, Statist. Sci. 10, 273–304.CrossRefGoogle Scholar
Chang, K.-H. (2012), Stochastic Nelder–Mead simplex method: A new globally convergent direct search method for simulation optimization, European J. Oper. Res. 220, 684–694.CrossRefGoogle Scholar
Chekuri, C., Vondrák, J. and Zenklusen, R. (2014), Submodular function maximization via the multilinear relaxation and contention resolution schemes, SIAM J. Comput. 43, 1831–1879.CrossRefGoogle Scholar
Chen, X., Wang, C., Zhou, Z. and Ross, K. (2021), Randomized ensembled double Q-learning: Learning fast without a model, in 9th International Conference on Learning Representations (ICLR 2021). Available at https://openreview.net/forum?id=AY8zfZm0tDd.Google Scholar
Chevalier, C. and Ginsbourger, D. (2013), Fast computation of the multi-points expected improvement with applications in batch selection, in Learning and Intelligent Optimization , Vol. 7997 of Lecture Notes in Computer Science, Springer, pp. 59–69.CrossRefGoogle Scholar
Chowdhary, A., Tong, S., Stadler, G. and Alexanderian, A. (2023), Sensitivity analysis of the information gain in infinite-dimensional Bayesian linear inverse problems. Available at arXiv:2310.16906.Google Scholar
Christen, J. A. and Nakamura, M. (2003), Sequential stopping rules for species accumulation, J. Agric. Biol. Environ. Statist. 8, 184–195.CrossRefGoogle Scholar
Clyde, M. A. (2001), Experimental design: Bayesian designs, in International Encyclopedia of the Social & Behavioral Sciences (Smelser, N. J. and Baltes, P. B., eds), Science Direct, pp. 5075–5081.CrossRefGoogle Scholar
Cohn, D. A., Ghahramani, Z. and Jordan, M. I. (1996), Active learning with statistical models, J. Artificial Intelligence Res. 4, 129–145.CrossRefGoogle Scholar
Conforti, M. and Cornuéjols, G. (1984), Submodular set functions, matroids and the greedy algorithm: Tight worst-case bounds and some generalizations of the Rado–Edmonds theorem, Discrete Appl. Math. 7, 251–274.CrossRefGoogle Scholar
Conn, A. R., Scheinberg, K. and Vicente, L. N. (2009), Introduction to Derivative-Free Optimization , SIAM.CrossRefGoogle Scholar
Cook, R. D. and Nachtsheim, C. J. (1980), A comparison of algorithms for constructing exact D-optimal designs, Technometrics 22, 315–324.CrossRefGoogle Scholar
Cotter, S. L., Roberts, G. O., Stuart, A. M. and White, D. (2013), MCMC methods for functions: Modifying old algorithms to make them faster, Statist. Sci. 28, 424–446.CrossRefGoogle Scholar
Cover, T. A. and Thomas, J. A. (2006), Elements of Information Theory , second edition, Wiley.Google Scholar
Cox, R. T. (1946), Probability, frequency and reasonable expectation, Amer. J. Phys. 14, 1–13.CrossRefGoogle Scholar
Craig, C. C. and Fisher, R. A. (1936), The design of experiments, Amer. Math. Monthly 43, 180.CrossRefGoogle Scholar
Cui, T. and Tong, X. T. (2022), A unified performance analysis of likelihood-informed subspace methods, Bernoulli 28, 2788–2815.CrossRefGoogle Scholar
Cui, T., Dolgov, S. and Zahm, O. (2023), Scalable conditional deep inverse Rosenblatt transports using tensor trains and gradient-based dimension reduction, J. Comput. Phys. 485, art. 112103.CrossRefGoogle Scholar
Cui, T., Law, K. J. H. and Marzouk, Y. M. (2016), Dimension-independent likelihood-informed MCMC, J. Comput. Phys. 304, 109–137.CrossRefGoogle Scholar
Cui, T., Martin, J., Marzouk, Y. M., Solonen, A. and Spantini, A. (2014), Likelihood-informed dimension reduction for nonlinear inverse problems, Inverse Problems 30, art. 114015.CrossRefGoogle Scholar
Czyż, P., Grabowski, F., Vogt, J., Beerenwinkel, N. and Marx, A. (2023), Beyond normal: On the evaluation of mutual information estimators, in Advances in Neural Information Processing Systems 36 (Oh, A. et al., eds), Curran Associates, pp. 16957–16990.Google Scholar
Das, A. and Kempe, D. (2011), Submodular meets spectral: Greedy algorithms for subset selection, sparse approximation and dictionary selection, in Proceedings of the 28th International Conference on Machine Learning (ICML 2011) (Getoor, L. and Scheffer, T., eds), ACM, pp. 1057–1064.Google Scholar
DasGupta, A. (1995), Review of optimal Bayes designs. Technical report, Purdue University, West Lafayette, IN.Google Scholar
Dasgupta, S. (2011), Two faces of active learning, Theoret. Comput. Sci. 412, 1767–1781.CrossRefGoogle Scholar
Dashti, M. and Stuart, A. M. (2017), The Bayesian approach to inverse problems, in Handbook of Uncertainty Quantification (Ghanem, R., Higdon, D. and Owhadi, H., eds), Springer, pp. 311–428.CrossRefGoogle Scholar
Dashti, M., Law, K. J., Stuart, A. M. and Voss, J. (2013), MAP estimators and their consistency in Bayesian nonparametric inverse problems, Inverse Problems 29, art. 095017.CrossRefGoogle Scholar
Dewaskar, M., Tosh, C., Knoblauch, J. and Dunson, D. B. (2023), Robustifying likelihoods by optimistically re-weighting data. Available at arXiv:2303.10525.Google Scholar
Dick, J., Kuo, F. Y. and Sloan, I. H. (2013), High-dimensional integration: The quasi-Monte Carlo way, Acta Numer. 22, 133–288.CrossRefGoogle Scholar
Dong, J., Jacobsen, C., Khalloufi, M., Akram, M., Liu, W., Duraisamy, K. and Huan, X. (2024), Variational Bayesian optimal experimental design with normalizing flows. Available at arXiv:2404.13056.Google Scholar
Donsker, M. D. and Varadhan, S. R. S. (1983), Asymptotic evaluation of certain Markov process expectations for large time IV, Commun. Pure Appl. Math. 36, 183–212.CrossRefGoogle Scholar
Dror, H. A. and Steinberg, D. M. (2008), Sequential experimental designs for generalized linear models, J. Amer. Statist. Assoc. 103, 288–298.CrossRefGoogle Scholar
Drovandi, C. C., McGree, J. M. and Pettitt, A. N. (2013), Sequential Monte Carlo for Bayesian sequentially designed experiments for discrete data, Comput. Statist. Data Anal. 57, 320–335.CrossRefGoogle Scholar
Drovandi, C. C., McGree, J. M. and Pettitt, A. N. (2014), A sequential Monte Carlo algorithm to incorporate model uncertainty in Bayesian sequential design, J. Comput. Graph. Statist. 23, 3–24.CrossRefGoogle Scholar
Duncan, T. E. (1970), On the calculation of mutual information, SIAM J. Appl. Math. 19, 215–220.CrossRefGoogle Scholar
Duong, D.-L., Helin, T. and Rojo-Garcia, J. R. (2023), Stability estimates for the expected utility in Bayesian optimal experimental design, Inverse Problems 39, art. 125008.CrossRefGoogle Scholar
El Moselhy, T. A. and Marzouk, Y. M. (2012), Bayesian inference with optimal maps, J. Comput. Phys. 231, 7815–7850.CrossRefGoogle Scholar
Elfving, G. (1952), Optimum allocation in linear regression theory, Ann. Math. Statist. 23, 255–262.CrossRefGoogle Scholar
Englezou, Y., Waite, T. W. and Woods, D. C. (2022), Approximate Laplace importance sampling for the estimation of expected Shannon information gain in high-dimensional Bayesian design for nonlinear models, Statist. Comput. 32, art. 82.CrossRefGoogle Scholar
Eskenazis, A. and Shenfeld, Y. (2024), Intrinsic dimensional functional inequalities on model spaces, J. Funct. Anal. 286, art. 110338.CrossRefGoogle Scholar
Fan, K. (1967), Subadditive functions on a distributive lattice and an extension of Szász’s inequality, J. Math. Anal. Appl. 18, 262–268.CrossRefGoogle Scholar
Fan, K. (1968), An inequality for subadditive functions on a distributive lattice, with application to determinantal inequalities, Linear Algebra Appl. 1, 33–38.CrossRefGoogle Scholar
Fedorov, V. V. (1972), Theory of Optimal Experiments , Academic Press.Google Scholar
Fedorov, V. V. (1996), Design of spatial experiments: Model fitting and prediction. Technical report, Oak Ridge National Laboratory, Oak Ridge, TN.Google Scholar
Fedorov, V. V. and Flanagan, D. (1997), Optimal monitoring network design based on Mercer’s expansion of covariance kernel, J. Combin. Inform. System Sci. 23, 237–250.Google Scholar
Fedorov, V. V. and Hackl, P. (1997), Model-Oriented Design of Experiments , Vol. 125 of Lecture Notes in Statistics, Springer.CrossRefGoogle Scholar
Fedorov, V. V. and Müller, W. G. (2007), Optimum design for correlated fields via covariance kernel expansions, in mODa 8: Advances in Model-Oriented Design and Analysis (López-Fidalgo, J., Rodríguez-Díaz, J. M. and Torsney, B., eds), Contributions to Statistics, Physica, Springer, pp. 57–66.CrossRefGoogle Scholar
Feldman, D. and Langberg, M. (2011), A unified framework for approximating and clustering data, in Proceedings of the 43rd Annual ACM Symposium on Theory of Computing (STOC 2011) , ACM, pp. 569–578.Google Scholar
Feng, C. and Marzouk, Y. M. (2019), A layered multiple importance sampling scheme for focused optimal Bayesian experimental design. Available at arXiv:1903.11187.Google Scholar
Fisher, M. L., Nemhauser, G. L. and Wolsey, L. A. (1978), An analysis of approximations for maximizing submodular set functions II, Math. Program. 8, 73–87.CrossRefGoogle Scholar
Fisher, R. A. (1936), Design of experiments, Brit. Med. J. 1(3923), 554.CrossRefGoogle Scholar
Ford, I., Titterington, D. M. and Kitsos, C. P. (1989), Recent advances in nonlinear experimental design, Technometrics 31, 49–60.CrossRefGoogle Scholar
Foster, A., Ivanova, D. R., Malik, I. and Rainforth, T. (2021), Deep adaptive design: Amortizing sequential Bayesian experimental design, in Proceedings of the 38th International Conference on Machine Learning (ICML 2021) (Meila, M. and Zhang, T., eds), Vol. 139 of Proceedings of Machine Learning Research, PMLR, pp. 3384–3395.Google Scholar
Foster, A., Jankowiak, M., Bingham, E., Horsfall, P., Teh, Y. W., Rainforth, T. and Goodman, N. (2019), Variational Bayesian optimal experimental design, in Advances in Neural Information Processing Systems 32 (Wallach, H. et al., eds), Curran Associates, pp. 14036–14047.Google Scholar
Foster, A., Jankowiak, M., O’Meara, M., Teh, Y. W. and Rainforth, T. (2020), A unified stochastic gradient approach to designing Bayesian-optimal experiments, in Proceedings of the 23rd International Conference on Artificial Intelligence and Statistics , Vol. 108 of Proceedings of Machine Learning Research, PMLR, pp. 2959–2969.Google Scholar
Frazier, P. I. (2018), Bayesian optimization, INFORMS TutORials in Operations Research 2018, 255–278.Google Scholar
Freund, Y., Seung, H. S., Shamir, E. and Tishby, N. (1997), Selective sampling using the query by committee algorithm, Mach. Learn. 28, 133–168.CrossRefGoogle Scholar
Fujishige, S. (2005), Submodular Functions and Optimization , Vol. 58 of Annals of Discrete Mathematics, second edition, Elsevier.Google Scholar
Gantmacher, F. R. and Kreĭn, M. G. (1960), Oszillationsmatrizen, Oszillationskerne und kleine Schwingungen mechanischer Systeme , Vol. 5, Akademie.CrossRefGoogle Scholar
Gao, W., Oh, S. and Viswanath, P. (2018), Demystifying fixed k-nearest neighbor information estimators, IEEE Trans. Inform. Theory 64, 5629–5661.CrossRefGoogle Scholar
Gautier, R. and Pronzato, L. (2000), Adaptive control for sequential design, Discuss. Math. Probab. Statist. 20, 97–113.Google Scholar
Ghattas, O. and Willcox, K. (2021), Learning physics-based models from data: Perspectives from inverse problems and model reduction, Acta Numer. 30, 445–554.CrossRefGoogle Scholar
Giles, M. B. (2015), Multilevel Monte Carlo methods, Acta Numer. 24, 259–328.CrossRefGoogle Scholar
Giné, E. and Nickl, R. (2021), Mathematical Foundations of Infinite-Dimensional Statistical Models , Cambridge University Press.Google Scholar
Ginebra, J. (2007), On the measure of the information in a statistical experiment, Bayesian Anal. 2, 167–212.CrossRefGoogle Scholar
Giraldi, L., Le Maître, O. P., Hoteit, I. and Knio, O. M. (2018), Optimal projection of observations in a Bayesian setting, Comput. Statist. Data Anal. 124, 252–276.CrossRefGoogle Scholar
Gneiting, T. and Raftery, A. E. (2007), Strictly proper scoring rules, prediction, and estimation, J. Amer. Statist. Assoc. 102, 359–378.CrossRefGoogle Scholar
Go, J. and Isaac, T. (2022), Robust expected information gain for optimal Bayesian experimental design using ambiguity sets, in Proceedings of the 38th Conference on Uncertainty in Artificial Intelligence , Vol. 180 of Proceedings of Machine Learning Research, PMLR, pp. 718–727.Google Scholar
Goda, T., Hironaka, T., Kitade, W. and Foster, A. (2022), Unbiased MLMC stochastic gradient-based optimization of Bayesian experimental designs, SIAM J. Sci. Comput. 44, A286–A311.CrossRefGoogle Scholar
Goda, T., Hironaka, T. and Iwamoto, T. (2020), Multilevel Monte Carlo estimation of expected information gains, Stoch. Anal. Appl. 38, 581–600.CrossRefGoogle Scholar
Gorodetsky, A. and Marzouk, Y. (2016), Mercer kernels and integrated variance experimental design: Connections between Gaussian process regression and polynomial approximation, SIAM/ASA J. Uncertain. Quantif. 4, 796–828.CrossRefGoogle Scholar
Gramacy, R. B. (2020), Surrogates: Gaussian Process Modeling, Design, and Optimization for the Applied Sciences , CRC Press.CrossRefGoogle Scholar
Gramacy, R. B. (2022), plgp: Particle learning of Gaussian processes. Available at https://cran.r-project.org/package=plgp.Google Scholar
Gramacy, R. B. and Apley, D. W. (2015), Local Gaussian process approximation for large computer experiments, J. Comput. Graph. Statist. 24, 561–578.CrossRefGoogle Scholar
Grünwald, P. and van Ommen, T. (2017), Inconsistency of Bayesian inference for misspecified linear models, and a proposal for repairing it, Bayesian Anal. 12, 1069–1103.CrossRefGoogle Scholar
Gürkan, G., Özge, A. Y. and Robinson, S. M. (1994), Sample-path optimization in simulation, in Proceedings of the 1994 Winter Simulation Conference (WSC ’94) (Sadowski, D. A. et al., eds), ACM, pp. 247–254.Google Scholar
Haber, E., Horesh, L. and Tenorio, L. (2008), Numerical methods for experimental design of large-scale linear ill-posed inverse problems, Inverse Problems 24, art. 055012.CrossRefGoogle Scholar
Haber, E., Horesh, L. and Tenorio, L. (2009), Numerical methods for the design of large-scale nonlinear discrete ill-posed inverse problems, Inverse Problems 26, art. 025002.CrossRefGoogle Scholar
Hainy, M., Drovandi, C. C. and McGree, J. M. (2016), Likelihood-free extensions for Bayesian sequentially designed experiments, in mODa 11: Advances in Model-Oriented Design and Analysis (Kunert, J., Müller, C. and Atkinson, A., eds), Contributions to Statistics, Springer, pp. 153–161.Google Scholar
Hainy, M., Price, D. J., Restif, O. and Drovandi, C. (2022), Optimal Bayesian design for model discrimination via classification, Statist. Comput. 32, art. 25.CrossRefGoogle ScholarPubMed
Hairer, M., Stuart, A. M. and Voss, J. (2011), Signal processing problems on function space: Bayesian formulation, stochastic PDEs and effective MCMC methods, in The Oxford Handbook of Nonlinear Filtering (Crisan, D. and Rozovskii, B., eds), Oxford University Press, pp. 833–873.Google Scholar
Harari, O. and Steinberg, D. M. (2014), Optimal designs for Gaussian process models via spectral decomposition, J. Statist. Plann. Infer. 154, 87–101.CrossRefGoogle Scholar
He, X. D., Kou, S. and Peng, X. (2022), Risk measures: Robustness, elicitability, and backtesting, Annu. Rev. Statist. Appl. 9, 141–166.CrossRefGoogle Scholar
Healy, K. and Schruben, L. W. (1991), Retrospective simulation response optimization, in Proceedings of the 1991 Winter Simulation Conference (WSC ’91) (Nelson, B. L. et al., eds), IEEE Computer Society, pp. 901–906.CrossRefGoogle Scholar
Hedayat, A. (1981), Study of optimality criteria in design of experiments, in Statistics and Related Topics: International Symposium Proceedings (Csorgo, M., ed.), Elsevier Science, pp. 39–56.Google Scholar
Helin, T. and Kretschmann, R. (2022), Non-asymptotic error estimates for the Laplace approximation in Bayesian inverse problems, Numer. Math. 150, 521–549.CrossRefGoogle Scholar
Helin, T., Hyvönen, N. and Puska, J.-P. (2022), Edge-promoting adaptive Bayesian experimental design for X-ray imaging, SIAM J. Sci. Comput. 44, B506–B530.CrossRefGoogle Scholar
Herrmann, L., Schwab, C. and Zech, J. (2020), Deep neural network expression of posterior expectations in Bayesian PDE inversion, Inverse Problems 36, art. 125011.CrossRefGoogle Scholar
Hoang, T. N., Low, B. K. H., Jaillet, P. and Kankanhalli, M. (2014), Nonmyopic ε-Bayes-optimal active learning of Gaussian processes, in Proceedings of the 31st International Conference on Machine Learning (ICML 2014) , Vol. 32 of Proceedings of Machine Learning Research, PMLR, pp. 739–747.Google Scholar
Hochba, D. S. (1997), Approximation algorithms for NP-hard problems, ACM SIGACT News 28, 40–52.CrossRefGoogle Scholar
Huan, X. (2015), Numerical approaches for sequential Bayesian optimal experimental design. PhD thesis, Massachusetts Institute of Technology.Google Scholar
Huan, X. and Marzouk, Y. M. (2013), Simulation-based optimal Bayesian experimental design for nonlinear systems, J. Comput. Phys. 232, 288–317.CrossRefGoogle Scholar
Huan, X. and Marzouk, Y. M. (2014), Gradient-based stochastic optimization methods in Bayesian experimental design, Int. J. Uncertain. Quantif. 4, 479–510.CrossRefGoogle Scholar
Huan, X. and Marzouk, Y. M. (2016), Sequential Bayesian optimal experimental design via approximate dynamic programming. Available at arXiv:1604.08320.Google Scholar
Huang, C.-W., Chen, R. T. Q., Tsirigotis, C. and Courville, A. (2020), Convex potential flows: Universal probability distributions with optimal transport and convex optimization. Available at arXiv:2012.05942.Google Scholar
Huggins, J., Campbell, T. and Broderick, T. (2016), Coresets for scalable Bayesian logistic regression, in Advances in Neural Information Processing Systems 29 (Lee, D. et al., eds), Curran Associates, pp. 4080–4088.Google Scholar
Huggins, J. H. and Miller, J. W. (2023), Reproducible model selection using bagged posteriors, Bayesian Anal. 18, 79–104.CrossRefGoogle ScholarPubMed
Ivanova, D. R., Foster, A., Kleinegesse, S., Gutmann, M. U. and Rainforth, T. (2021), Implicit deep adaptive design: Policy-based experimental design without likelihoods, in Advances in Neural Information Processing Systems 34 (Ranzato, M. et al., eds), Curran Associates, pp. 25785–25798.Google Scholar
Jacob, P. E., Murray, L. M., Holmes, C. C. and Robert, C. P. (2017), Better together? Statistical learning in models made of modules. Available at arXiv:1708.08719.Google Scholar
Jagalur-Mohan, J. and Marzouk, Y. (2021), Batch greedy maximization of non-submodular functions: Guarantees and applications to experimental design, J. Mach. Learn. Res. 22, 11397–11458.Google Scholar
Jaynes, E. T. and Bretthorst, G. L. (2003), Probability Theory: The Logic of Science , Cambridge University Press.CrossRefGoogle Scholar
Johnson, C. R. and Barrett, W. W. (1985), Spanning-tree extensions of the Hadamard–Fischer inequalities, Linear Algebra Appl. 66, 177–193.CrossRefGoogle Scholar
Johnson, M. E. and Nachtsheim, C. J. (1983), Some guidelines for constructing exact D-optimal designs on convex design spaces, Technometrics 25, 271–277.Google Scholar
Johnson, M. E., Moore, L. M. and Ylvisaker, D. (1990), Minimax and maximin distance designs, J. Statist. Plann. Infer. 26, 131–148.CrossRefGoogle Scholar
Jones, D. R., Schonlau, M. and Welch, W. J. (1998), Efficient global optimization of expensive black-box functions, J. Global Optim. 13, 455–492.CrossRefGoogle Scholar
Joseph, V. R., Gul, E. and Ba, S. (2015), Maximum projection designs for computer experiments, Biometrika 102, 371–380.CrossRefGoogle Scholar
Joseph, V. R., Gul, E. and Ba, S. (2020), Designing computer experiments with multiple types of factors: The MaxPro approach, J. Qual. Technol. 52, 343–354.CrossRefGoogle Scholar
Jourdan, A. and Franco, J. (2010), Optimal Latin hypercube designs for the Kullback–Leibler criterion, AStA Adv. Statist. Anal. 94, 341–351.CrossRefGoogle Scholar
Kaelbling, L. P., Littman, M. L. and Cassandra, A. R. (1998), Planning and acting in partially observable stochastic domains, Artif. Intell. 101, 99–134.CrossRefGoogle Scholar
Kaelbling, L. P., Littman, M. L. and Moore, A. W. (1996), Reinforcement learning: A survey, J. Artif. Intell. Res. 4, 237–285.CrossRefGoogle Scholar
Kaipio, J. and Kolehmainen, V. (2013), Approximate marginalization over modelling errors and uncertainties in inverse problems, in Bayesian Theory and Applications (Damien, P. et al., eds), Oxford University Press, pp. 644–672.CrossRefGoogle Scholar
Kaipio, J. and Somersalo, E. (2006), Statistical and Computational Inverse Problems , Vol. 160 of Applied Mathematical Sciences, Springer.Google Scholar
Kaipio, J. and Somersalo, E. (2007), Statistical inverse problems: Discretization, model reduction and inverse crimes, J. Comput. Appl. Math. 198, 493–504.CrossRefGoogle Scholar
Karaca, O. and Kamgarpour, M. (2018), Exploiting weak supermodularity for coalition-proof mechanisms, in 2018 IEEE Conference on Decision and Control (CDC) , IEEE, pp. 1118–1123.CrossRefGoogle Scholar
Karhunen, K. (1947), Über lineare Methoden in der Wahrscheinlichkeitsrechnung, Am. Acad. Sci. Fennicade, Ser. A, I 37, 3–79.Google Scholar
Kelmans, A. K. and Kimelfeld, B. N. (1983), Multiplicative submodularity of a matrix’s principal minor as a function of the set of its rows and some combinatorial applications, Discrete Math. 44, 113–116.CrossRefGoogle Scholar
Kennamer, N., Walton, S. and Ihler, A. (2023), Design amortization for Bayesian optimal experimental design, in Proceedings of the 37th AAAI Conference on Artificial Intelligence (Williams, B., Chen, Y. and Neville, J., eds), AAAI Press, pp. 8220–8227.Google Scholar
Kennedy, M. C. and O’Hagan, A. (2001), Bayesian calibration of computer models, J. R. Statist. Soc. Ser. B. Statist. Methodol. 63, 425–464.CrossRefGoogle Scholar
Kiefer, J. (1958), On the nonrandomized optimality and randomized nonoptimality of symmetrical designs, Ann. Math. Statist. 29, 675–699.CrossRefGoogle Scholar
Kiefer, J. (1959), Optimum experimental designs, J. R. Statist. Soc. Ser. B. Statist. Methodol. 21, 272–304.CrossRefGoogle Scholar
Kiefer, J. (1961a), Optimum designs in regression problems II, Ann. Math. Statist. 32, 298–325.CrossRefGoogle Scholar
Kiefer, J. (1961b), Optimum experimental designs V, with applications to systematic and rotatable designs, in Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability , Vol. 1, University of California Press, pp. 381–405.Google Scholar
Kiefer, J. (1974), General equivalence theory for optimum designs (approximate theory), Ann. Statist. 2, 849–879.CrossRefGoogle Scholar
Kiefer, J. and Wolfowitz, J. (1952), Stochastic estimation of the maximum of a regression function, Ann. Math. Statist. 23, 462–466.CrossRefGoogle Scholar
Kiefer, J. and Wolfowitz, J. (1959), Optimum designs in regression problems, Ann. Math. Statist. 30, 271–294.CrossRefGoogle Scholar
Kiefer, J. and Wolfowitz, J. (1960), The equivalence of two extremum problems, Canad. J. Statist. 12, 363–366.Google Scholar
Kim, W., Pitt, M. A., Lu, Z.-L., Steyvers, M. and Myung, J. I. (2014), A hierarchical adaptive approach to optimal experimental design, Neural Comput. 26, 2565–2492.CrossRefGoogle ScholarPubMed
King, J. and Wong, W.-K. (2000), Minimax D-optimal designs for the logistic model, Biometrics 56, 1263–1267.CrossRefGoogle ScholarPubMed
Kleijn, B. J. K. and van der Vaart, A. W. (2012), The Bernstein–von-Mises theorem under misspecification, Electron. J. Statist. 6, 354–381.CrossRefGoogle Scholar
Kleinegesse, S. and Gutmann, M. U. (2020), Bayesian experimental design for implicit models by mutual information neural estimation, in Proceedings of the 37th International Conference on Machine Learning (ICML 2020) (Daumé, H. and Singh, A., eds), Vol. 119 of Proceedings of Machine Learning Research, PMLR, pp. 5316–5326.Google Scholar
Kleinegesse, S. and Gutmann, M. U. (2021), Gradient-based Bayesian experimental design for implicit models using mutual information lower bounds. Available at arXiv:2105.04379.Google Scholar
Kleinegesse, S., Drovandi, C. and Gutmann, M. U. (2021), Sequential Bayesian experimental design for implicit models via mutual information, Bayesian Anal. 16, 773–802.CrossRefGoogle Scholar
Kleywegt, A. J., Shapiro, A. and Mello, T. Homem-de (2002), The sample average approximation method for stochastic discrete optimization, SIAM J. Optim. 12, 479–502.CrossRefGoogle Scholar
Knapik, B. T., van der Vaart, A. W. and van Zanten, J. H. (2011), Bayesian inverse problems with Gaussian priors, Ann. Statist. 39, 2626–2657.CrossRefGoogle Scholar
Knothe, H. (1957), Contributions to the theory of convex bodies, Michigan Math. J. 4, 39–52.CrossRefGoogle Scholar
Ko, C.-W., Lee, J. and Queyranne, M. (1995), An exact algorithm for maximum entropy sampling, Oper. Res. 43, 684–691.CrossRefGoogle Scholar
Kobyzev, I., Prince, S. J. and Brubaker, M. A. (2020), Normalizing flows: An introduction and review of current methods, IEEE Trans. Pattern Anal. Mach. Intell. 43, 3964–3979.CrossRefGoogle Scholar
Konda, V. R. and Tsitsiklis, J. N. (1999), Actor–critic algorithms, in Advances in Neural Information Processing Systems 12 (Solla, S. et al., eds), MIT Press, pp. 1008–1014.Google Scholar
Korkel, S., Bauer, I., Bock, H. G. and Schloder, J. P. (1999), A sequential approach for nonlinear optimum experimental design in DAE systems, in Scientific Computing in Chemical Engineering II (Keil, F. et al., eds), Springer, pp. 338–345.Google Scholar
Kotelyanskiĭ, D. M. (1950), On the theory of nonnegative and oscillating matrices, Ukrains’kyi Matematychnyi Zhurnal 2, 94–101.Google Scholar
Kouri, D. P., Jakeman, J. D. and Huerta, J. Gabriel (2022), Risk-adapted optimal experimental design, SIAM/ASA J. Uncertain. Quantif. 10, 687–716.CrossRefGoogle Scholar
Koval, K., Alexanderian, A. and Stadler, G. (2020), Optimal experimental design under irreducible uncertainty for linear inverse problems governed by PDEs, Inverse Problems 36, art. 075007.CrossRefGoogle Scholar
Koval, K., Herzog, R. and Scheichl, R. (2024), Tractable optimal experimental design using transport maps. Available at arXiv:2401.07971.Google Scholar
Kozachenko, L. F. and Leonenko, N. N. (1987), A statistical estimate for the entropy of a random vector, Probl. Inf. Transm. 23, 9–16.Google Scholar
Kraskov, A., Stögbauer, H. and Grassberger, P. (2004), Estimating mutual information, Phys. Rev. E 69, art. 066138.CrossRefGoogle ScholarPubMed
Krause, A. and Golovin, D. (2014), Submodular function maximization, in Tractability, Practical Approaches to Hard Problems (Bordeaux, L., Hamadi, Y. and Kohli, P., eds), Cambridge University Press, pp. 71–104.CrossRefGoogle Scholar
Krause, A., Singh, A. and Guestrin, C. (2008), Near-optimal sensor placements in Gaussian processes: Theory, efficient algorithms and empirical studies, J. Mach. Learn. Res. 9, 235–284.Google Scholar
Kuhn, D., Esfahani, P. M., Nguyen, V. A. and Shafieezadeh-Abadeh, S. (2019), Wasserstein distributionally robust optimization: Theory and applications in machine learning, in Operations Research & Management Science in the Age of Analytics , INFORMS, pp. 130–166.CrossRefGoogle Scholar
Kushner, H. J. and Yin, G. G. (2003), Stochastic Approximation and Recursive Algorithms and Applications , second edition, Springer.Google Scholar
Lam, R. and Willcox, K. (2017), Lookahead Bayesian optimization with inequality constraints, in Advances in Neural Information Processing Systems 30 (Guyon, I. et al., eds), Curran Associates, pp. 1890–1900.Google Scholar
Larson, J., Menickelly, M. and Wild, S. M. (2019), Derivative-free optimization methods, Acta Numer. 28, 287–404.CrossRefGoogle Scholar
Lau, L. C. and Zhou, H. (2020), A spectral approach to network design, in Proceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing (STOC 2020) , ACM, pp. 826–839.CrossRefGoogle Scholar
Lau, L. C. and Zhou, H. (2022), A local search framework for experimental design, SIAM J. Comput. 51, 900–951.CrossRefGoogle Scholar
Le Cam, L. (1964), Sufficiency and approximate sufficiency, Ann. Math. Statist. 35, 1419–1455.CrossRefGoogle Scholar
Lehmann, E. L. and Casella, G. (1998), Theory of Point Estimation , Springer Texts in Statistics, Springer.Google Scholar
Leskovec, J., Krause, A., Guestrin, C., Faloutsos, C., VanBriesen, J. and Glance, N. (2007), Cost-effective outbreak detection in networks, in Proceedings of the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , ACM, pp. 420–429.CrossRefGoogle Scholar
Letizia, N. A., Novello, N. and Tonello, A. M. (2023), Variational f-divergence and derangements for discriminative mutual information estimation. Available at arXiv:2305.20025.Google Scholar
Lewis, D. D. (1995), A sequential algorithm for training text classifiers: Corrigendum and additional data, SIGIR Forum 29, 13–19.CrossRefGoogle Scholar
Li, F., Baptista, R. and Marzouk, Y. (2024a), Expected information gain estimation via density approximations: Sample allocation and dimension reduction. Forthcoming.Google Scholar
Li, M. T. C., Marzouk, Y. and Zahm, O. (2024b), Principal feature detection via ϕ-Sobolev inequalities. To appear in Bernoulli. Available at https://bernoullisociety.org/publications/ bernoulli-journal/bernoulli-journal-papers.CrossRefGoogle Scholar
Liepe, J., Filippi, S., Komorowski, M. and Stumpf, M. P. H. (2013), Maximizing the information content of experiments in systems biology, PLoS Comput. Biol. 9, e1002888.CrossRefGoogle ScholarPubMed
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D. and Wierstra, D. (2016), Continuous control with deep reinforcement learning, in Proceedings of the 4th International Conference on Learning Representations (ICLR 2016) (Bengio, Y. and LeCun, Y., eds).Google Scholar
Lindley, D. V. (1956), On a measure of the information provided by an experiment, Ann. Math. Statist. 27, 986–1005.CrossRefGoogle Scholar
Loève, M. (1948), Fonctions aléatoires du second ordre, in Processus Stochastique et Mouvement Brownien (Lévy, P., ed.), Gauthier-Villars.Google Scholar
Long, Q., Scavino, M., Tempone, R. and Wang, S. (2013), Fast estimation of expected information gains for Bayesian experimental designs based on Laplace approximations, Comput. Methods Appl. Mech. Engrg 259, 24–39.CrossRefGoogle Scholar
Loredo, T. J. (2011), Rotating stars and revolving planets: Bayesian exploration of the pulsating sky, in Bayesian Statistics 9: Proceedings of the Ninth Valencia International Meeting (Bernardo, J. M., Bayarri, M. J. and Berger, J. O., eds), Oxford University Press, pp. 361–392.CrossRefGoogle Scholar
Lovász, L. (1983), Submodular functions and convexity, in Mathematical Programming The State of the Art: Bonn 1982 (Bachem, A., Korte, B. and Grötschel, M., eds), Springer, pp. 235–257.CrossRefGoogle Scholar
Lovász, L. (2007), Combinatorial Problems and Exercises , second edition, American Mathematical Society.Google Scholar
MacKay, D. J. C. (1992), Information-based objective functions for active data selection, Neural Comput. 4, 590–604.CrossRefGoogle Scholar
Madan, V., Nikolov, A., Singh, M. and Tantipongpipat, U. (2020), Maximizing determinants under matroid constraints, in 2020 IEEE 61st Annual Symposium on Foundations of Computer Science (FOCS) , IEEE, pp. 565–576.Google Scholar
Madan, V., Singh, M., Tantipongpipat, U. and Xie, W. (2019), Combinatorial algorithms for optimal design, in Proceedings of the 32nd Conference on Learning Theory , Vol. 99 of Proceedings of Machine Learning Research, PMLR, pp. 2210–2258.Google Scholar
Mak, W.-K., Morton, D. P. and Wood, R. K. (1999), Monte Carlo bounding techniques for determining solution quality in stochastic programs, Oper. Res. Lett. 24, 47–56.CrossRefGoogle Scholar
Manole, T., Balakrishnan, S., Niles-Weed, J. and Wasserman, L. (2021), Plugin estimation of smooth optimal transport maps. Available at arXiv:2107.12364.Google Scholar
Markowitz, H. (1952), Portfolio selection, J. Finance 7, 77–91.Google Scholar
Marzouk, Y. and Xiu, D. (2009), A stochastic collocation approach to Bayesian inference in inverse problems, Commun. Comput. Phys. 6, 826–847.CrossRefGoogle Scholar
Marzouk, Y., Moselhy, T., Parno, M. and Spantini, A. (2016), Sampling via measure transport: An introduction, in Handbook of Uncertainty Quantification (Ghanem, R., Higdon, D. and Owhadi, H., eds), Springer, pp. 1–41.Google Scholar
McAllester, D. and Stratos, K. (2020), Formal limitations on the measurement of mutual information, in Proceedings of the 23rd International Conference on Artificial Intelligence and Statistics , Vol. 108 of Proceedings of Machine Learning Research, PMLR, pp. 875–884.Google Scholar
McGree, J., Drovandi, C. and Pettitt, A. (2012), A sequential Monte Carlo approach to the sequential design for discriminating between rival continuous data models. Technical report, Queensland University of Technology, Brisbane, Australia.Google Scholar
McKay, M. D., Beckman, R. J. and Conover, W. J. (1979), A comparison of three methods for selecting values of input variables in the analysis of output from a computer code, Technometrics 21, 239–245.Google Scholar
Mertikopoulos, P., Hallak, N., Kavis, A. and Cevher, V. (2020), On the almost sure convergence of stochastic gradient descent in non-convex problems, in Advances in Neural Information Processing Systems 33 (Larochelle, H. et al., eds), Curran Associates, pp. 1117–1128.Google Scholar
Meyer, R. K. and Nachtsheim, C. J. (1995), The coordinate-exchange algorithm for constructing exact optimal experimental designs, Technometrics 37, 60–69.CrossRefGoogle Scholar
Miller, J. W. and Dunson, D. B. (2019), Robust Bayesian inference via coarsening, J. Amer. Statist. Assoc. 114, 1113–1125.CrossRefGoogle ScholarPubMed
Mirzasoleiman, B., Badanidiyuru, A., Karbasi, A., Vondrák, J. and Krause, A. (2015), Lazier than lazy greedy, in Proceedings of the 29th AAAI Conference on Artificial Intelligence (AAAI 2015) , pp. 1812–1818.Google Scholar
Mirzasoleiman, B., Karbasi, A., Sarkar, R. and Krause, A. (2013), Distributed submodular maximization: Identifying representative elements in massive data, in Advances in Neural Information Processing Systems 26 (Burges, C. J. et al., eds), Curran Associates, pp. 2049–2057.Google Scholar
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., Petersen, S., Beattie, C., Sadik, A., Antonoglou, I., King, H., Kumaran, D., Wierstra, D., Legg, S. and Hassabis, D. (2015), Human-level control through deep reinforcement learning, Nature 518, 529–533.CrossRefGoogle ScholarPubMed
Močkus, J. (1975), On Bayesian methods for seeking the extremum, in Optimization Techniques IFIP Technical Conference (Marchuk, G. I., ed.), Springer, pp. 400–404.CrossRefGoogle Scholar
Mohamed, S., Rosca, M., Figurnov, M. and Mnih, A. (2020), Monte Carlo gradient estimation in machine learning, J. Mach. Learn. Res. 21, 5183–5244.Google Scholar
Morrison, R. E., Oliver, T. A. and Moser, R. D. (2018), Representing model inadequacy: A stochastic operator approach, SIAM/ASA J. Uncertain. Quantif. 6, 457–496.CrossRefGoogle Scholar
Müller, P., Berry, D. A., Grieve, A. P., Smith, M. and Krams, M. (2007), Simulation-based sequential Bayesian design, J. Statist. Plann. Infer. 137, 3140–3150.CrossRefGoogle Scholar
Müller, P., Duan, Y. and Tec, M. Garcia (2022), Simulation-based sequential design, Pharma. Statist. 21, 729–739.CrossRefGoogle ScholarPubMed
Murphy, S. A. (2003), Optimal dynamic treatment regimes, J. R. Statist. Soc. Ser. B. Statist. Methodol. 65, 331–366.CrossRefGoogle Scholar
Myung, J. I. and Pitt, M. A. (2009), Optimal experimental design for model discrimination, Psychol. Rev. 116, 499–518.CrossRefGoogle ScholarPubMed
Nelder, J. A. and Mead, R. (1965), A simplex method for function minimization, Comput. J. 7, 308–313.CrossRefGoogle Scholar
Nemhauser, G. L. and Wolsey, L. A. (1978), Best algorithms for approximating the maximum of a submodular set function, Math. Oper. Res. 3, 177–188.CrossRefGoogle Scholar
Nemhauser, G. L., Wolsey, L. A. and Fisher, M. L. (1978), An analysis of approximations for maximizing submodular set functions I, Math. Program. 14, 265–294.CrossRefGoogle Scholar
Nesterov, Y. (1983), A method of solving a convex programming problem with convergence rate $\mathcal{O}\left(1/{k}^2\right)$ , Soviet Math. Doklady 27, 372–376.Google Scholar
Ng, A. and Russell, S. (2000), Algorithms for inverse reinforcement learning, in Proceedings of the 17th International Conference on Machine Learning (ICML 2000) , Morgan Kaufmann, pp. 663–670.Google Scholar
Nguyen, X., Wainwright, M. J. and Jordan, M. I. (2010), Estimating divergence functionals and the likelihood ratio by convex risk minimization, IEEE Trans. Inform. Theory 56, 5847–5861.CrossRefGoogle Scholar
Niederreiter, H. (1992), Random Number Generation and Quasi-Monte Carlo Methods , SIAM.CrossRefGoogle Scholar
Nikolov, A. and Singh, M. (2016), Maximizing determinants under partition constraints, in Proceedings of the 48th Annual ACM Symposium on Theory of Computing (STOC 2016) , ACM, pp. 192–201.Google Scholar
Nikolov, A., Singh, M. and Tantipongpipat, U. (2022), Proportional volume sampling and approximation algorithms for A-optimal design, Math. Oper. Res. 47, 847–877.CrossRefGoogle Scholar
Nocedal, J. and Wright, S. J. (2006), Numerical Optimization, Springer.Google Scholar
Norkin, V., Pflug, G. and Ruszczynski, A. (1998), A branch and bound method for stochastic global optimization, Math. Program. 83, 425–450.CrossRefGoogle Scholar
O’Hagan, A., Buck, C. E., Daneshkhah, A., Eiser, J. R., Garthwaite, P. H., Jenkinson, D. J., Oakley, J. E. and Rakow, T. (2006), Uncertain Judgements: Eliciting Experts’ Probabilities , Wiley.CrossRefGoogle Scholar
Orozco, R., Herrmann, F. J. and Chen, P. (2024), Probabilistic Bayesian optimal experimental design using conditional normalizing flows. Available at arXiv:2402.18337.Google Scholar
Overstall, A. M. (2022), Properties of Fisher information gain for Bayesian design of experiments, J. Statist. Plann. Infer. 218, 138–146.CrossRefGoogle Scholar
Overstall, A. M. and Woods, D. C. (2017), Bayesian design of experiments using approximate coordinate exchange, Technometrics 59, 458–470.CrossRefGoogle Scholar
Overstall, A. M., McGree, J. M. and Drovandi, C. C. (2018), An approach for finding fully Bayesian optimal designs using normal-based approximations to loss functions, Statist. Comput. 28, 343–358.CrossRefGoogle Scholar
Owen, A. B. (1992), Orthogonal arrays for computer experiments, integration and visualization, Statist. Sinica 2, 439–452.Google Scholar
Owen, A. B. (2013), Monte Carlo theory, methods and examples. Available at https://artowen.su.domains/mc/.Google Scholar
Papadimitriou, C. H. and Steiglitz, K. (1998), Combinatorial Optimization: Algorithms and Complexity , Courier Corporation.Google Scholar
Papamakarios, G., Nalisnick, E., Rezende, D. J., Mohamed, S. and Lakshminarayanan, B. (2021), Normalizing flows for probabilistic modeling and inference, J. Mach. Learn. Res. 22, 2617–2680.Google Scholar
Pardo-Igúzquiza, E. (1998), Maximum likelihood estimation of spatial covariance parameters, Math. Geol. 30, 95–108.CrossRefGoogle Scholar
Peters, J. and Schaal, S. (2008), Natural actor–critic, Neurocomput. 71, 1180–1190.CrossRefGoogle Scholar
Pilz, J. (1991), Bayesian Estimation and Experimental Design in Linear Regression Models , Wiley.Google Scholar
Polyak, B. T. and Juditsky, A. B. (1992), Acceleration of stochastic approximation by averaging, SIAM J. Control Optim. 30, 838–855.CrossRefGoogle Scholar
Pompe, E. and Jacob, P. E. (2021), Asymptotics of cut distributions and robust modular inference using posterior bootstrap. Available at arXiv:2110.11149.Google Scholar
Pooladian, A.-A. and Niles-Weed, J. (2021), Entropic estimation of optimal transport maps. Available at arXiv:2109.12004.Google Scholar
Poole, B., Ozair, S., Van Den Oord, A., Alemi, A. and Tucker, G. (2019), On variational bounds of mutual information, in Proceedings of the 36th International Conference on Machine Learning (ICML 2019) , Vol. 97 of Proceedings of Machine Learning Research, PMLR, pp. 5171–5180.Google Scholar
Powell, W. B. (2011), Approximate Dynamic Programming: Solving the Curses of Dimensionality , second edition, Wiley.CrossRefGoogle Scholar
Prangle, D., Harbisher, S. and Gillespie, C. S. (2023), Bayesian experimental design without posterior calculations: An adversarial approach, Bayesian Anal. 18, 133–163.CrossRefGoogle Scholar
Pronzato, L. and Müller, W. G. (2012), Design of computer experiments: Space filling and beyond, Statist. Comput. 22, 681–701.CrossRefGoogle Scholar
Pronzato, L. and Thierry, É. (2002), Sequential experimental design and response optimisation, Statist. Methods Appl. 11, 277–292.CrossRefGoogle Scholar
Pronzato, L. and Walter, E. (1985), Robust experiment design via stochastic approximation, Math. Biosci. 75, 103–120.CrossRefGoogle Scholar
Pukelsheim, F. (2006), Optimal Design of Experiments , SIAM.CrossRefGoogle Scholar
Rahimian, H. and Mehrotra, S. (2019), Distributionally robust optimization: A review. Available at arXiv:1908.05659.Google Scholar
Raiffa, H. and Schlaifer, R. (1961), Applied Statistical Decision Theory , Wiley.Google Scholar
Rainforth, T., Cornish, R., Yang, H., Warrington, A. and Wood, F. (2018), On nesting Monte Carlo estimators, in Proceedings of the 35th International Conference on Machine Learning (ICML 2018) , Vol. 80 of Proceedings of Machine Learning Research, PMLR, pp. 4267–4276.Google Scholar
Rainforth, T., Foster, A., Ivanova, D. R. and Smith, F. B. (2023), Modern Bayesian experimental design, Statist. Sci. 39, 100–114.Google Scholar
Rasmussen, C. E. and Williams, C. K. I. (2006), Gaussian Processes for Machine Learning , MIT Press.Google Scholar
Rhee, C. H. and Glynn, P. W. (2015), Unbiased estimation with square root convergence for SDE models, Oper. Res. 63, 1026–1043.CrossRefGoogle Scholar
Riis, C., Antunes, F., Hüttel, F., Azevedo, C. Lima and Pereira, F. (2022), Bayesian active learning with fully Bayesian Gaussian processes, in Advances in Neural Information Processing Systems 35 (Koyejo, S. et al., eds), Curran Associates, pp. 12141–12153.Google Scholar
Riley, Z. B., Perez, R. A., Bartram, G. W., Spottswood, S. M., Smarslok, B. P. and Beberniss, T. J. (2019), Aerothermoelastic experimental design for the AEDC/VKF Tunnel C: Challenges associated with measuring the response of flexible panels in high-temperature, high-speed wind tunnels, J. Sound Vib. 441, 96–105.CrossRefGoogle Scholar
Robbins, H. and Monro, S. (1951), A stochastic approximation method, Ann. Math. Statist. 22, 400–407.CrossRefGoogle Scholar
Robertazzi, T. and Schwartz, S. (1989), An accelerated sequential algorithm for producing D-optimal designs, SIAM J. Sci. Statist. Comput. 10, 341–358.CrossRefGoogle Scholar
Rockafellar, R. T. and Royset, J. O. (2015), Measures of residual risk with connections to regression, risk tracking, surrogate models, and ambiguity, SIAM J. Optim. 25, 1179–1208.CrossRefGoogle Scholar
Rockafellar, R. T. and Uryasev, S. (2002), Conditional value-at-risk for general loss distributions, J. Bank. Finance 26, 1443–1471.CrossRefGoogle Scholar
Rockafellar, R. T. and Uryasev, S. (2013), The fundamental risk quadrangle in risk management, optimization and statistical estimation, Surv. Oper. Res. Manag. Sci. 18, 33–53.Google Scholar
Rosenblatt, M. (1952), Remarks on a multivariate transformation, Ann. Math. Statist. 23, 470–472.CrossRefGoogle Scholar
Royset, J. O. (2022), Risk-adaptive approaches to learning and decision making: A survey. Available at arXiv:2212.00856.Google Scholar
Rudolf, D. and Sprungk, B. (2018), On a generalization of the preconditioned Crank–Nicolson Metropolis algorithm, Found. Comput. Math. 18, 309–343.CrossRefGoogle Scholar
Ruppert, D. (1988), Efficient estimations from a slowly convergent Robbins–Monro process. Technical report, Cornell University. Available at http://ecommons.cornell.edu/bitstream/handle/1813/8664/TR000781.pdf?sequence=1http://ecommons.cornell.edu/bitstream/handle/1813/8664/TR000781.pdf?sequence=1.Google Scholar
Ruthotto, L., Chung, J. and Chung, M. (2018), Optimal experimental design for inverse problems with state constraints, SIAM J. Sci. Comput. 40, B1080–B1100.CrossRefGoogle Scholar
Ryan, E. G., Drovandi, C. C., McGree, J. M. and Pettitt, A. N. (2016), A review of modern computational algorithms for Bayesian optimal design, Int. Statist. Rev. 84, 128–154.CrossRefGoogle Scholar
Ryan, K. J. (2003), Estimating expected information gains for experimental designs with application to the random fatigue-limit model, J. Comput. Graph. Statist. 12, 585–603.CrossRefGoogle Scholar
Sacks, J., Welch, W. J., Mitchell, T. J. and Wynn, H. P. (1989), Design and analysis of computer experiments, Statist. Sci. 4, 118–128.Google Scholar
Santambrogio, F. (2015), Optimal Transport for Applied Mathematicians: Calculus of Variations, PDEs, and Modelin g, Vol. 87 of Progress in Nonlinear Differential Equations and their Applications, Springer.CrossRefGoogle Scholar
Santner, T. J., Williams, B. J. and Notz, W. I. (2018), The Design and Analysis of Computer Experiments , second edition, Springer.CrossRefGoogle Scholar
Sargsyan, K., Huan, X. and Najm, H. N. (2019), Embedded model error representation for Bayesian model calibration, Int. J. Uncertain. Quantif. 9, 365–394.CrossRefGoogle Scholar
Sargsyan, K., Najm, H. N. and Ghanem, R. G. (2015), On the statistical calibration of physical models, Int. J. Chem. Kinet. 47, 246–276.CrossRefGoogle Scholar
Schein, A. I. and Ungar, L. H. (2007), Active learning for logistic regression: An evaluation, Mach. Learn. 68, 235–265.CrossRefGoogle Scholar
Schillings, C. and Schwab, C. (2016), Scaling limits in computational Bayesian inversion, ESAIM Math. Model. Numer. Anal. 50, 1825–1856.CrossRefGoogle Scholar
Schillings, C., Sprungk, B. and Wacker, P. (2020), On the convergence of the Laplace approximation and noise-level-robustness of Laplace-based Monte Carlo methods for Bayesian inverse problems, Numer. Math. 145, 915–971.CrossRefGoogle Scholar
Schrijver, A. (2003), Combinatorial Optimization: Polyhedra and Efficiency, Springer.Google Scholar
Sebastiani, P. and Wynn, H. P. (2000), Maximum entropy sampling and optimal Bayesian experimental design, J. R. Statist. Soc. Ser. B. Statist. Methodol. 62, 145–157.CrossRefGoogle Scholar
Seo, S., Wallat, M., Graepel, T. and Obermayer, K. (2000), Gaussian process regression: Active data selection and test point rejection, in Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks (IJCNN 2000) , Springer, pp. 27–34.Google Scholar
Settles, B. (2009), Active learning literature survey. Computer Sciences Technical Report 1648, University of Wisconsin–Madison.Google Scholar
Shah, K. R. and Sinha, B. K. (1989), Theory of Optimal Designs , Vol. 54 of Lecture Notes in Statistics, Springer.CrossRefGoogle Scholar
Shahriari, B., Swersky, K., Wang, Z., Adams, R. P. and de Freitas, N. (2016), Taking the human out of the loop: A review of Bayesian optimization, Proc. IEEE 104, 148–175.CrossRefGoogle Scholar
Shapiro, A. (1991), Asymptotic analysis of stochastic programs, Ann . Oper. Res. 30, 169–186.CrossRefGoogle Scholar
Shapiro, A., Dentcheva, D. and Ruszczynski, A. (2021), Lectures on Stochastic Programming: Modeling and Theory , third edition, SIAM.CrossRefGoogle Scholar
Shashaani, S., Hashemi, F. S. and Pasupathy, R. (2018), ASTRO-DF: A class of adaptive sampling trust-region algorithms for derivative-free stochastic optimization, SIAM J. Optim. 28, 3145–3176.CrossRefGoogle Scholar
Shen, W. (2023), Reinforcement learning based sequential and robust Bayesian optimal experimental design. PhD thesis, University of Michigan.Google Scholar
Shen, W. and Huan, X. (2021), Bayesian sequential optimal experimental design for nonlinear models using policy gradient reinforcement learning. Available at arXiv:2110.15335.Google Scholar
Shen, W. and Huan, X. (2023), Bayesian sequential optimal experimental design for nonlinear models using policy gradient reinforcement learning, Comput. Methods Appl. Mech. Engrg 416, art. 116304.CrossRefGoogle Scholar
Shen, W., Dong, J. and Huan, X. (2023), Variational sequential optimal experimental design using reinforcement learning. Available at arXiv:2306.10430.Google Scholar
Shewry, M. C. and Wynn, H. P. (1987), Maximum entropy sampling, J. Appl. Statist. 14, 165–170.CrossRefGoogle Scholar
Siade, A. J., Hall, J. and Karelse, R. N. (2017), A practical, robust methodology for acquiring new observation data using computationally expensive groundwater models, Water Resour. Res. 53, 9860–9882.CrossRefGoogle Scholar
Silver, D., Lever, G., Heess, N., Degris, T., Wierstra, D. and Riedmiller, M. (2014), Deterministic policy gradient algorithms, in Proceedings of the 31st International Conference on Machine Learning (ICML 2014) , Vol. 32 of Proceedings of Machine Learning Research, PMLR, pp. 387–395.Google Scholar
Singh, M. and Xie, W. (2020), Approximation algorithms for D-optimal design, Math. Oper. Res. 45, 1512–1534.CrossRefGoogle Scholar
Solonen, A., Haario, H. and Laine, M. (2012), Simulation-based optimal design using a response variance criterion, J. Comput. Graph. Statist. 21, 234–252.CrossRefGoogle Scholar
Song, J. and Ermon, S. (2020), Understanding the limitations of variational mutual information estimators, in Proceedings of the 8th International Conference on Learning Representations (ICLR 2020). Available at https://openreview.net/forum?id=B1x62TNtDS.Google Scholar
Song, J., Chen, Y. and Yue, Y. (2019), A general framework for multi-fidelity Bayesian optimization with Gaussian processes, in Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics , Vol. 89 of Proceedings of Machine Learning Research, PMLR, pp. 3158–3167.Google Scholar
Spall, J. C. (1998a), Implementation of the simultaneous perturbation algorithm for stochastic optimization, IEEE Trans. Aerosp. Electron. Systems 34, 817–823.CrossRefGoogle Scholar
Spall, J. C. (1998b), An overview of the simultaneous perturbation method for efficient optimization, Johns Hopkins APL Tech . Digest 19, 482–492.Google Scholar
Spantini, A., Bigoni, D. and Marzouk, Y. (2018), Inference via low-dimensional couplings, J. Mach. Learn. Res. 19, 2639–2709.Google Scholar
Spantini, A., Cui, T., Willcox, K., Tenorio, L. and Marzouk, Y. (2017), Goal-oriented optimal approximations of Bayesian linear inverse problems, SIAM J. Sci. Comput. 39, S167–S196.CrossRefGoogle Scholar
Spantini, A., Solonen, A., Cui, T., Martin, J., Tenorio, L. and Marzouk, Y. (2015), Optimal low-rank approximations of Bayesian linear inverse problems, SIAM J. Sci. Comput. 37, A2451–A2487.CrossRefGoogle Scholar
Spielman, D. A. and Srivastava, N. (2008), Graph sparsification by effective resistances, in Proceedings of the 40th Annual ACM Symposium on Theory of Computing (STOC 2008) , ACM, pp. 563–568.Google Scholar
Spöck, G. (2012), Spatial sampling design based on spectral approximations to the random field, Environ. Model. Softw. 33, 48–60.CrossRefGoogle Scholar
Spöck, G. and Pilz, J. (2010), Spatial sampling design and covariance-robust minimax prediction based on convex design ideas, Stoch. Env. Res. Risk Assessment 24, 463–482.Google Scholar
Spokoiny, V. (2023), Dimension free nonasymptotic bounds on the accuracy of high-dimensional Laplace approximation, SIAM/ASA J. Uncertain. Quantif. 11, 1044–1068.CrossRefGoogle Scholar
Sprungk, B. (2020), On the local Lipschitz stability of Bayesian inverse problems, Inverse Problems 36, art. 055015.CrossRefGoogle Scholar
Sriver, T. A., Chrissis, J. W. and Abramson, M. A. (2009), Pattern search ranking and selection algorithms for mixed variable simulation-based optimization, European J. Oper. Res. 198, 878–890.CrossRefGoogle Scholar
Steinberg, D. M. and Hunter, W. G. (1984), Experimental design: Review and comment, Technometrics 26, 71–97.CrossRefGoogle Scholar
Stone, M. (1959), Application of a measure of information to the design and comparison of regression experiments, Ann. Math. Statist. 30, 55–70.CrossRefGoogle Scholar
Strutz, D. and Curtis, A. (2024), Variational Bayesian experimental design for geophysical applications: Seismic source location, amplitude versus offset inversion, and estimating ${\mathrm{CO}}_2$ saturations in a subsurface reservoir, Geophys. J. Int. 236, 1309–1331.CrossRefGoogle Scholar
Stuart, A. and Teckentrup, A. (2018), Posterior consistency for Gaussian process approximations of Bayesian posterior distributions, Math. Comp. 87, 721–753.CrossRefGoogle Scholar
Stuart, A. M. (2010), Inverse problems: A Bayesian perspective, Acta Numer. 19, 451–559.CrossRefGoogle Scholar
Sun, N.-Z. and Yeh, W. W. (2007), Development of objective-oriented groundwater models, 2: Robust experimental design, Water Resour. Res. 43, 1–14.Google Scholar
Sutton, R. S. and Barto, A. G. (2018), Reinforcement Leaning , second edition, MIT Press.Google Scholar
Sutton, R. S., McAllester, D., Singh, S. P. and Mansour, Y. (1999), Policy gradient methods for reinforcement learning with function approximation, in Advances in Neural Information Processing Systems 12 (Solla, S. et al., eds), MIT Press, pp. 1057–1063.Google Scholar
Sviridenko, M. (2004), A note on maximizing a submodular set function subject to a knapsack constraint, Oper. Res. Lett. 32, 41–43.CrossRefGoogle Scholar
Sviridenko, M., Vondrák, J. and Ward, J. (2017), Optimal approximation for submodular and supermodular optimization with bounded curvature, Math. Oper. Res. 42, 1197–1218.CrossRefGoogle Scholar
Tang, B. (1993), Orthogonal array-based Latin hypercubes, J. Amer. Statist. Assoc. 88, 1392–1397.CrossRefGoogle Scholar
Tec, M., Duan, Y. and Müller, P. (2023), A comparative tutorial of Bayesian sequential design and reinforcement learning, Amer. Statist. 77, 223–233.CrossRefGoogle Scholar
Terejanu, G., Upadhyay, R. R. and Miki, K. (2012), Bayesian experimental design for the active nitridation of graphite by atomic nitrogen, Experimental Therm. Fluid Sci. 36, 178–193.CrossRefGoogle Scholar
Torczon, V. (1997), On the convergence of pattern search algorithms, SIAM J. Optim. 7, 1–25.CrossRefGoogle Scholar
Tzoumas, V., Carlone, L., Pappas, G. J. and Jadbabaie, A. (2021), LQG control and sensing co-design, IEEE Trans. Automat. Control 66, 1468–1483.CrossRefGoogle Scholar
van den Oord, A., Li, Y. and Vinyals, O. (2018), Representation learning with contrastive predictive coding. Available at arXiv:1807.03748.Google Scholar
Vazirani, V. V. (2001), Approximation Algorithms , Springer.Google Scholar
Villa, U., Petra, N. and Ghattas, O. (2021), HIPPYlib: An extensible software framework for large-scale inverse problems governed by PDEs I: Deterministic inversion and linearized Bayesian inference, ACM Trans. Math. Software 47, 1–34.CrossRefGoogle Scholar
Villani, C. (2009), Optimal Transport: Old and New, Springer.CrossRefGoogle Scholar
von Toussaint, U. (2011), Bayesian inference in physics, Rev. Modern Phys. 83, 943–999.CrossRefGoogle Scholar
Vondrák, J. (2008), Optimal approximation for the submodular welfare problem in the value oracle model, in Proceedings of the Fortieth Annual ACM Symposium on Theory of Computing (STOC 2008) , ACM, pp. 67–74.Google Scholar
Vondrák, J. (2010), Submodularity and curvature: The optimal algorithm, RIMS Kokyuroku Bessatsu B23, 253–266.Google Scholar
Vondrák, J., Chekuri, C. and Zenklusen, R. (2011), Submodular function maximization via the multilinear relaxation and contention resolution schemes, in Proceedings of the Forty-Third Annual ACM Symposium on Theory of Computing (STOC 2011) , ACM, pp. 783–792.Google Scholar
Wald, A. (1943), On the efficient design of statistical investigations, Ann. Math. Statist. 14, 134–140.CrossRefGoogle Scholar
Walker, S. G. (2016), Bayesian information in an experiment and the Fisher information distance, Statist. Probab. Lett. 112, 5–9.CrossRefGoogle Scholar
Wang, J., Clark, S. C., Liu, E. and Frazier, P. I. (2020), Parallel Bayesian global optimization of expensive functions, Oper. Res. 68, 1850–1865.CrossRefGoogle Scholar
Wang, S. and Marzouk, Y. (2022), On minimax density estimation via measure transport. Available at arXiv:2207.10231.Google Scholar
Wang, X., Jin, Y., Schmitt, S. and Olhofer, M. (2023), Recent advances in Bayesian optimization, ACM Comput. Surv. 55, 1–36.Google Scholar
Wathen, J. K. and Christen, J. A. (2006), Implementation of backward induction for sequentially adaptive clinical trials, J. Comput. Graph. Statist. 15, 398–413.CrossRefGoogle Scholar
Weaver, B. P. and Meeker, W. Q. (2021), Bayesian methods for planning accelerated repeated measures degradation tests, Technometrics 63, 90–99.CrossRefGoogle Scholar
Weaver, B. P., Williams, B. J., Anderson-Cook, C. M. and Higdon, D. M. (2016), Computational enhancements to Bayesian design of experiments using Gaussian processes, Bayesian Anal. 11, 191–213.CrossRefGoogle Scholar
Whittle, P. (1973), Some general points in the theory of optimal experimental design, J. R. Statist. Soc. Ser. B. Statist. Methodol. 35, 123–130.CrossRefGoogle Scholar
Wolsey, L. A. and Nemhauser, G. L. (1999), Integer and Combinatorial Optimization , Wiley.Google Scholar
Wu, J. and Frazier, P. (2019), Practical two-step lookahead Bayesian optimization, in Advances in Neural Information Processing Systems 32 (Wallach, H. et al., eds), Curran Associates, pp. 9813–9823.Google Scholar
Wu, K., Chen, P. and Ghattas, O. (2023a), An offline–online decomposition method for efficient linear Bayesian goal-oriented optimal experimental design: Application to optimal sensor placement, SIAM J. Sci. Comput. 45, B57–B77.CrossRefGoogle Scholar
Wu, K., O’Leary-Roseberry, T., Chen, P. and Ghattas, O. (2023b), Large-scale Bayesian optimal experimental design with derivative-informed projected neural network, J. Sci. Comput. 95, art. 30.CrossRefGoogle Scholar
Wynn, H. P. (1972), Results in the theory and construction of D-optimum experimental designs, J. R. Statist. Soc. Ser. B. Statist. Methodol. 34, 133–147.CrossRefGoogle Scholar
Wynn, H. P. (1984), Jack Kiefer’s contributions to experimental design, Ann. Statist. 12, 416–423.CrossRefGoogle Scholar
Xu, Z. and Liao, Q. (2020), Gaussian process based expected information gain computation for Bayesian optimal design, Entropy 22, art. 258.CrossRefGoogle ScholarPubMed
Yates, F. (1933), The principles of orthogonality and confounding in replicated experiments, J. Agric. Sci. 23, 108–145.CrossRefGoogle Scholar
Yates, F. (1937), The design and analysis of factorial experiments. Technical Communication no. 35, Imperial Bureau of Soil Science.Google Scholar
Yates, F. (1940), Lattice squares, J. Agric. Sci. 30, 672–687.CrossRefGoogle Scholar
Zahm, O., Cui, T., Law, K., Spantini, A. and Marzouk, Y. (2022), Certified dimension reduction in nonlinear Bayesian inverse problems, Math. Comp. 91, 1789–1835.CrossRefGoogle Scholar
Zhan, D. and Xing, H. (2020), Expected improvement for expensive optimization: A review, J. Global Optim. 78, 507–544.CrossRefGoogle Scholar
Zhang, J., Bi, S. and Zhang, G. (2021), A scalable gradient-free method for Bayesian experimental design with implicit models, in Proceedings of the 24th International Conference on Artificial Intelligence and Statistics , Vol. 130 of Proceedings of Machine Learning Research, PMLR, pp. 3745–3753.Google Scholar
Zhang, X., Blanchet, J., Marzouk, Y., Nguyen, V. A. and Wang, S. (2022), Distributionally robust Gaussian process regression and Bayesian inverse problems. Available at arXiv:2205.13111.Google Scholar
Zheng, S., Hayden, D., Pacheco, J. and Fisher, J. W. (2020), Sequential Bayesian experimental design with variable cost structure, in Advances in Neural Information Processing Systems 33 (Larochelle, H. et al., eds), Curran Associates, pp. 4127–4137.Google Scholar
Zhong, S., Shen, W., Catanach, T. and Huan, X. (2024), Goal-oriented Bayesian optimal experimental design for nonlinear models using Markov chain Monte Carlo. Available at arXiv:2403.18072.Google Scholar
Zhu, Z. and Stein, M. L. (2005), Spatial sampling design for parameter estimation of the covariance function, J. Statist. Plann. Infer. 134, 583–603.CrossRefGoogle Scholar
Zhu, Z. and Stein, M. L. (2006), Spatial sampling design for prediction with estimated parameters, J. Agric. Biol. Environ. Statist. 11, 24–44.CrossRefGoogle Scholar
Zong, C. (1999), Sphere Packings, Springer.Google Scholar