Hostname: page-component-76d6cb85b7-lcgwf Total loading time: 0 Render date: 2026-07-19T06:23:30.613Z Has data issue: false hasContentIssue false
  • English
  • Français

Information-theoretic convergence of extreme values to the Gumbel distribution

Part of: Communication, information Limit theorems Sufficiency and information

Published online by Cambridge University Press:  21 June 2023

Oliver Johnson Open the ORCID record for Oliver Johnson [Opens in a new window]
Oliver Johnson*
Affiliation:
University of Bristol
*
*Postal address: School of Mathematics, University of Bristol, Fry Building, Woodland Road, Bristol, BS8 1UG, UK. Email: O.Johnson@bristol.ac.uk
Get access Rights & Permissions [Opens in a new window]

Abstract

We show how convergence to the Gumbel distribution in an extreme value setting can be understood in an information-theoretic sense. We introduce a new type of score function which behaves well under the maximum operation, and which implies simple expressions for entropy and relative entropy. We show that, assuming certain properties of the von Mises representation, convergence to the Gumbel distribution can be proved in the strong sense of relative entropy.

Keywords

entropyextreme value theoryinformation theoryvon Mises representation

MSC classification

Primary: 60F99: None of the above, but in this section
Secondary: 62B10: Information-theoretic topics 94A15: Information theory, general

Information

Type
Original Article
Information
Journal of Applied Probability , Volume 61 , Issue 1 , March 2024 , pp. 244 - 254
Check for updates
Copyright
© The Author(s), 2023. Published by Cambridge University Press on behalf of Applied Probability Trust

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

Arnold, B. C., Balakrishnan, N. and Nagaraja, H. N. (2008). A First Course in Order Statistics. SIAM, Philadelphia.CrossRefGoogle Scholar
Arulkumaran, K., Deisenroth, M. P., Brundage, M. and Bharath, A. A. (2017). A brief survey of deep reinforcement learning. Preprint, arXiv:1708.05866.Google Scholar
Barron, A. R. (1986). Entropy and the central limit theorem. Ann. Prob. 14, 336–342.CrossRefGoogle Scholar
Bartholmé, C. and Swan, Y. (2013). Rates of convergence towards the Fréchet distribution. Preprint, arXiv:1311.3896.Google Scholar
Bercher, J.-F. and Vignat, C. (2008). An entropic view of Pickands’ theorem. In Proc. 2008 IEEE Int. Symp. Information Theory (ISIT). IEEE, New York, pp. 2625–2628.CrossRefGoogle Scholar
Blachman, N. M. (1965). The convolution inequality for entropy powers. IEEE Trans. Inf. Theory 11, 267–271.CrossRefGoogle Scholar
Block, H. W., Savits, T. H. and Singh, H. (1998). The reversed hazard rate function. Prob. Eng. Inf. Sci. 12, 69–90.CrossRefGoogle Scholar
Brown, L. D. (1982). A proof of the central limit theorem motivated by the Cramér–Rao inequality. In Statistics and Probability: Essays in Honour of C. R. Rao, eds G. Kallianpur, P. R. Krishnaiah and J. K. Ghosh. North-Holland, New York, pp. 141–148.Google Scholar
Cardone, M., Dytso, A. and Rush, C. (2023). Entropic central limit theorem for order statistics. IEEE Trans. Inf. Theory 69, 2193–2205.CrossRefGoogle Scholar
Cover, T. M. and Thomas, J. A. (1991). Elements of Information Theory. John Wiley, New York.Google Scholar
De Haan, L. and Ferreira, A. (2007). Extreme Value Theory: An Introduction. Springer, New York.Google Scholar
De Haan, L. and Resnick, S. I. (1982). Local limit theorems for sample extremes. Ann. Prob. 396–413.Google Scholar
Embrechts, P., Klüppelberg, C. and Mikosch, T. (2013). Modelling Extremal Events: For Insurance and Finance (Stochastic Modelling and Appl. Prob. 33). Springer, New York.Google Scholar
Gnedenko, B. (1943). Sur la distribution limite du terme maximum d’une série aléatoire. Ann. Math. 44, 423–453.CrossRefGoogle Scholar
Johnson, O. T. (2004). Information Theory and the Central Limit Theorem. Imperial College Press, London.CrossRefGoogle Scholar
Johnson, O. T. (2020). Maximal correlation and the rate of Fisher information convergence in the central limit theorem. IEEE Trans. Inf. Theory 66, 4992–5002.CrossRefGoogle Scholar
Johnson, O. T. and Barron, A. R. (2004). Fisher information inequalities and the central limit theorem. Prob. Theory Relat. Fields 129, 391–409.CrossRefGoogle Scholar
Kontoyiannis, I., Harremoës, P. and Johnson, O. T. (2005). Entropy and the law of small numbers. IEEE Trans. Inf. Theory 51, 466–472.CrossRefGoogle Scholar
Kotz, S. and Nadarajah, S. (2000). Extreme Value Distributions: Theory and Applications. World Scientific, Singapore.CrossRefGoogle Scholar
Kullback, S. (1967). A lower bound for discrimination information in terms of variation. IEEE Trans. Inf. Theory 13, 126–127.CrossRefGoogle Scholar
Pickands, III, J. (1986). The continuous and differentiable domains of attraction of the extreme value distributions. Ann. Prob. 14, 996–1004.Google Scholar
Ravi, S. and Saeb, A. (2012). A note on entropies of l-max stable, p-max stable, generalized Pareto and generalized log-Pareto distributions. ProbStat Forum 5, 62–79.Google Scholar
Ravi, S. and Saeb, A. (2014). On convergence of entropy of distribution functions in the max domain of attraction of max stable laws. Preprint, arXiv:1402.0277.Google Scholar
Reiss, R.-D. (2012). Approximate Distributions of Order Statistics: With Applications to Nonparametric Statistics. Springer, New York.Google Scholar
Resnick, S. I. (2013). Extreme Values, Regular Variation and Point Processes. Springer, New York.Google Scholar
Saeb, A. (2014). Rates of convergence for Rényi entropy in extreme value theory. Preprint, arXiv:1402.4316.Google Scholar
Stam, A. J. (1959). Some inequalities satisfied by the quantities of information of Fisher and Shannon. Inf. Control 2, 101–112.CrossRefGoogle Scholar
Stein, C. (1972). A bound for the error in the normal approximation to the distribution of a sum of dependent random variables. In Proc. 6th Berkeley Symp. Math. Statist. Prob., Vol. 2. University of California Press, Berkeley, CA.Google Scholar
Wong, K. M. and Chen, S. (1990). The entropy of ordered sequences and order statistics. IEEE Trans. Inf. Theory 36, 276–284.CrossRefGoogle Scholar