Skip to main content Accessibility help

We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.

Close cookie message
×
Hostname: page-component-cd9895bd7-fscjk Total loading time: 0 Render date: 2024-12-25T04:52:02.043Z Has data issue: false hasContentIssue false

14 - Sorting and Fast Algorithms

Published online by Cambridge University Press:  01 June 2011

John F. Monahan
John F. Monahan
Affiliation:
North Carolina State University
Get access

Summary

Introduction

The theme of this chapter is a simple one: there may be better, faster ways of computing something than you may ever have thought of. One of the maxims of computer science is that just a few programs use most of the resources. Early in the history of computing, people recognized that much of the computing resources went into the common task of sorting a list of numbers. If this task could be done faster, then everything could be done better. A concentrated effort on improving sorting algorithms led to several breakthroughs, all following the principle known as “divide and conquer.” In simple terms, to solve a large task, break it into smaller tasks of the same kind. Cleverly done, the resulting algorithm can be more efficient than anyone would have expected, earning the jargon adjective “fast.” The principle of divide and conquer will be discussed in the next section, followed by a discussion of fast algorithms for sorting. Section 14.4 comprises statistical applications of divide and conquer. Another great breakthrough, the fast Fourier transform (FFT), will be discussed in Section 14.5. Using the FFT to compute convolutions will be discussed in Section 14.6, followed by some interesting applications of the FFT to statistics in Section 14.7. This chapter will close with some topics that are important but don't really fit elsewhere: algorithms for constructing permutations and combinations.

Type
Chapter
Information
Numerical Methods of Statistics , pp. 403 - 438
Publisher: Cambridge University Press
Print publication year: 2011

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Aho, Alfred V., Hopcroft, John E., and Ullman, Jeffrey D. (1974), The Design and Analysis of Computer Algorithms. Reading, MA: Addison-Wesley.Google Scholar
Baglivo, Jenny, Olivier, Donald, and Pagano, Marcello (1992), “Methods for Exact Goodness-of-Fit Tests,” Journal of the American Statistical Association 87: 464–9.CrossRefGoogle Scholar
Bluestein, L. I. (1970), “A Linear Filtering Approach to the Computation of the Discrete Fourier Transform,” IEEE Transactions on Audio and Electroacoustics 18: 451–5.CrossRefGoogle Scholar
Bohman, Harald (1975), “Numerical Inversions of Characteristic Functions,” Scandinavian Actuarial Journal, pp. 121–4.CrossRefGoogle Scholar
Boothroyd, J. (1967), “Algorithm 29: Permutation of the Elements of a Vector,” Computer Journal 10: 311.Google Scholar
Cooley, J. W., Lewis, P. A., and Welch, P. D. (1967), “Historical Notes on the Fast Fourier Transform,” IEEE Transactions on Audio and Electroacoustics 15: 76–9.CrossRefGoogle Scholar
Cooley, J. W. and Tukey, J. W. (1965), “An Algorithm for the Machine Calculation of Complex Fourier Series,” Mathematics of Computation 19: 297–301.CrossRefGoogle Scholar
Cox, D. R. and Hinkley, D. V. (1974), Theoretical Statistics. London: Chapman & Hall.CrossRefGoogle Scholar
David, F. N. and Barton, D. E. (1962), Combinatorial Chance. New York: Hafner.Google Scholar
Davies, R. B. (1975), “Numerical Inversion of a Characteristic Function,” Biometrika 60: 415–17.CrossRefGoogle Scholar
Davies, Robert B. (1980), “ASS 155: The Distribution of a Linear Combination of Chi-Squared Random Variables,” Applied Statistics 29: 323–33.CrossRefGoogle Scholar
Good, I. J., Gover, T. N., and Mitchell, G. J. (1970), “Exact Distribution for χ2 and for the Likelihood-Ratio Statistic for the Equiprobable Multinomial Distribution,” Journal of the American Statistical Association 65: 267–83.Google Scholar
Hoare, C. A. R. (1961), “Algorithm 63 (PARTITION) and Algorithm 65 (FIND),” Communications of the ACM 4: 321–2.CrossRefGoogle Scholar
Hodges, J. L. and Lehmann, E. L. (1963), “Estimates of Location Based on Rank Tests,” Annals of Mathematical Statistics 34: 598–611.CrossRefGoogle Scholar
Imhof, J. P. (1961), “Computing the Distribution of Quadratic Forms in Normal Variables,” Biometrika 48: 419–26.CrossRefGoogle Scholar
Johnson, D. B. and Kashdan, S. D. (1978), “Lower Bounds for Selection in X + Y and Other Multisets,” Journal of the ACM 25: 556–70.CrossRefGoogle Scholar
Knott, Gary D. (1974), “A Numbering System for Combinations,” Communications of the ACM 17: 45–6.CrossRefGoogle Scholar
Knuth, Donald E. (1997), The Art of Computer Programming (vol. 2: Seminumerical Algorithms), 3rd ed. Reading, MA: Addison-Wesley.Google Scholar
Knuth, Donald E. (1998), The Art of Computer Programming (vol. 3: Sorting and Searching), 2nd ed. Reading, MA: Addison-Wesley.Google Scholar
Zeng-Hua, Lu and King, Maxwell L. (2002), “Improving the Numerical Technique for Computing the Accumulated Distribution of a Quadratic Form in Normal Variables,” Econometric Reviews 21: 149–65.Google Scholar
McKean, J. W. and Ryan, T. A. (1977), “Algorithm 516: An Algorithm for Obtaining Confidence Intervals and Point Estimates Based on Rank in a Two-Sample Location Problem,” ACM Transactions on Mathematical Software 3: 183–5.CrossRefGoogle Scholar
Mehta, Cyrus R. and Patel, Nitin R. (1986), “FEXACT: A Fortran Subroutine for Fisher's Exact Test on Unordered r × c Contingency Tables,” ACM Transactions on Mathematical Software 12: 154–61.CrossRefGoogle Scholar
Monahan, John F. (1984), “Fast Computation of the Hodges–Lehmann Location Estimator,” ACM Transactions on Mathematical Software 10: 265–70.CrossRefGoogle Scholar
Monahan, John F. (1987), “An Alternative Method for Computing Overflow Probabilities,” Communications in Statistics A 16: 3355–7.CrossRefGoogle Scholar
Nijenhius, A. and Wilf, H. S. (1978), Combinatorial Algorithms. New York: Academic Press.Google Scholar
Pagano, Marcello and Tritchler, David (1983), “On Obtaining Permutation Distributions in Polynomial Time,” Journal of the American Statistical Association 78: 435–40.CrossRefGoogle Scholar
Rabiner, L. R., Schafer, R. W., and Rader, C. M. (1969), “The Chirp-z Transform Algorithm and Its Applications,” Bell System Technical Journal 48: 1249–92.CrossRefGoogle Scholar
Ramakrishna, M. V. (1987), “Computing the Probability of Hash Table/Urn Overflow,” Communications in Statistics A 16: 3343–53.CrossRefGoogle Scholar
Robinson, Ian and Sheather, Simon (1988), “Fast Computation of the Hodges–Lehmann Estimator and Its Associated Confidence Limits,” American Statistical Association Proceedings of the Statistical Computing Section, pp. 187–91.Google Scholar
Shen, Mok-Kong (1963), “Algorithm 202: Generation of Permutations in Lexicographical Order,” Communications of the ACM 6: 517.CrossRefGoogle Scholar
Silverman, B. W. (1982), “Kernel Density Estimation Using the Fast Fourier Transform,” Applied Statistics 31: 93–7.CrossRefGoogle Scholar
Singleton, R. C. (1969), “An Algorithm for Computing the Mixed-Radix Fast Fourier Transform,” IEEE Transactions on Audio and Electroacoustics 17: 93–103.CrossRefGoogle Scholar
Spino, Cathie and Pagano, Marcello (1991), “Efficient Calculation of the Permutation Distribution of Trimmed Means,” Journal of the American Statistical Association 86: 729–37.CrossRefGoogle Scholar
Tritchler, David (1984a), “On Inverting Permutation Tests,” Journal of the American Statistical Association 79: 200–7.CrossRefGoogle Scholar
Tritchler, David (1984b), “An Algorithm for Exact Logistic Regression,” Journal of the American Statistical Association 79: 709–11.CrossRefGoogle Scholar
Wells, Mark B. (1961), “Generation of Permutations by Transposition,” Mathematics of Computation 15: 192–5.CrossRefGoogle Scholar

Save book to Kindle

To save this book to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Available formats
×

Save book to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.

Available formats
×

Save book to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.

Available formats
×