Skip to main content Accessibility help

We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.

Close cookie message
×
Hostname: page-component-6bb9c88b65-t28k2 Total loading time: 0 Render date: 2025-07-18T16:31:26.777Z Has data issue: false hasContentIssue false

State space modeling for analysis of behavior in learning experiments

from Part I - State space methods for neural data

Published online by Cambridge University Press:  05 October 2015

By
A. C. Smith
Edited by
Zhe Chen
A. C. Smith
Affiliation:
University of Arizona
Zhe Chen
Affiliation:
New York University
Get access

Summary

Introduction

During the process of learning the brain undergoes changes that can be observed at both the cellular and systems level. Being able to track accurately simultaneous changes in behavior and neural activity is key to understanding how the brain learns new tasks and information.

Learning is studied in a large number of experimental paradigms involving, for example, testing effects on learning of brain lesions (Whishaw & Tomie 1991; Dias et al. 1997; Dusek & Eichenbaum 1997; Wise & Murray 1999; Fox et al. 2003; Kim & Frank 2009; Kayser & D'Esposito 2013), attentional modulation (Cook & Maunsell 2002; Hudson et al. 2009), optogenetic manipulation (Warden et al. 2012) and pharmacological interventions (Stefani et al. 2003). Studies are also performed to understand how learning is affected by aging (Harris & Wolbers 2012), stroke (Panarese et al. 2012) and psychological conditions including autism (Solomon et al. 2011) and synesthesia (Brang et al. 2013). The learning process is also studied in relation to changes in neural activity in specific brain regions (Jog et al. 1999; Wirth et al. 2003; Suzuki & Brown 2005; Brovelli et al. 2011; Mattfeld & Stark 2011).

In most cases, the response accuracy of a subject is binary, with a one representing a correct response and a zero representing an incorrect response. In its raw form, binary response accuracy can be difficult to visualize, especially if the time series is long, and the exact time when learning occurs can be difficult to identify. Typically, an experimenter is interested in deriving two things from the learning data: a learning trial and a learning curve. The first item is the time point at which responses significantly change relative to a baseline value such as chance performance. The second is estimation of a curve that defines the probability of a correct response as a function of trial. From these estimates, it is possible to compare changes in learning with other measurements such as, for example, localized brain oxygen consumption (via fMRI) or electrical activity.

Information

Type
Chapter
Information
Advanced State Space Methods for Neural and Clinical Data , pp. 231 - 254
Publisher: Cambridge University Press
Print publication year: 2015

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Book purchase

Temporarily unavailable

References

Abanto-Valle, C.A. & Dey, D.K. (2014). State space mixed models for binary responses with scale mixture of normal distributions links. Computational Statistics & Data Analysis 71, 274–287.Google Scholar
Ahmadian, Y., Pillow, J.W. & Paninski, L. (2011). Efficient Markov chain Monte Carlo methods for decoding neural spike trains. Neural Computation 23(1), 46–96.Google Scholar
Albert, J.H. & Chib, S. (1993). Bayesian analysis of binary and polychotomous response data. Journal of American Statistical Association 88, 669–679.Google Scholar
Baker, K. (2011). Determining resident clinical performance: getting beyond the noise. Anesthesiology 115(4), 862–878.Google Scholar
Barnes, T.D., Kubota, Y., Hu, D., Jin, D. Z. & Graybiel, A. M. (2005). Activity of striatal neurons reflects dynamic encoding and recoding of procedural memories. Nature 437, 1158–1161.Google Scholar
Bishop, C. M. (2006). Pattern Recognition and Machine Learning, New York: Springer.
Brang, D., Ghiam, M. & Ramachandran, V. (2013). Impaired acquisition of novel grapheme-color correspondences in synesthesia. Frontiers In Human Neuroscience 7.Google Scholar
Brooks, S. P. & Gelman, A. (1998). General methods for monitoring convergence of iterative simulations. Journal of Computational and Graphical Statistics 7(4), 434–455.Google Scholar
Brovelli, A., Nazarian, B., Meunier, M. & Boussaoud, D. (2011). Differential roles of caudate nucleus and putamen during instrumental learning. Neuroimage 57(4), 1580–1590.Google Scholar
Casella, G. & George, E. I. (1992). Explaining the Gibbs sampler. The American Statistician 46(3), 167–174.Google Scholar
Congdon, P. (2003). Applied Bayesian Modelling, New York: Wiley.
Congdon, P. (2005). Bayesian Models for Categorical Data, New York: Wiley.
Congdon, P. (2007). Bayesian Statistical Modelling, New York: Wiley.
Congdon, P. D. (2010). Applied Bayesian Hierarchical Methods, Boca Raton, FL: CRC Press.
Cook, E. P. & Maunsell, J. H. R. (2002). Attentional modulation of behavioral performance and neuronal responses in middle temporal and ventral intraparietal areas of macaque monkey. Journal of Neuroscience 22(5), 1994–2004.Google Scholar
Dayan, P. & Yu, A. J. (2003). Uncertainty and learning. IETE Journal of Research 49(2-3), 171–181.Google Scholar
De Jong, P. & Mackinnon, M. J. (1988). Covariances for smoothed estimates in state space models. Biometrika 75(3), 601–602.Google Scholar
Dempster, A. P., Laird, N. M. & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM Algorithm. Journal of the Royal Statistical Society, Series B 39(1), 1–38.Google Scholar
Dias, R., Robbins, T.W. & Roberts, A. C. (1997). Dissociable forms of inhibitory control within prefrontal cortex with an analog of the Wisconsin card sort test: Restriction to novel situations and independence from “on-line” processing. Journal of Neuroscience 17, 9285–9297.Google Scholar
Ditterich, J. (2006). Evidence for time-variant decision making. European Journal of Neuroscience 24(12), 3628–3641.Google Scholar
Duane, S., Kennedy, A., Pendleton, B. & Roweth, D. (1987). Hybrid Monte Carlo. Physics Letters B 195(2), 216–222.Google Scholar
Dusek, J. A. & Eichenbaum, H. (1997). The hippocampus and memory for orderly stimulus relations. Proceedings of the National Academy of Sciences USA 94, 7109–7114.Google Scholar
Eden, U. T. & Brown, E. N. (2008). Mixed observation filtering for neural data. In Proceedings of ICASSP, pp. 5201–5203.Google Scholar
Eldar, E., Morris, G. & Niv, Y. (2011). The effects of motivation on response rate: a hidden semi-Markov model analysis of behavioral dynamics. Journal of Neuroscience Methods 201(1), 251–261.Google Scholar
Fahrmeir, L., Tutz, G. & Hennevogl, W. (1994). Multivariate Statistical Modelling based on Generalized Linear Models, New York: Springer.
Fox, M. T., Barense, M. D. & Baxter, M. G. (2003). Perceptual attentional set-shifting is impaired in rats with neurotoxic lesions of posterior parietal cortex. Journal of Neuroscience 23, 676–681.Google Scholar
Gallistel, C. R., Mark, T. A., King, A. P. & Latham, P. E. (2001). The rat approximates an ideal detector of changes in rates of reward: implications for the law of effect. Journal of Experimental Psychology-Animal Behavior Processes 27(4), 354–372.Google Scholar
Gelfand, A. E. & Smith, A. F. M. (1990). Sampling-based approaches to calculating marginal densities. Journal of American Statistical Association 85, 398–409.Google Scholar
Gelman, A. (2006). Prior distributions for variance parameters in hierarchical models. Bayesian Analysis 1(3), 515–534.Google Scholar
Gelman, A. & Rubin, D. B. (1992). Inference from iterative simulation using multiple sequences. Statistical Science 7, 457–472.Google Scholar
Geman, S. & Geman, D. (1984). Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence 6(6), 721–741.Google Scholar
Gilks, W. R. &Wild, P. (1992). Adaptive rejection sampling for gibbs sampling. Applied Statistics 41(2), 337–348.Google Scholar
Harris, M. A. & Wolbers, T. (2012). Ageing effects on path integration and landmark navigation. Hippocampus 22, 1770–1780.Google Scholar
Hastings, W. K. (1970). Monte carlo sampling methods using markov chains and their applications. Biometrika 57(1), 97–109.Google Scholar
Haykin, S. (2001). Kalman Filtering and Neural Networks, New York: Wiley.
Hudson, A. E., Schiff, N. D., Victor, J. D. & Purpura, K. P. (2009). Attentional modulation of adaptation in V4. European Journal of Neuroscience 30, 151–171.Google Scholar
Jog, M. S., Kubota, Y., Connolly, C. I., Hillegaart, V. & Graybiel, A.M. (1999). Building neural representations of habits. Science 286, 1745–1749.Google Scholar
Jones, R. H. (1993). Longitudinal Data with Serial Correlation: A State-Space Approach, Boca Raton, FL: CRC Press.
Jungbacker, B. & Koopman, S. J. (2007). Monte Carlo estimation for nonlinear non-Gaussian state space models. Biometrika 94(4), 827–839.Google Scholar
Kakade, S. & Dayan, P. (2002). Acquisition and extinction in autoshaping. Psychological Review 109(3), 533–544.Google Scholar
Kass, R. E., Carlin, B. P., Gelman, A. & Neal, R. M. (1998). Markov chain Monte Carlo in practice: a roundtable discussion. The American Statistician 52(2), 93–100.Google Scholar
Kayser, A. S. & D'Esposito, M. (2013). Abstract rule learning: the differential effects of lesions in frontal cortex. Cerebral Cortex 23, 230–240.Google Scholar
Kim, S. M. & Frank, L. M. (2009). Hippocampal lesions impair rapid learning of a continuous spatial alternation task. PLoS ONE 4, e5494.Google Scholar
Kitagawa, G. & Gersh, W. (1996). Smoothness Priors Analysis of Time Series, New York:Springer.
Klein, B. M. (2003). State space models for exponential family data, PhD, University of Southern Denmark, Department of Statistics.
Lambert, P., Sutton, A., Burton, P., Abrams, K. & Jones, D. (2005). How vague is vague? A simulation study of the impact of the use of vague prior distributions in MCMC using WinBUGS. Statistics in Medicine 24(15), 2401–2428.Google Scholar
Lunn, D. J., Thomas, A., Best, N. & Spiegelhalter, D. (2000). WinBUGS – a Bayesian modelling framework: concepts, structure, and extensibility. Statistics and Computing 10, 325–337.Google Scholar
Lunn, D., Jackson, C., Best, N., Thomas, A. & Spiegelhalter, D. (2012). The BUGS Book: A Practical Introduction to Bayesian Analysis, Boca Raton, FL: CRC Press.
MacKay, D. J.C. (2003). Information Theory, Inference and Learning Algorithms, Cambridge: Cambridge University Press.
Mattfeld, A. & Stark, C. (2011). Striatal and medial temporal lobe functional interactions during visuomotor associative learning. Cerebral Cortex 21(3), 647–658.Google Scholar
Metropolis, N., Rosenbluth, A.W., Rosenbluth, M. N., Teller, A. H. & Teller, E. (2004). Equation of state calculations by fast computing machines. Journal of Chemical Physics 21(6), 1087–1092.Google Scholar
Neal, R. M. (2003). Slice sampling. Annals of Statistics 31(3), 705–741.Google Scholar
Ntzoufras, I. (2011). Bayesian Modeling Using WinBUGS, New York: Wiley.
Panarese, A., Colombo, R., Sterpi, I., Pisano, F. & Micera, S. (2012). Tracking motor improvement at the subtask level during robot-aided neurorehabilitation of stroke patients. Neurorehabilitation and Neural Repair 26, 822–833.Google Scholar
Paninski, L., Ahmadian, Y., Ferreira, D. G., Koyama, S., Rad, K. R., Vidne, M., Vogelstein, J. &Wu, W. (2010). A new look at state-space models for neural data. Journal of Computational Neuroscience 29, 107–126.Google Scholar
Prerau, M. J., Smith, A. C., Eden, U. T., Kubota, Y., Yanike, M., Suzuki, W., Graybiel, A. M. & Brown, E. N. (2009). Characterizing learning by simultaneous analysis of continuous and binary measures of performance. Journal of Neurophysiology 102(5), 3060–3072.Google Scholar
Schiff, N.D., Giacino, J. T., Kalmar, K., Victor, J., Baker, K., Gerber, M., Fritz, B., Eisenberg, B., O'Connor, J., Kobylarz, E. J., Farris, S., Machado, A., McCagg, C., Plum, F., Fins, J. J. & Rezai, A. (2007). Behavioural improvements with thalamic stimulation after severe traumatic brain injury. Nature 448, 600–603.Google Scholar
Shah, S. A., Baker, J. L., Ryou, J.-W., Purpura, K. P. & Schiff, N. D. (2009).Modulation of arousal regulation with central thalamic deep brain stimulation. In Proceedings of IEEE Engineering in Medicine and Biology, pp. 3314–3317.Google Scholar
Shephard, N. & Pitt, M. K. (1997). Likelihood analysis of non-Gaussian measurement time series. Biometrika 84(3), 653–667.Google Scholar
Shumway, R. H. & Stoffer, D. S. (1982). An approach to time series smoothing and forecasting using the EM algorithm. Journal of Time Series Analysis 3(4), 253–264.Google Scholar
Siegel, S. & Castellan, N. J. (1988). Nonparametric Statistics for the Behavioral Sciences, New York: McGraw-Hill.
Smith, A. C. & Brown, E. N. (2003). Estimating a state-space model from point process observations. Neural Computation, 15, 965–991.Google Scholar
Smith, A.C., Frank, L. M., Wirth, S., Yanike, M., Hu, D., Kubota, Y., Graybiel, A. M., Suzuki, W. A. & Brown, E. N. (2004). Dynamic analysis of learning in behavioral experiments. Journal of Neuroscience 24, 447–461.Google Scholar
Smith, A. C., Shah, S.A., Hudson, A. E., Purpura, K. P., Victor, J. D., Brown, E.N. & Schiff, N. D. (2009). A Bayesian statistical analysis of behavioral facilitation associated with deep brain stimulation. Journal of Neuroscience Methods 183, 267–276.Google Scholar
Smith, A. C., Stefani, M. R., Moghaddam, B. & Brown, E. N. (2005). Analysis and design of behavioral experiments to characterize population learning. Journal of Neurophysiology 93, 1776–1792.Google Scholar
Smith, A. C., Wirth, S., Suzuki, W. A. & Brown, E. N. (2007). Bayesian analysis of interleaved learning and response bias in behavioral experiments. Journal of Neurophysiology 97, 2516–2524.Google Scholar
Solomon, M., Frank, M. J., Smith, A. C., Ly, S. & Carter, C. S. (2011). Transitive inference in adults with autism spectrum disorders. Cognitive Affective & Behavioral Neuroscience 11(3), 437–449.Google Scholar
Spiegelhalter, D. J., Best, N., Carlin, B. & Van Der Linde, A. (2002). Bayesian measures of model complexity and fit. Journal of the Royal Statistical Society, Series B 64(4), 583–639.Google Scholar
Stefani, M. R., Groth, K. & Moghaddam, B. (2003). Glutamate receptors in the rat medial prefrontal cortex regulate set-shifting ability. Behavioral Neuroscience 117, 728–737.Google Scholar
Stone, M. (1977). An asymptotic equivalence of choice of model by cross-validation and Akaike's criterion. Journal of the Royal Statistical Society, Series B 39(1), 44–47.Google Scholar
Suzuki, W. A. & Brown, E. N. (2005). Behavioral and neurophysiological analysis of dynamic learning processes. Behavioral Cognitive Neuroscience Review 4, 67–95.Google Scholar
Thomas, A., O'Hara, B., Ligges, U. & Sturtz, S. (2006). Making BUGS open. R News 6(1), 12–17.Google Scholar
Usher, M. &McClelland, J. L. (2001). The time course of perceptual choice: the leaky, competing accumulator model. Psychological Review 108(3), 550–592.Google Scholar
Warden, M. R., Selimbeyoglu, A., Mirzabekov, J. J., Lo, M., Thompson, K. R., Kim, S.-Y., Adhikari, A., Tye, K. M., Frank, L. M. & Deisseroth, K. (2012). A prefrontal cortex-brainstem neuronal projection that controls response to behavioural challenge. Nature 492, 428–432.Google Scholar
Whishaw, I. Q. & Tomie, J. A. (1991). Acquisition and retention by hippocampal rats of simple, conditional, and configural tasks using tactile and olfactory cues: implications for hippocampal function. Behavioral Neuroscience 105, 787–797.Google Scholar
Wirth, S., Avsar, E., Chiu, C. C., Sharma, V., Smith, A. C., Brown, E. N. & Suzuki, W. A. (2009). Trial outcome and associative learning signals in the monkey hippocampus. Neuron 61(6), 930–940.Google Scholar
Wirth, S., Yanike, M., Frank, L. M., Smith, A. C., Brown, E. N. & Suzuki, W. A. (2003). Single neurons in the monkey hippocampus and learning of new associations. Science 300, 1578–1584.Google Scholar
Wise, S. P. & Murray, E. A. (1999). Role of the hippocampal system in conditional motor learning: mapping antecedents to action. Hippocampus 9(2), 101–117.Google Scholar
Wong, K. F. K., Smith, A. C., Pierce, E. T., Harrell, P. G., Walsh, J. L., Salazar, A. F., Tavares, C. L., Cimenser, A., Prerau, M. J., Mukamel, E. A., Sampson, A., Purdon, P. L. & Brown, E. N. (2011). Bayesian analysis of trinomial data in behavioral experiments and its application to human studies of general anesthesia. In Proceedings of IEEE Engineering in Medicine and Biology, pp. 4705–4708.Google Scholar
Wong, K. F. K., Smith, A. C., Pierce, E. T., Harrell, P. G., Walsh, J. L., Salazar-Gómez, A. F., Tavares, C. L., Purdon, P. L. & Brown, E. N. (2014). Statistical modeling of behavioral dynamics during propofol-induced loss of consciousness. Journal of Neuroscience Methods 227, 65–74.Google Scholar

Accessibility standard: Unknown

Accessibility compliance for the PDF of this book is currently unknown and may be updated in the future.

Save book to Kindle

To save this book to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Available formats
×

Save book to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.

Available formats
×

Save book to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.

Available formats
×