Book contents
- Frontmatter
- Contents
- Preface to the second edition
- Acknowledgments
- 1 The evolution, development, and modification of behavior
- 2 Variation and selection: kineses
- 3 Reflexes
- 4 Direct orientation and feedback
- 5 Operant behavior
- 6 Reward and punishment
- 7 Feeding regulation: a model motivational system
- 8 The optimal allocation of behavior
- 9 Choice: dynamics and decision rules
- 10 Foraging and behavioral ecology
- 11 Stimulus control and cognition
- 12 Stimulus control and performance
- 13 Molar laws
- 14 Time and memory, I
- 15 Time and memory, II
- 16 Template learning
- 17 Learning, I
- 18 Models of classical conditioning
- 19 Learning, II
- 20 Learning, III: procedures
- 21 Comparative cognition
- Index
13 - Molar laws
Published online by Cambridge University Press: 05 March 2016
- Frontmatter
- Contents
- Preface to the second edition
- Acknowledgments
- 1 The evolution, development, and modification of behavior
- 2 Variation and selection: kineses
- 3 Reflexes
- 4 Direct orientation and feedback
- 5 Operant behavior
- 6 Reward and punishment
- 7 Feeding regulation: a model motivational system
- 8 The optimal allocation of behavior
- 9 Choice: dynamics and decision rules
- 10 Foraging and behavioral ecology
- 11 Stimulus control and cognition
- 12 Stimulus control and performance
- 13 Molar laws
- 14 Time and memory, I
- 15 Time and memory, II
- 16 Template learning
- 17 Learning, I
- 18 Models of classical conditioning
- 19 Learning, II
- 20 Learning, III: procedures
- 21 Comparative cognition
- Index
Summary
Every operant conditioning experiment is a feedback system. Response and reinforcer measures are neither independent nor dependent variables – causes and effects – but interdependent variables. Because of feedback, the relations between response and reinforcer rates, for example, reveal little about true causes, the processes internal to the organism that allow it to react to the stimuli and reinforcers the environment provides. But, also because of feedback, the relations between response and reinforcer measures in these experiments are often extremely orderly. This chapter describes a very simple model for the reliable molar regularities of simultaneous and successive free operant choice (concurrent and multiple variable-interval [VI] schedules). I seek the order in the empirical laws, skipping over the molecular/procedural details necessary to obtain them, such as the effects of changeover delay, interresponse time reinforcement, etc., which were discussed in Chapter 9.
Matching and optimality
Chapter 9 showed how diminishing marginal utility could account for response functions (steady-state response rate vs. reinforcer rate functions) obtained on numerous simple schedules. Chapter 10 showed how marginal utility can explain optimal choice between food patches. A version of this approach can be easily applied to concurrent schedules (simultaneous discrimination).
Matching can be derived in a way that allows for interim activities and can be generalized so as to account for performance on multiple schedules (i.e., successive discrimination). In a two-component concurrent schedule, the components share an interim activity, which occurs in both. I assume that in each component, the balance between operant reinforcement and the interim activity adjusts so as to minimize cost (maximize value), in the sense defined in Chapter 8.
The condition for optimal (maximizing) behavior is that the marginal change in reinforcement rate be equal to the marginal change in the value of the interim activity, formally
dR (x) /dx = dV (z) /dz,
where V(z) is the value of the interim activity as a function of its level, z, x is the level of responding in one component, and R(x) is its rate of reinforcement. If we assume that the marginal value of the interim activity is constant across both components, the optimality condition is that
dR (x) /dx = dR (y) /dy,
where y is the response rate in the other component.
- Type
- Chapter
- Information
- Adaptive Behavior and Learning , pp. 377 - 397Publisher: Cambridge University PressPrint publication year: 2016