Game theory provides a tool for studying frequency-dependent selection. In particular, whenever the replicator dynamics is a Shahshahani gradient, then so is the corresponding selection equation. For games with two pure strategies, game theory and population genetics agree both for the continuous time model and for the discrete time model. For more than two strategies and more than three alleles, open problems remain. But under appropriate conditions, an ESS corresponds to an attracting set of rest points in the gene space, and long-term stability implies evolutionary stability.

Strategy and genetics

So far, we have not specified the connection between game-theoretical modelling and population genetics. The underlying idea is, of course, that genotypes specify strategies as behavioural phenotypes. But the replicator dynamics studied in part II of this book did not take the intricacies of Mendelian inheritance into account. It was firmly based on the implicit assumption of asexual reproduction. This simplified the analysis considerably; moreover, in the absence of detailed knowledge on the genetic background of a behavioural trait, all corresponding assumptions are bound to be arbitrary, and may obfuscate the essential aspects of the model. On the other hand, the neglect of Mendelian inheritance entails a serious loss of realism. For the Battle of the Sexes or the sex ratio game, an asexual model may seem paradoxical.

It is obvious that a genetic mechanism may prevent the establishment of an ESS.