Book contents
- Frontmatter
- Contents
- List of Panels
- Preface
- PART I A LONG-PONDERED OUTFIT
- PART II THE EVALUATION DISCORDANCE
- PART III THE ALGORITHMIC CONFLUENCE
- 7 Intelligence and Algorithmic Information Theory
- 8 Cognitive Tasks and Difficulty
- 9 From Tasks to Tests
- 10 The Arrangement of Abilities
- 11 General Intelligence
- PART IV THE SOCIETY OF MINDS
- PART V THE KINGDOM OF ENDS
- References
- Index
- Plate section
8 - Cognitive Tasks and Difficulty
from PART III - THE ALGORITHMIC CONFLUENCE
Published online by Cambridge University Press: 19 January 2017
- Frontmatter
- Contents
- List of Panels
- Preface
- PART I A LONG-PONDERED OUTFIT
- PART II THE EVALUATION DISCORDANCE
- PART III THE ALGORITHMIC CONFLUENCE
- 7 Intelligence and Algorithmic Information Theory
- 8 Cognitive Tasks and Difficulty
- 9 From Tasks to Tests
- 10 The Arrangement of Abilities
- 11 General Intelligence
- PART IV THE SOCIETY OF MINDS
- PART V THE KINGDOM OF ENDS
- References
- Index
- Plate section
Summary
There are two theorems in Levin's “Universal Sequential Search Problems” (1973). The first states the now well-known principle of NP completeness and is followed by an outline of a proof. The second gives a solution to a very broad class of mathematical problems, but, partly because no proof was suggested in the paper, its great importance is not widely appreciated.
– Ray J. Solomonoff, Optimum Sequential Search (1984a)THE FORMAL DERIVATION of a test and the analysis of item difficulty using algorithmic information theory establishes an embryo for universal psychometrics. However, this proves almost nothing if we cannot generalise these ideas to the computational definition of other tasks. We must aim at a well-founded universal notion of cognitive task from which task and item difficulty can be derived. In this chapter we will see a characterisation of cognitive tasks as interactive (stochastic) machines, which is mirrored by the characterisation of the subjects that take the tasks as interactive machines too. The view of tasks as stochastic, featuring many instances, is crucial for the distinction of a solution for a single instance and a policy that achieves an acceptable result for the whole task. Looking at the policies instead of the task is the key issue leading to a computational notion of difficulty, in terms of (the logarithm of) the number of computational steps required to find the simplest acceptable policy.
INTERPRETING TASKS AND INSTANCES
In Chapter 1we saw that cognitive evaluation is performed through instruments, or tests, which are composed of cognitive tasks. In subsequent chapters we also saw that some subjects are designed (by evolution for natural systems or by programming for artificial systems) to do some tasks, whereas other systems are able to learn to do new tasks, usually through several trials and some kind of feedback. A proper notion of task should accommodate the different administrations and evaluation goals seen in Chapters 3–6.
Let us start with a very simple task. Panel 8.1 explains the “relative numerousness” task, whose goal is to tell which quantity is lowest.
- Type
- Chapter
- Information
- The Measure of All MindsEvaluating Natural and Artificial Intelligence, pp. 201 - 233Publisher: Cambridge University PressPrint publication year: 2017