Bowers et al. do the field a service with their thought-provoking commentary. If the problems currently characterizing deep neural network (DNN) models of human vision laid out in the target article are not adequately addressed, the field risks another winter. Bowers et al. sketch one important way forward: Building DNNs that can account for psychological data. I put forward that developing DNNs of human vision will foremost require a conceptual shift: From approaching perception as the outcome of a largely stimulus-driven process of feature detection and object recognition to treating perception as an active, top-down-guided process.

Building on the traditional notion of perception as a largely bottom-up process, mainstream computational cognitive neuroscience currently embraces the idea that simply feeding blank slate DNNs large amounts of training data will produce human-like vision. Yet, as Bowers et al.'s overview shows, this may not yet be the case. Based on the literature on human perceptual learning and action-oriented theories of perception, I contend that this may be a direct result from the manner in which DNNs are typically trained to “perceive”: In a passive, data-driven manner. This approach typically does not induce perceptual learning that generalizes to new stimuli or tasks in humans (Lu & Dosher, Reference Lu and Dosher2022), is very different from how babies learn to perceive (Emberson, Reference Emberson and Benson2017; Zaadnoordijk, Besold, & Cusack, Reference Zaadnoordijk, Besold and Cusack2022), and does not take into account the action-oriented nature of perception (Friston, Reference Friston2009; Gibson, Reference Gibson2014; Hurley, Reference Hurley2001).

The most consistent finding in the literature on visual perceptual learning in human adults is that learning is highly specific to the trained stimuli and tasks (Lu & Dosher, Reference Lu and Dosher2022). For example, improvements are often not observed if the test stimulus has a different orientation or contrast than the trained stimulus or when the trained stimulus is relocated or rotated (Fahle, Reference Fahle2004; Fiorentini & Berardi, Reference Fiorentini and Berardi1980). These findings indicate that the typical outside-in approach used in perceptual learning studies in which participants are presented with stimuli to detect or categorize tends to induce learning at too low levels in the processing hierarchy to support feature-, stimulus-, or view-independent learning. Indeed, more recent research suggests that transfer learning can be enhanced when learning can be top-down guided and connect to higher levels in the processing hierarchy (Tan, Wang, Sasaki, & Watanabe, Reference Tan, Wang, Sasaki and Watanabe2019). For example, when the training procedure allowed for more abstract rule formation, complete transfer of learning between physically different stimuli was observed (Wang et al., Reference Wang, Wang, Zhang, Xie, Yang, Luo and Li2016). These observations fit with recent findings that perceptual learning involves higher cognitive areas (Shibata, Sagi, & Watanabe, Reference Shibata, Sagi and Watanabe2014; Zhang et al., Reference Zhang, Zhang, Xiao, Klein, Levi and Yu2010) and proposals that perceptual learning is a top-down-guided process (Ahissar & Hochstein, Reference Ahissar and Hochstein2004). Perceptual development in infants is also more top-down guided than traditionally assumed (Emberson, Reference Emberson and Benson2017) and perception continues to develop through childhood based on acquired knowledge across a range of tasks (Milne et al., Reference Milne, Lisi, McLean, Zheng, Groen and Dekker2022). Yet the building of models of human vision still typically starts from the notion that human-like vision will simply arise by feeding blank slate DNNs many supervised training images, which may cause learning at too low levels in the processing hierarchy. Indeed, as Bowers et al. summarize, DNNs can be fooled by additive noise (Heaven, Reference Heaven2019), have difficulty generalizing learning to novel objects, and do not form transformation-tolerant object identity representations at higher layers (Xu & Vaziri-Pashkam, Reference Xu and Vaziri-Pashkam2021). These problems conceivably reflect insufficient top-down-guided learning.

Research also shows that perception and action are interdependent processes, in particular during the development of perception (Zaadnoordijk et al., Reference Zaadnoordijk, Besold and Cusack2022). For example, kittens that are passively moved around, do not develop depth perception (Held & Hein, Reference Held and Hein1963), just like DNN's that are fed visual input do not perceive depth (Jacob, Pramod, Katti, & Arun, Reference Jacob, Pramod, Katti and Arun2021). Humans are not passive perceivers, but continuously build on past experiences to actively predict and generate their own sensory information through action, thereby top-down driving their own learning (Boonstra & Slagter, Reference Boonstra and Slagter2019; Buzsáki, Reference Buzsáki2019; Friston, Reference Friston2009; Gibson, Reference Gibson1988). That perception incorporates expectations about the sensory outcome of actions is demonstrated by the fact that humans who wear goggles that flip the visual field from left to right, do not perceive a normal world (albeit flipped left-right), but experience distorted perception (Kohler, Reference Kohler1963), caused by the disruption of normal sensorimotor contingencies. Moreover, recent studies show that responses in early visual cortex also reflect actions (Schneider, Reference Schneider2020). These findings cannot easily be explained by the classical view of the brain as processing information serially from sensory to cognitive to motor control stages (Hurley, Reference Hurley2001), each subserved by distinct brain regions – a view that currently still drives much of DNN research. Rather, they indicate that perception emerges from dynamic feedback relations between input and output, and does not merely entail the encoding of environmental statistics, but also the statistics of agent–environment interactions (Friston, Reference Friston2010). Yet DNNs are generally trained in a passive way. This may cause shortcut learning or DNNs to latch onto features that do not matter to humans in categorizing objects. DNNs may focus on texture (Geirhos et al., Reference Geirhos, Rubisch, Michaelis, Bethge, Wichmann and Brendel2022) or local rather than global shape (Baker, Lu, Erlikhman, & Kellman, Reference Baker, Lu, Erlikhman and Kellman2018), because they never had to interact with objects, for which global shape knowledge is important. Notably, the development of global shape representations may depend on the dorsal stream (Ayzenberg & Behrmann, Reference Ayzenberg and Behrmann2022).

To develop models of human-like vision, the field thus needs to turn the notion of perception on its head: From bottom-up driven to top-down guided and fundamentally serving agent–environment interactions. Important steps are already taken in this direction. For example, DNN architectures wired to top-down infer their sensory input has been shown to work at scale (Millidge, Salvatori, Song, Bogacz, & Lukasiewicz, Reference Millidge, Salvatori, Song, Bogacz and Lukasiewicz2022). There are also exciting developments in robotics, in which artificial systems equipped with the possibility to predict and generate their sensory information through action can top-down drive their own learning (Lanillos et al., Reference Lanillos, Meo, Pezzato, Meera, Baioumy, Ohata and Tani2021). DNNs have the potential to provide powerful ways to study the human brain and behavior, but this will require the incorporation of biologically realistic, action-oriented learning algorithms, grounding vision on interactions with the environment.