Joerg Fingerhut

EX TENDED IM AGERY, EX TENDED ACCESS,

OR SOMETHING ELSE?
Pictures and the Extended Mind Hypothesis

Introduction
Up until now pictures have not been a focus of what has come to be known as
the enactive, embodied, embedded, extended, and affective philosophy of mind
(following common usage henceforth referred to as 4EA approaches). While
pictures have enjoyed notable treatment in subfields for example in accounts
that emphasize the role of bodily mediated emotions for aesthetic experience of
pictures,1 or in Ed Hutchins classical treatment of the role maps and charts play
in the navigation of a ship2 they have rarely gained center stage in philosophical theories.3
4EA approaches counterbalance certain exaggerations, address certain
neglected problems, and in some instances question the very foundation of what has
long been the standard cognitivist picture of the mind. In this standard view
championed by most of cognitive science in the second half of the 20th century
the mind is first and foremost an information-processing device that operates

2
3

See e.g. David Freedberg/Vittorio Gallese: Motion, Emotion and Empathy in

Esthetic Experience, in: Trends in Cognitive Sciences 11/5 (2007), pp. 197203, for
an approach that discusses the importance of simulated embodiment and empathic
engagement with works of art. See also Jesse Prinz: Emotion and Aesthetic Value,
in: Arthur E. Shimamura/Stephen E. Palmer (eds.): The Aesthetic Mind, New York
2011, pp. 7188, for a contemporary version of sentimentalism in aesthetics, i.e. the
view that emotions are the central component of our aesthetic appraisals of works
of art.
Edwin Hutchins: Cognition in the Wild, Cambridge, MA 1995.
This holds for the main proponents of 4EA approaches. For two notable exceptions
that discuss pictures against the backdrop of 4EA ideas see John Michael Krois:
Bildkrper und Krperschema. Schriften zur Verkrperungstheorie ikonischer
Formen, ed. by Horst Bredekamp/Marion Lauschke, Berlin 2011 (Actus et Imago 2)
and Frederik Stjernfelt: Diagrammatology, Dordrecht 2007.

34

JOERG FINGERHUT

upon inner representational states and whose main operations consist of transformations of those inner states. The mind is therefore tucked in between
perceptual inputs and motor outputs of the brain. Any science of the mind or
so proponents of this view argue has best to concern itself with those processes
and operations and the way they make up an intelligent system in order for this
science to strive and fully mature.4
This image of the mind is rejected by 4EA approaches. What they provide instead, is a view of the mind that sees it as being scaffolded by environmental structures in order to perform many of its tasks (embedded), as being
realized not only in the brain but also in the skillful body and its motor systems
(embodied), or sometimes even including elements outside the body as proper
parts of the mechanisms that constitute mental states (extended). The mind is
seen as actively generating meaning through active engagement with its environment (enactive) and one therefore also should take the fundamental evaluative environmental relations that underlie these engagements into account,
namely the affectivity of every organism and its needs and strivings (affective).
I am not going to repeat the main arguments for these different views, just as I
am not trying to identify a common denominator.5 Instead I will look into certain upshots that such views of the mind have for explanations of our interactions
with pictures and artifacts, and vice versa.
This paper introduces pictures more generally into the discussion of
cognition and mind. I will argue that pictures play a decisive role in shaping our
mental lives because they have changed (and constantly keep changing) the
ways we access the world. Focusing on pictures will therefore also shed new
light on various claims within the field of embodied cognition. In the first half
of this paper I address the question of whether, and in what possible ways, pictures might be considered to be part of our extended mind. We will see however,
that the explanatory means contingent upon the extended mind thesis i.e. the
claim that the vehicles of cognition are not confined to the boundaries of the
individual organism can only take us so far. Beyond such claims it will be pivotal to understand in what specific ways pictures might be regarded as being at
the basis of certain perceptions of and interactions with the world. I will therefore address, in the second half of this paper, in what ways enactive and affective
elements should inform our theory of the pictorial mind. In the course of this
discussion it will become apparent that pictures are strange objects because they
differ profoundly from other objects surrounding us. And it will also turn out
4
5

See Jerry Fodor: LOT 2: The Language of Thought Revisited, New York 2008 for a
current defense of such a strategy.
See for this the introduction in Joerg Fingerhut/Rebekka Hufendiek/Markus Wild
(eds.): Philosophie der Verkrperung. Grundlagentexte zu einer aktuellen Debatte,
Berlin 2013, pp. 9104, esp. pp. 6491.

35

EXTENDED IMAGERY, EXTENDED ACCESS, OR SOMETHING ELSE?

that pictures beyond the fact that they can be considered to be tools for our
mind (in the sense that they facilitate our access to the world) are rather strange
or stubborn tools6 in that something in them resists full integration into our
cognitive routines.

Complementarity and Parity

The extended mind hypothesis is the claim that extra-neural elements and even
elements outside the boundaries of the human organism are proper parts of
mechanisms that constitute cognitive processes and mental states. This is not
just to say that our mind is about the world, but that the mind literally can leap
into the environment: components of the external world are sometimes vehicles
of our mental states. 7
This hypothesis has been prominently and elaborately defended by Andy
Clark. In the same year that the (now classic) Extended Mind paper8 was published, Clark also wrote a small paper that contains a defense against a critic who
emphasized the striking and apparent differences between external (extended)
and neural (brainbound) components.9 According to this criticism, such incongruities clearly speak against the claim that external elements could ever be a part
of the physical machinery that realizes cognitive processes.
In my view Clarks twofold response elucidates quite well what the
extended mind thesis was intended to cover. Firstly, he insists that what speaks
in favor of the extended mind claim is that sometimes extra-neural components

8
9

I am very much indebted to discussions with Alva No on the concept of strange

tools, a concept that he has developed with respect to works of art and that has
been working on over the last years. He currently is preparing a book on this topic
with the title: Strange Tools. Art and Human Nature.
Corresponding discussions of the extended mind claim outside the field of philosophy sometimes connect to the above claim only loosely. However, it is this claim in
particular that has initiated an intense debate and has triggered strong objections
in both philosophy of mind and philosophy more generally, including those camps
that propagate other elements of the 4EA approaches. For a view that is e.g. in favor
of embodiment and embeddedness yet opposes the extended mind thesis (with the
argument based on the threatened simplicity and conservatism of the psychological sciences that elements outside the organism should not become a central
focus in these sciences) see Robert Rupert: Cognitive Systems and the Extended
Mind, New York 2009.
See Andy Clark/David Chalmers: The Extended Mind, in: Analysis 58/1 (1998),
pp. 719.
This paper is part of a special journal symposium on Andy Clarks: Being There.
Putting Brain, Body, and World Together Again, Cambridge, MA 1997. See Andy
Clark: Being There. Putting Philosopher, Researcher and Student Together Again
(Authors Response), in: Metascience 7/1 (1998), pp. 95104.

36

JOERG FINGERHUT

can play a role similar to internal ones.10 Clark directs our attention to the
computational and functional organization of larger problem solving wholes
that not only include resources of the brain but also those of the body of the
cognizer and those of her environment. From this perspective we are able to
identify elements outside the organism that play roles that resemble those normally filled by mechanisms within the brain. It is this similarity that provides
support for the claim that external elements are also proper parts of cognitive
routines.
Secondly, Clark argues that once we take such a perspective on the whole organism-environment nexus we can also take into account that external elements may play a role different from, but complementary to, the inner ones.11
Simply by virtue of the pivotal functions certain bits and pieces of the environment occupy in our cognitive problem solving routines one has good reason to
argue that they should be considered parts of the mechanisms that realize our
cognitive states (and not only those elements that play similar roles as parts
within the brain).
While both of these elements are present in his account, the second point
seems to be of particular importance to him:
The argument for the extended mind thus turns primarily on the way
disparate inner and outer components may co-operate so as to yield
integrated larger systems capable of supporting various (often quite advanced) forms of adaptive success. The external factors and operations, in this
model, are most unlikely to be computationally identical to the ones supported directly in the wetware indeed, the power of the larger system
depends very much on the new kinds of storage, retrieval and transformation made possible by the use of extra-neural resources []. These
new operations, however, may often be seen as performing kinds of tasks
which, were they but done in the head, we would have no hesitation in
labeling cognitive.12
Although we might sometimes be able to identify a similarity between external
and internal components (i.e. the neural processes that are normally seen as
realizers of mental states), it is nonetheless the interplay between very different
kinds of components that especially facilitates cognitive problem solving. This
interaction with elements that have something to add beyond the operations of
the normal wetware in the brain might have been key to the evolution and the
ontogenetic development of our minds. From this viewpoint, external elements
10
11
12

Ibid., p. 99 (my emphasis).

Ibid. (my emphasis).
Ibid. (my emphasis).

37

EXTENDED IMAGERY, EXTENDED ACCESS, OR SOMETHING ELSE?

that are functionally isomorphic to inner states and processes might constitute
a rather small, even neglectable class.
It should be clear that in Clarks account of the extended mind the biological organism (and its neuronal parts) still plays a central, even indispensable
role. One can therefore distinguish three different claims with respect to the
boundaries of cognitive mechanisms, all of which are in accordance with Clarks
theory. A cognitive mechanism can
(a)
(b)

(c)

reside fully within the organism (e.g. in cases where all components of
the mechanism are neuronally realized);
be partially realized within the boundaries of the organism yet include
external parts whose functional roles resemble the role inner components already play; and
be partially realized within, yet include external parts whose role is
quite different from any functions that previously have been attributed
to parts of the brain and body of the organism.

John Sutton distinguishes a first and a second wave within theories of the extended mind based on this difference between parity of contribution of inner and
outer elements on the one hand (b), and complementarity of outer elements to
the inner ones on the other (c).13 When one considers claims such as the one cited
above, one could also argue that the second wave might be equiprimordial (if not
prior) to the first, yet there is some justification for Suttons usage: it has been
parity claims that have received most attention in the literature. But why is it
despite the above pledge for complementarity that arguments based on similarity and the promises and pitfalls of what has been labeled the parity principle14
are getting so much leverage within current philosophy?
One reason might be that it seems dialectically advantageous to lean on
a form of conservatism in the argumentation: what parity arguments could
show is that even if one focuses on what cognitive science already sees as the
13

14

See John Sutton: Exograms and Interdisciplinarity. History, the Extended Mind,
and the Civilizing Process, in: Richard Menary (ed.): The Extended Mind, Cambridge, MA 2010, pp. 189222.
See Clark/Chalmers: The Extended Mind (as fn. 8), p. 8, for what later has been called
the parity principle: If, as we confront some task, a part of the world functions as
a process which, were it done in the head, we would have no hesitation in recognizing as part of the cognitive process, then that part of the world is (so we claim) part
of the cognitive process. Based on this formulation one could also consider to
distinguish similarity claims (rather than parity claims) from complementarity
claims. Parity of treatment does not require external elements to be similar to ones
that can be found inside the head (and could therefore comprise similarity and complementarity). I will however retain the notion of parity as it is most commonly
used in the literature, namely as the more confined concept based on similarity.

38

JOERG FINGERHUT

central components of cognitive states, one might nonetheless be able to demonstrate an inconsistency in the standard view of cognition. The inconsistency
consists in the following: cognitive science draws an arbitrary line (at the boundary of skull and skin) that excludes components that it otherwise would see as
explanatorily central elements and that were they realized within the boundary of the brain would count as part of a cognitive mechanism. The moral of the
parity argument is this: components should be considered a part of a cognitive
mechanism (or not a part of that mechanism for that matter) independent of the
brute fact of their location.
Consider the discussion of memory retrieval in the Extended Mind paper
by Clark and Chalmers.15 Generally, if one wants to tackle human memory it
seems that one should include components such as the means or mechanisms by
which the brain accesses information as well as the information itself (stored in
the brain). The latter then is the content of an episode of remembering and could
also be characterized as a belief. Clark and Chalmers discuss the now famous
(and slightly overused) fictional case of Otto who suffers from a mild form of
Alzheimers. Although his memory is impaired due to his illness, he retrieves
information from a notebook when needed (in the original thought example: the
location of the Museum of Modern Art in New York, MoMA, on 53rd Street). He
does so effortlessly and can act on the basis of such information with similar
success as his healthy counterpart Inga who uses her biological memory (both
are able to find the MoMA). Ottos old-fashioned pen and paper notebook, so the
story goes, is meticulously updated and always kept close by, which guarantees
constant availability and the reliability of the stored information. Clark and
Chalmers emphasize the comparable roles that inner (in the case of Inga) and
outer (in the case of Otto) elements play in both scenarios. Ottos seamless
interactions with his notebook seem sufficiently similar to the cognitive operations of a healthy subject (at a certain level of description) and this might warrant the claim that Ottos dispositional beliefs are literally in his notebook.
What this argument therefore is meant to demonstrate is that even paradigmatic mental states (such as beliefs) do have wide realizations.16
One important line of counterarguments focuses, as could be expected,
on the various ways that Ottos memory nonetheless differs from the one realized in Ingas brain.17 Yet, Clark stands his ground by arguing that there is comparability on the level of description that is central for identifying the relevant
15
16
17

Ibid., pp. 1216.

See Robert Wilson: Two Views of Realization, in: Philosophical Studies 104/1
(2001), pp. 131.
See e.g. Fred Adams/Ken Aizawa: The Bounds of Cognition, in: Philosophical Psychology 14/1 (2001), p. 4364. They make a distinction between derived (i.e. conventional) and non-derived content and argue that the latter is, as a matter of con-

39

EXTENDED IMAGERY, EXTENDED ACCESS, OR SOMETHING ELSE?

components of cognitive mechanisms. It is this coarse functional poise of the

stored information (i.e. what role the respective information can play) that was,
in both cases, sufficiently similar and that warrants similarity of treatment.
Otto believed (even before consulting his notebook) that MoMA was on 53rd
Street and acts accordingly. This similarity cancels out what might then seem as
rather superficial differences in both cases.

Pictures as Extended Imagery

Although pictures have not been primarily addressed in the extended mind literature, they might nonetheless fall under the parity story outlined above. Consider, for example, the possibility that Otto consults a map or an image (instead
of a linguistic inscription) in his notebook in order to retrieve the relevant information about the location of MoMA. One might claim that this also resembles
a mental operation because representations in the brain have, according to some
theories of cognitive processing, been considered to be image-like.
Stephen Kosslyn, for example, has argued since the 1970s for the depictive character of mental representations and has outlined a theory of how pictorial elements influence information processing and how they are neuronally
realized. He defines a depictive representation as a type of picture, which specifies the locations and values of configurations of points in a space.18 Kosslyn
conducted experiments in which subjects were asked to mentally scan (mentally
look at) imagined objects. He was able to show that the relative size of subparts of an imagined object measurably affects the time that it takes to assess
and report on those sub-parts: larger features are reported more quickly than
smaller ones. Yet if information would be stored as propositional knowledge,
then all information should arguably be available to be retrieved in equal ways
and within the same time span.19 Kosslyn therefore argues that imagery is a sui
generis form of mental representation that possesses properties that are distinct
from those of linguistic or purely conceptual representations.
Based on such a view of how mental representations are realized in the
brain and how their set-up affects the way we access them, one can build an
argument for a parity of usage of inner and outer pictorial representations. This
means, one could argue for a similarity of the accessing of a map in Ottos notebook and the accessing of a mental representation. It seems that the details of

18
19

tingent fact, only realized in brain-bound processes. This kind of content constitutes
a mark of the cognitive (something that lacks in Clarks account).
Stephen M. Kosslyn: Image and Brain. The Resolution of the Imagery Debate,
Cambridge, MA 1994, p. 16.
See Stephen M. Kosslyn: Can Imagery Be Distinguished From Other Forms of
Internal Representation?, in: Memory and Cognition 4/3 (1976), pp. 291297.

40

JOERG FINGERHUT

such an account could easily be filled in and what just has been described admits
of a relatively straightforward translation into the original fictional Otto case
study.
Although such a parity argument is perfectly feasible it can be argued
that it might not be very illuminative with respect to what pictures actually can
do for our mind. Kosslyn defines pictures as pictures due to their preservation of
certain spatial features that also pertain to the object that they depict. Yet a
picture has to maintain only some (if any) spatial features in order to represent.
Consider an example often used by Kosslyn: it takes measurably longer to move
from the trunk of an imagined elephant to its tail (when e.g. asked questions
about the respective shapes of both body parts while imagining to see these
animals from the side) compared to the time it takes to shift mentally from a
rabbits nose to its tail. Yet this is not due to anything that pertains to pictures
per se. One can, for example, easily draw a rabbit and an elephant such that they
occupy the same space on a piece of paper. Such effects are rather based on properties we encounter with real elephants and rabbits and that are preserved in
imagery. One could therefore say that what explains the above effects of visual
imagery are foremost visual elements (or more precisely those of our standard
visual exploration of objects in our environment) and not primarily pictorial ones.
At times Kosslyn is very well aware of this when he states that the picture
metaphor of mental images wrongly implies that images are reperceived much
as pictures are perceived.20 Mental images, in his view, do not require figureground separation or contour sharpening in order to be perceived. To this I
would add that mental images also do not afford the perceptual relation of seeing-in when viewing them. The concept of seeing-in has been defended by
Richard Wollheim (among others) who argues for the twofoldness thesis of our
experience of pictures: the visual awareness [is] not only of what is represented
but also of the surface qualities of the representation.21 In mental imagery this
twofoldness does not seem to be part of the experience. When we imagine something we do not pick up on the properties of the object that does the representing. (After all: what would this object be? The respective brain state does not
seem to be a good candidate for this.)22 With real pictures we do: we not only
20

21
22

Stephen M. Kosslyn/James Pomerantz: Imagery, Propositions, and the Form of

Internal Representations, in: Cognitive Psychology 9/1 (1977), pp. 5276, p. 54. See
also Kosslyn: Image and Brain (as fn. 18), p. 13: Clearly, mental images are not
actual pictures in the head (there is no light up there, and besides, who would look
at them?).
Richard Wollheim: Art and Its Objects, Cambridge, MA 21980, pp. 214f.
Things get more complicated in cases where one specifically imagines to look at a
picture (to imagine it with a frame, canvas, etc.). This might constitute a case of
imagery where something akin to seeing-in is happening. But this nonetheless
seems quite hard to do. To test the possibility of imagining pictures in the mind,

41

EXTENDED IMAGERY, EXTENDED ACCESS, OR SOMETHING ELSE?

grasp the pictorial content but we also interact with the very materiality of
images (the canvas, the wall, etc.). We are even physiologically equipped to do
so. The two kinds of information (of the material properties of the object that we
share the same action space with and of the pictorial space) are processed in
parallel, i.e. in two separate pathways of our visual brain.23 To sum up: if we
endorse a strong account of pictures, such as the one just outlined, then there are
no pictures in the brain. Pictures are particular objects out there in the world
with properties that specifically pertain to them and we will have to focus on the
special actions they afford us.
We should however not be too hasty in dismissing parity claims completely. The parity argument does work on a certain level (Otto picks up visual
information from his notebook in a way that roughly resembles his internal
operations), and it may even work well as a means of initiating some doubts with
respect to a fully internalist view of the mind (just as the original Otto case
does). It nonetheless only takes us so far. With respect to pictures one could say
that parity accounts that start on the inside (e.g. by focusing on highly localized operations of the brain and on the capacity to evoke imagery in the absence
of direct interactions with the world) simply put the proverbial cart before the
horse. Instead our starting point should be that our embodied mind already acts
in a world that is occupied by artifacts and that it is intimately intermingled
with such a world (to use a phrase by John Haugeland24). We should thus focus
on those existing relations to pictures and how they might have enabled us to
achieve certain performances that only could have accrued from specific perceptual and bodily engagements with pictorial objects. Importantly, pictures are
objects that afford certain kinds of interactions. The desideratum therefore
should be to investigate directly these interactions and thereby to focus especially on those specifics that are not, or cannot be, reproduced in imagery.25

23
24

25

one could conduct a study and test the hypothesis that it should not take longer for
the head-to-tail jump when an elephant is imagined to be drawn on a piece of
paper where it occupies the same size as a rabbit. For something along this line see
the discussion of mental imagery in Evan Thompson: Mind in Life, Cambridge,
MA 2007, pp. 267311.
For a discussion of the Two Visual System Hypothesis and picture perception see
Mohan Matthen: Seeing, Doing, and Knowing, Oxford 2005, pp. 306318.
See John Haugeland: Mind Embodied and Embedded, in: Acta Philosophica Fennica 58 (1995), pp. 233267 reprinted in Haugeland: Having Thought. Essays in the
Metaphysics of Mind, Cambridge MA 1998, pp. 207237, p. 223: I want to suggest
that the human mind may be more intimately intermingled with its body and its
world than is any other, and that this is one of its distinctive advantages.
That imagery falls short of certain operations that we can undergo with external
pictures is also addressed in Andy Clark: Mindware, New York 22013. He refers to
experiments that show that our mental images seem to be more interpretively
fixed: less enabling of the discovery of novel forms and components (ibid., p. 186).

42

JOERG FINGERHUT

Homo Depictor
Artifacts, and among them pictures, make up what could be considered our cultural and cognitive niches. Such niches are constructed, not just occupied, and
they (as well as the objects they contain) fuel the power of the composite system
cognizer and object in ways that could not be explained by only making reference to processes within the organism. We attach meanings and offload information to objects that occupy our world. We, for example, rely on public maps
and road signs to find our way around cities; we sometimes draw complicated
arrangements and complex relations between thoughts we have on a piece of
paper in order to check whether things work out the way we imagine them. By
externalizing our ideas this way we communicate them to ourselves at the same
time as we are communicating them to others.
In the following, I will address the question of how external props complement our inner resources with a special focus on pictorial props and how
inner and outer resources together make us the kinds of minds we are. While
one excellent way to understand the complex workings of pictures would be to
address their role in the specific evolution of certain thoughts and ideas and by
excavating some important changes in concepts they engendered (as it is sometimes done in the history of art and pictures and in media theory) I want to take
a rather more fundamental perspective. By this I mean the way philosophers
(before the advent of 4EA approaches in the philosophy of mind) explored how
our minds might be defined by the very objects that we either pick up in the
environment or that we construe ourselves. Consider for example what Ian
Hacking calls his anthropological fancy. Hacking conjures up an image of
man that goes beyond the widespread focus on language as the key element of
what makes our minds human. He highlights another, also specifically human
way to build relations to our environment: modern humans became capable of
making and interacting with objects that were likenesses of the things around
them. In other words, humans make and interact with objects of a particular
kind, namely those that have the power to represent. Hacking emphasizes the
material status of these representations and the ways in which they afford
exploration. He writes in regard to this: When I speak of representations I first
of all mean physical objects: figurines, statues, pictures, engravings, objects that
One example of an operation that can be better performed with external objects
than with mental images is the switching between two versions of a bi-stable
image (like the famous duck/rabbit). Several participants of a study were only able
to switch between images once they actually drew them based on their memory
and perceived them externally. See Deborah Chambers/Daniel Reisberg: Can Mental Images Be Ambiguous?, in: Journal of Experimental Psychology 11/3 (1985),
pp. 317328.

43

EXTENDED IMAGERY, EXTENDED ACCESS, OR SOMETHING ELSE?

are themselves to be examined, regarded.26 Language is sometimes considered

to be the uncircumventable a priori of our mentality.27 In contrast, Hacking
claims while nonetheless keeping language and the concept of humans as speakers as a necessary and important contrast class that we are first and foremost
homo depictors.28 If we underwrite the claim that the relation to pictorial artifacts is in such a way constitutive for the modern human, it should also be
expected that related claims become important explanatory elements in theories
of our mind.
There are at least three ways in which our mental relations to the world
might be determined or even constituted by pictures. Firstly, pictures are part of
our external tool set and help us master cognitive tasks (i.e. they are complementary to our inner cognitive apparatus). This is a development of Clarks initial extended mind idea (i.e. before its curtailment on parity that is based on
similarities to inner processes), which was discussed in the first section of this
paper. Pictures on such an account constitute an extended solution space. Secondly over and above their role in specific solutions to cognitive problems
pictures may alter our perceptual skills and thereby our experiential relationship to the world: we might phenomenally experience the world differently by
virtue of being schooled by and having interacted with pictures in our evolutionary past as well as in the culture that we are brought up in. Pictures are
therefore part of our extended perceptual access. The third aspect I want to
highlight gives this experiential element a further spin. A central, yet often
overlooked, corollary of the homo depictor account is the idea that since the
prehistoric past we have filled our world with objects that not only have representational but also emotional and affective powers: objects fascinate us, address

26
27

28

Ian Hacking: Representing and Intervening. Introductory Topics in the Philosophy

of Natural Science, Cambridge, MA 1983, p. 133.
As, e.g., in the Wittgensteinian sense that our access to mental states is necessarily
mediated by a (public) language. But also in the following way: paradigmatic mental states are often construed along the lines of inner sentences of a language of
thought and it has been claimed that only as such they exhibit the features of systematicity and productivity that characterize our minds. See Fodor: LOT 2 (as fn. 4).
See for this concept: Hacking: Representing and Intervening (as fn. 26), pp. 132ff.
Recently the rising field of cognitive archeology has also addressed such claims.
One has to be careful, however, to not collapse Hackings homo depictor idea into
the more common homo faber aspect of hominid evolution. The latter concept and
its relation to the extended mind thesis is discussed in Lambros Malafouris: How
Things Shape the Mind, Cambridge, MA 2013. To claim that the profound complexity of our engagements with tools and technologies defines the modern
human (ibid., p. 154) in my view undercuts the homo depictor idea by not acknowledging the important roles tools can take on beyond their initial (practical and
cognitive) usefulness when they e.g. become ornamented or are seen as representations in Hackings sense.

44

JOERG FINGERHUT

us, they become items of veneration and worship. My conjecture is that a theory
of the pictorial mind can only claim its proper place within 4EA approaches
when it attends to all of these three aspects. They provide important bootstraps
for the multiple ways pictures could be theoretically incorporated into the sciences of the mind. I will address the three claims now in turn.

1. Pic t ures as Tools

From the viewpoint of the extended solutions space our visual environment
(and with it pictures) could be seen to be an extension of our possibilities for
thought and action, and as enhancing our cognitive efficiency and accuracy. This
aspect is, for example, reflected in the treatment of the role of pictures as a
means to bundle, reduce, or alter information to an ordered set of simpler pattern-completing operations of the kind our brains are most comfortable with.29
Artifacts that structure information in these ways become intimately integrated
in our problem solving routines. Signs, diagrams, charts, radiographs, etc. constitute information clusters that are not (and often cannot be) internally reproduced. Pictures have the power to depict previously inaccessible details, as well
as abstract contents and complex topics and make these all available in ways that
would not be graspable in the perception of a natural scene or in imagery alone.
Consider how Daniel Dennett in a paragraph entitled Making Things to
Think With discusses high-speed still photography as a revolutionary technological advance for science because it permitted human beings, for the first
time ever, to examine complicated temporal phenomena not in real time but in
their own good time in leisurely, methodical, backtracking analysis of the traces
they had created of those complicated events.30 Here pictures become a tool in
that they overcome the limitations of our visual perception for certain epistemic
tasks. This tool idea elucidates the extended mind claim most directly, since
artifacts here become integrated into cognitive problem solving routines that
otherwise would not be possible, were it not for features that they exhibit as
external objects. And it is not least the indispensability of these external features to a cognitive solution that makes it feasible to treat them as proper parts of
a cognitive process. External artifacts make us smarter by transforming the
cognitive tasks to something our biological brain can more readily deal with
(and sometimes enable it to operate in novel and advantageous ways) and it therefore seems increasingly arbitrary to single out the contribution of these external

29
30

Clark: Mindware (as fn. 25), p. 168.

Daniel C. Dennett: Kinds of Minds, New York 1996, pp. 134147, p. 143.

45

EXTENDED IMAGERY, EXTENDED ACCESS, OR SOMETHING ELSE?

elements as mere inputs to cognitive processes as opposed to them being part of

the process itself. 31

2. Pic t ures and Percept ion

Each of the three claims I am discussing builds on the fact that we have a complex and intricate sensorimotor, cognitive, affective apparatus that is concerned
with vision. This biological set-up constrains our cognitive and experiential life,
yet it also constitutes a large playing field of possible structured interactions
with the world. Based on this, one can wonder whether the tools and artifacts we
interact with could be considered to be changing vision itself. This idea needs
some introductory notes: Vision in some theories of perception especially in
those that highlight the enactive aspect within the 4EA approaches is primarily understood as a skill-based way of interacting with the environment. According to such theories, it is our structured interaction with the environment that
realizes perceptual meaning and not the brain itself (e.g. by constructing inner
representations of the environment). Consider the credo of sensorimotor enactivism: experience is a temporally extended activity of exploration mediated by
the perceivers knowledge of sensorimotor contingencies.32 They claim that the
content and character of what we perceive is fundamentally set up and constrained by what we can do.
First of all, consider how our very movements (e.g. involuntary saccades
of the eye or movements of the head and body) and the interdependent relations
between movements, proprioceptive feedback, and sensory stimulations play a
decisive role in this (as well as the morphological design of our eyes, head, and
body that only allow for certain kinds of information processing). These structured interactions determine what we experience but also how we experience
the world (the patterns underlying vision are e.g. quite different from those of

31

32

Such hybrid solutions for cognitive problems clearly fall under the scope of the
sciences of the mind, yet some additional qualifications i.e. specific coupling conditions between organism and artifact might be needed to justify referring to
them as proper parts of cognitive mechanisms. There is no fact of the matter that
settles such a question but rather a continuum that spans from cases where artifacts are so intimately integrated into a problem solving routine that they are a
proper part to cases where they rather function as scaffoldings to it. In the latter
sense, the mind is characterized as embedded in a world of artifacts and not in the
literal sense extended into it. For the embedded view see Kim Sterelny: Minds:
Extended or Scaffolded?, in: Phenomenology and the Cognitive Sciences 9/4 (2010),
pp. 465481.
J. Kevin ORegan/Alva No: A Sensorimotor Account of Vision and Visual Consciousness, in: The Behavioral and Brain Sciences 24/5 (2001), pp. 9391031, p. 961.

46

JOERG FINGERHUT

hearing or of taste). Such elements can be labeled prenoetic: they structure our
conscious experience without themselves necessarily becoming the topic of the
experience.33
Once we accept that the very means of interaction themselves constitute
our experience, vision itself no longer needs to be understood as being limited to
our biological apparatus. Vision includes and can change according to the different mediums (tools, technologies) through which we gain access to the world
and through which the world affects us. They are not just input to the system
but essential parts of the means of our access.
Based on this one can also consider how pictures give our perception
novel structures and thereby liberate us from biologically basic interactions
with the world.34 We should therefore take into account, for example, how art
can make certain connections visible that had not previously been conceived in
this way. Or think of the way pictures might even have made our ways of perceiving perceivable. Take depictions of forest wildlife in renaissance paintings
that cluster different animals as well as different stages of blossoming plants
(that could not coexist at the same space or time) as an example of the first. Take
textbook discussions of what impressionism or cubism did to our conception of
vision as a case of the latter (think e.g. of how a face-perception might be different before and after you have seen a deconstruction of a face in a Picasso painting).35
This might initially sound as if all that has been gained by such an account
is that pictures make new contents of perception available. But this would be a
rather trivial fact. I am gesturing instead towards the possibility that pictures
might also have a larger impact on us, namely that we have learned to perceive
differently (e.g. by picking up patterns of saccadic jumps pictures impose upon
us or by having made more conceptual relations perceivable that then become
employed in the exploration of the non-artifact world more generally) and that
our faculty of perceiving thereby has been enlarged and changed.36

33
34

35

36

See Shaun Gallagher: How the Body Shapes the Mind, New York 2005, pp. 133152.
An analogy might help to illuminate the role of pictures in structuring our access
to the world: the advent of the microscope not only allowed us to study new objects
(like micro bacteria), it also made a new skill-set available at the moment we
learned to understand what we were seeing and thereby enlarged our world: the
world we have access to, the world we can now more directly interact with, and the
world we understand as having an impact on us. The latter adds an important affective element to the story.
Or, in order to include a more contemporary example, consider how the perception
of color-constancy or light more generally is not the same after having been in one
of the color rooms conceived of by James Turrell.
This is of course a very contentious claim and one that would seem difficult to
prove either right or wrong. Proponent of the impenetrability of perception, e.g.,
would allow that we notice different things by being schooled by art and modern

47

EXTENDED IMAGERY, EXTENDED ACCESS, OR SOMETHING ELSE?

Pictures not only directly influence (motor-)action-and-perception cycles

through habituation with new viewing patterns but also more indirectly via the
understanding that guides our explorations of the environment (an understanding that is also subject to change). Our skill-based active exploration of features
of the environment therefore does not only rely on knowledge of expected
changes in inputs due to different bodily movements, but also on our more
intellectual faculties that alter what is accessible to us and thus changes what we
experience.37 This could constitutes an additional way that pictures and art might
affect changes on our experience of the world: by making novel things and ideas
visible that make in turn new means of accessing the world available to our
understanding.
Importantly, vision in the views just discussed is not only evolutionary
and developmentally plastic but is constantly subject to change even in the hereand-now.
We must of course be careful how we develop this perceptual access idea
and the role pictures might play therein. On the one hand, we should not underplay the stable and invariant aspect of visual experience (the very elements that
are exploited by different visual artforms) by carrying too far the alluring possibility that vision is changed by different media we interact with. But on the
other hand we should also take our plasticity seriously enough and not simply
treat pictorial objects as concurring to our biological visual system.
To illustrate this latter point consider how neuroaesthetics, in its early
stages, has embraced the idea of art as an extension of our perceptual skill set.
One of its founding fathers, Semir Zeki, claimed that the overall function of
art is an extension of the function of the brain.38 This seems like a welcome
inclusion of pictures as part of our access to the world. Given what I have discussed this far however, it should be clear that we must be careful not to subscribe to a view that ascribes to art (and pictures for that matter) very much the
same functions as it does to the brain. Zeki claims that both (brain and art)
identify the constant and essential features of objects and maintains that no
theory of aesthetics is likely to be complete, let alone profound, unless it is based

37

38

media, yet still would claim that we see the same thing and that our faculty of
perception remained unchanged. See Nol Carroll: Modernity and the Plasticity of
Perception, in: The Journal of Aesthetics and Art Criticism 59/1 (2001), pp. 1117.
For the role of understanding in enactivism, or actionism as he more recently calls
it, see Alva No: Varieties of Presence, Cambridge, MA 2012, pp. 9ff. I discuss
these issues in more detail, although not with respect to pictures in: The Body and
the Experience of Presence, in: Joerg Fingerhut/Sabine Marienberg (eds.): Feelings
of Being Alive, Berlin/Boston 2012, pp. 167199.
Semir Zeki: Art and the Brain, in: Journal of Consciousness Studies 6/67 (1999),
pp. 7696, p. 76.

48

JOERG FINGERHUT

on an understanding of the workings of the brain.39 I take the latter to be somewhat trivially true if one sees aesthetics as being informed by psychological
accounts at all. However, as long as Zeki models his view of art on the basis of
the internal brain processes underpinning vision, he inevitably underplays the
role of external artifacts. Art, in his view, must conform to the organizational
and functional principles of the brain. Yet there are operations that the biological brain would not be able to achieve were it not for some special relations that
have become possible due to the emergence of certain artifacts and were it not
for the special relation we hold to them in the here and now. The nature of such
operations would be necessarily left out in any account that complies with Zekis
model.

3. Pic t ures as Externa l Embod iment

By including questions of how we might construe new forms of perceptual
meaning through our enactive engagements with pictorial objects we already
shifted our scope away from the extended mind idea and its basic claims with
respect to extra-bodily vehicles of mental states. Clarks extended mind hypothesis is not a claim about the extended vehicles of experiential states. Such states
in his view are realized by a highly dense interplay of neural resources within
the central nervous system and not by loops through the environment.40 Clarks
extension claim gains its strength insofar as it relies on the idea that the mental refers to more than that which is consciously occurrent (as seen in the
example of dispositional beliefs) and should include the computationally salient
roles certain vehicles can play. It could therefore be said that both, Clarks complementary and his parity account, argue for a cognitive externalism that features the ways in which the widespread use of cognitive technologies (in a wide
sense) is able to expand and reshape the space of human reason and cognition.
I have suggested that we should in addition consider how pictures might
have changed our perceptual skills and how we consciously perceive the world.
The enactive theory I discussed would claim that experiences do not supervene
on the brain and that [t]he substrate of experience may include the non-brain
body, and the world.41 This claim has been defended by means of denying that
experiences are properties of states at all. According to the theory, it is a category error to consider brain states (or any thing more generally) as generating

39
40
41

Ibid., p. 94.
See Andy Clark: Spreading the Joy? Why the Machinery of Consciousness is
(Probably) Still in the Head, in: Mind 118/472 (2009), pp. 963993
See Alva No: Experience Without the Head, in: Tamar Gendler/John Hawthorne
(eds.): Perceptual Experience, New York 2006, pp. 411433, p. 430.

49

EXTENDED IMAGERY, EXTENDED ACCESS, OR SOMETHING ELSE?

experiences. Rather, experiences should be understood as modes of interactions

with (or modes of access to) the world.
What the two aspects discussed so far have in common is that they both
address pictures as being (or becoming) integrated into certain routines. Pictures might therefore be usefully understood as tools of the mind in the sense
that they provide an extended cognitive solution space and in the sense that they
structure our skillful perceptual relations to the world. However, pictures are
recalcitrant and stubborn tools; something in them fundamentally resists integration and invites further exploration.42 Seeing pictures primarily through
extended mind goggles (or through the lenses of other similar claims) would not
only disguise this key insight but also the fact that we stand in a particular relation to pictures as external objects, a relation that is importantly dissimilar to
that which we normally entertain in regard to external objects. This is because
some pictures from early cave paintings to works in the history of art could
be considered paradoxical objects. We experience them as being both close and
transcending space and time, as being both inanimate and artificial and emotionally engaging us up to the point that we experience and treat them as
agents.43 From this it should be clear that pictures might not have been exclusively selected for their cognitive functions.
That pictures bear affective meanings, meanings that address us in specific ways, is the third and final aspect I want to highlight. Pictures might be just
the kind of objects that, at some point in our evolutionary history, have developed the power to look back. By this I mean that they were so full of meaning,
emotion, and affective potential that they fell somewhat out of the natural order
of things (think of the advent of animism but also of the emotional power that
great works of art can exert upon us) while they at the same time somehow
remained to be objects. Such objects therefore do not just modify our cognitive
computing power, nor do they just make things visible that previously had not
been in sight and teach us to see differently. Critically, they also occupy our
world with a very different kind of object. This is why I think that their external
embodiment should be a crucial part of our account: pictures stand out insofar
as they present us with man-made objects (or material structures in the environment that were picked up as meaningful) that we experience as beautiful, as
uncanny, as challenging, and as inviting emotional and cognitive discovery.
They are objects that embody ideas and contents in specific pictorial ways and

42
43

As Alva No (see fn. 6) and Andy Clark (see fn. 25) seem to acknowledge with
respect to specific aspects of pictures.
See for this Horst Bredekamp: Theorie des Bildakts. Frankfurter Adorno-Vorlesungen 2007, Frankfurt/M. 2010, and his article in the present volume.

50

JOERG FINGERHUT

therefore also afford specific kinds of engagements.44 What I want to take away
from this aspect of our interactions with pictures is that pictures gain pictorial
status not least because they are not integrated fully into our cognitive or skillbased routines.

Conclud ing Remarks

While the aspects outlined above are still somewhat speculative, I believe that
they gesture towards promising research questions. Once we change the direction of explanation and address our interactions with pictures more directly, we
can include aspects whose relevance for philosophy of mind has up until now
been largely undervalued. Yet, as seen in the last sections, they might not be
captured well under the general umbrella of the extended mind hypothesis.
There is one final element that was implicit in the above claims but deserves
some special attention to defend against possible misconceptions of the homo
depictor account. It should be clear that pictures might originally have started
out as representations, as likenesses in Hackings words, but from the very
beginning they have also presented us with styles, modes of access, and powers
that work independently of any rigid relation that they have to what they depict.
What follows from this is that all mainly representational accounts of pictures
must fall short of what a theory of pictures in the sense envisioned here
should aim at. I mention this in closing because it underscores my plea in favor
of an account that primarily addresses our interactions with pictures and that
defines pictures primarily with respect to specific organism-object relations.
Only when we grasp such characteristic properties of our interactions with pictorial objects more directly will we be able to relate them to our other embodied,
enactive relations to the world. Only then will a more general account of pictures be able to make headway and truly inform us about the specific kinds of
minds we are.45

44

45

We already encountered one type of engagement with pictures that in my view is

fundamental and that could also be brought to bear on explanations of the paradoxical nature of pictures, namely that of seeing-in. Pictures enable us to see
something in an object that we share a common space with, yet that at the same
time transcends this space.
I am grateful to Mog Stapleton for helpful comments on drafts of this paper.