There are entire academic programs devoted to the psychology of seeing [however] in reality it is still unclear to us how at all we see anything. This fact is hardly ever communicated to the students (Crick, 1997)
At the interface of psychology, humanities, and neuroscience
Although images created intentionally by humans dominate contemporary civilization, psychologists are present in the discourse on the specificity of the influence of the image on the mind of the man only to a slight extent. In 1946 an eminent German art historian, Max Jakob Friedländer, wrote: “Because art is a domain of the spirit, every scientific study on it shall belong to psychology.” It may as well belong to other fields of science, yet psychology will always be one of them” (pp. 128).
Nonetheless, the 20th century saw a paradoxical situation — studies on the image, referring to theses which definitely derive from cognitive psychology, perception, emotions and personality, and even clinical psychology as well as neuropsychology, were not conducted by psychologists, but by anthropologists (for example Belting, 2007; Freedberg, D., 2005; Rose, 2010), philosophers (e.g. Eliade, 2009; Florenski, 1984; Ingarden, 1958; Sartre, 2012; Wunenburger, 2011), semiologists (e.g. Barthes, 1996; Eco, 1996), art historians (e.g. Arnheim, 1978; 2011; Didi-Huberman, 2011a; 2011b; 2012; Gage, 2010; Gombrich, 1981; 2005; 2009; 2011) and artists/theorists (e.g. Berger, 1999; 2009; Kandinsky, 1986; Strzemiński, 1974).
Over and above that, an overwhelming part of these studies were based on speculative, empirically unverified concepts which freely — and sometimes quite superficially — refer to psychoanalytical, structuralist hypotheses (for example gestalt), and since the middle of the 20th century — also cognitive and neurocognitive ones. These references most often concern results of studies that were not conducted on complex, real images, but rather simplified visual stimuli, used in psychological experiments. Only since less than 20 years a dozen or so monographs concerning perception of image, understood as the subject of an aesthetic experience, that were written by cognitive psychologists have been published. Even though, to a large extent, they still refer to general rules concerning seeing, while confronting them with examples taken from museums they increasingly often present also the results of experiments in which paintings constituted the subject matter (for example Kandel, 2012; Livingstone, 2002; Ramachandran and Hirstein, 1999; Zeki, 1993; 1999; 2009) and cognitive psychologists (Efland, 2002; Solso, 1996).
To summarise, on the map of the contemporary knowledge, somewhere at the border of psychology, humanities (philosophy, history of art, anthropology) and natural sciences (neurobiology, neurophysiology), there is a white spot of empirically unverified knowledge related to human reactions to an intentionally created image. This book is a proposal to explore this terra incognita.
It is worth beginning the studies on neurocognitive foundations of perception of a visual image by presenting several fundamental findings regarding the nature of the image and its relation to its creator and the recipient. I treat them as assumptions that explicitly express my views regarding the order of things in the field of knowledge in question.
Definition of image
An image is a flat or spacious object, which is either static or dynamic, and has been created by a man (most often an artist) in order to encode a certain content (sense) within it, using methods which vision is sensitive to. Most often, the content of the image is a representation of something or someone using a specific technique (i.e. painting, photography, television, sculpting or theatre), however, its content may be also abstract (non-representing). Regardless of the content and creation technique, the final matter that paintings are made of is light, in the range of approximately 400–700 nanometres of the electromagnetic wave length of variable intensity.
Within this monograph, the notion of painting is limited to motionless and flat visual representations, such as paintings, photographs, posters, billboards or drawings, regardless of whether they represent particular objects, people or scenes, or are of abstract nature. Such understanding of image limits the scope of mental and neurophysiological mechanisms to those that are responsible for seeing shapes and colors, and — to a certain extent — the depth of the depicted scene, on the basis of the so-called monocular indicators. Thus, it removes from the field of interest the issues related to perception of movement and three-dimensionality of the image available stereoscopically (binocularly). This limitation results from the complexity of the subject concerning vision, especially the object is an image, created intentionally by a man. Even though multiple vision mechanisms are equally involved in viewing images as well as real scenes, human visual system often faces completely new challenges, due to the forms of image creation and exposure, and to the content of images. Incidentally, it is one of the basic functions of visual arts.
Real visual scene vs. image
From the perspective of an observer, images constitute a subset of objects that are presented to him or her during a subjective experience of seeing, when his or her eyes are open. Many rules regarding visual perception are, therefore, equally related to all visual scenes, including images. A photograph can represent a famous person. However, the basic difference between seeing them “in person” and in a picture boils down to the fact that in the first situation there is at least one intermediary less than in the second one. We usually do not realize that seeing an object, person or scene that is captured within the frame of painting on canvas, a photograph, printed sheet or a computer screen, is an entirely different visual experience for the observer than looking at those things directly, without the intermediation of the listed media. Thus, the way of depicting a real visual scene in an image modifies its understanding, interpretation and value attribution in comparison with the understanding, interpretation and value attribution of a scene that is not mediated by it. It is due to the fact that the intent of a creator and the means of expression applied by them create an entirely new reality enclosed in the frames of an image.
If we assume that the photograph by Pablo Picasso, presented on the right in Fig. 1 is an unmediated visual experience (that is, let us presume for a moment that we can see the artist “in person”), then the caricature on the left illustrates well the concept of an image as a distorted representation of an original visual scene.
In reality, however, the image on the right is burdened with similar distortions with respect to the original as the caricature with respect to the photograph. Merely the fact that the person who was alive at the time the photograph was taken presents himself to us as frozen in time, and constantly available in the same form makes it an unreal object. Moreover, the person is monochromatic, which, in confrontation with an everyday visual experience, is rather unique, because only in unfavourable lighting conditions “the world becomes colorless”. Being used to seeing photographs we no longer pay attention to such details as, for example, the fact that the person lacks not only hands below the elbows, but the entire bottom part of his body. Of course, in our everyday experience we do not always see entire silhouettes of people, because they are most often hidden behind something. The rule, however, does not apply to the analysed photograph. Pablo Picasso is not hidden behind anything, unless we, quite valiantly, assume that in the photograph his entire body is hidden behind the edges of the rectangular hole cut out in paper.
The question is, however: what obscures most of the body of Frank Kramer, the showman, while he is pictured on the billboard of a famous California radio show entitled The Heidi and Frank show (Fig. 2)?
Individual differences in visual perception
A visual scene — mediated or unmediated by an image — is the content of a subjective experience of an observer who disposes of a biological device able to register and process the intensity and the length of an electromagnetic wave in the range of visible light. The observer of the image is not only its recipient, but also the creator who, in the act of creation, constantly gazes at the results of his work and conforms them to his vision. Even though the structure and functioning of the neural visual pathways in different people is generally similar, significant differences that influence the process of creation and reception of images may occur already at the neurophysiological and neuroanatomical level.
It can be exemplified by the oeuvre of Vincent van Gogh who suffered from temporal lobe epilepsy for many years (see: Arnold, 1992; Gastaut, 1956). The disease presents itself with intense, often peculiar sensations (for instance visual), accompanied by powerful emotions. The paintings of van Gogh perfectly reflect the way in which he most likely saw the world during the attacks that accompanied his work. His visual experiences shaped his well-known painting style, an example of which could be “Starry Night”, painted one month after he was admitted to the Saint-Paul-de-Mausole hospital close to Saint-Rémy (Fig. 3). Despite numerous theories, in the light of which the painting is interpreted, it first and foremost was a product of the poor condition of the artist’s central nervous system, more precisely — the complexes of cooperating neurons located at various levels of visual pathway.
The case of Vincent van Gogh is by no means isolated. Many people, including both the creators and recipients of images, suffer from various vision-related conditions. For instance, astigmatism can significantly affect the shape of seen objects. In turn, damage to some parts of the brain’s visual cortex may lead to shape agnosia, i.e. partial or complete loss of the ability to recognise and copy objects. A special case of shape agnosia is prosopagnosia, i.e. the inability to recognise faces.
The electromagnetic wave in the range of continuous visual spectrum can cause various color effects in different people due to their visual system status. For example, yellowing of the cornea in old age reduces the sensitivity to blue color, causing that the world is seen in a more autumnal tone. Achromatopsia, in turn, manifests itself in the form of an inability to differentiate colors. It is caused by defects in the structure and functioning of the retina or damage to one of the brain’s centres dealing with the analysis and interpretation of the data on wavelength of light.
A serious spatial orientation disorder in the visual scene can be, e.g. hemispatial neglect, which manifests itself in difficulties in perceiving half of the space located on the opposite side of the hemisphere brain damage. Even a slight damage to the cerebral cortex in one part of the parietal lobe may lead to akinetopsia, that is, partial or complete loss of vision. There is a long list of vision disturbances that could be elaborated on.
Although all the mentioned damage to the visual pathways and vision deficits and disturbances can significantly affect both the perception of the visual scene and its creation in the form of an image, they nevertheless will not be the subject of any detailed analysis in this book. Discussing them would require a separate monograph, modelled on excellent studies concerning this issue, including the ones by Michael F. Marmor and James G. Ravin (1997; 2009).
Neural stages of visual pathway
Understanding, interpreting and evaluating the content of images are derivatives of a multistage neurophysiological process. It begins with recording the light – reflected or emitted by objects in the visual scene – using photoreceptors found in the retinas of both eyeballs. From the moment when the photon beam enters the eye’s interior, the physiological response of photosensitive cells becomes the subject of complex neural processes that occur first in the retina and then in many subcortical structures of the brain. This is called the early stage of visual data processing. It results in relatively ordered information on the specifics of the lighting of the space occupied by the observer, gradually transferred to the cortex structures of the brain. More precisely, the cerebral cortex is constantly being informed about the distribution of the light recorded but also processed by millions of neurons forming the early stage of the visual pathway. Their task is not only to transfer, but also process data flowing from the photoreceptors to the brain.
The second, so-called higher stage of visual processing takes place in the cerebral cortex of the observer. When signals from the eyes reach its various parts, the most mysterious process is initiated, the results of which we experience as the subjective experience of seeing. Its components comprise, among other things, recognition, understanding, interpretation of scenes, mood, intensity and direction of emotional arousal and value attribution. These sensations result not only from the stimulation of the photoreceptors in the retina. On the contrary, one of the most important neurocognitive processes, which underlies the experience of seeing, is the process of comparing the incoming sensory data with the data that the recipient has already stored in their brain.
On the one hand, these neuronal carriers of visual memory are a legacy of our species evolution. They are similar in all human brains, both in terms of their structure and function. The neuroanatomical structure, location and function of neural groups, which analyse the color or shape of the objects viewed may be an example of the above.
On the other hand, brain records of visual memories are shaped by means of subjective experiences of the observer within their entire life. The frequency of visual contacts with a given class of objects and their features is of decisive importance for them. All these brain records are carriers of individual visual memory, namely of the knowledge of the way the world looks, which to a large extent determines the quality of the subjective visual experience.
Therefore, determining to what extent the painting can be sensibly interpreted in the light of the possessed visual knowledge is of key importance for understanding the contents of a painting. Only as a result of detecting a similarity between the inflowing sensory data and the data which were stored in the memory before, is the brain able to “formulate” and then “verify” the hypotheses concerning the contents of a given painting. Generally speaking, the things that constitute the content of the visual experience hardly reflect what is currently present in the observer’s field of vision, but are the most probable hypothesis concerning this reality. This hypothesis was formulated on the basis of the level of conformity between the inflowing sensory data and the possessed visual knowledge.
The secret of the subjective experience of seeing
The experience of seeing is undoubtedly the result of the brain’s work. Its endings, which are sensitive to light, are placed in the eye’s retina of the observer. It is worth remembering, however, that as much as we currently know a lot about the organisation and the function of the so-called neural visual pathways, running from the photoreceptors to multiple areas in the cerebral cortex, we still have no idea whatsoever how we experience seeing resulting from the activity of the neurons forming these pathways. To put it briefly, the contemporary science currently does not possess any kind of empirically verifiable hypothesis which would bring us closer to solving the problem of the mind-brain relationship.
The knowledge on the matter that we currently have is the result of research aimed at searching for relationships between the contemporary states of mind (e.g. seeing something) and the activity of various brain structures involved in the process of seeing, recorded at those moments. The starting point for seeking these relationships are the results of studies conducted on the basis of biological sciences, for example anatomy, physiology or biochemistry, as well as psychology, especially cognitive psychology, emotions and neuropsychology and many other sciences and humanities. The contemporary cognitive neuroscience lies where those research areas meet. Its basic aim is to search for the neural correlates of various mental processes (Bremer, 2005; Chalmers 2000; Crick and Koch, 1998).
Today, it is difficult to predict whether the knowledge gathered on the basis of neuroscience will ever allow us to understand how it is possible that billions of neurons, the sole task of which is receiving and sending bioelectric signals to other cells, generate a state that we experience as seeing. Regardless of that, however, the results of neurocognitive studies already allow us to understand better the nature of those subjective experiences. Knowing, for example, the distribution of various types of photoreceptors in the retina of the eye and how they are connected with each other we can understand why while looking at a visual scene we see some of its elements sharply and others are more blurred, why in certain lighting conditions we see colors and in others — we do not, or how we see that something is moving.
The presented findings determine the framework of this book. They constitute a foundation on which I shall place blocks of knowledge on biological basics of seeing, and, in particular, seeing in the acts of creation and painting reception. The fundamental question that this monograph answers is the question of how the human visual system, responsible for every act and aspect of the subjective experience of seeing, analyses, interprets and values seen images. Here, the visual system means both the neuroanatomical basis for the flow of the sensory data from the photoreceptors to all structures which become activated during the acts of seeing, that is everything that is most often referred to as a visual pathway(s) or neural paths and structures as well as the so-called higher mental processes (cognitive, emotional and other) which constitute the essence of the subjective experience of seeing.
While discussing the subsequent stages of visual data processing, I will focus only on the structures whose functions are important for understanding of the image viewing process. In other words, if I assume that the reader should know, for instance, the biochemical basics of some biological process, then I will present them in detail. On the contrary, if the functions of some element of the visual pathway structure is not recognised well or its detailed presentation will make it possible to explain a given mental phenomenon related to seeing images only to a small extent, then I will not focus on it in a detailed way, referring a curious reader to specialist literature on the subject.
In order to understand the content of this book, a minimum level of knowledge on the structure and functioning of neurons and the human brain, from a high school biology course, is sufficient. A little knowledge on mathematics and optics can also be useful. References to paintings which constitute illustrations or the subject of the experimental research also do not require advanced knowledge of history of art and visual communication.
What is this book about?
The book consists of five chapters and in a sense reflects two orders. The first one is determined by the chronology of events on the visual pathways in the acts of seeing, and the other is related to the method of narration, that is top-down approach.
In chapter I entitled “Neurocognitive basis of seeing”, I discuss two basic visual systems: the analysis of the contents of a painting and the framing of a visual scene. Activity of both systems is presented in the light of neurobiological research results, which I relate to various cognitive functions.
Chapter II, III and IV concern three most important features of every visual scene, namely the shape of the objects seen, their color and order in a three-dimensional space. The description lacks one more characteristic of a scene — motion. Because static images are the subject of the analyses in the book, I also omit the issue of image dynamic, solely discussing the issues related to the dynamic of the oculomotor system.
In the final chapter — the 5th — I discuss the issues related to aesthetic valuing of paintings by presenting results of oculographic studies. They are aimed at searching for oculomotor correlates of beauty.
Instead of an ending, I present a list of topics and research areas, which this monograph should be extended by in its following issues.