INTRODUCTION

There are entire aca­d­e­m­ic pro­grams devot­ed to the psy­chol­o­gy of see­ing [how­ev­er] in real­i­ty it is still unclear to us how at all we see any­thing. This fact is hard­ly ever com­mu­ni­cat­ed to the stu­dents (Crick, 1997)

At the interface of psychology, humanities, and neuroscience

Although images cre­at­ed inten­tion­al­ly by humans dom­i­nate con­tem­po­rary civ­i­liza­tion, psy­chol­o­gists are present in the dis­course on the speci­fici­ty of the influ­ence of the image on the mind of the man only to a slight extent. In 1946 an emi­nent Ger­man art his­to­ri­an, Max Jakob Friedlän­der, wrote: “Because art is a domain of the spir­it, every sci­en­tif­ic study on it shall belong to psy­chol­o­gy.” It may as well belong to oth­er fields of sci­ence, yet psy­chol­o­gy will always be one of them” (pp. 128).

Nonethe­less, the 20th cen­tu­ry saw a para­dox­i­cal sit­u­a­tion — stud­ies on the image, refer­ring to the­ses which def­i­nite­ly derive from cog­ni­tive psy­chol­o­gy, per­cep­tion, emo­tions and per­son­al­i­ty, and even clin­i­cal psy­chol­o­gy as well as neu­ropsy­chol­o­gy, were not con­duct­ed by psy­chol­o­gists, but by anthro­pol­o­gists (for exam­ple Belt­ing, 2007; Freed­berg, D., 2005; Rose, 2010), philoso­phers (e.g. Eli­ade, 2009; Flo­ren­s­ki, 1984; Ingar­den, 1958; Sartre, 2012; Wunen­burg­er, 2011), semi­ol­o­gists (e.g. Barthes, 1996; Eco, 1996), art his­to­ri­ans (e.g. Arn­heim, 1978; 2011; Didi-Huber­man, 2011a; 2011b; 2012; Gage, 2010; Gom­brich, 1981; 2005; 2009; 2011) and artists/theorists (e.g. Berg­er, 1999; 2009; Kandin­sky, 1986; Strzemińs­ki, 1974).

Over and above that, an over­whelm­ing part of these stud­ies were based on spec­u­la­tive, empir­i­cal­ly unver­i­fied con­cepts which freely — and some­times quite super­fi­cial­ly — refer to psy­cho­an­a­lyt­i­cal, struc­tural­ist hypothe­ses (for exam­ple gestalt), and since the mid­dle of the 20th cen­tu­ry — also cog­ni­tive and neu­rocog­ni­tive ones. These ref­er­ences most often con­cern results of stud­ies that were not con­duct­ed on com­plex, real images, but rather sim­pli­fied visu­al stim­uli, used in psy­cho­log­i­cal exper­i­ments.  Only since less than 20 years a dozen or so mono­graphs con­cern­ing per­cep­tion of image, under­stood as the sub­ject of an aes­thet­ic expe­ri­ence, that were writ­ten by cog­ni­tive psy­chol­o­gists have been pub­lished.  Even though, to a large extent, they still refer to gen­er­al rules con­cern­ing see­ing, while con­fronting them with exam­ples tak­en from muse­ums they increas­ing­ly often present also the results of exper­i­ments in which paint­ings con­sti­tut­ed the sub­ject mat­ter (for exam­ple  Kan­del, 2012; Liv­ing­stone, 2002; Ramachan­dran and Hirstein, 1999; Zeki, 1993; 1999; 2009) and cog­ni­tive psy­chol­o­gists (Efland, 2002; Sol­so, 1996).

To sum­marise, on the map of the con­tem­po­rary knowl­edge, some­where at the bor­der of psy­chol­o­gy, human­i­ties (phi­los­o­phy, his­to­ry of art, anthro­pol­o­gy) and nat­ur­al sci­ences (neu­ro­bi­ol­o­gy, neu­ro­phys­i­ol­o­gy), there is a white spot of empir­i­cal­ly unver­i­fied knowl­edge relat­ed to human reac­tions to an inten­tion­al­ly cre­at­ed image. This book is a pro­pos­al to explore this ter­ra incog­ni­ta.

It is worth begin­ning the stud­ies on neu­rocog­ni­tive foun­da­tions of per­cep­tion of a visu­al image by pre­sent­ing sev­er­al fun­da­men­tal find­ings regard­ing the nature of the image and its rela­tion to its cre­ator and the recip­i­ent. I treat them as assump­tions that explic­it­ly express my views regard­ing the order of things in the field of knowl­edge in question.

Definition of image

An image is a flat or spa­cious object, which is either sta­t­ic or dynam­ic, and has been cre­at­ed by a man (most often an artist) in order to encode a cer­tain con­tent (sense) with­in it, using meth­ods which vision is sen­si­tive to.  Most often, the con­tent of the image is a rep­re­sen­ta­tion of some­thing or some­one using a spe­cif­ic tech­nique (i.e. paint­ing, pho­tog­ra­phy, tele­vi­sion, sculpt­ing or the­atre), how­ev­er, its con­tent may be also abstract (non-rep­re­sent­ing).  Regard­less of the con­tent and cre­ation tech­nique, the final mat­ter that paint­ings are made of is light, in the range of approx­i­mate­ly 400–700 nanome­tres of the elec­tro­mag­net­ic wave length of vari­able intensity.

With­in this mono­graph, the notion of paint­ing is lim­it­ed to motion­less and flat visu­al rep­re­sen­ta­tions, such as paint­ings, pho­tographs, posters, bill­boards or draw­ings, regard­less of whether they rep­re­sent par­tic­u­lar objects, peo­ple or scenes, or are of abstract nature.  Such under­stand­ing of image lim­its the scope of men­tal and neu­ro­phys­i­o­log­i­cal mech­a­nisms to those that are respon­si­ble for see­ing shapes and col­ors, and — to a cer­tain extent — the depth of the depict­ed scene, on the basis of the so-called monoc­u­lar indi­ca­tors. Thus, it removes from the field of inter­est the issues relat­ed to per­cep­tion of move­ment and three-dimen­sion­al­i­ty of the image avail­able stereo­scop­i­cal­ly (binoc­u­lar­ly). This lim­i­ta­tion results from the com­plex­i­ty of the sub­ject con­cern­ing vision, espe­cial­ly the object is an image, cre­at­ed inten­tion­al­ly by a man.  Even though mul­ti­ple vision mech­a­nisms are equal­ly involved in view­ing images as well as real scenes, human visu­al sys­tem often faces com­plete­ly new chal­lenges, due to the forms of image cre­ation and expo­sure, and to the con­tent of images.  Inci­den­tal­ly, it is one of the basic func­tions of visu­al arts.

Real visual scene vs. image

From the per­spec­tive of an observ­er, images con­sti­tute a sub­set of objects that are pre­sent­ed to him or her dur­ing a sub­jec­tive expe­ri­ence of see­ing, when his or her eyes are open. Many rules regard­ing visu­al per­cep­tion are, there­fore, equal­ly relat­ed to all visu­al scenes, includ­ing images.  A pho­to­graph can rep­re­sent a famous per­son. How­ev­er, the basic dif­fer­ence between see­ing them “in per­son” and in a pic­ture boils down to the fact that in the first sit­u­a­tion there is at least one inter­me­di­ary less than in the sec­ond one. We usu­al­ly do not real­ize that see­ing an object, per­son or scene that is cap­tured with­in the frame of paint­ing on can­vas, a pho­to­graph, print­ed sheet or a com­put­er screen, is an entire­ly dif­fer­ent visu­al expe­ri­ence for the observ­er than look­ing at those things direct­ly, with­out the inter­me­di­a­tion of the list­ed media.  Thus, the way of depict­ing a real visu­al scene in an image mod­i­fies its under­stand­ing, inter­pre­ta­tion and val­ue attri­bu­tion in com­par­i­son with the under­stand­ing, inter­pre­ta­tion and val­ue attri­bu­tion of a scene that is not medi­at­ed by it. It is due to the fact that the intent of a cre­ator and the means of expres­sion applied by them cre­ate an entire­ly new real­i­ty enclosed in the frames of an image.

If we assume that the pho­to­graph by Pablo Picas­so, pre­sent­ed on the right in Fig. 1 is an unmedi­at­ed visu­al expe­ri­ence (that is, let us pre­sume for a moment that we can see the artist “in per­son”), then the car­i­ca­ture on the left illus­trates well the con­cept of an image as a dis­tort­ed rep­re­sen­ta­tion of an orig­i­nal visu­al scene.

In real­i­ty, how­ev­er, the image on the right is bur­dened with sim­i­lar dis­tor­tions with respect to the orig­i­nal as the car­i­ca­ture with respect to the pho­to­graph. Mere­ly the fact that the per­son who was alive at the time the pho­to­graph was tak­en presents him­self to us as frozen in time, and con­stant­ly avail­able in the same form makes it an unre­al object. More­over, the per­son is mono­chro­mat­ic, which, in con­fronta­tion with an every­day visu­al expe­ri­ence, is rather unique, because only in unfavourable light­ing con­di­tions “the world becomes col­or­less”. Being used to see­ing pho­tographs we no longer pay atten­tion to such details as, for exam­ple, the fact that the per­son lacks not only hands below the elbows, but the entire bot­tom part of his body.  Of course, in our every­day expe­ri­ence we do not always see entire sil­hou­ettes of peo­ple, because they are most often hid­den behind some­thing. The rule, how­ev­er, does not apply to the analysed pho­to­graph. Pablo Picas­so is not hid­den behind any­thing, unless we, quite valiant­ly, assume that in the pho­to­graph his entire body is hid­den behind the edges of the rec­tan­gu­lar hole cut out in paper.

The ques­tion is, how­ev­er: what obscures most of the body of Frank Kramer, the show­man, while he is pic­tured on the bill­board of a famous Cal­i­for­nia radio show enti­tled The Hei­di and Frank show (Fig. 2)?

Individual differences in visual perception

A visu­al scene — medi­at­ed or unmedi­at­ed by an image — is the con­tent of a sub­jec­tive expe­ri­ence of an observ­er who dis­pos­es of a bio­log­i­cal device able to reg­is­ter and process the inten­si­ty and the length of an elec­tro­mag­net­ic wave in the range of vis­i­ble light. The observ­er of the image is not only its recip­i­ent, but also the cre­ator who, in the act of cre­ation, con­stant­ly gazes at the results of his work and con­forms them to his vision. Even though the struc­ture and func­tion­ing of the neur­al visu­al path­ways in dif­fer­ent peo­ple is gen­er­al­ly sim­i­lar, sig­nif­i­cant dif­fer­ences that influ­ence the process of cre­ation and recep­tion of images may occur already at the neu­ro­phys­i­o­log­i­cal and neu­roanatom­i­cal level.

It can be exem­pli­fied by the oeu­vre of Vin­cent van Gogh who suf­fered from tem­po­ral lobe epilep­sy for many years (see: Arnold, 1992; Gas­taut, 1956). The dis­ease presents itself with intense, often pecu­liar sen­sa­tions (for instance visu­al), accom­pa­nied by pow­er­ful emo­tions.  The paint­ings of van Gogh per­fect­ly reflect the way in which he most like­ly saw the world dur­ing the attacks that accom­pa­nied his work.  His visu­al expe­ri­ences shaped his well-known paint­ing style, an exam­ple of which could be “Star­ry Night”, paint­ed one month after he was admit­ted to the Saint-Paul-de-Mau­sole hos­pi­tal close to Saint-Rémy (Fig. 3). Despite numer­ous the­o­ries, in the light of which the paint­ing is inter­pret­ed, it first and fore­most was a prod­uct of the poor con­di­tion of the artist’s cen­tral ner­vous sys­tem, more pre­cise­ly — the com­plex­es of coop­er­at­ing neu­rons locat­ed at var­i­ous lev­els of visu­al pathway.

The case of Vin­cent van Gogh is by no means iso­lat­ed. Many peo­ple, includ­ing both the cre­ators and recip­i­ents of images, suf­fer from var­i­ous vision-relat­ed con­di­tions. For instance, astig­ma­tism can sig­nif­i­cant­ly affect the shape of seen objects. In turn, dam­age to some parts of the brain’s visu­al cor­tex may lead to shape agnosia, i.e. par­tial or com­plete loss of the abil­i­ty to recog­nise and copy objects. A spe­cial case of shape agnosia is prosopag­nosia, i.e. the inabil­i­ty to recog­nise faces.

The elec­tro­mag­net­ic wave in the range of con­tin­u­ous visu­al spec­trum can cause var­i­ous col­or effects in dif­fer­ent peo­ple due to their visu­al sys­tem sta­tus. For exam­ple, yel­low­ing of the cornea in old age reduces the sen­si­tiv­i­ty to blue col­or, caus­ing that the world is seen in a more autum­nal tone. Achro­matop­sia, in turn, man­i­fests itself in the form of an inabil­i­ty to dif­fer­en­ti­ate col­ors. It is caused by defects in the struc­ture and func­tion­ing of the reti­na or dam­age to one of the brain’s cen­tres deal­ing with the analy­sis and inter­pre­ta­tion of the data on wave­length of light.

A seri­ous spa­tial ori­en­ta­tion dis­or­der in the visu­al scene can be, e.g. hemis­pa­tial neglect, which man­i­fests itself in dif­fi­cul­ties in per­ceiv­ing half of the space locat­ed on the oppo­site side of the hemi­sphere brain dam­age. Even a slight dam­age to the cere­bral cor­tex in one part of the pari­etal lobe may lead to akine­top­sia, that is, par­tial or com­plete loss of vision. There is a long list of vision dis­tur­bances that could be elab­o­rat­ed on.

Although all the men­tioned dam­age to the visu­al path­ways and vision deficits and dis­tur­bances can sig­nif­i­cant­ly affect both the per­cep­tion of the visu­al scene and its cre­ation in the form of an image, they nev­er­the­less will not be the sub­ject of any detailed analy­sis in this book. Dis­cussing them would require a sep­a­rate mono­graph, mod­elled on excel­lent stud­ies con­cern­ing this issue, includ­ing the ones by Michael F. Mar­mor and James G. Ravin (1997; 2009).

Neural stages of visual pathway

Under­stand­ing, inter­pret­ing and eval­u­at­ing the con­tent of images are deriv­a­tives of a mul­ti­stage neu­ro­phys­i­o­log­i­cal process. It begins with record­ing the light – reflect­ed or emit­ted by objects in the visu­al scene – using pho­tore­cep­tors found in the reti­nas of both eye­balls. From the moment when the pho­ton beam enters the eye­’s inte­ri­or, the phys­i­o­log­i­cal response of pho­to­sen­si­tive cells becomes the sub­ject of com­plex neur­al process­es that occur first in the reti­na and then in many sub­cor­ti­cal struc­tures of the brain. This is called the ear­ly stage of visu­al data pro­cess­ing. It results in rel­a­tive­ly ordered infor­ma­tion on the specifics of the light­ing of the space occu­pied by the observ­er, grad­u­al­ly trans­ferred to the cor­tex struc­tures of the brain.  More pre­cise­ly, the cere­bral cor­tex is con­stant­ly being informed about the dis­tri­b­u­tion of the light record­ed but also processed by mil­lions of neu­rons form­ing the ear­ly stage of the visu­al path­way. Their task is not only to trans­fer, but also process data flow­ing from the pho­tore­cep­tors to the brain.

The sec­ond, so-called high­er stage of visu­al pro­cess­ing takes place in the cere­bral cor­tex of the observ­er.  When sig­nals from the eyes reach its var­i­ous parts, the most mys­te­ri­ous process is ini­ti­at­ed, the results of which we expe­ri­ence as the sub­jec­tive expe­ri­ence of see­ing. Its com­po­nents com­prise, among oth­er things, recog­ni­tion, under­stand­ing, inter­pre­ta­tion of scenes, mood, inten­si­ty and direc­tion of emo­tion­al arousal and val­ue attri­bu­tion. These sen­sa­tions result not only from the stim­u­la­tion of the pho­tore­cep­tors in the reti­na. On the con­trary, one of the most impor­tant neu­rocog­ni­tive process­es, which under­lies the expe­ri­ence of see­ing, is the process of com­par­ing the incom­ing sen­so­ry data with the data that the recip­i­ent has already stored in their brain.

On the one hand, these neu­ronal car­ri­ers of visu­al mem­o­ry are a lega­cy of our species evo­lu­tion.  They are sim­i­lar in all human brains, both in terms of their struc­ture and func­tion. The neu­roanatom­i­cal struc­ture, loca­tion and func­tion of neur­al groups, which analyse the col­or or shape of the objects viewed may be an exam­ple of the above.

On the oth­er hand, brain records of visu­al mem­o­ries are shaped by means of sub­jec­tive expe­ri­ences of the observ­er with­in their entire life.  The fre­quen­cy of visu­al con­tacts with a giv­en class of objects and their fea­tures is of deci­sive impor­tance for them. All these brain records are car­ri­ers of indi­vid­ual visu­al mem­o­ry, name­ly of the knowl­edge of the way the world looks, which to a large extent deter­mines the qual­i­ty of the sub­jec­tive visu­al experience.

There­fore, deter­min­ing to what extent the paint­ing can be sen­si­bly inter­pret­ed in the light of the pos­sessed visu­al knowl­edge is of key impor­tance for under­stand­ing the con­tents of a paint­ing.  Only as a result of detect­ing a sim­i­lar­i­ty between the inflow­ing sen­so­ry data and the data which were stored in the mem­o­ry before, is the brain able to “for­mu­late” and then “ver­i­fy” the hypothe­ses con­cern­ing the con­tents of a giv­en paint­ing. Gen­er­al­ly speak­ing, the things that con­sti­tute the con­tent of the visu­al expe­ri­ence hard­ly reflect what is cur­rent­ly present in the observer’s field of vision, but are the most prob­a­ble hypoth­e­sis con­cern­ing this real­i­ty. This hypoth­e­sis was for­mu­lat­ed on the basis of the lev­el of con­for­mi­ty between the inflow­ing sen­so­ry data and the pos­sessed visu­al knowledge.

The secret of the subjective experience of seeing

The expe­ri­ence of see­ing is undoubt­ed­ly the result of the brain’s work. Its end­ings, which are sen­si­tive to light, are placed in the eye­’s reti­na of the observ­er. It is worth remem­ber­ing, how­ev­er, that as much as we cur­rent­ly know a lot about the organ­i­sa­tion and the func­tion of the so-called neur­al visu­al path­ways, run­ning from the pho­tore­cep­tors to mul­ti­ple areas in the cere­bral cor­tex, we still have no idea what­so­ev­er how we expe­ri­ence see­ing result­ing from the activ­i­ty of the neu­rons form­ing these path­ways. To put it briefly, the con­tem­po­rary sci­ence cur­rent­ly does not pos­sess any kind of empir­i­cal­ly ver­i­fi­able hypoth­e­sis which would bring us clos­er to solv­ing the prob­lem of the mind-brain relationship.

The knowl­edge on the mat­ter that we cur­rent­ly have is the result of research aimed at search­ing for rela­tion­ships between the con­tem­po­rary states of mind (e.g. see­ing some­thing) and the activ­i­ty of var­i­ous brain struc­tures involved in the process of see­ing, record­ed at those moments. The start­ing point for seek­ing these rela­tion­ships are the results of stud­ies con­duct­ed on the basis of bio­log­i­cal sci­ences, for exam­ple anato­my, phys­i­ol­o­gy or bio­chem­istry, as well as psy­chol­o­gy, espe­cial­ly cog­ni­tive psy­chol­o­gy, emo­tions and neu­ropsy­chol­o­gy and many oth­er sci­ences and human­i­ties.  The con­tem­po­rary cog­ni­tive neu­ro­science lies where those research areas meet.  Its basic aim is to search for the neur­al cor­re­lates of var­i­ous men­tal process­es (Bre­mer, 2005; Chalmers 2000; Crick and Koch, 1998).

Today, it is dif­fi­cult to pre­dict whether the knowl­edge gath­ered on the basis of neu­ro­science will ever allow us to under­stand how it is pos­si­ble that bil­lions of neu­rons, the sole task of which is receiv­ing and send­ing bio­elec­tric sig­nals to oth­er cells, gen­er­ate a state that we expe­ri­ence as see­ing.  Regard­less of that, how­ev­er, the results of neu­rocog­ni­tive stud­ies already allow us to under­stand bet­ter the nature of those sub­jec­tive expe­ri­ences.   Know­ing, for exam­ple, the dis­tri­b­u­tion of var­i­ous types of pho­tore­cep­tors in the reti­na of the eye and how they are con­nect­ed with each oth­er we can under­stand why while look­ing at a visu­al scene we see some of its ele­ments sharply and oth­ers are more blurred, why in cer­tain light­ing con­di­tions we see col­ors and in oth­ers — we do not, or how we see that some­thing is moving.

Summary

The pre­sent­ed find­ings deter­mine the frame­work of this book.  They con­sti­tute a foun­da­tion on which I shall place blocks of knowl­edge on bio­log­i­cal basics of see­ing, and, in par­tic­u­lar, see­ing in the acts of cre­ation and paint­ing recep­tion. The fun­da­men­tal ques­tion that this mono­graph answers is the ques­tion of how the human visu­al sys­tem, respon­si­ble for every act and aspect of the sub­jec­tive expe­ri­ence of see­ing, analy­ses, inter­prets and val­ues seen images. Here, the visu­al sys­tem means both the neu­roanatom­i­cal basis for the flow of the sen­so­ry data from the pho­tore­cep­tors to all struc­tures which become acti­vat­ed dur­ing the acts of see­ing, that is every­thing that is most often referred to as a visu­al pathway(s) or neur­al paths and struc­tures as well as the so-called high­er men­tal process­es (cog­ni­tive, emo­tion­al and oth­er) which con­sti­tute the essence of the sub­jec­tive expe­ri­ence of seeing.

While dis­cussing the sub­se­quent stages of visu­al data pro­cess­ing, I will focus only on the struc­tures whose func­tions are impor­tant for under­stand­ing of the image view­ing process. In oth­er words, if I assume that the read­er should know, for instance, the bio­chem­i­cal basics of some bio­log­i­cal process, then I will present them in detail. On the con­trary, if the func­tions of some ele­ment of the visu­al path­way struc­ture is not recog­nised well or its detailed pre­sen­ta­tion will make it pos­si­ble to explain a giv­en men­tal phe­nom­e­non relat­ed to see­ing images only to a small extent, then I will not focus on it in a detailed way, refer­ring a curi­ous read­er to spe­cial­ist lit­er­a­ture on the subject.

In order to under­stand the con­tent of this book, a min­i­mum lev­el of knowl­edge on the struc­ture and func­tion­ing of neu­rons and the human brain, from a high school biol­o­gy course, is suf­fi­cient.  A lit­tle knowl­edge on math­e­mat­ics and optics can also be use­ful.  Ref­er­ences to paint­ings which con­sti­tute illus­tra­tions or the sub­ject of the exper­i­men­tal research also do not require advanced knowl­edge of his­to­ry of art and visu­al communication.

What is this book about?

The book con­sists of five chap­ters and in a sense reflects two orders. The first one is deter­mined by the chronol­o­gy of events on the visu­al path­ways in the acts of see­ing, and the oth­er is relat­ed to the method of nar­ra­tion, that is top-down approach.

In chap­ter I enti­tled “Neu­rocog­ni­tive basis of see­ing”, I dis­cuss two basic visu­al sys­tems: the analy­sis of the con­tents of a paint­ing and the fram­ing of a visu­al scene. Activ­i­ty of both sys­tems is pre­sent­ed in the light of neu­ro­bi­o­log­i­cal research results, which I relate to var­i­ous cog­ni­tive functions.

Chap­ter II, III and IV con­cern three most impor­tant fea­tures of every visu­al scene, name­ly the shape of the objects seen, their col­or and order in a three-dimen­sion­al space.  The descrip­tion lacks one more char­ac­ter­is­tic of a scene — motion.   Because sta­t­ic images are the sub­ject of the analy­ses in the book, I also omit the issue of image dynam­ic, sole­ly dis­cussing the issues relat­ed to the dynam­ic of the ocu­lo­mo­tor system.

In the final chap­ter — the 5th — I dis­cuss the issues relat­ed to aes­thet­ic valu­ing of paint­ings by pre­sent­ing results of ocu­lo­graph­ic stud­ies. They are aimed at search­ing for ocu­lo­mo­tor cor­re­lates of beauty.

Instead of an end­ing, I present a list of top­ics and research areas, which this mono­graph should be extend­ed by in its fol­low­ing issues.