The idea of writ­ing this book arose sev­er­al years ago, evolv­ing from my inter­est in how audio-visu­al media, espe­cial­ly tele­vi­sion infor­ma­tion pro­grams, influ­ence the way their view­ers under­stand their con­tent. While sum­maris­ing an overview of psy­cho­log­i­cal, soci­o­log­i­cal and media stud­ies the­o­ries as well as the results of my stud­ies on under­stand­ing tele­vi­sion broad­cast, I noticed two char­ac­ter­is­tic ten­den­cies (Fran­cuz, 2002). First and fore­most, it seemed to me that audio-visu­al media ana­lysts focus much more often on the tex­tu­al (ver­bal) ele­ment of the broad­cast than on the visu­al data. I also con­clud­ed that they refer much more often to the con­tent of media broad­cast rather than to its structure.

The ana­lyt­i­cal approach, accord­ing to which the mean­ing of the audio-visu­al mes­sage can be found almost exclu­sive­ly in words con­tained in it, while the mean­ing of an image — if it is tak­en into account at all — is treat­ed as a mar­gin­al and insignif­i­cant addi­tion to the mes­sage con­veyed, rep­re­sents a sub­stan­tial mis­un­der­stand­ing of the medi­um.   Images and words are equal­ly impor­tant in terms of con­vey­ing mean­ing and they deter­mine the sense of audio-visu­al com­mu­ni­ca­tion. What is more, the mar­gin­al­iza­tion of the role of the image in cre­at­ing the mean­ing of audio-visu­al com­mu­ni­ca­tion leads to a fail­ure to see the com­plex seman­tic inter­ac­tions occur­ring between the mean­ing of words and the mean­ing of accom­pa­ny­ing images. More­over, their foun­da­tion, or form, is often ignored in stud­ies of audio-visu­al media.

Both those find­ings drew my atten­tion to the image as a car­ri­er of mean­ing, and in par­tic­u­lar, to the asso­ci­a­tion between the struc­ture of an image and the mean­ing it con­veys. In the first ver­sion of this book, my aim was to describe the rela­tion­ship between the con­tent of audio-visu­al com­mu­ni­ca­tion, which con­sists of both the mean­ing of the image (video) and the mean­ing of word­ing (audio) and the struc­ture of both meth­ods of encod­ing mean­ing. Specif­i­cal­ly, my aim was to describe all these rela­tions in the con­text of the results of stud­ies on the neu­ro­phys­i­o­log­i­cal cor­re­lates of visu­al and audi­al per­cep­tion and per­cep­tu­al inte­gra­tion. In the first decade of the twen­ty-first cen­tu­ry, stud­ies on the sub­ject were in abun­dance, because cog­ni­tive neu­ro­science was devel­op­ing in an uncan­ni­ly dynam­ic way. While study­ing the lit­er­a­ture on the sub­ject, I quick­ly came to real­ize that in this ver­sion, it would be vir­tu­al­ly impos­si­ble to car­ry out the project in a rea­son­able time frame. The rapid­ly increas­ing num­ber of reports from stud­ies was not the only prob­lem; the high lev­el of doubt regard­ing the reli­a­bil­i­ty of all the results pub­lished in them was of greater impor­tance, some­thing that is, inci­den­tal­ly, typ­i­cal for every new field of science.

I, there­fore, decid­ed to lim­it the scope of my work to the descrip­tion of the neu­rocog­ni­tive basis of image per­cep­tion. At this point, how­ev­er, I had to make some oth­er deci­sions restrict­ing the scope of the prob­lem for the same rea­sons I had to lim­it my work only to visu­al analy­sis of media con­tent.  At first, I decid­ed not to dis­cuss mat­ters relat­ed to the motion of objects in a visu­al scene.

The pri­ma­ry rea­son for this was the fact that motion in films, tele­vi­sion, or the­atre stages com­pli­cates the process of form­ing the com­mu­ni­ca­tion of mean­ing in an incom­pa­ra­bly greater way than motion­less forms enclosed in sin­gle, still frames.

The sec­ond rea­son was the fact that audio-visu­al media analy­sis which con­sid­ers only the visu­al aspect may lead to arti­facts, as the basis for this type of mes­sages to be formed is pre­cise­ly the inter­ac­tive influ­ence of both path­ways of mean­ing encod­ing. Tak­ing mobile visu­al com­mu­ni­ca­tions as an exam­ple, we would find that their scope would have to be lim­it­ed only to cer­tain silent films from the begin­ning of the twen­ti­eth cen­tu­ry (and, for that mat­ter, only those that were entire­ly stripped of their tex­tu­al ele­ment), pan­tomime or dance (exclud­ing music), or tele­vi­sion reports of no com­ment type, which do not con­tain com­men­taries expressed in words, but are filled with all sound effects, i.e., ones.  To put it briefly, con­tem­po­rary cul­ture has cre­at­ed rel­a­tive­ly few mobile media com­mu­ni­ca­tions which are entire­ly deprived of text and audio elements. 

Final­ly, study­ing move­ment in visu­al com­mu­ni­ca­tion leads to the neces­si­ty of under­tak­ing deep­ened stud­ies on the neu­rocog­ni­tive cor­re­lates of see­ing appar­ent move­ment, opti­cal illu­sions asso­ci­at­ed with move­ment and actu­al move­ment, both of the objects in a visu­al scene and the observ­er, which is an issue that is both fas­ci­nat­ing and immense­ly complicated. 

The sec­ond and last fun­da­men­tal lim­i­ta­tion that I imposed on this work is the deci­sion not to dis­cuss the mat­ters relat­ed to the three-dimen­sion­al space of the visu­al stage. First and fore­most, three-dimen­sion­al­i­ty is very close­ly asso­ci­at­ed with move­ment, both in the sense of the sub­jec­tive expe­ri­ence of see­ing and its neu­ro­bi­o­log­i­cal basis. Sculp­tures or archi­tec­ture, per­ceived in the real three-dimen­sion­al space can be observed not only using the stereo­scop­ic per­cep­tion but also from unlim­it­ed points of view, which pro­duces entire­ly new inter­pre­ta­tion oppor­tu­ni­ties, com­pared to observ­ing them in a pho­to­graph or a paint­ing. There­fore, study­ing three-dimen­sion­al com­mu­ni­ca­tions impos­es the neces­si­ty of tak­ing the loca­tion of the observ­er on the stage as well as their move­ment and mul­ti­ple char­ac­ter­is­tics relat­ed to their pos­ture into account. The sit­u­a­tion is addi­tion­al­ly com­pli­cat­ed by the prob­lem of per­cep­tion of three-dimen­sion­al stage, gen­er­at­ed by new 3D tech­nolo­gies.  As it is in the case of see­ing move­ment in a film, which is based on entire­ly dif­fer­ent prin­ci­ples than see­ing move­ment in real life, the expe­ri­ences of three-dimen­sion­al space in a 3D cin­e­ma and in the real world use dif­fer­ent mech­a­nisms of vision.

Hav­ing intro­duced all these lim­i­ta­tions to the orig­i­nal ver­sion of this mono­graph, I stood before a flat image, and I real­ized that from neu­rocog­ni­tive per­spec­tive it was the most appro­pri­ate sub­ject of stud­ies to start with. Sev­er­al facts con­vinced me on this point.

First and fore­most, flat images — cre­at­ed by painters, pho­tog­ra­phers and drafts­men — have been pro­duced by man for more than 30 thou­sand years. The num­ber of exam­ples of images under­stood this way is essen­tial­ly uncount­able, which pro­duces lim­it­less oppor­tu­ni­ties for illus­trat­ing issues relat­ed to see­ing these images.

Sec­ond­ly, I real­ized that flat images are the sub­jects of advanced stud­ies in var­i­ous fields of knowl­edge, such as his­to­ry of art, anthro­pol­o­gy, aes­thet­ics, phi­los­o­phy as well as social and bio­log­i­cal sci­ences. Such stud­ies are con­duct­ed not only by sci­en­tists, but also by the cre­ators of the images. I also noticed that the num­ber of bio­log­i­cal­ly-ori­en­tat­ed researchers show­ing inter­est in the visu­al arts has rapid­ly increased due to the fact that the ways of pic­tur­ing real­i­ty by artists of almost all epochs con­sti­tute an inspi­ra­tion for stud­ies on visu­al per­cep­tion in gen­er­al as well as its neu­rocog­ni­tive foun­da­tions. There is a rea­son why artists are some­times referred to as ‘uncon­scious neu­ro­bi­ol­o­gists’ (Ramachan­dran and Hirstein, 1999). Art his­to­ri­ans, anthro­pol­o­gists and even aes­thetes have also noticed the new per­spec­tives that the con­tem­po­rary neu­ro­science opens before the tra­di­tion­al meth­ods of analysis.

Third­ly, when I want­ed to illus­trate the effects relat­ed to see­ing dur­ing my lec­tures on per­cep­tion, I often pre­ferred to use the exam­ples of pieces of visu­al art rather than the stim­uli cre­at­ed arti­fi­cial­ly for the pur­pos­es of psy­cho­log­i­cal and neu­rocog­ni­tive stud­ies. In those pieces of art, I quite often noticed glimpses of inge­nious and sur­pris­ing intu­ition relat­ed to the things which artists saw and per­pet­u­at­ed in their canvas.

The fourth aspect is the fact that stud­ies on the neu­rocog­ni­tive foun­da­tions of image per­cep­tion con­sti­tute a com­bi­na­tion of my two great­est inter­ests. On the one hand, they are pro­fes­sion­al inter­ests in per­cep­tion, in par­tic­u­lar of visu­al per­cep­tion, on the oth­er an inter­est in art as a hob­by. I have been devel­op­ing these inter­ests some­what inde­pen­dent­ly. The idea of writ­ing this book moti­vat­ed me to use my knowl­edge on both domains with­in one mono­graph. The open­ing of the Psy­choneu­ro­phys­i­o­log­i­cal Lab­o­ra­to­ry by the Depart­ment of Exper­i­men­tal Psy­chol­o­gy of the Lublin Catholic Uni­ver­si­ty run be me, in 2008, was also an addi­tion­al moti­va­tor. It opened up the pos­si­bil­i­ty of con­duct­ing inde­pen­dent exper­i­men­tal research on media by means of ocu­lo­graph­ic and elec­troen­cephalo­graph­ic techniques.

Writing by means of text and image

In the pref­ace to the twelfth edi­tion of his book The Sto­ry of Art (2005) Ernst Gom­brich wrote, ‘From the very begin­ning, the book was sup­posed to speak of art, using both words and images, enabling the read­ers to look at the illus­tra­tion under dis­cus­sion with­out turn­ing the page, if it was only pos­si­ble’ (p. 10). This sen­tence became some­thing of an idée fixe to me while I was writ­ing this book. Its first parts indi­cate the fun­da­men­tal rule of writ­ing about images, both using text and images. The idea is not obvi­ous for many authors and pub­lish­ers of this type of mono­graphs who view text as the most impor­tant source of knowl­edge, includ­ing the knowl­edge of images.  In par­tic­u­lar, the imple­men­ta­tion of the idea of mak­ing use of images as con­vey­ors of mean­ing equal to text is still dis­tant in the case of aca­d­e­m­ic works, both books and arti­cles, in which the space for illus­tra­tions is lim­it­ed to a min­i­mum. It is a dis­play of exact­ly same ten­den­cies referred to above. Per­haps finan­cial con­sid­er­a­tions could be of impor­tance here as well.

Be that as it may, in my opin­ion, the image and the space it occu­pies in a pub­li­ca­tion, espe­cial­ly one devot­ed to images, is like the text and the space of the para­graph it occu­pies.  Images quite often con­tain much more use­ful infor­ma­tion than even the rich­est text devot­ed to describ­ing things that can be sim­ply seen.  This is how I approach the illus­tra­tions pre­sent­ed in this book. The imple­men­ta­tion of the rule of writ­ing using text and image is not easy and requires con­stant con­trol of the space of the page, on which frag­ments of text and images are placed in a log­i­cal sequence, based on a prin­ci­ple sim­i­lar to that upon which the author orders his thoughts in sub­se­quent para­graphs. It is not always suc­cess­ful in the case of images that can­not be bro­ken into two halves, so that the sec­ond half can be moved to the next page.

The sec­ond thought from that quo­ta­tion by Ernst Gom­brich con­cerns con­tent con­trol, the so-called cen­ter­fold, name­ly two pages of an open book that are con­nect­ed at the spine of the book. The con­trol of the cen­ter­fold pages con­sists in dis­trib­ut­ing the text and illus­tra­tions on them in such a way that the pages do not have to be turned in order to see the things that are is being dis­cussed. This is anoth­er lim­i­ta­tion that I imposed on myself while writ­ing this mono­graph. Main­tain­ing this prin­ci­ple was just as dif­fi­cult as main­tain­ing the pre­vi­ous one, but I did my very best to fol­low it.

If I want­ed to com­ply with both prin­ci­ples I had to break the text and the illus­tra­tions on sub­se­quent pages of the book on my own. At least at this stage of project imple­men­ta­tion I could not real­ly imag­ine the neces­si­ty of con­tin­u­ous­ly mak­ing agree­ments with the pub­lish­er regard­ing how much text I should remove and how much text I should add on every page so that an illus­tra­tion of an opti­mal size could be placed under a giv­en para­graph and, in addi­tion, with­in the same cen­ter­fold. Indeed, the clas­sic print­ing tech­nique is a true chal­lenge for the author who tries to make the con­tent and the form of its pre­sen­ta­tion conform. 

The next rule that I was try­ing to fol­low con­cerned the appro­pri­ate choice of pos­si­ble high qual­i­ty of the illus­tra­tions. Car­ry­ing out this assump­tion took almost as much time as writ­ing the texts the illus­tra­tions were accom­pa­nied by. At this point, I would like to thank my co-work­er, Paweł Augustynow­icz, for help­ing me pre­pare over 30 illus­tra­tions. Many of them were devel­oped on a three-dimen­sion­al mod­el of the brain and the eye pro­duced by Anatomi­umTM — 3D Dig­i­tal Human Brain. With­out Paweł’s help, I would not have been able to pre­pare all the illus­tra­tions that are includ­ed in the book. 

The illus­tra­tions pre­sent­ed in the book are clas­si­fied into sev­er­al cat­e­gories, which include pat­terns, two and three-dimen­sion­al mod­els, pho­tographs and draw­ings from var­i­ous sci­en­tif­ic pub­li­ca­tions, paint­ing repro­duc­tions and graph­ic depic­tions of the results of own stud­ies. All illus­tra­tions, which are pre­sent­ed with infor­ma­tion regard­ing their source, have been added to the text by means of quo­ta­tion. If they graph­i­cal­ly illus­trate the results of own stud­ies, they also con­tain infor­ma­tion regard­ing the pro­ce­dure of their cre­ation. It par­tic­u­lar­ly con­cerns the images that were processed using Adobe Pho­to­shop in order to reveal their inter­est­ing struc­tur­al prop­er­ties, such as, for exam­ple, edges of the object depict­ed in them or sep­a­ra­tion of col­ors used for their painting.

I cre­at­ed some images illus­trat­ing the results of the ocu­lo­graph­ic stud­ies by com­bin­ing two images with the use of the Adobe and OGAMA 4.3 or iViewX pro­grams. Work on a dozen or so illus­tra­tions resem­bled an attempt to sep­a­rate, for instance, a piece of cere­bral cor­tex using the tech­nique of cytochrome oxi­dase in order to be able to see its inter­nal struc­ture. All the illus­tra­tions that pre­sent­ed the effects of pro­cess­ing their con­tent using com­put­er pro­grams were devel­oped using the same algo­rithm, which enables their com­par­i­son in the book as well as pro­vid­ing inter­est­ed read­ers with an oppor­tu­ni­ty to cre­ate illus­tra­tions on their own.

The method that I read­i­ly used in this book is anal­o­gy. I believe that at the present stage of devel­op­ment of the inter­dis­ci­pli­nary field which is, after all, a meet­ing point of dis­tant sci­ences such as his­to­ry of art, media sci­ences and neu­ro­science along with cog­ni­tive psy­chol­o­gy, pro­lif­ic analo­gies, which will allow the dis­ci­pline to devel­op, unre­strained by the meth­ods of the above fields, are need­ed.  I am par­tic­u­lar­ly grate­ful to Pro­fes­sor Adam Biela for draw­ing my atten­tion to this aspect of the nar­ra­tion that I applied. I am also grate­ful for his sug­ges­tions based on his read­ing of the entire manuscript. 

Some words of gratitude

The first offi­cial read­ers of this book were its edi­to­r­i­al review­ers: Prof. Ane­ta R. Borkows­ka of the UMCS (Maria Curie-Skłodows­ka Uni­ver­si­ty), a clin­i­cal psy­chol­o­gist and neu­ropsy­chol­o­gist, and Prof. Dobrosław Bag­ińs­ki, a spe­cial­ist in visu­al com­mu­ni­ca­tion. Both of them, on the basis of two very dif­fer­ent points of view, drew my atten­tion to sev­er­al impor­tant mat­ters, clar­i­fi­ca­tion of which enabled me to artic­u­late my thoughts better.

Among the many remarks I received from my review­ers, there is one I would like to dwell on for a bit longer. The man­u­script of this book was giv­en to them under the title of A Neu­rocog­ni­tive The­o­ry of Images. It seemed to me that the title was appro­pri­ate­ly adjust­ed to the book’s con­tent. After I wrote the pref­ace I real­ized that, due to the numer­ous lim­i­ta­tions which I pre­sent­ed in it, I could not refer to the con­tent of the mono­graph as the­o­ry, in the sense of a coher­ent con­cept sys­tem in the field of cog­ni­tive neu­ro­science. It is, how­ev­er, a work that is clear­ly aimed at devel­op­ing such a con­cept. For this rea­son, the pre­vi­ous title has become a sub­ti­tle and was addi­tion­al­ly com­ple­ment­ed with the expres­sion: TOWARDS a Neu­rocog­ni­tive The­o­ry of Images.

The per­son who helped me for­mu­late the new title was Pro­fes­sor Dobrosław Bag­ińs­ki, with whom I often talked about the issues dis­cussed in the book. Dur­ing one of such dis­cus­sions he unex­pect­ed­ly brought for­ward the word ima­gia to define the mag­ic tricks that the brain uses in the process of see­ing. Clos­er look at the word revealed some incred­i­bly accu­rate ref­er­ences to the prob­lems explored in this mono­graph. Its name clear­ly refers to the Latin word imāgō, which denotes image. Unlike the Eng­lish or French-sound­ing words image or imagé, the word ima­gia sounds more famil­iar and brings to mind asso­ci­a­tions with the word mag­ic the Latin ori­gin of which was, in turn, asso­ci­at­ed with the mag­i­cal arts (ars mag­i­ca). This book is part­ly about the art and mag­ic of paint­ings, and about the mag­ic tricks of the visu­al sys­tem, which — like an illu­sion­ist — con­jures up new knowl­edge on the way the world looks. Final­ly, there is the pre­fix, i‑, which locates the asso­ci­a­tions in the area of the Inter­net, the most pow­er­ful con­tem­po­rary medi­um, and forms a new word — iMag­ic — derived from iPhone, iTunes as well as iBook, iPer­fumy, iTV etc.  Fur­ther­more, the word ima­gia has one more advan­tage – it is short. For all those rea­sons the book was enti­tled imagia.

I would like to take this oppor­tu­ni­ty to also thank the Rec­tor of the Lublin Catholic Uni­ver­si­ty, Pro­fes­sor Antoni Dębińs­ki, whose short fiat paved my way to pub­lish­ing an obvi­ous­ly cost­ly book in full col­or at the University’s pub­lish­ing house. I would also like to thank my co-work­ers, post­grad­u­ate stu­dents and the employ­ees of the Depart­ment of Exper­i­men­tal Psy­chol­o­gy who under­take research projects con­cern­ing art per­cep­tion with great com­mit­ment. I would also like to express my grat­i­tude to Mr Paweł Jaska­nis, the direc­tor of the Muse­um of Palace in Wilanów, War­saw, and Mrs Elż­bi­eta Gry­giel, the head of the Depart­ment of Social Com­mu­ni­ca­tion at the Wilanów Muse­um for their kind­ness and help in orga­niz­ing the mate­r­i­al for a research on beau­ty and for mak­ing a documentary. 

Last but not least, my great­est grat­i­tude goes out to my wife, Graży­na. Had she not tak­en on the siz­able bur­den of near­ly all the house­hold chores, this book would have nev­er been writ­ten. She also found enough time to read it from cov­er to cov­er, and to pro­vide lin­guis­tic cor­rec­tion (the respon­si­bil­i­ty for all errors and over­sights that may still appear, how­ev­er, remains with me). Either way, this book is ded­i­cat­ed to her.

Piotr Fran­cuz, 10 June 2013

Table of content