[-empyre-] neuroaesthetics

[tina gonsalves]

Thanks Michele for inviting me to be part of this months discussion. I  
won’t be able to spend much time on it over the next week – am getting  
married...;) but will be back in the next ten days or so hopefully  
with a clearer head. But, I will start with a brief intro to my past  
work and current practice.

  With my work, I have concentrated on exploring the emotional/ 
empathic body. Empathy is the capacity to recognise or understand  
another's state of mind or emotion. Although a definition of what  
emotions are still lies on shifting ground, most researchers agree  
that emotions are pretty complex, comprising of subjective experience,  
expressive behaviour and specific physiological components. Being  
angry makes our blood pressure rise, therefore we frown, our palms  
sweat and our heart jumps a beat, further causing us to feel and look  
anxious. Emotions influence all aspect of behaviour and subjective  
experience. They grab our attention, enhance or block memories, and  
are often blamed for swaying logical thought.

My interest in this area began a decade ago. I used various techniques  
to translate emotional feelings into a metaphorical artistic moving  
image form, creating many short films. This lead to early wearable  
works (Medulla Intimata 2002-2004) that monitored and probed the  
wearer’s emotional body. I quickly realized that if I was going to  
explore ‘emotion’ sensing works, I needed to become further  
empirically informed in order to monitor, assess, and provoke the  
emotional body in more ‘intelligent’ and ‘meaningful’ ways. When I say  
‘intelligent’, I wanted to work with neuroscientists to assess the  
data emitted by the body, to understand how it relates to a feeling  
state of the participant, and match this information to drive more  
meaningful moving images. This information is often personal, leading  
to a feeling of vulnerability in the participant – To me, it needed to  
be used sensitively. Also the interaction design of these works means  
that the viewer often had to be constrained, dressed in obtrusive  
technology to monitor the body, which obviously effects the emotions  
of the body. By working with affective computing scientists, human  
computer interaction specialists and neuroscientists, the aim was to  
research and develop more naturalistic and transparent monitoring  
techniques. Also, by ‘meaningful’  - artists often use ambiguous  
generative abstract moving images or sound to respond to the data of  
the body. I know pulsating big yellow circles or shifting colours to  
represent emotion may generate a sense of meaning due to its  
ambiguity, but I was more interested in working with scientists to  
produce figurative and emotionally narrative based video works that  
could engage, reflect and provoke the feelings of the viewer.  All of  
these aims are complex, and each are many phd’s, so I have ended up  
collaborating with a range of various disciplines over the last few  
years.
I began my role as artist in resident at the Wellcome Department of  
Neuroimaging (WDIN) (2005-10) at the Institute of Neurology at UCL in  
the UK and Visiting Artist at the Affective Computing Group the MIT  
Media Lab in the US (2008-). I built an ever-shifting collaborative  
group including emotion neuroscientist Prof Hugo Critchley, social  
neuroscientist Prof Chris Frith, psychologist and clinical hypnotist,  
Prof David Oakley, affective computer scientists Prof Rosalind Picard  
and Dr Rana El Kaliouby, and a range of very talented computer  
scientists, HCI specialists, sensor manufacturers and programmers. Our  
different disciplines gave alternative insights into our cross over  
interest areas of social networks, empathy, affect and computing. We  
embarked on the “Feel” Series, an interconnected progression of short  
films and interactive sketches aiming to sense, translate and provoke  
the psycho-physiology of the audience. Darren Tofts writes “(with  
“Feel” Series), Gonsalves’ artistic sensibility absorbs scientific  
hypothesis and technological possibility into an interface, a psycho- 
somatic stage, at once theatre of cruelty, emotional catharsis and  
critical insight”.[i]

My current project Chameleon[ii] explores emotional contagion – the  
idea of how we infect each other with our emotions. We are producing  
it in small stages, and the last iteration should conclude late next  
year. “Chameleon” is a more ambitious multi-participant video  
installation using mind reading technology, video and emotional  
algorithms to assess, respond and provoke the emotional states of the  
audience. With the MIT affective computing group, we are developing  
facial emotion expression reading (called mind reading technology) and  
this monitors the audience. I am working closely with Chris Frith to  
map how people respond in different emotional states. With emotion HCI  
experts we are currently testing Frith’s hypothesis in the lab. These  
algorithms will form an intelligent emotional response system in the  
video engine (which talks to the mind reading technology), in order to  
build ‘empathy’ with the participant. We are currently attempting to  
build intent/desire into the algorithms. At the Banff Center I am  
working with actors and artists to build an emotional expression  
moving image database, a more expressive one then the ‘Ekman’ and  
'Karolinska' ones often used in neuroscientific studies. When  
participating in  “Chameleon”, individuals become intimately connected  
and implicated into varying emotionally provocative and reflexive  
social interactions. I haven’t had the time to update the website with  
the work in progress - hopefully I will over the next couple of weeks  
- but its been a really fantastic piece to work on -a huge learning  
experience. It’s a complex piece which I would glad to talk about more  
of the details later.

Anyway, that’s a beginning. Sorry it was rushed. Talk soon.


[i]Tofts, D, “Tina Gonsalves: Unleashing Emotion”, Artlink, vol 28 no  
2 Melbourne, AUSTRALIA


[ii] Chameleon is funded by the Wellcome Trust Large Art Award,  
Australian Network for Art and Technology Synapse Award, Arts Council  
England, Australia Arts Council visual and inter-arts board and  
supported by Banff New Media Institute, MIT Media Lab, Wellcome  
Department of Neuroimaging and Brighton and Sussex Medical School.  
Collaborative Group includes the core members of Tina Gonsalves, Chris  
Frith, Hugo Critchley, Rosalind Picard, Rana El Kaliouby and Helen  
Sloan.

tina gonsalves
http://www.tinagonsalves.com