[-empyre-] "synaesthetic media"

[Mitchell Whitelaw <mitchell.whitelaw@canberra.edu.au>]

On 10/12/2004, at 2:14 AM, Nancy Paterson wrote:

> “the most salient and vital aspect of interacting with computer 
> systems is consistently overlooked, that is, the importance of 
> computer systems as perceptual rather than conceptual tools. Insofar 
> as people interact with them, computer systems function primarily as 
> sensual transducers which I term synaesthetic media and not as 
> so-called cognitive-artifacts.”

Great stuff - can we have the full reference to this research Nancy?

For a few years I've been observing new media / experimental music work 
which explores exactly this function of digital media - their ability 
to transduce signal / information between sensory modes. These works 
are examples of "synaesthetic media" in the sense that they demonstrate 
a kind of machine synaesthesia and evoke a human sensory fusion / 
crossover. What interests me is that such work is evoking human 
synaesthesia through a non-human process - via the particular 
audio/visual crossovers offered by digital/analog media technologies, 
rather than the crossovers generated in our neural structures. It's 
interesting also that this evocation is powerful in spite of what might 
be considered the "artefacts" of transduction - for example the 50Hz 
hum that results from turning an analog video signal into audio. In 
fact it's the properties of the signal *per se* which the works direct 
us to...

Two of my favourite examples... both happen to be Australian (in 
memoriam OzCo New Media Arts)

Robin Fox, Backscatter (DVD) - Synaesthesia Records, 2004. (see for eg 
http://www.synrecords.com/synaesthesia/featured/ - I can't find any 
video online!) This work is truly astounding: live generated digital 
waveforms plugged into an oscilloscope in "polar" mode... Fox designs 
sounds and gestures for visual as well as sonic results - staggering.

Andrew Gadow, INVERSION (2001). Again not online as far as I know. 
Gadow generates video live with an old Fairlight CVI (digital video 
"synthesiser") and simply routes the video signal out as audio. 
Abstract, flickering, noisy, degraded video, sound similar, end result 
is hardcore sensory fusion.

Cheers,

Mitchell
http://creative.canberra.edu.au/mitchell