A multisensory view on cinema refutes the idea of experiencing films in only mono-sensory way, mainly, through vision only. Researchers in the audiovisual film study discipline agree with this willingly, as their premise takes into account sound as well as vision. Multisensory research, on the other hand, adds sense of touch to vision. While of course touching a film is impossible (we can touch the screen of our TVs or laptops, or the white screen of the cinema, without actually touching anything on the film itself), multisensory film research applies touch as a point of reference within the film or, indeed, as a form of haptic visuality (Marks). While we can research how touch, in all its possible ways, is conveyed within the storyline of the film, haptic visuality brings the experience home to the viewer of the film. It is still uncommon to find multisensory cinema research as applying to audiovisuality as well, however. Only a handful of (music) scholars have made the connection between hapticity and audiovisuality of films. Those include Michel Chion, John Richardson, Anahid Kassabian, and Amy Herzog, to mention a few. Following in their footsteps, my goal is to bridge the gap especially between touch and hearing, if possible, and thus convey a truly multisensoral view on films.
Multisensory analysis of cinema has been discussed since very early writings of film theory. According to Vivian Sobchack (2000), the earlier philosophical and quantitative research on film reception was quite frequent, from analysing Sergei Eisenstein’s films to measuring bodily responses while watching a film. Contemporary film theory, on the other hand, seems to be less oriented to multisensory experiences. According to Sobchack (2000), bodily responses seem to be too crude or banal to be worthy of further notice. There are, however, quite a few scholars that disagree with the crudeness of bodily feedback, and hold it to be an important field of study. They are most often influenced by phenomenology, and use it as a point of departure when approaching the multisensoral aspects of cinema.
Thomas Elsaesser and Malte Hagener (2010, 4) have written their book Film Theory: An Introduction Through the Senses of different film theories focusing on the triangle between cinema, perception and human body, and how these three are negotiated in specific film study disciplines. They make a case in their introduction that the matters of the receiving body cannot be overseen. From the fact that a human body is usually doing the perception, and that it usually is situated within a certain physical space while observing, to the bodies that are displayed on film, and to the philosophical issues of perception and temporality related to the human body, corporeal realities cannot be left out of the cinematic experience. Elsaesser and Hagener discuss the audiovisual sphere in a chapter, but leave the connections between the tactility of cinema and audiovisuality untouched. This is understandable, however, as within the space of one introductory book, the task perhaps fell outside of the interest of their book.
When multisensory cinema is in question, quite often there are two ways to approach it. One, a more neutral way, is to perceive and analyse how touch is involved within the story of a film. Is it unnoticeable or made neutral, does touching cause reactions in the actors, what sort of power balances are implied by touching, etc. Another, more phenomenological way, is to interpret the film in connection to the researcher’s own bodily responses. Vivian Sobchack (2000) writes eloquently about her tactile sense understanding the opening scene of Jane Campion’s The Piano before her vision or rational thinking identified the blurry, reddish figures as digits placed on one’s eyes. Linda Williams (1991, 7) considers the “bodily genres” of melodrama, horror and porn as the most affective ones, or excessive, when human responses are concerned. It can happen with crying while the characters onscreen cry, jumping from horror, or getting aroused by pornography. Williams’s three bodily genres of course rely on bodily identification in a wider range of responses, that is, not only through touch, but through affective reactions of the body as well.
Laura Marks (1998) has discussed hapticity in film in particular. She has noted that in the visual sphere, haptic perception of cinema is a combination of tactile, kinaesthetic and proprioceptive functions. In short, haptic perception allows the audience to sense the image in an embodied way, as the image invokes sensory memories of touching and/or being touched. Tactility and hapticity are relatively new terms through which film and music are analysed. They are closely connected with phenomenology, as the experience thereof is always instant and affective, automatic through the senses. The tactile film experience “opens up the possibility of cinema as an intimate experience and of our relationship with cinema as a close connection, rather than as a distant experience of observation” (Barker 2009, 2). Such a view foregoes the claim that film is mainly enjoyed through visual perception, and theorizes how the image on screen can affect the whole body, be it to the skin or on deeper levels.
When the auditive side of films is concerned, nothing comes as close to multisensory (or touch – vision –related) cinema as Michel Chion’s concept of rendering. Sound effects, sometimes accentuated to the point of sounding inauthentic but naturalized in the audiovisual contract and synchronisation, bring about an effect what Michel Chion (1994, 109) calls “rendered sounds.” Rendering is the phenomenon of sound editing that emphasizes not only the “naturalness” of a sound effect, but also conveys through multisensory perception “the event’s impact and even its very appearance” (Chion 2009, 237). This effect is based on the fact that very few real life perceptions are purely visual or purely auditory, and film production is based on the reconstitution of this multisensory experience. The sound effect is not what it would be in real life, but rather renders what the sound feels like, creating thus an embodied feeling also in the viewer/listener of the film. Thus some sounds can make you shiver with cold (say, a wind blowing), while others render the feelings of pain or erotic touch.
Chion returns to his term also with a more recent article, “Sensory Aspects of Contemporary Cinema” (2013), where he builds a more sustainable bridge between audiovisuality and hapticity of cinema. Chion (2013, 325) argues that a better term for audiovisuality would be “audio-visiogenic”, which – inbuilt with rendering aspects of the film – suggests that cinema “consists of sensations created by combinations of sounds and images, greater than the sum of the parts”. Chion suggests that superimposed on each other, sound and vision reach beyond only hearing and seeing, and enable other sensory responses in the audience. This is of course not a special talent of cinema only, as Chion (ibid. 326–327) points out. It seems to be a common trait of any art that at some point it will try to “reproduce what is sensorily unavailable to it” (ibid. 326): say, a painting depicting a musical event, instrumental music telling a story, or a sculpture depicting motion. These Chion (2013, 330; 2009, 493) calls “trans-sensory perceptions: (…) perceptions that belong to no one particular sense but which may travel via one sensory channel or another without their content or their effect being limited to this one sense”. Trans-sensory perceptions are, for example, rhythm (involving hearing, touch, movement), texture (touch, temperature, balance) and grain. Chion (2013, 330) argues that trans-sensory perception should not be marginalised from scholarly writing of film studies. I read his suggestion as belonging to audiovisual research especially, as the multisensory readings of films need to include sonic aspects as well, before they can truly be called “multi”sensory.
Laura Marks (1998) has written about the hapticity and the erotic potential thereof with film visuals in the article “Video Haptics and Erotics,” where she theorizes how haptic perception can be an erotic experience when conveyed by the visual image of the film. Marks (ibid. 332) begins her theorisation by distinguishing “haptic perception” in three different functions: the tactile, the kinesthic, and the proprioceptive functions. “Haptic visuality” thereby operates with the eyes working similarly to the organs of touch. Seeing a tender touch on film creates not only a visual image in the spectator’s mind, but the kinaesthetic memory of touching and being touched as well. Marks (1998, 337) is wary of calling the haptic perception a feminine modality (which would be justified if theorizing it through Irigaray’s claims of tactility as representing a feminine form of experience; ibid; see also Irigaray 1985, 26), preferring to think of it as a feminist strategy of perception. It focuses on surfaces and texture more than depth or form (Marks 1998, 338). Thus it goes against the formalist masculine, “objective” perception: “Cinematic perception is not merely (audio)visual but synaesthetic, an act in which the senses and the intellect are not conceived of as separate” (ibid. 339). As far as erotics is concerned, Marks thinks that all haptic visuals are erotic despite their content because of the intersubjective relationship between the image and its beholder: “Haptic visuality implies making oneself vulnerable to the image, reversing the relation of mastery that characterizes optical viewing” (Marks 1998, 341). In a way, Marks’s depiction of haptic visuality reminds me of the Romantic conception of art perception, which consists of losing oneself to the beauty of the art and thereby sublimating it to a sphere of eroticism and/or divinity.
As all academic tools, multisensory cinema has also faced some criticism. As a highly phenomenological way of analysing, criticism of phenomenology also applies to multisensory cinema. Criticism usually focuses on phenomenology’s possible slip towards overly personal interpretation, or downright incoherent results that are born of a “narcissistic–autistic interest of knowledge” (Torvinen 2008, 11). Therefore it is important for phenomenological studies to be explicit and inductive by nature (ibid.), but also critical and self-reflexive at the same time. Regarding multisensory cinema itself, it has been criticized as ”…sometimes [being] in danger of celebrating a big-tent, inclusive feel-good-theory of sensory empowerment” (Elsaesser & Hagener 2010, 128). Thus multisensory cinema can be seen as a group of self-involved film critics who take films “at heart” but without ever really focusing on what exactly makes them affectively feel the experience. Perhaps here audiovisual research side can assist multisensory cinema to become more accurate, more event-based and more analytic than merely the vision-touch-related analysis tool does.
Recommended films: Gravity, The Piano, Secretary, Harry Potter and the Prisoner of Azkaban, Perfume – The Story of a Murderer, Chocolat, The Labyrinth, Ratatouille, 127 Days
Barker, Jennifer M. 2009. The Tactile Eye: Touch and the Cinematic Experience. Berkeley: University of California Press.
Chion, Michel 2013. Sensory Aspects of Contemporary Cinema. In Richardson, John; Claudia Gorbman & Carol Vernallis (eds), The Oxford Handbook of New Audiovisual Aesthetics. Oxford: Oxford UP. 325 – 330.
___ 2009 . Film, A Sound Art. Translated by Claudia Gorbman. New York: Columbia UP.
___1999. The Voice in Cinema. Translated by Claudia Gorbman. New York: Columbia UP.
___1994 . Audio-Vision: Sound on Screen. Translated by Claudia Gorbman. New York: Columbia UP.
Corbett, John & Terry Kapsalis 1996. Aural sex: The female orgasm in popular sound. The Drama Review, vol. 40 no 3: 102–111.
Elsaesser, Thomas & Malte Hagener 2010. Film Theory: An Introduction through the senses. New York: Routledge.
Härmä, Sanna & Leppänen, Taru 2006. Miltä porno kuulostaa? Sukupuoli, äänet ja musiikki valtavirran pornoelokuvissa. Lähikuva no. 3: 6–18.
Kristeva, Julia 1980. Revolution in Poetic Language. New York: Columbia UP.
Marks, Laura U. 2002. Touch – Sensuous Theory and Multisensory Media. Minneapolis: University of Minnesota Press.
___2000, The Skin of the Film: Intercultural Cinema, Embodiment, and the Senses. London: Duke UP.
___1998. Video Haptics and Erotics. Screen, vol. 39, no. 4 (Winter): 331–348.
Richardson, John 2012. An Eye for Music. Popular Music and the Audiovisual Surreal. New York: Oxford UP.
Sobchack, Vivian 2000. What My Fingers Knew: The Cinesthetic Subject, or Vision in the Flesh. Senses of Cinema, Issue 5. http://sensesofcinema.com/2000/5/fingers/
Torvinen, Juha 2008. Fenomenologinen musiikintutkimus: Lähtökohtia kriittiseen keskusteluun. Musiikki no. 1: 1–17.
___2006. Musiikkianalyysi ontologiana II. Fenomenologisen musiikintutkimuksen metodologisista lähtökohdista. Musiikki no. 3: 70–92.
Williams, Linda 1999 . Hard Core. Power, Pleasure, and the “Frenzy of the Visible.” Expanded Edition. Berkeley: University of California Press.
___ 1991. Film Bodies: Genre, Gender, and Excess. Film Music Quarterly, Vol. 44, No. 4 (Summer): 2–13.