Sonic Plasticity, An Introduction

Sonic Plasticity proposes the use of sound as a malleable material — one that can be stretched in all dimensions, encompassing height, width, and depth, with curves, edges, and changing geometries. The works produced within this framework serve as the context in which a person is encouraged to (a) contemplate the act of listening as a phenomenon; (b) be actively present in the moment, observing the inner cognitive and emotional challenges involved in receiving the new art form in detail; and ultimately (c) overcome such challenges, extending the person’s ability to sense the world, thus widening the sensing spectrum that encompasses the human experience.

Sound is the result of dynamic actions, periodic vibrations, sudden impacts, or oscillating resonances. Everyday sounds like clapping, dragging a chair, or a dog bark produce a pressure wave that propagates from the source, encountering multiple objects, all made of different materials. At each collision point, the particles forming the pressure wave bounce and change directions forming new, reflective waves. These interactions get encoded into the final waves that make our eardrums resonate, which contain information not only about the source — the event that originated the wave — but also about the space surrounding it.

Our mind is constantly scanning for sources of sound around us. The simple act of listening allows us to estimate where sound-emitting sources are in relation to the listener, as well as whether they are moving, which direction they are moving in, and if they represent a possible threat. Our ability to locate a sound’s source through listening has evolved and developed over thousands of years and has been fundamental to our survival.

Sound localization is very useful to our daily lives but is rarely considered when creating artistic experiences. Audio engineers use studio processes like panning and reverb to recreate the placement of musicians during a performance by recording their individual sonic contributions and placing them within the stereo field formed by two speakers. When this is done correctly, it can make the listener feel closer to the music they are experiencing, almost as if they are in the studio.

But once a sound has been recorded, it becomes disassociated from the source that produced it. This means that the dynamic event that initiated the sound is no longer responsible for creating the pressure wave described above. Instead, an electronic setup is needed to reproduce and amplify the recorded material. Our ability to identify the physical location of a sound produced through electronic means depends on the dimensions, surfaces, objects, materials, and environmental noise level of a listening space as well as speaker placement, panning technique, how sensitive and trained our ears are to decode sonic spatial cues, and how willing we are to take in the experience with an open mind. The frequency balance of the sound being amplified also plays an important part in our ability to locate it. These factors interact with each other in very intricate ways, and fully describing such interactions requires in-depth interdisciplinary research.

To illustrate the complexity of the interaction among all of these factors simply, let’s consider a minimal setup: an audio player connected to a single speaker in a quiet, midsize, neutral-sounding space, such as a professional sound studio, a low-reverb college classroom, or a serene outdoor space. Imagine a sound file is reproduced through such a setup. In this scenario we experience a match between where we see the speaker standing and where we perceive the sound to be coming from, and therefore we correctly attribute the source of sound to the speaker. Some may think of this as an obvious assumption, but the relationship between the location of a speaker and our spatial perception of a sound’s location can quickly become counterintuitive.

Consider a different scenario: In the same space described above, an audio player is directly connected to two speakers. The same audio signal is sent to both speakers with the same volume level. Imagine that a listener is standing in front of the speakers, separated from them at a similar distance to that between the speakers, forming a triangle (see figure below). In this new scenario, a person would see two possible sound sources but hear one virtual source between the two speakers, as if the sound is coming from a place where there is no speaker!

Recorded sounds exist in the form of information, and this information can be transformed by adding electronic and/or algorithmic processes modeled from our observation of how sound and space interact. Audio engineers and composers use reverb, delay, and filters to add information to a reproduced sound that our minds interpret as spatial cues, similar to the ones added by a physical space. Such processes allow the creation of dynamic virtual spaces. The shapes and properties of these spaces depend on the parameters fed to the constituting processes over time, creating a sense of space. At times, the chain of processes can make the sound feel as if it has been produced in a very large space, and immediately after, fade into a small, narrow space. Virtual spaces are dynamic. In contrast, the effect a space has on sound is mostly static, as it depends on the dimensions, objects, and materials of the space, which seldom change over time.

The same evolutive mechanisms we use to decode the properties of a space through listening can be fooled in order to create new aesthetic experiences. Using multichannel speaker systems, like a commercial surround system, virtual sources can be placed anywhere within the periphery of the plane formed by the speakers. The more speakers in the plane, equally distributed, the more resolution a person has to place a virtual source in space. Diffusion techniques like Ambisonics, Vector-based Amplitude Panners (VBAP), and Distance-based Amplitude Panners (DBAP) allow for the elevation of a virtual source above and below the more traditional surround sound speaker arrangement. Wave Field Synthesis allows virtual sources to be taken out of the periphery of the sound system and inside the space surrounded by the system, allowing a person to specify the location and direction of a virtual source anywhere inside the listening space.

These technologies not only facilitate the placement of a virtual source in a single point in space — that is, that we perceive sound coming from a single place — but can also make this point move seamlessly over the plane formed by a sound system. Most importantly, such technologies also allow for the shaping of the virtual source itself, meaning that sound is no longer perceived as produced by a single point in space, but by areas with curves and edges. The shaping of a virtual source is fundamental to the ideas described in this essay.

There are multiple examples that come to mind when attempting to describe the experience of sonic geometry, the most graphic being the analogy of the beehive. Imagine hundreds of bees flying around a beehive with seemingly random trajectories. One could step far back and focus on the sound emanating from the beehive as a single source, with almost no sense of movement, or walk closer to it and pay attention to each bee’s sonic contribution as they move around the beehive in all directions, or stand next to it and focus on how the sound of hundreds of bees gets amalgamated into a unified source with height, width, and depth. There are different techniques for sound synthesis that resemble the movement and behavior described in this example, with parameters that permit the curving and shaping of this cumulative virtual source within the confines of a sound system.

As a result of studying and experimenting with such technologies and diffusion techniques, I propose Sonic Plasticity, a new paradigm for observing and creating sonic experiences. This perspective acknowledges our innate ability to sense space through listening, and offers a series of concepts, tools, and workflows that explore the artistic potential of such phenomenon. This framework proposes the use of sound as a malleable material — one that can be stretched in all dimensions, encompassing height, width, and depth, with curves, edges, and changing geometries. The works produced within this framework serve as the context in which a person is encouraged to (a) contemplate the act of listening as a phenomenon; (b) be actively present in the moment, observing the inner cognitive and emotional challenges involved in receiving the new art form in detail; and ultimately (c) overcome such challenges, extending the person’s ability to sense the world, thus widening the sensing spectrum that encompasses the human experience.

Using Sonic Plasticity as a framework, I have identified four interrelated lines of research exploring the malleability of sound and the use of sound as a building block to create spatial experiences. Each line of research comes in the form of a series, each of them with multiple works set to shine a light on different phenomena. I call these four series Aural Fields, Resonant Bodies, Sound as Object (sound-object), and Sound as Space (sound-space).

Aural Fields

The Aural Fields series is composed of multiple geometric sculptures of different sizes, shapes, materials, and finishes, hosting an embedded multichannel sound system. Each sculpture is accompanied by an ever-changing sound composition created specifically for it.

The sculptures’ specifications — weight, size, and shape — emphasize the spatial quality of the sounds generated from within. At the same time, these sounds highlight the sculptures’ dimensions as they get stretched throughout the sculptures’ panels, sonically illuminating its edges. The tight relationship between the sculptures and their sounds is explored over time.

The intricate combination of sounds and their movements produce spatial patterns. By focusing one’s attention on the area these sounds are coming from, audience members will be able to trace the location of these sounds over time, allowing them to experience volumes forming and moving in space — volumes whose contours cannot be seen, but only heard.

Resonant Bodies

The Resonant Bodies series is made up of multiple geometric sculptures, which are mechanically excited into a vibrating state, turning their whole body into sound sources. The ever-changing drone produced by the sculptures’ vibration presents an audience with the opportunity to experience the objects’ spatial properties — dimensions, curves and edges — sonically; in other words, to stand in front of a sculpture and to hear its geometry. The sculptures comprising the Resonant Bodies series come in different sizes, shapes, materials, and finishes.

Sound-Object

In the Sound-Object series, sound-only works use the standing wave phenomenon as a composition tool. Standing waves occur in an enclosed listening space as first-generation sound waves interfere with reflected sound, producing volumes (or large virtual sources) in space where certain tones become amplified, modulated, or canceled. As the audience explores the listening space, they step in and out of these sonic volumes. Each volume has distinct tonal qualities. By walking around and navigating the listening space, audience members can choose which volume to experience or avoid, or even focus on the in-betweens, thus shaping their own experience. By identifying the limits of a volume, an audience member senses its contours and shape.

Sound-Space

Lastly, the Sound-Space series explores the role visual and sonic stimuli have in our mind’s ability to perceive space. Works in this series begin within a constructed and controlled space with a simple, generic architecture that functions as a spatial reference point. Composed sounds are played into the space in succession. These sounds aim to disjoint visual and aural perception, blurring and extending beyond the visible limits of the space, making the space sound at times larger, taller, or narrower than it looks.

Sound-Space is a ramification of the ideas behind Aural Fields, and yet the perspective is different enough to require its own series. In Aural Fields, we experience a static object with sound emanating from within. These sounds are created by virtual sources, which normally move within the front panel of the object (up and down, left and right). We experience the sounds’ movement while standing in front of and within proximity to the object. In Sound-Space, we see a constructed space, maybe similar in size to a shipping container. We walk into this space and notice it is sonically insulated inside. In such space, we also hear moving sounds going up-down, left-right, back-forward, but these are placed within changing virtual spaces. At times, a sound can feel like it is coming from behind one of the space’s walls while the effect of the wall upon the sound is not perceptible, as if the wall is sonically transparent. Or maybe we feel a sound is moving towards us from far away, much farther than the wall is. The resulting sounds challenge our spatial perception of such space, as well as our understanding of the inner processes involved in spatial perception.

There are many works that have influenced the conception of Sonic Plasticity as a framework. On the sound side, the works of La Monte Young, especially Dream House; Maryanne Amacher’s writing, music, and lectures; Bernhard Gunter’s Monochrome White / Polychrome W/Neon Nails; Iannis Xenakis’ Concret PH; and Michael J. Schumacher’s Room Pieces have been fundamental to this research, most importantly because of the artists’ use of sound as a material, and their promotion of active and spatial listening, bringing the listening space into the composition. Walter de Maria’s Broken Kilometer and Earth Room, Dan Flavin’s Monument, Richard Serra’ Sequence and James Turrell’s Afrum I (White) have also been great influences to Sonic Plasticity, especially because of their commitment to material, form, and perception, to fusing the art object and the environment into an enveloping spatial experience, and for their unapologetic inclination toward ideas of monumental scale. I look up to their artistic output, sometimes for comfort, as I continue to move forward over the rocky path towards realizing my most ambitious works.

Earlier in the essay, we briefly looked over the factors that enable sonic localization and provided a couple simple examples to illustrate the complex interplay among them. I provided some directions that can help the reader further explore the technical, physical, and cognitive phenomena involving spatial listening. The books Spaces Speak, Are You Listening? by Barry Blesser and Linda-Ruth Salter and The Soundscape by R. Murray Schafer are excellent resources to further dig into this type of research.

Apart from the physical ability to participate in and understand spatial listening is a personal and cultural factor that could limit our capacity to hear geometry; namely, our willingness to take in the experience with an open mind. I think it is fair to say that to most people, regardless of profession and educational level, think of any creative endeavor involving the order of sound (and silence) over time as the same as music (and, if not music, then what else could it be?). Furthermore, our idea of what music is has been heavily informed by the rhythm, timbers, tonal structures, voices, narratives, and emotional motives accessible to us within our personal cultures. These two propositions combined leave little room in our imagination for an artform that explores sound outside of the common limits of music.

This is a problem for artists working on sound outside of music, since the lack of a referential system to compare and analyze these new works makes it difficult for funders to assign value to these works, and without funding opportunities it is difficult to develop such works to their full potential. As sound technologies are also expensive, the production of such work is even more difficult. For example, just the fabrication, technology, and material cost of producing one Aural Fields piece can easily go over $10,000. To help continue my work despite the lack of institutional support, I cofounded The Honk-Tweet, a production company that creates experiential content, and became a technical advisor for the company. Like most artists, I would prefer to focus on my research and the creation of my work on a full-time basis.

Music provides comfort, accompaniment, and entertainment; it helps us at times to feel more human, to connect with one another. Music empowers us, challenges us emotionally and intellectually; it is a field that has produced incredible artistic value to humanity. At the same time, music’s long history, traditions, and symbolism have cemented our preconceived notions, informing how and why we participate in a musical experience. To many, thinking of sound outside of these boundaries means entering new territories. While I am hopeful that music will continue to exist as a living art, with living and expanding borders, I think it is important to acknowledge that sound art has long been dominated by music, similar to how painting dominated the visual arts for centuries.

It is not within the scope of this essay to suggest a classification system for the field of sound art that would allow sonic explorations outside music to thrive. Neither do I suggest that categories have an inherent value other than clarifying the artist’s intent behind the creation of the work, and hopefully setting up the expectation to appreciate the art. Moreover, certain works are meant to exist within classifications while others resist classification altogether. There are many examples in the visual arts of works that fall in between categories; Frank Stella’s Bogoria I and Barnett Newman’s Vir Heroicus Sublimis are two examples that come to mind. There is a painted canvas against the wall, but the style, form, and colors on it exist to highlight the canvas’ dimensions and edges, turning the canvas into an object that is present, that has geometry.

The intentions behind Sonic Plasticity are not musical. Using music as a context to think of and talk about Sonic Plasticity does not help a general audience get in the proper mindset to fully experience works created within the framework. If anything, such context will confuse the audience, as their preconceived expectation will not match their experience, resulting in alienation. Instead, Sonic Plasticity invites the audience to leave preconceived notions behind, providing a unique opportunity to experience sound in a new light. It invites the audience to be present, to interrogate their very own listening experience as a phenomenon. The sonic material and other elements comprising these works are purposefully there to stimulate and foster such behavior. This is the reason why the sonic content produced by the pieces is meant to be constantly changing and evolving over time, to be algorithmically created in the moment. This is also why the sounds used are abstract, allowing the narrative resulting from the transition between these sounds to be as self referential as possible. The sculpture is also there to provide a visual frame of reference to host the sounds. Ultimately, Sonic Plasticity is not about its sounds and sculptures themselves but an inner experience.

Sonic Plasticity happens within, as we allow ourselves to hear geometry. We seldom question the phenomena involved in how we see the contours, shape, and volume of a sculpture, mostly because our mind paints the picture for us almost effortlessly. But the phenomena involved in seeing geometry is not simpler than the one studied by Sonic Plasticity. The potential to hear geometry exists in all people who do not have a hearing disability (and even some who do), but converting that potential into action requires willingness. In many cases, our ability to hear geometry might be limited by our inability to be present in a situation we might not be able to control immediately; in other words, the fear of looking and/or feeling powerless. If anything, I believe these works provide an opportunity for transformative change, to earn a new perspective of the world and widen the sensing spectrum that encompasses the human experience.

Imposing musical expectations into the work — specifically, expectation on how musical sounds should sound — as well as expectations of musical narrative and emotional motives, can result in a cognitive brick wall, capable of reducing the rich details present in the sounds into plain, featureless noise. Our ability to concentrate and be attentive over an extended period of time can also impact our ability to hear geometry. Sometimes, the quality of a sound used in a work may trigger a memory, an image, or a thought. Appreciating that and then coming back to the moment gives people the opportunity to fully take in the experience. Our ability to experience sounds as they are, as well as sonic geometry, becomes easier and more natural over time.

The narrative created within Sonic Plasticity is nonprescriptive, in that there is no single way to experience these works. Moreover, there is no way to fully experience them, to exhaust their possibilities. Engaging with these works offers unique opportunities for the discovery of both the external space and our internal perception processes. I invite you to experience these works with an open mind and the willingness to be enveloped by them, similar to how we appreciate the changing color of a sunset in the open sky of an autumn evening.