The Art of a Scientist is an annual exhibit curated by Duke University graduate students who are interested in promoting dialogue between art and science. STEM graduate students submit images from their scientific research to the AoS Committee. I answered a call for artists to work with the project, and was paired with a graduate medical student who submitted a video.
The video is of the vascular system of a mouse hind leg. It begins with a view of the murine vascular tree branching out in red. We see the side of the leg rotating, then it tilts and rotates around, and disappears. An angled plane (that resembles a microscope slide) moves from bottom left of screen to top right of screen, to reveal the leg with the soft tissue enclosing the vascular system. The image rotates, then the layers melt away as the leg disappears. Then the image reverses and the layers swirl back together. The image stops before the layers finish, rotates a half turn and is complete.
The video is a collaboration between Hasan Abbas, an MD/PhD student, and the Shared Materials Instrumentation Facilities at Duke University. The mouse leg visualization can be used to model healthy and diseased cardiovascular systems. When I asked Hasan what he heard while creating the video, he said, “scholarship”, “elegance” and “discovery”. I love this as a jumping off point for a soundscape – “elegant discovery”.
And then there is the mouse! My first thought while curating samples for this project was to “give voice to the mouse”. Modern medical research is built on the backs of mice, so it seems right to honor and acknowledge their participation. I found hours of recordings of mice squeaks and scratches. Another element I wanted to capture was the branching of the vascular structure at the beginning of the video. So the creak and cracking of a large body of ice was layered into the sound bed. These sounds were synthesized into a liquidy flowing underbed (suggestive of bloodflow) over which orchestral voices swell in wonder. This piece is called O Men and Mice.
Hasan Abbas and I had several email correspondences. I sent him my first soundscape and he gave wonderfully useful feedback. In all of his correspondence, Hasan spoke with keen interest about the technology used to create the video. The method is called diceCT and the technology is a micro-CT scanner. Hasan referred to the technology as “a thousand tiny X-ray” images stitched together to create the detailed 3D image of the mouse hind leg. This made me wonder how 1000 Tiny X-rays might sound. So a second soundscape was born.
For this piece, while the primary idea is the sound of 1000 tiny x-rays, I also wanted to convey a sense of excitement and pride in an amazing technological accomplishment. diceCT is a new way of seeing living matter that could reveal hidden organic structures or systems. Drum rolls, claps and cymbal crashes are iconic sounds of triumph, so these were used as the sound source. In the video, when the tilted slide-like plane moves from bottom left to top right, the full leg emerges, and there is a feeling of a “great reveal”. This feeling is emphasized by a drum roll and cymbal splash into a moment of silence in the soundscape. For the sound of thousands of X-ray images being taken, granular synthesis was applied to the drum sounds as they built up in dense layers. Interestingly, granular processing does a similar thing to audio as the diceCT method does to matter. Hasan provided me with a video that was slower in pace for this piece. The layers of the whole leg system as they swirl away and return are so beautiful and perfectly fitted together, I wanted it to take more time.
The Art of a Scientist will open Saturday April 6 at the Golden Belt Grand Gallery (800 Taylor St. Durham) and will run through June 23, 2019.
My quest to synthesonize Ableton Live has taken an exciting new turn. Last Sunday, we discovered that by micing The Bells at the Central Park School Soundgarden, I can run that sound through Ableton and into the various synth modules and FX racks I am building. What happens is that the Abeju Synth Modules and FX Racks capture most of the harmonics that arise from Eleanor’s bell playing. The harmonics can be shaped by envelopes and attenuation and, of course, granular synthesis. My goal is to gradually shape the bell harmonics into a watery stream sound. This will be part of the soundscape for The Place ReSounds of Water (TPRSW) on April 14th at 4pm for SITES Season 2018-19.
When iBoD first started playing with The Bells, I recorded and analyzed their harmonic content. These bells are former compressed air tanks with the bottoms cut off, so the metal is not pure, it is some kind of alloy. This translates to lots of harmonic AND enharmonic content! A pure metal would render more pure harmonics. These pure harmonics are pretty, often beautiful, but my ear grows tired of the stasis of it all. The idea of purity in all of its forms is an illusion that leads to much misunderstanding and anguish in the world. Think about what striving for purity has given us: genocides, fascism, chronic autoimmune diseases, disconnection from and attempts to conquer nature, diminished empathy, and on and on. It is my prayer that riding and faithfully playing All the en/harmonic waveforms will encourage evolutionary growth. That is what I am going for!
TPRSW is my first attempt to sync up with the National Water Dance. My timing is off as this is not the year for National Water Dance, however I am hoping this will kickoff some interest for 2020. The idea for TPRSW is to give prolonged loving attention to water in the form of sound, light and the liquid itself. The soundscape will consist of Eleanor Mills playing The Bells, dejacusse aka Jude Casseday capturing and playing the en/harmonic waves from The Bells and morphing them into a watery feeling soundbed. Then Susanne Romey will play Native American flute over that for a while, then we start the wave again. The movers will pour water from vessel to vessel. An altar of flowers may be built. The whole thing is a mystery.
Our location at the Soundgarden at Central Park School gets full afternoon sun, so the visuals might include sparkles and shimmers of water. We could be lit up! If it is overcast, the air will be moist and the sounds of water will carry more clearly. If it threatens rain on Sunday, we will do it on Saturday instead! Or, perhaps, we will figure something else out and perform as it rains.
My first pass at working with Ableton as a modular synth was to explore “dummy” clips. These are tracks that contain an audio clip with the actual content muted. The clip is then transformed into a container for audio effects automation that can be applied to any audio routed through whatever audio track the clip is in. The clip becomes an Envelope template that can be pulled into other projects.
Ableton Live allows the sound designer to see and manipulate many parameters within each individual clip. This is most amazing, and something I have not paid enough attention to. Here is a picture of the clip console from the Ableton Manual:
For Audio Animation Clips, the focus is on the Sample box and the Envelopes box. The slider in the sample box is used to mute the audio by simply turning it completely down. Now the clip is ready to be shaped into an envelope. (That is how Audio Animation Clips will function in my Abeju Synth Station – as Envelope Generators (AAC/EG). The two drop down windows in the Envelope box give the sound designer access to all the effects and mix processing available on the track for this clip only. So whatever I draw onto the audio clip itself only happens when that particular clip is playing. In the case of AAC/EG, the clip will play the animation of the envelope and the envelope will be applied to any audio routed through the track.
In order to create an AAC/EG, we need an audio track and an audio clip. The length of the clip doesn’t seem to matter cause it loops. (Currently, I am sticking with 8-12-24 bars and I hear potential for longer clips with more complex movement in the automation in the future). Use the volume slider to mute the audio within the clip. (Addendum: Experience has taught me that the original audio in the clip should be low volume. I am going to start experimenting with creating audio specifically for these clips.) Then assemble a rack of audio effects you want to use to shape sound and insert them on the track. In the orginal audio clip, use the drop down box in the clip to access each effect, and turn them all to Device Off by moving the red line in the clip box to the off position. Now you have an open audio sleeve with no effects enabled. When source audio is routed through this clip, the audio will sound as is, with no effects present. Next, duplicate the clip. This will be the first AAC/EG. Go to the drop down box and turn on the devices you want to use. The top box takes you to the main effect, and the second will bring up every adjustable parameter within the effect. When you identify a parameter that you want to change, go to the red line in the audio clip window (on the right above) and draw in the animation you want.
Once I finally understood all this, I began to design AAC/EG Modules. Each module is a track with 2 to 4 different envelope template clips. Each instance of the clip can be shaped by adjusting the automation (I am calling animation) of the effects that are on the track. One technique I like is layering three effects over three clips. The track contains three effects (a delay, an EQ and a reverb), so the first clip has one effect on, the second has two, the third has three. Initially, I tried linking several modules (tracks containing the clips) together but found this too cumbersome. The option to layer modules is in the routing in each particular set.
To use the modules, insert another audio track that will hold the samples you are using. I label this track Source. Now there are several routing options, but the main idea is that the AAC/EG modules need sound coursing through them in order to perform. On the In/Out tab on the track, audio can be routed from someplace and routed to some place. The best mixing option so far is to have the AAC/EG modules receive audio from source and then everybody sends audio to the Master Volume. This allows the faders to act as dry/wet attenuators. The Source can be heard as is or through the AAC/EG Modules and Clip Templates.
So far, I have created four modules. Combed, Mangled, Stutter Delay and Harmonics Flinger. [Post Publication Addendum: More modules are added to above and all are being fined tuned. Our April 14th SITES event was rained out (rescheduled for May 19th) so the debut of Abeju Synth Station will be tonight as part of Electronic Appetizers program at Arcana. I am playing a piece called Perimeter Centre, which will feature the Ripplemaker semi-modular ios synth as my sound source for the Abeju Synth Station. Come listen!
Glenna Batson is back in town and we have started a monthly Human Origami Jam at ADF Studios on Broad Street in Durham. Join us this Friday, February 15th for an exploration guided by the foldings of cells – the building blocks of nature. While Glenna guides you through a macroscopic to microscopic sense of the cell, dejacusse will sculpt sonic forms in the atomosphere of the room. The soundscape will swath you in harmonics, whispers, bounce and back to the ambient sound of the room. Sound as water is my theme, and since cells are mostly made of water…
A major soundscape creation for 2019 is to sonify data for the Sourdough Project. The Rob Dunn Lab at NCSU, the Ben Wolfe Lab at Tufts, and the Noah Fierer Lab at the University of Colorado are collaborating to further the study of microbiomes in sourdough starters. The Sourdough Project has gathered starters from many parts of the world in order to study the bacteria and yeast interactions that create the fermenting acids and leavening gases necessary for the creation of sourdough bread.
In October 2018, the Sourdough Project Team and two artists met at the As If Center in Bakersville NC. The As If Center (Art and science In the field) is the burgeoning vision of Nancy Lowe, who is keenly interested in exploring this fertile collaborative area.The other artist was Ferne Johannssen, freshly graduated from college, and off to see what life outside of Vermont has to offer. Ferne is a visual artist/printmaker. [Interestingly, Ferne made a print on a scoby (symbiotic colony of bacteria and yeast) which grows and ferments in kombucha tea.] The purpose of our meeting was to seed an artistic and scientific direction for sharing the data from the Sourdough Project. All three labs were represented and we spent most of our time sharing information and structuring the research paper that will come from this study. Here is a description of the study from the Rob Dunn Lab website:
There are millions of kinds of bacteria and fungi on Earth. We have found several thousand species in human belly buttons alone. Yet if you mix flour and water, the community of organisms that colonize the resulting concoction is almost always composed of a small handful of organisms that are able to leaven bread, yielding a sourdough starter. How this happens is one of civilizations great mysteries, a mystery at the heart of the bread making (and, for that matter, traditional beer brewing). Yet, while bakers understand how to make starters, the underlying biology of the species in these starters remains mysterious. Starters can produce similar effects on bread (and similar flavors), despite being composed of different species, a key different ingredient. Conversely, starters composed of the same species sometimes yield different flavors. Then there is the issue of what happens to starters over time. The organisms in starters are hypothesized, by some, to stay the same over time—an old growth forest of miniatures—even if their living conditions change. Few ecosystems are so (apparently) stable. Then again, starters can change through time, sometimes suddenly. Starters are, if anything, predictably mysterious. But not for long. We aim to understand the biology underlying the differences among starters and the changes (or lack of change) in starters through time.
The last sentence of this description is what I honed in on. My current sense of how to render data as sound is that it would be most effective with data changes (or lack of) across a timeline. The other word that caught my eye is biology. What is biology? The science of living matter in all forms and phenomena, with special reference to origins, growth, structure, behavior and reproduction. The bases of biology are macromolecules (proteins, lipids, nucleic acids and carbohydrates), cells, and evolutionary changes creating phylogenic families across species. With sourdough starters, we are at the microbial layer of life. On the microbial level, diversity rules and it may have something to teach us. That is what I hope!
The Sourdough Project team had a conference call a few weeks ago, where we saw some of the data analysis of the samples, and received updates from each of the labs. Patterns are starting to emerge as the data is narrowed and focused into categorical relationships. This is the crossroads where it all comes together in the question: What do I want from this data? This most interesting question was posed our first night at As If Center, as we sat around an outdoor fire: what is your currency? what do you want from this project? I can’t remember ever having been asked that before.
The bakers who sent in samples want to know the microbiotic fingerprint of their particular starter. The scientists want to discover some new information about the ecologies of sourdough starters in general. The artists are interested in translation, transposition, representation of the discoveries found in the fingerprints. For myself, I am looking to identify a timeline and voice the bacteria-yeast exchange that is fermentation and leavening. Here is a diagram of a potential time frame:
which give rise (the timeline but also a phase within the process)
to VOCs (Volatile Organic Compounds) aroma
To my ear this begins with the very lively interaction of the organisms that changes over time into a lighter, gaseous state. There is an alchemy that takes place and we are trying to hear and understand that.
Still looking at TRIC (Terry Riley’s In C) as a template for orchestrating interesting timbral relationships in this context. Pattern 35 is a possible frame for rise which seems to be the name of this piece. Pattern 35 jumps to a start with an eighth note run. This is the organism interaction phase. Then the mid section is where the rise happens with more space and elevation in tone. Then the aromatic texture is very open and light and unfinished.
What other sound elements might lend to this soundscape? There are likely real live sound samples to be had from this process. Another thought is what if each starter could have its own microbiome sounded out? To do this, I need to see more deeply into the data then I have to this date.
Reading deeply into sound, physics, chemistry, and nuerobiology, it becomes clear that lived experience is comprised of waveforms modulating each other. These waveforms weave around and through us. The waves are different frequencies and different amplitudes, they have different shapes and resonances AND they interact with each other, all of which creates various densities of matter. As extreme examples, there is a diamond and a thought. These extremes define the boundaries of material experience – the hatd shell of the matrix we exist within. There is more beyond these boundaries, and the more seems to be the field of conciousness.*
With this as a basic understanding, it is my intention to mind the oscillations, particularly those that eminate from my human being/my earth suit/the event horizon of my world. The waveforms I conjur come into phase relationships with themselves and with the waveforms of other human beings as we event our common reality/the hard-shell matrix. This is an amazing realization! This realization sheds light on a major question I have danced with in my life:
How can I be more loving? So much of my living is spent seeking love from “the world”. That was my quest for the first half of my living experience. While it does hurt so good, I finally realized the question is framed in an unhelpful way. So I am moving from how can I get more love, to how can I be more loving?
My most important practice for exploring this question is awareness and presence in the moment. We have been listening to Eckhart Tolle go on and on about this. He works very hard to articulate awareness/presence in all the ways that might possibly bounce around (or cut through) the mind. Listening to ET, I am aware of the trinity of self, egoic mind and conciousness all walking into a bar. (It is a good practice for them to stick together, although the bar was probably egoic mind’s idea.) These three are waveforms of their own, and their interactions eminate into manifest world! These two thought-fields (waveforms of matter and awareness through being) intersect at my heart.
One definition of oscillation (from physics) is
regular variation in magnitude or position around a central point
This aptly describes the interplay of these two planes intersecting through me at heart center. I feel this most intensely when I am sounding harmonics through a space. Especially using modular synthesis – with all the operators, envelopes, slopes, attenuations, triggers, gates and frequencies- to create actual sonic forms in space. At present I play with Ripplemaker ios app from Bram Bos. It is great to learn on. Ableton can function as a synthesizer. This is my focus at present- to configure Ableton into a synthesizer. (I am deeply desiring a modular system, but must go slowly due to funds. So work with what you have!)
This first week of 2019 – a new year (marking earth’s intersecting oscillations with the sun, moon and stars) brings some interesting mantras –
Listen more deeply, mind the oscillations, propagate harmonic vibrations, learn to play with forms, depersonalize the whole thing! The WoW
Alrighty then!!…on with the emanations!
*These books have informed my thinking about all this:
Long, Manya J. The Psyche as Interaction (Electromagnetic Patterns of ConsciousEnergy)
Lanza, Dr. Robert Biocentrism and Beyond Biocentrism
Since excitedly sharing the results of a sonic analysis of Lemur Gut Microbiomes, I have been working up a soundscape based on the quick sketch included in the first Sonifications and Life Forms post. (You can hear both below.) In order to get feedback on the work, I sent the first blog post to Mark Ballora, with whom I had taken a data sonification workshop in May. His response helped me realize the need to clarify my sonification process. So here is a description of the project:
The purpose of the sonification is to illustrate the changes in baby lemur microbiomes from birth to weaning. Microbiome data was captured through fecal samples taken at birth, through nursing, introduction to solid foods, regular solid foods, and two times while the babies were weaning. The sonification will illustrate changes in the type and amount of bacterial phyla present at each of the six sampling stages for all three lemur babies. In addition, the mother’s microbiome was sampled at the time she gave birth, so her profile, which was assumed not to change, provides a baseline adult profile with which to compare the babies’ changes.
There were 255 strains of bacteria collected over the course of the study. These fell into 95 classes and 35 phylum. I focused on the phylum, as my plan was to assign a note value to each bacterial data point, so I needed a smaller data set. The data set was narrowed further (and made more interesting) by focusing on a family: a mother Pryxis, and her triplets, Carne, Puck and Titan. This group allows us to not only hear the variety of changes in the babies’ microbiomes, but compare the changes as well.
The original data set included 9 lemur babies and 7 mothers. So the first step was to go through the phylum data sheets and pull out the profiles for Pryxis, Carne, Puck and Titan. A phylum profile would be the type and amount of each phyla present at each data collection point. The profile changes over time at each collection point. The microbiomes of these four lemurs housed 15 phylum (at a density of >.001) out of the 35 found in the entire study group.
The next step was to assign a note value to each phyla. Since there are only 13 notes available in the chromatic scale, some phylum would need to be on the same note, albeit a different octave. Same note, different octave will lend a tonal consonance to the profiles. So what might this consonance represent? There were 5 phylum that had the greatest density and presence in all the samples, so I assigned those to the note G from octave 1 to 5. The remaining 10 phylum were assigned note values based on their presence throughout the profiles, and on their consonance/dissonance with the tonal center G.
In order to capture the density of each phyla, a midi velocity range was aligned with the decimal percentage of the phyla in each profile. Midi velocity settings determine the force with which the note is played. Thus the velocity ranges render a clear sense of presence or loudness to each note played. The decimal percentages ran from .001 to 1.0 and the midi velocity range runs from 1 – 127. Here is a chart of how these ranges overlap:
So for example, Protobacteria present at .25473 would be represented by the note G at octave 3 set at 40 velocity. The largest sample in all the data points captured for this project was around .9 and the smallest was .001 (this was a cutoff point as there were bacterial phylum present down to .0001 ranges.) Here is the chart for Titan showing note assignment and density values through each sample stage:
My sounding board for this data comparison is Ableton Live, a digital audio workstation (DAW). The individual lemurs are represented by a “voice”/midi instrument in Ableton. Tuck, Titan and Carne are bell-like voices that blend together, while Pryxis, the mother, is a warm, pervasive woodwind. She envelops and contains the changes in the babies’ phylum profiles.
All lemurs had a Phylum Profile Chart like the one above. In the DAW, the instrument track for Titan, for Puck and for Carne contains a midi-clip of notes of the phylum colonies present at each stage of dietary change, which were then laid out as a “scene” in Ableton. As example, Titan’s Phylum Profile at birth was
Protobacteria (Note value=G3) set at 101/127 in intensity
Euryarcheatae (Note Value=A2) set at 34/127
Firmicutes (Note Value=G2) set at 11/127
Cyanobacteria (Note Value=A#4) set at 1/127
Other Bacteria (Note Value=B3) set at 1/127
Spirochaetae (Note Value=G5) set at 1/127
Titan’s Birth Phylum Profile is the multi octave chord GAA#B. Three of the phylum were barely present, so those tones are almost inaudible in the chord. However, 2 Gs and the A ring out. The total number of phylum present in each dietary stage varied from 3 to 14, so the multi octave chord becomes more dense and dissonant when the phylum are so varied. Here is a look at the tracks (individual lemur voices) and the “scenes” (which are the phylum profiles from all 3 babies at each stage.)
The first sketch was just the mother’s phylum profile droning under the three babies’ profiles expressed as a stacked megachord. All 3 baby profiles rang out together four times at each stage, starting with birth and ending with the second wean. What could be heard was a homogeneity and consonance between the Mother and babies at birth that gradually became more diverse and dissonant as solid food was introduce. However, by the second wean, the babies’ and mother’s profiles become more consonant again. The researcher said this illustrated the conclusions of her study.
As a soundscape artist, I felt there was more here than just that basic chordal movement. The babies’ phylum profiles were quite different from each other as well, which is lost in the chord presentation. For example, Carne’s birth profile has only 3 phylum, while Titan has twice that amount. One way to hear this level of contrast in the baby profiles is to articulate the chords into riffs. Now we can hear the interplay of the changes in their microbiomes. In addition, we can hear how consonnant/dissonant and dense the phylum become as outside food is introduced into their systems. Titan’s phylum profiles arpeggiate down, Puck’s go up and Carne’s go down then up. A practiced deep listener could key in on a particular profile and follow it through to the end. I played around with rhythmic shifts to create more movement in the stages where the phylum profile were incredibly dense and diverse. The last two arpeggiating riffs you will hear are all of the phylum notes sounding through twice. And listen for the elevated levels of Protobacteria in all 3 profiles at birth – that G3 rings out at that point.
As I put this full family profile together, another more nuanced movement in the data appeared. In the chord rendering, I heard the data get more dissonant and dense from nursing through first wean, and then the phylum thinned out and became more consonant at the last wean. In the riff rendering, I can hear a contraction and more consonance at the Intro to Solid Foods stage as well as the second Wean. That was not clear in the chord presentation. When I checked my data records, there was a drop in the number of phylum present between Nurse and Intro stages. I love that a nuance appeared in the listening that made me go back and check the data. That is exactly how I hope this process will work.
Some other things for future consideration:
Aligning each phylum tone to a particular beat might help the listener hear the differences from stage to stage more clearly.
When assigning notes to data points, closer attention to the harmonic overtone series might help clarify the role consonance and dissonance play in hearing the data.
The voices of the baby profiles have similar timbre as a unifying element. The profiles could have very distinct voices which might make the variances in their profiles more audible.