DAVID GREEN, CRAIG J. RODGER, JAMES B. BRUNDELL,
STEVEN MILLS AND PETER BROOK ¹

Scientific Reification

 

For full text of this article with illustrations included please download PDF at http://www.junctures.org/index.php/junctures/article/view/228/410

Figure 1. Artist’s documentation of Embodied Earth, an installation shown as part of the Art and Light Exhibition held in the HD Skinner Annex, Otago Museum, Dunedin, New Zealand, August 2015.

INTRODUCTION

In the Embodied Earth installation, part of the Art and Light Exhibition held 15-30 August 2015 in the HD Skinner Annex of the Otago Museum in Dunedin, the viewer sensorially experiences lightning strikes in synchronicity with actual terrestrial lightning events occurring over a large swathe of the Earth’s surface via a live data stream. Viewers face a large projection screen on which they can see themselves in silhouette. My design intends the viewer to don a haptic jacket and move freely, as a live data stream, translated into animated lightning flashes, tracks the viewer’s screen position, appearing to strike the wearer’s upper body. In association with with the visual cue, the viewer would feel a strong vibration at the point of apparent lightning contact, concurrent with a synchronised subwoofer signal that pulsates their body with a short burst of low-frequency sound.

EMBODIED COGNITION

The idea behind the installation emerged from a series of lectures and discussions about neuro- evolution and neuro-ethics led by Associate Professor Mike Paulin (Department of Zoology, University of Otago) and Professor Grant Gillett (Division of Health Sciences, University of Otago) that began in February 2013. These talks often referenced the concept of “embodied cognition.” According to the Stanford Encyclopedia of Philosophy, “Many features of cognition are embodied in that they are deeply dependent upon characteristics of the physical body of an agent, such that the agent’s beyond-the-brain body plays a significant causal role, or a physically constitutive role, in that agent’s cognitive processing.”²

During the Cambrian Period—about 541 million years ago—neural networks exploded onto the Earthly scene and suddenly, from a small variety of relatively neuron-free biota, there emerged the brain-mediated world of the hunter and the hunted. From their genesis our neural networks evolved synergistically and in concert with the whole of our physical apparatus, fully in tune with specific physical environments. The result is that our ways of knowing are fully interwoven into our whole bodies; our whole bodies are fully interwoven into the whole material world.

Embodied neural networks reconfirm our boundaries of self and maintain bodily homeostasis through multiple sensory inputs, analysis and response strategies: oxygen and carbon dioxide levels, blood pressure, heart rate, metabolic rate, salt balance, immunity, pupil diameter, thermoregulation, proprioception, hunger, thirst, sleep, smell, taste, feeling, hearing, sight …

Beyond these complex functions, experiments revealing the existence of other phenomena—such as blindsight—indicate that our neural networks also collect volumes of subliminal information at any given moment. Through diverse receptors, we experience far more of the world around us than is readily apparent to the conscious processes of our brain—in more parts of the electromagnetic spectrum than is simply accounted for by audible sound or visible light. This expanded net informs intuitive feelings that affect our responses to a given situation and help determine which way we will jump at a critical moment. In this way, we call upon an extended array of neural resources to preserve ourselves, as generations before us have done, under unpredictable and dynamic Earthly conditions.

I am interested in exploring more fully embodied approaches to reading and intuiting data, making use of mechanoreceptors, thermoreceptors and chemoreceptors. If one can feel, taste, and/or smell complex datasets, perhaps a more integrated and invested response would emerge from the readings taken. These sensory tools and their complex neural pathways enabled generation upon generation of our human forebears to survive long enough to reproduce, despite relentless environmental predicaments. Like all biomechanical systems, they demand an energy budget to operate and maintain; as they have not been discarded in the process of evolution we, as terrestrial creatures, underutilise these assets at our own risk. The installation Embodied Earth is intended as a small step towards the development of such an expanded reading.

While scientific visualisation and sonification are well-established ways of making complex datasets accessible, a new rubric seems necessary to describe this concept. The word “reify” means to embody, to make palpable something which we otherwise cannot sense. It is a term called upon to perform a number of tasks within diverse disciplines. Notably, “reification” has been used in Marxist theory to denote modernist economies that objectify, quantify and concretise social, economic, political and material relationships. I would like here to appropriate and reposition the “thingification” word (in German: Verdinglichung), when used in association with the term “scientific,” to mean a technologically mediated affordance created from a complex dataset. Made “sense-able” in at least three ways (eye, ears, plus), it would bring our more intimate experiences of proximal stimuli (feel, taste, smell, movement and propriosense) into cognitive play with our presently over-mediated and generally more distal stimuli reception (vision/hearing), in order for us to better intuit dynamic and complex systems. This is the meaning I intend for the descriptor “scientific reification.”

In “Scientific Misprision,”³ I put forward the argument that when art engages with science the outcome should transcend a straight illustration of a scientific concept (the task of science communication), but instead offer a creative misreading or repositioning of the ideas presented. In this way, rather than reiterating an accepted paradigm the artist offers the possibility of a new thinking tool. I am interested in encouraging the development and practical application of the “scientific reification” of a number of correlated datasets, working in conjunction with scientific visualisation and sonification, in order to facilitate an intuitive, more fully embodied engagement with the Earth that we increasingly deconstruct and reconfigure.⁴ Perhaps such mediations will help cultivate the cognitive development of a new generation of systems practitioners who might learn to operate more delicately and responsively in a space between positivism and intuition.

A FLASH OF LIGHTNING

Writing in 2014 in the online journal LiveScience,⁵ Becky Oskin reported that radio waves, created by lightning strikes over large regions of the Earth (as distant from New Zealand as Alaska), travel along Earth’s magnetic fields and have been found to pass through Dunedin, New Zealand, as the magnetosphere re-enters the planet. In Dunedin, the strikes are recorded within a VLF (Very Low Frequency) signal in real time by Craig Rodger’s team at the Physics Department, University of Otago, and shared with an international research consortium, the World Wide Lightning Location Network (WWLLN) (see Panel 1).

Figure 3. Artist’s documentation of Embodied Earth, an installation shown as part of the Art and Light Exhibition held in the HD Skinner Annex, Otago Museum, Dunedin, New Zealand, August 2015.

Lightning is a spectacular dynamic that all of life on Earth has evolved around, if not through. It is an elemental force that is still not entirely understood, particularly in its close association with volcanic eruptions. In an example of remarkable mid-twentieth-century thinking, Stanley Miller and Harold Urey hypothesised that lightning strikes, mixed with the basic chemistry of a young Earth, may have kick-started life on our planet.⁶ Their famous experiment attempted to create a laboratory model of the basic conditions of Earth’s early atmosphere by heating and electrocuting water, hydrogen, ammonia and methane over a number of days. An updated version of their experiment, reconfigured by their former students to include additional gases expected from volcanic eruptions, resulted in the spontaneous creation of 22 different amino acids, 20 of which form basic components of complex proteins found in all living things.⁷

I wanted to build an artwork that would feature an embodied experience using a live-streamed realtime dataset. With its strong local connections, the WWLLN became the prompt.⁸

EMBODIED EARTH

What if we could extend our nerve endings and neural processes to include the whole Earth? What if our experience of the Earth could span geological time?

If we experienced our planet as an extension of our body, might we treat it differently?

Craig Rodger’s lightning datastream was the first essential ingredient of this project. I contacted him and James Brundell through Dr Ruth Napper (Department of Anatomy, University of Otago) who, along with Peter Stupples (Art History and Theory, The Dunedin School of Art), was responsible for organising the Art and Light Exhibition in 2015 (see Panel 1).

Following our discussion, Steven Mills, of the Computer Science Department, University of Otago, and I talked over the basic viewer/haptics/screen interaction I was hoping to achieve. Steven immediately identified the Kinect sensor as the appropriate mapping device for the viewer’s position, and began to write software to work in conjunction with the VLF signal and the sensor (see Panel 2).

I wanted viewers to experience global lightning strikes in real time and, in a larger sense, to intuitively identify with the Earth, if only for a moment. For this purpose I wanted them to see a projected image of each VLF-derived strike hitting their arms and upper torso while at the same time experiencing a specific haptic sensation at the place of visual impact. I briefly explored the idea of using static electricity, but it proved to be too complicated and, for some viewers, potentially unsafe. I then considered Arduino-driven cellphone vibrators which I had seen Mike Paulin use in conjunction with his bristlebot project.⁹

At this point in its development, the proximal sensory aspect of the project hinged on the controller- driven haptic interface. I was eventually directed to Peter Brook, who teaches embedded systems at Otago Polytechnic (see Panel 3). He was interested in the project, but wasn’t sure about completing a prototype within the time frame given. I continued to explore and consider other opportunities for engaging the viewer’s mechanoreceptors, deciding that the simplest backup would be a subwoofer linked to the VLF feed. At a minimum, this would safely and effectively pulsate the viewer’s body in sync with the real-time lightning animation, creating a rudimentary haptic interface. Ben Watson, a second-year student at the Dunedin School of Art, in conjunction with Steve Mills, worked through a number of technical issues in order to identify a workable signal for the subwoofer. A commercial audiovisual company, Strawberry Sound of Dunedin, generously allowed Ben to experiment at their workshop, and provided us with an amplifier and subwoofer for the duration of the exhibition.

At the time of publication, the haptic jacket has yet to be brought into sync with the streaming data that drives the visuals. This aspect of “proof of concept” has yet to be completed. However, the subwoofer pulsation, sound and body-tracked lightning strike visuals all operate in sync and work convincingly in themselves. The haptic jacket prototype tests, working in near-sync, were successful enough to make us feel confident that, once connectivity to the streaming data is achieved, this deeper experiential engagement in real time will only gain resonance.

BACK TO THE FUTURE

Like a tsunami, the clearest view of a new technology or medium seems to occur as it looms up on the horizon and just before it hits. Once the new medium truly engulfs a culture and settles into more predictable currents, production generally falls into a series of locked and repetitive steps. (This is clearly illustrated in the early history of the cinema.) Neglected opportunities may be explored later, as one medium falls out of relevance and all heads turn in unison to the next. (For a recent example, note the explosion of the complex long-form narrative on television coinciding with the rise of the Internet.) Similarly, new media can rejuvenate old media—for example, radio has become a teaser and a navigational tool for directing viewers to deeper explorations on the Internet. This both illustrates and adds resonance to McLuhan’s famous dictum that “the ‘content’ of any medium is always another medium.”¹⁴ Processes of media integration and succession accelerated dramatically from the mid-1990s.

From the late 1940s through the 1970s, computers were steadily completing the transition from human to machine, from science fiction fantasy to the quotidian. At the same time, the seismic wave of broadcast television was still breaking over the global social infrastructure.

In the midst of these technological revolutions, “systems art” emerged within the conceptual art movements that had started in the early 1960s. In his seminal book Beyond Modern Sculpture,¹⁵ Jack Burnham suggests that artistic expression is limited only by the technology of a particular period and its conceptual paradigms. In the financial and industrial boom that followed the Second World War, new frameworks of thinking and new media technologies suddenly made it possible for artists to experiment with different sorts of materiality to produce and present artworks in novel ways.

Beginning with Rachel Carson,¹⁶ who instigated the broader environmental discussion in 1962 with her book Silent Spring, this postwar, pre-home-computer period produced a number of prescient and cautionary voices, each of whom sought to find new ways of helping humans process and respond to post-modernity’s looming challenges.

Figure 4. The Fuller Projection Map13 is a trademarked design of the Buckminster Fuller Institute. ©1938, 1967 & 1992. All rights reserved, reproduced with permission, www.bfi.org.

Either obliquely or directly, these prophetic voices described fully embodied technological mediations developed in order to respond dynamically to larger environmental changes. One such voice was R Buckminster Fuller. Polymath, environmentalist and thinker, he was responsible for a number of dynamic and innovative ideas, perhaps most famously the geodesic dome. He also began to popularise the “Spaceship Earth” concept from the mid-1960s.¹⁷ The “Dymaxion” map (see Figure 2), designed in 1938, is an early example of one of his global thinking tools used in association with his “World Game” concept.¹⁸ The map creates one continuous path out of the Earth’s landmasses and cleverly folds into a 20-faced polyhedron.

According to Fuller:

Up to the Twentieth Century, reality was everything humans could touch, smell, see, and hear. Since the initial publication of the chart of the electromagnetic spectrum, humans have learned that what they can touch, smell, see, and hear is less than one-millionth of reality. Ninety-nine percent of all that is going to affect our tomorrows is being developed by humans using instruments and working in ranges of reality that are nonhumanly sensible.¹⁹

From the 1960s, Buckminster Fuller feverishly sought out media pulpits to augur imminent disaster as the price of ignoring humanity’s responsibility to care for the Earth. He reframed our planet as a fragile and unique life-affording bubble, making its way alone in an indifferent universe. Arts and media writers such as György Kepes, Jack Burnham and Marshall McLuhan were simultaneously disseminating concepts of ecological (and political) salvation through technological remediation. Responding to these ideas were interdisciplinary arts/technology collectives including Pulsa, USCO and the Ant Farm.²⁰

Unfortunately, Fuller’s inability to pause between thoughts seriously impeded his message from being disseminated to the masses during the early years of television. He gained little television airtime from the major networks. On the other hand, Marshall McLuhan received a good deal of network airtime in the 1960s and early 1970s. He understood more about the new medium of television than his academic colleagues or its practitioners. Although constantly pressed by interviewers, he diligently refused to make any value judgments whatsoever. More importantly, he stopped talking to accommodate commercial breaks. As he put it:

Our new electric technology that extends our senses and nerves in a global embrace has large implications for the future of language. Electric technology does not need words any more than the digital computer needs numbers. Electricity points the way to an extension of the process of consciousness itself, on a world scale, and without any verbalization whatsoever.²¹

The downfall of the sculpted object will represent one of many climactic symbols for our civilization— among them a realization that the old form-shaping approaches are no longer sufficient. By rendering the invisible visible through systems consciousness, we are beginning to accept responsibility for the well-being and continued existence of life upon the Earth.²²

In 1968, Pulsa made an untitled work “propositionising”²⁴ an autonomic system for balancing the health of a city (in this case Boston). They intended to do this by gathering data from its entropic boundaries and bringing them back to its more responsive centre. They would then display this data abstractly through patterns of light and sound in order to maintain a sort of urban “environmental homeostasis.” (This was the particular battle cry of artist György Kepes, former student of Moholy- Nagy and founder of the Center for Advanced Visual Studies at MIT, who vociferously advocated sustainability through technologically attuned artwork).

Pulsa’s design for accomplishing their aims for the city was part postmodern and part magical thinking. It involved miles of electrical wire joined to complex homemade circuitry firing a data-addled array of underwater strobe lights in conjunction with a matrix of land-based speakers. Although the urban planning problem they identified was real, judging from surviving documentation the installation was highly stimulating if impossible to read. Although their “system” concept offered a vatic view of the theoretical space unfolding before them, they were at least half a century ahead of a workable technology. The recurrent problem with systems art, particularly at the time, was that it often didn’t quite function as propositionised—often to the confusion and dismay of its patrons, promoters and the viewing public.

At the same time as the Art and Light Exhibition was being shown in Dunedin, London’s Tate Britain invited four visitors at a time to experience a fully embodied art installation entitled Tate Sensorium, “an immersive display featuring four paintings from the Tate collection. You can experience sounds, smells, tastes and physical forms inspired by the artworks, and record and review your physiological responses through sophisticated measurement devices.”²⁵

For this installation, Tom Pursey, Tim Partridge and Peter Law, co-founders of the creative agency Flying Object, chose four works by painters Francis Bacon, David Bomberg, John Latham and Richard Hamilton to explore sensually with viewers by designing a variety of embodied experiences, including the use of a new form of haptic technology that involve interacting with ultrasonic interference patterns in mid-air.²⁶

Environmental and ecoart practitioners are extending the discourses of anthropocentrism and the anthropocene through artworks, interventions and symposia. In 2013 the Haus der Kulturen der Welt, in cooperation with the Max Planck Gesellschaft, Deutsches Museum, the Rachel Carson Center for Environment and Society, and the Institute for Advanced Sustainability Studies sponsored two years of interdisciplinary “situations for engagement” called the Anthropocene Project,²⁷ which was based in Berlin. Ideas of “virtual” second nature and island analogue logistics emerged notably as themes.

Embodied Earth engages with a discourse closely related to other contemporary artworks and practices, such as the work of Queensland-based new media artist Keith Armstrong.²⁸ Armstrong is very attentive to embodiment issues, using haptic feedback and proprioception to devise interactive works that operate with similar environmental ends in mind, but using quite different means. He focuses on the remediation and repair of our lost social and environmental fabric through artworks and interventions that try to identify and reinvigorate former connections. I agree with Armstrong’s view that identifying and reinforcing relevant contemporary practices offers the best way to reinstate traditional modes of connectivity. However, I am also inclined to agree with McLuhan in the sense that our technologically enhanced, ever-extending nervous system promotes an ongoing generational reconfiguring of young neural networks through engagement with new media during normal periods of brain development (“firing and wiring”). This process is described by Mark BN Hansen as “human technogenesis.”²⁹ The dynamic plasticity of our brains moves us ever onward. New connective modalities are required. While I agree with Armstrong that reawakening our fully embodied faculties of engagement and connectedness is critical, I believe that the only direction for our subspecies is forward.

Looking back, it is clear that the glue of traditional tribal connectedness relies too heavily on a tight control of information, a certain maintenance of ignorance, and the culture of bullying that this dichotomy enables. Looking further back, our highly successful primeval environmental strategy of moving on to the next pristine territory—after having stripped the former one bare—in large part explains why we find ourselves in our current position. We now know far more about our Earthly predicament than ever before. Returning to Buckminster Fuller: the infinite expands elsewhere; we inhabit a small closed loop.

MIND TRAP

In Understanding Media, McLuhan describes how each new technologically mediated affordance acts as an extension of the human sensory apparatus—for example, radio reception extends the ear, while television reception extends both ear and eye. In so doing, he stressed the idea that the specific type of messaging each new medium enables dramatically alters our neural balance and loading—the shape of human perception. As a direct result, our experience of the world, as beings in the world, changes dramatically, particularly within the generation whose developing brains are “firing and wiring” to newly mediated data streams. However, as Buckminster Fuller reminds us, human beings have always experienced the world through a multiplicity of senses—not just through sight and sound.

As human beings, our evolutionary success is in large part due to the fact that we have such highly adaptive neural networks. But this can also be a vulnerability; a very simple idea can utterly transform us.

Over the last 350 years of European colonisation, the “liberated” world has been systematically influenced by what is now commonly regarded as the Cartesian fallacy of mind–body dualism. Residual evidence of this invasive concept remains embedded in the sphere of Western influence, despite centuries of robust philosophical criticism and decades of glaring empirical evidence to the contrary.

By broadly and loosely adopting this particular thinking tool we, as a subspecies, have disembodied our “selves” and have thereby undermined our own platform for living. Operating within its conventions, we obsessively accumulate mountains of material “wealth” through massive global interventions, all the while maintaining a magical belief in the mind’s transcendent non-materiality: a highly toxic combination. By detaching our “selves” from the rules that govern the material world, we lose the opportunity to make an integrated response to proximal and distal evidence of incremental ecological disaster. According to Antonio Damasio,“the mind exists in and for an integrated organism; our minds would not be the way they are if it were not for the interplay of body and brain during evolution, during individual development, and at the current moment. The mind had to be first about the body, or it could not have been.”³⁰

Neuroscience opposes the Cartesian mind–body division. As Damasio points out, our neural networks evolve in direct conjunction with more apparent biomechanical (and tool-enabling) developments, alongside, and in response to, other agents within the changing boundaries of specific biomes. Up until now our embodied neural networks, by necessity, have reaffirmed tight boundaries of self.

Our subspecies’ survival has also relied on the fact that we experience far more of the world than will ever meet our consciousness. This is expressed in unconscious (“black box”) processes that can directly determine how we respond to complex and dynamic situations. In this way we have managed to preserve our subspecies, both individually and as collaborative groups, on a day-to-day basis.

It is worth noting that in Plato’s allegory of the cave, people are chained to the floor, their experiencing bodies withheld from the world they witness only as a projection of flickering shadow and light. If we accept McLuhan’s caveats about the mediated extensions of man, we will understand that our

mediated experience of the world can also be distorted into a dangerous sort of tunnel vision. In 1968, Jack Burnham wrote, “Scientists and technicians are not converted into ‘artists,’ rather the artist becomes a symptom of the schism between art and technics. Progressively the need to make ultrasensitive judgements as to the uses of technology and scientific information becomes ‘art’ in the most literal sense.”³¹

Having irrevocably transformed the planet, it is probable that surviving our success will require that we dramatically retool our neural networks. Fifty years of further development in computer processing, sensory/haptic technologies and the promise of smart fabrics have certainly brought the systems concepts promoted by Fuller, McLuhan, Kepes and Burnham more within reach than they have ever been before.

Ongoing collaborative interdisciplinary projects by artists and scientists may help us devise the fully embodied, technologically mediated extensions necessary to cultivate and enable more environmentally relevant neural configurations in the next generation of data-readers and problem- solvers. We will require a new generation of enabled decision-makers in order to realise Kepes’ dream of environmental homeostasis, if such a thing is still possible.

David Green is an artist and filmmaker who lectures in Electronic Arts at The Dunedin School of Art.

____________________

Panel 1.

The Science behind the Signal: Professor Craig J. Rodger and Dr James B. Brundell

For a physicist, “light” is an electromagnetic wave and “colour” is an indication of the wavelength. Humans have evolved to be able to detect a small range of wavelengths which are close to the peak wavelength7 output of the Sun; we call this “visible” light.
But there is a vast range of other wavelengths which we cannot detect with our human senses. We need to build devices to help us. Over time, we have given some of these wavelengths special names such as, infra-red, microwave, radio wave, X-ray, and the like. The Otago Space Physics group makes use of extremely long wavelengths, in the order of tens to hundreds of kilometres, which are generally termed Very Low Frequency (or VLF). VLF electromagnetic waves can be produced by natural processes in Space and near the Earth’s surface, and are also exploited by some nations to allow transmission over huge distances to submerged submarines.
The bulk of the electromagnetic energy is trapped between the conducting Earth’s surface and the lower part of the ionosphere, which is the charged part of the atmosphere starting at ~65-70 km. VLF waves, radiated by a source, man-made or natural, can be detected thousands, or even tens of thousands of kilometres away. From the radio wave received one can learn more about the source and about the electrical properties of the route between the source and receiver. The Space Physics group is part of two international networks exploiting VLF waves. The first of these is the Antarctic–Arctic Radiation-belt (Dynamic) Deposition—VLF Atmospheric Research Konsortium (AARDDVARK). The joint NZ–UK AARDDVARK monitors powerful manmade communications transmitters and provides continuous long-range observations of the lower ionosphere. The Konsortia sensors detect changes in ionisation levels from ~30-85 km altitude, with the goal of increasing the understanding of energy coupling between the Earth’s atmosphere, Sun and Space. We use the upper atmosphere as a gigantic energetic particle detector to observe and understand changing energy flows. This scientific field impacts our knowledge of the polar climate, Space weather and navigation.
We are also active in the World Wide Lightning Location Network8 (WWLLN), which is pronounced “Woollen.” The technology behind WWLLN was developed in New Zealand, and the lead programmer is still based in Dunedin. WWLLN is a set of ~65 internet-linked VLF radio receivers. The receiver detects the strong VLF pulse from a lightning discharge, and sends the precise GPS-locked time of this event to the WWLLN central processing computers at the University of Otago and the University of Washington in the US. By combining multiple event times WWLLN can detect the location of the lightning.
The installation Embodied Earth relies on the live VLF radio feed collected by the WWLLN antenna on the roof of the Physics Department, University of Otago. While we have not evolved physical senses to detect VLF waves, they lie in the same wavelength range as audible sound. So by playing these waves directly into speakers we can appreciate the VLF wave activity. The result is a crackling, snapping, popping noise from radio waves launched by lightning discharges, probably located thousands or tens of thousands of kilometres away. The VLF radio signature from lightning is called a “sferic,” which comes from “atmospheric,” a contraction adopted in the earliest days of radio research.

Professor Craig J. Rodger and Dr James B. Brundell Department of Physics
University of Otago

____________________

Panel 2. The Science behind the Interface: Dr Steven Mills

In order to give an impression of embodiment, allowing the viewer to experience the electromagnetic forces that act in the atmosphere, we use a variety of sensory modes. The primary mode is visual, but this is reinforced with audio and haptic cues.
The visual experience is based on the use of Microsoft’s Kinect sensor to track the user’s body.¹⁰ This provides a video feed of the user, with information about where they are and how their limbs are positioned. We use this to animate virtual lighting strikes, which are triggered from a live feed of the VLF (Very Low Frequency) electromagnetic activity in the atmosphere. The VLF activity is in the same frequency as audible sound, but is an electromagnetic rather than mechanical wave. However, this means that it can be translated directly into sound, which gives a hissing, popping noise which has been compared to the sound of bacon frying. When the VLF “sound” is loudest, we generate a lighting strike—the louder the sound the bigger the strike. Since the VLF feed does not provide location information, these strikes are generated to strike the user at a random position along their arms and shoulders.

Figure 2. A visitor experiences the exhibit while Steven Mills explains the tracking system during a floor talk.

Each strike is animated as a bolt of lightning coming in from the edge of the projected image to the viewer’s body. This is accompanied by a brightening of the projected image to give the impression of a lightning flash. Since the lightning strike is transient, a lasting impression is provided by a series of sparks travelling along the viewer’s arms using a particle simulation.¹¹ Adding additional sensory cues can increase the viewer’s sense of presence or immersion.¹² We add two additional sensory elements—one audible and one haptic.

The audio cue is a deep bass sound delivered by a subwoofer. While the VLF signal has a direct audio representation, the individual events that trigger the lighting animation are hard to discern. An additional audio cue, synchronised to the animation, provides the viewer with a more direct experience, and using a subwoofer means that the sound can be felt as well as heard. This tactile experience is further enhanced when a haptic jacket is worn. This jacket contains small motors which vibrate in sequence with the locations of the simulated lightning strikes. Together, these features provide direct tactile sensations, reinforcing the feeling that the viewer is directly experiencing the phenomenon rather than merely observing it.

In the system presently in use, the haptic jacket and the Kinect-based software are not communicating directly. A Bluetooth communication system is being developed, but was not reliable enough for the initial exhibition. However, since the VLF feed does not provide location information, the positions of the strikes on the body are randomly placed. By pre-generating a sequence of random locations, the jacket and the Kinect-based software can generate visual and haptic effects which affect the same part of the body simultaneously.

Dr Steven Mills
Department of Computer Science
University of Otago

____________________

Panel 3. Haptic Jacket: Peter Brook

The platform used for the haptics was the ubiquitous Arduino Uno, which was inserted into the jacket behind the collar. Over 100 soldered or wire-wrapped connections and five metres of wiring went into the electronics, that were mounted on a textile strip and inserted into the lining of the jacket behind the neck and down both arms. A 7.3-volt battery sat in one of jacket pockets to power the Arduino and vibrators.
I experimented with a variety of vibrators and resonating containers. Mike Paulin supplied the vibrators that worked the best. When placed in a rigid container, these units produced an experience similar to a vibrating cellphone. They were then glued to a copper-etched strip board that contained the transistor-based circuit that drives the vibrator for 450 milliseconds when signaled to do so by the Arduino.
The jacket itself presented all the usual challenges of wearable computing. The main problem, apart from the Bluetooth connection, resulted from delicate electronics sitting inside a dynamic textile environment. During scheduled one-hour sessions over a number of days, the jacket was tested on 60 people of various shapes and sizes. The lining of the jacket was cut to admit nine boxes. Parts of the lining were re-sewn prior to testing, but apertures had to be maintained in order to service the circuits. People’s arms often found these apertures and their arms then had to be reinserted carefully so as not to disconnect wires.
A program was written into the Arduino that would read the lightning data via Bluetooth from a PC receiving a live VLF feed of real-time lightning strikes. This proved to be a problematic channel, and the problem was not solved before the opening of the exhibition. As a result, a secondary program that would play an historic sequence in coordination with the lightning displays was burned in for testing the prototype.

Peter Brook
Lecturer Embedded Systems
Otago Polytechnic

____________________

ENDNOTES

1. Authorial responsibility is as follows: David Green (main text), Craig Rodger and James Brundell (panel 1), Steven Mills (panel 2) and Peter Brook (panel 3). The first author would like to thank Elaine Reese, Bridie Lonie, Grant Gillett, Peter Stupples, Michele Beevors and Leoni Schmidt for their interest in this project, their ideas, conversation and valuable support.
2. Robert A Wilson and Lucia Foglia, “Embodied Cognition,” Stanford Encyclopedia of Philosophy, ed. Edward N Zalta, Fall 2011 ed., http://plato.stanford.edu/archives/ fall2011/entries/embodied-cognition (accessed 8 Sept 2015).
3. David Green, “Scientific Misprision,” Scope: ContemporaryResearch Topics, art & design 9 (November 2014), http://www.thescopes.org/home/art-and-design-9/ new-scopes-article-page-19 [accessed 15 Sept 2015].
4. H-DV Boehm and F Siegert, “Ecological Impact of the One Million Hectare Rice Project in Central Kalimantan, Indonesia, Using Remote Sensing and GIS,” paper presented at the 22nd Asian Conference on Remote Sensing, 5-9 November 2001, Singapore, http://www. crisp.nus.edu.sg/~acrs2001/pdf/126boehm.pdf. (accessed 15 September 2015)
5. Becky Oskin, “‘Whistling’ Volcanic Lightning Heard Halfway around the World,” LiveScience, 24 July 2014, http://www.livescience.com/46995-volcano-lighting- whistlers-earth-chorus.html (accessed 8 Sept 2015).
6. Phil Berardelli, “Did Volcanos Spark Life on Earth?” http:// news.sciencemag.org/earth/2008/10/did-volcanoes- spark-life-earth (accessed 21 Oct 2015).
7. A Nasa Earth Observatory feature article, “Incoming Sunlight,” describes the concept of peak radiation, http:// earthobservatory.nasa.gov/Features/EnergyBalance/ page2.php (accessed 11 Nov 2015).
8. For further information, see the World Wide Lightning Location Network website at wwlln.net (accessed 11 Nov 2015).
9. Bristlebots are inexpensively produced miniature robots designed to move via controlled bristle vibration. “Bristelbotics Ltd  aims to increase the number of schools offering robotics programmes while supporting and enhancing those that already do. We offer inexpensive robotics  options  and  use  open  source software, encouraging modification of our materials and programmes and sharing between students and schools,” http://bristlebotics.com/about-us/press  (accessed  11 Nov 2015).
10. Z Zhang,  “Microsoft Kinect  Sensor  and its  Effect,” MultiMedia, IEEE, 19:2 (2012), 4-10.
11. WT Reeves, “Particle Systems – A Technique for Modeling a Class of Fuzzy Objects,” ACM Transactions on Graphics, 2:2 (1983), 91-108.
12. Huong Q Dinh et al., “Evaluating the Importance of Multi- Sensory Input on Memory and the Sense of Presence in Virtual Environments,” Proceedings of IEEE Virtual Reality, 1999 (1999), 222-8.
13. The Fuller Projection Map (Dymaxion) was devised by R Buckminster Fuller as a thinking tool for use in solving global humanitarian and environmental crises. See https://bfi.org/about-fuller/big-ideas/dymaxion-world/ dymaxion-map (accessed 11 Nov 2015).
14. M McLuhan, Understanding Media: The Extensions of Man (New York: McGraw-Hill, 1966), 8.
15. Jack Burnham, Beyond Modern Sculpture: The Effects ofTechnology and Science on the Sculpture of this Century(New York: Braziller, 1968).
16. Rachel Carson (1907-64), a marine biologist turned environmentalist, responded courageously to first-hand evidence of the long-term and far-reaching destructive effects of the large-scale use of synthetic chemical pesticides in the United States following the Second World War. Her book Silent Spring led directly to a national ban on DDT and other pesticides, despite strenuous opposition from the chemical industry.
17. Richard Buckminster Fuller, Operating Manual forSpaceship Earth (Carbondale, ILL: Southern Illinois University Press, 1968).
18. Also referred to as the “great logistics game” or “world peace game,” Fuller proposed it as an inclusive and egalitarian platform for creating and exploring systems designed to resolve emergent world problems in a holistic way. 19.
19. R. Buckminster Fuller on Education, eds Robert Kahn and Peter Wagschal (Amherst: University of Massachusetts Press, 1979), 130.
20. Yates McKee, “The Public Sensoriums of Pulsa: Cybernetic Abstraction and the Biopolitics of Urban Survival,” Art Journal, 67:3 (Fall 2008) 46-67.
21. McLuhan: 1966, 80.
22. Burnham: 1968, 370.
23. “Sensorium: The totality of those parts of the brain that receive, process and interpret sensory stimuli. The sensorium is the supposed seat of sensation, the place to which impressions from the external world are conveyed and perceived. The sensorium also refers to the entire sensory apparatus of the body,” http://www.medicinenet. com/script/main/art.asp?articlekey=15732  (accessed 8 Sept 2015).
24. “[John Hughlings Jackson] argues that higher mental functions make use of ‘propositionising’ to articulate complex actions in the social and interpersonal sphere where such actions are attuned to a ‘third thing’ – the focus of shared attention – in triadic relations where communication occurs about what is being dealt with and how to respond.” G Gillett and E Franz, “Evolutionary Neurology, Responsive Equilibrium, and the Moral Brain,” Consciousness and Cognition, 22 October 2014, 1053-8100. (See also R Saxe, “Uniquely Human Social Cognition,” Current Opinion in Neurobiology, 16 (2006), 235-9.) Coined in the late nineteenth century, this term is useful in describing the intentionality behind contemporary art-making.
25. IK Prize 2015: Tate Sensorium, http://www.tate.org. uk/whats-on/tate-britain/display/ik-prize-2015-tate- sensorium (accessed 15 Sept 2015).
26. 26.  Ibid.

27. See http://www.hkw.de/media/en/texte/pdf/2013_2/ programm_6/anthropozaen/booklet_anthropozaen_ eine_eroeffnung.pdf (accessed 8 Sept 2015).
28. Keith M Armstrong, “‘Grounded Media:’ Expanding the Scope of Ecological Art Practices within New Media Arts Culture,” in Proceedings PerthDAC 2007: The 7thInternational Digital Arts and Culture Conference: ‘TheFuture of Digital Media Culture,’ ed. Andrew Hutchinson (Perth: Curtin University of Technology, 2007) 21-31, http://www.academia.edu/8914451/Grounded_ Media_Expanding_the_scope_of_ecological_art_ practices_within_new_media_arts_culture. (accessed 15 September 2015).
29. Mark BN Hansen, “Media Theory,” Theory, Culture &Society, 23:2-3 (2006), 297-306.
30. Antonio Damasio, Descartes’ Error: Emotion, Reason,and the Human Brain (New York: Avon Books, 1994), xvi.
31. Jack Burnham, “Systems Esthetics,” Artforum, 7:1 (September 1968), 30-35.