Original Research Article
1 Laboratory for Neuroengineering, School of Mechanical Engineering,, Georgia Institute of Technology, USA
2 SymbioticA, School of Anatomy and Human Biology, University of Western Australia, Australia
2 SymbioticA, School of Anatomy and Human Biology, University of Western Australia, Australia
Here, we and others describe an unusual neurorobotic project, a merging of art and science called MEART, the semi-living artist.We built a pneumatically actuated robotic arm to create drawings, as controlled by a living network of neurons from rat cortex grown on a multielectrode array (MEA). Such embodied cultured networks formed a real-time closed-loop system which could now behave and receive electrical stimulation as feedback on its behavior.We used MEART and simulated embodiments, or animats, to study the network mechanisms that produce adaptive, goal-directed behavior. This approach to neural interfacing will help instruct the design of other hybrid neuralrobotic systems we call hybrots. The interfacing technologies and algorithms developed have potential applications in responsive deep brain stimulation systems and for motor prosthetics using sensory components. In a broader context, MEART educates the public about neuroscience, neural interfaces, and robotics. It has paved the way for critical discussions on the future of bio-art and of biotechnology.
Keywords: learning, embodiment, multi-electrode array, neural network, rat, art
Copyright: © 2007 Bakkum, Gamblen, Ben-Ary, Chao and Potter. This is an open-access article subject to an exclusive license agreement between the authors and the Frontiers Research Foundation, which permits unrestricted use, distribution, and reproduction in any medium, provided the original authors and source are credited.
*Correspondence: S. M. Potter, Laboratory for Neuroengineering, Coulter Department of Biomedical Engineering, Georgia Institute of Technology, Atlanta, Georgia 30332-0535, USA. e-mail: steve.potter@bme.gatech.edu
Citation: Bakkum DJ, Gamblen PM, Ben-Ary G, Chao ZC and Potter SM (2007) MEART: The semi-living artist. Front. Neurorobot. (2007) 1:5. doi:10.3389/neuro.12.005.2007
Received: 12 September 2007; paper pending published: 09 October 2007; accepted: 15 October 2007; published online: 02 November 2007.
Edited by:
Frederic Kaplan, Ecole Polytechnique Federale De Lausanne, Switzerland
Reviewed by:
Suguru N. Kudoh, National Institute of Advanced Industrial Science and Technology, Japan
Sergio Martinoia, University of Genova, Italy
Frederic Kaplan, Ecole Polytechnique Federale De Lausanne, Switzerland
Sergio Martinoia, University of Genova, Italy
Frederic Kaplan, Ecole Polytechnique Federale De Lausanne, Switzerland
*Correspondence: S. M. Potter, Laboratory for Neuroengineering, Coulter Department of Biomedical Engineering, Georgia Institute of Technology, Atlanta, Georgia 30332-0535, USA. e-mail: steve.potter@bme.gatech.edu
| Introduction | |||||||||||||||
“The most beautiful thing we can experience is the mysterious. It is the source of all true art and all science.”—Albert Einstein, 1931 The emergence of the mind from its biological substrate is one of the greatest and most complex mysteries. We study the brain using a synthetic approach, building from scratch a simple artificial animal, a new type of model for studying the brain. To be useful and easy to control and study, a model necessarily is a simpler version of what it models. Although our approach is fairly reductionistic, we assume that the complexity found in living brain cells is crucial to their function, including their network dynamics. Thus, our synthetic model system incorporates living neuronal networks, and is therefore a cybernetic organism, or cyborg. To distance this approach the culturally loaded conception of a cyborg, we prefer to call simple hybrid neural-robotic systems used for neurobiology research “hybrots” (Potter, 2002). We built a robotic drawing machine with two pneumatically actuated arms that move in concert to draw with ink markers on large sheets of paper (Figure 1) and designed software and hardware for it to converse with a network of rat cortical neurons grown in culture over a multi-electrode array (MEA, Figure 2) (Potter et al., 2006). The model system consisted of living neurons, growing in the laboratory for Neuroengineering at Georgia Tech, and connected by internet to the pen-wielding metal and plastic pair of arms behaving in gallery exhibitions around the world over the past 5 years. The whole system was named MEART, an acronym derived from Multi-Electrode Array aRT. This geographically distributed, “semi-living artist” was one of the first closed-loop neurally controlled animats with a robotic body (Manson, 2004; Potter et al., 1997; Reger et al., 2000). Neuronal action potentials recorded by an MEA in Atlanta were processed in real-time and used to command movement at different exhibitions in Perth, Melbourne, Bilbao, New York, Moscow, Atlanta, and Shanghai. (http://www.fishandchips.uwa.edu.au/exhibitions.html). Video images of the drawings in progress determined the subsequent feedback of electrical stimuli delivered to the neurons. |
|||||||||||||||
|
|||||||||||||||
Artists in Perth and scientists in Atlanta collaborated to construct MEART, a concept originating from scientific inquiries into hybrid bio-robotic technology (DeMarse et al., 2001), and artistic expressions by SymbioticA, an art–science collaboratory in the School of Anatomy and Human Biology at the University of Western Australia. Our common interest was to explore the essence, or primordial substrates, of creativity and intelligence. Because MEAs are so much more accessible than brains in animals, they allow researchers to manipulate and quantify underlying neural mechanisms of small (a few thousand neurons) networks, including the physical manifestations of learning and memory (Jimbo et al., 1999; Potter et al., 2001). |
|||||||||||||||
|
|||||||||||||||
The idea of controlling robots with brain cells removed from the body and interfaced to electronics went from the realm of science fiction to that of science when Mussa-Ivaldi and co-workers at Northwestern University interfaced the small, wheeled Khepera robot (K-Team) to a lamprey brainstem maintained Ex vivo (Reger et al., 2000), taking advantage of the existing vestibular circuitry in that part of the brain to study adaptation mechanisms. They re-mapped the lamprey brain's circuitry to take input from the robot's photosensors, and to control the motors with its vestibular response to this artificial input. “The semantics of the stimulus (gravity vs. light) is not likely to play any substantial role here,” they asserted (Karniel et al., 2005). This hybrot demonstrated phototaxis, and rudimentary learning, by changing its responses to light. When cultured networks serve as the brain of a hybrot, any intrinsic brain circuitry from the donor was lost during dissociation of the brain tissue during preparation of the cultures. A cortical culture lacks the 3D structure present in the brain and so lacks any computational advantages that this may have afforded. However, basic self-organizing principles and plasticity mechanisms such as spike timing-dependent plasticity (Bi and Poo, 1998) and homeostasis (Turrigiano and Nelson, 2000) persist and were the objects of our study. To what extent an organized network re-forms in vitro is still up for debate. However, we and others have shown that even dissociated networks of neurons have the ability to produce complex, repeating patterns of activity (Rolston et al., 2007; Wagenaar et al., 2006a, 2006b). In 2002, we presented a poster describing a simple approach-avoidance task executed by a Khepera interfaced to a cultured cortical network (DeMarse et al., 2002). Others using hybrots with cultured neurons as their brain include Kudoh and co-workers at the National Institute of Advanced Industrial Science and Technology in Japan (Kudoh and Taguchi, 2006) and Martinoia and co-workers at the University of Genoa in Italy (Martinoia et al., 2004, Novellino et al., 2007). Both of these groups also used the Khepera as the embodiment, in an obstacle-avoidance paradigm that included tetanic electrical stimulation to induce learning “Certain types of feedback stimulation caused suppression of spontaneous network electrical activities and drastic re-organization of functional connections between neurons, when these activities are initially almost synchronized. The result suggests that neurons in dissociated culture autonomously re-organized their functional neuronal networks [by interacting] with their environment. The spatio-temporal pattern of activity in the networks maybe a reflection of their external environment.”(Kudoh and Taguchi, 2006)This embodied cultured networks approach is intended to bridge a large gap that exists between in vivo behavioral studies of learning and memory, and in vitro studies of cellular plasticity. With a hybrot whose living brain can be easily probed and observed, behavior and learning can be observed in concert with the detailed and long-term multi-neuron electrophysiology available in vitro (Potter and DeMarse, 2001). We sought to find out whether MEART could learn something about the environment given to it, and whether a creative act could emerge from its interactions with this environment. We define learning in this context as a lasting change in behavior that results from experience. Here we present, along with artistic, philosophical, and scientific commentary, progress on engineering MEART's hardware, software, wetware, environment, and aesthetics. In experiments directed at making MEART learn, we applied patterned training stimuli (PTS) contingent on behavioral performance in order to achieve the goal-directed behavior of drawing geometrical shapes. Neural plasticity occurred, but successful learning did not. However, we modified the training algorithm using a living network connected to a simulated robot [an animat (Meyer and Guillot, 1994)]. Instead of a fixed transformation from sense data to stimuli, behavioral performance was used to continuously discover and refine effective sequences of PTSs, and in a preliminary experiment described below, an animat repeatedly learned to draw in different desired directions. By using more detailed sensation and motor output, we expect hybrots to demonstrate increasingly complex and interesting behaviors. What questions would be posed if MEART was eventually deemed to show intelligent or creative behavior? What would be the implications for biotechnology if its drawings were considered aesthetically beautiful? The unique nature of this art-science exploration in neurorobotics has stimulated wide-ranging discussion, about life, art, learning, embodiment, and other things, some of which is excerpted here. We hope that this discussion continues online via the Frontiers in NeuroRobotics web site. Methods: Making the Semi-Living Artist |
MEART was comprised of living neurons, recording and stimulating electronics, robotic drawing arms, electronic control circuits for a pneumatic actuation system, a CCD camera to feedback images of drawings, and software communicating between the neurons and robot over the internet (Figure 3). The simulated animat was made of living neurons, recording and stimulating hardware, and a simple virtual embodiment on a computer. It was used to develop protocols in the intervals between MEART exhibitions. Three major topics needed to be addressed to embody the cultured networks are as follows:
|
A Preparing and caring for MEART's brain We have developed techniques to maintain neuronal cultures and conduct experiments for many months using MEAs (Potter and DeMarse, 2001). We describe these briefly, and refer the enthusiast to that paper for more details. Cells were obtained from embryonic-day-18 rat cortex according to protocols approved by the NIH and the Georgia Institute of Technology animal care and use committee. Brain tissue was dissociated with enzymes and mechanical trituration, to prepare a dense suspension of neurons and glia. A droplet of this suspension containing about 50 000 cells was pipetted into MEAs coated with polyethylene imine and laminin, and cultured at high density (~3000 cells∕mm2) in serum-containing Dulbecco's Modified Eagle's Medium. The MEAs used were glass with silicon nitride insulation and 60 titanium nitride electrodes (multichannel systems). Neural activity was recorded using the MEA60 preamplifier and MCCard analog-to-digital converter (multichannel systems) with each of 60 channels being digitized at 25 kHz. All cultures were allowed to grow 3 weeks prior to experimentation, with weekly medium replacement. Neurons spontaneously began communicating electrically and chemically within a week, demonstrating an inherent goal to form a functional network (Van Pelt et al., 2004; Wagenaar et al., 2006a) and distinct repeating patterns of activity (Rolston et al., 2007; Wagenaar et al., 2006b). Sensory input to the networks was delivered via the substrate electrodes as voltage-controlled pulses. These were biphasic pulses of 400 μs duration and 500 mV magnitude per phase (Wagenaar et al., 2004) using a custom built all-electrode stimulator (Wagenaar and Potter, 2004). Data acquisition, visualization, artifact suppression, and spike detection were controlled using Meabench (Wagenaar et al., 2005a). Experiments were conducted using sealed-lid MEAs (Potter and DeMarse, 2001) inside an environmentally controlled incubator built around an optical microscope (Figure 4), allowing us to monitor and stimulate the networks continuously for many days.
|
B MEART's body Artistic design. The MEART data presented here were collected during the First Moscow Biennale of Contemporary Art at an exhibition entitled “art_digital_2004: I Click, Therefore I Am”, where MEART's goal of filling a square at the center of the drawing workspace was inspired by the Russian artist Kazimir Malevich's “Black Square” painting. From the art_digital_2004 program, “The action of MEART observing and drawing the Black Square explores the fundamentals of visual creativity and the way we communicate with the world through images, symbols, and their underlying meanings.” This goal behavior was a simplification of the mappings used during our previous exhibitions, to improve experimental controllability. In previous MEART exhibits, we added an element of interactivity by having MEART draw photographed faces of gallery attendees, entitled the “Portrait Series”. As with images from the drawing, the faces were pixilated and MEART's goal was to shade the drawing to match the grayscale pixel values. To give viewers a better understanding of MEART's brain, and the laboratory in which it was studied, live images from the laboratory, a close-up of the MEA, and a data display of neural activity were projected onto the exhibit walls. This, along with computers displaying the movement and feedback data streams, made the distributed nature of MEART more apparent. MEART's body was designed to closely resemble organic forms in function and aesthetics. Shapes were based on bones [influenced by the photographer Andreas Feininger (Feininger and Schlatter, 2003)], and sanded Perspex offered an elegant look that referred to a skeletal structure. Similarly, the pneumatic muscles paralleled biological muscles. The design had no covering, never attempting to hide or deny the underlying technology. Analogously, the complex biology of the rat was reduced to a few thousand neurons and glia, grown in vitro. MEART was thus a symbol of the reductionist nature of science and of the stripping down to expose the physical substrates of the creative process. Below, Emma McRae (2004) paints a verbal picture of MEART “1. Introduction to a Cybernetic Entity The soft popping sounds of air releasing, of the breaths taken between movements as the muscles contract and release on the mechanical structures at work on the table in the centre of the room, reach me first as I walk down the dark corridor in the Australian Centre for the Moving Image. I can see the plastic and metal arms and the tubes connected to two rows of valves—regular black garden hose valves—highlighted by a spotlight, that seem to create the movement of the arms. These arms (the creators call these structures arms, presumably because they hold pens and draw as human arms involved in drawing do) are busy drawing lines in apparently random directions with three different coloured pens on a large sheet of paper on the table. Behind the arms is a computer screen showing a photo of a man's face, a pixellated black and white image, a scrolling text box, and some graphs. The only other thing on the table is a camera which looks down over the arms at the picture they're drawing. A large screen on the wall behind the table shows a graph, a representation that looks like a glacial landscape and is constantly changing form, its peaks and troughs rising and falling in random motion, depicting varied intensities coloured in blue, yellow, white, and red. There are two smaller screens in the opposite corner of the room that intermittently display an image of a science laboratory, a close up of a petri-dish, a screen of 64 ECG-like blue tracking graphs, and a microscope view of cells”.Movement. The drawing machine consisted of two perpendicular, rigid, jointed arms (aluminum and acrylic Perspex) fixed by hinges at their ends to a 3 m by 3 m table actuating the X and Y positions of a group of pens over a sheet of paper (Figures 1 and 5). Similar to biceps and triceps, McKibben braided pneumatic artificial muscles could contract individually, allowing independent flexion or extension of each arm within approximately a 30 cm by 30 cm workspace. Similarly, activation of smaller muscles pressed pens to the paper; a dark pen marked target locations, while an optional lighter colored pen traced the movement trajectories. The supply line from an air compressor was split between three pressure regulators, one for each arm and one for the pens, to isolate pressure fluctuations. Air pressure and thus arm and pen movement was controlled by opening and closing 24 V AC pneumatic valves. Pneumatic muscles, while offering a high power to weight ratio, produce nonlinear motion difficult to predict. Therefore, arm location was tracked using joint encoders (10 k potentiometers), and a BASIC Stamp microcontroller (BS2SX-IC) modulated valve opening to increase movement accuracy as commanded by the living network (Figure 6).
|
|
|||||||||||
|
|||||||||||||||
|
Sensory feedback. A digital camera located above the movement workspace captured images of the drawing in progress. Fluctuations in light from shadows and clouds could strongly influence the image quality. Therefore, ambient and natural light sources were reduced or eliminated except for bright spotlights on the drawing itself. Image inhomogeneity due to imperfect lighting was corrected by subtracting from the captured images an image of the sheet of paper when blank, prior to a drawing. The accumulation of markings was recorded every 5 minutes by retracting the arms out of view and capturing an image, analogous to a painter stepping back from the canvas to check the work in progress. Internet communication. TCP∕IP sockets were used to send motor commands to the drawing machine and to return images of the progression of a drawing for feedback. To reduce internet bandwidth, 8 bit grayscale values of an 8 × 8 grid of pixels (isomorphic to the electrodes on the MEA) were transmitted over the internet and transformed into electrical stimulation feedback delivered to the neuronal network. | |||||||||||||||
| C Software development and experimental design
Motor transformation. For an animat to behave, sequences of neuronal action potentials need to be transformed into body movements, but understanding how such sequences might encode information is a subject of much scientific inquiry. Population vector coding is a candidate motor mapping found to occur in the motor cortex (Georgopoulos, 1994), premotor cortex (Caminiti et al., 1990), hippocampus (Wilson and McNaughton, 1993), and other cortical areas: the vector sum of firing rates of a group of broadly tuned neurons taken together provide a precisely tuned representation (e.g., to a preferred direction of arm movement). We have used a new statistic, the center of neural activity (CA, analogous to the center of mass) to reliably quantify neuronal network plasticity on an MEA by including spatial information (Chao et al., 2007). Movement of MEART or a simulated animat was calculated from the CA of 100 ms of responses after each probe stimulus:   The CA is the vector summation of action potentials at each electrode e (Ne) weighted by the spatial location of the electrode,  . The transformation, T, is a normalization matrix found prior to the closed-loop experiment to offset and scale the CAs (in electrode space) such that animat movement could produce a uniform distribution and the ability to place pen marks throughout the workspace (MEART) or move in any direction (simulated animat). Achieving a goal for either MEART or the animat required shifting the distribution of normalized CAs. Therefore, plasticity results were comparable. The responses to 1 Hz stimulation on a probe electrode were averaged between consecutive movements (every 4 second or 1/4 Hz) and used to command MEART pen location, while the responses to 1/4 Hz stimulation on a probe electrode were used to command the simulated animat. A single repeating probe electrode was used throughout an experiment.Movement could be commanded by absolute location (MEART) or in relative increments (simulated animat). For each case, the activity was normalized to equally distribute the distribution of CAs prior to experiments. For absolute location, this set the possible pen locations to be distributed throughout the whole workspace. For incremental movement, this set the possible movement directions to be distributed throughout 360 degree. Absolute pen location was used with MEART to avoid movement exceeding the workspace, which would introduce discontinuities in behavior. Incremental movement (Equation 2) was later used for the simulated animat as workspace size was not physically limited, and we were more interested in direction of movement than position. Training and sensory feedback. Previous MEART exhibits used a sensory mapping in which a camera's image, after reducing to 8 × 8 pixels, was directly mapped onto stimuli of the 8 × 8 grid of electrodes under the neuronal network. For the Moscow exhibit, the sensory system was simplified into a signal that merely indicated whether drawings were within a pre-defined square. Successful behavior was determined from comparisons between consecutive feedback images. If a larger proportion of markings occurred inside the target geometrical area than outside, behavior was considered successful. Otherwise, a change in the probe response was desired. For training, plasticity was induced by repetitive stimulation of paired electrodes, termed patterned training stimulation (PTS). A PTS was constructed by pairing the probe electrode with another active electrode (one that evokes network responses) 20 ms later, repetitively stimulated for 3 second with an inter-pair interval of 100 ms. For the simulated animat (Bakkum et al., 2007), the training algorithm was modified in two ways. A pool of candidate PTSs was formed by pairing the probe electrode with other electrodes (NE = 58) and inter-pulse intervals {−80, −40, −10, 10, 40, 80 ms} (NPTS = 58 × 6). The probabilities of choosing a given PTS were initially uniform and increased or decreased based on whether subsequent animat performance was successful or not. This allowed an iterative search for an appropriate training “solution” to direct neuronal plasticity. Second, plasticity can arise from both the PTS stimuli and ongoing spontaneous activity occurring between probes. In a model network, a random stimulation stabilized neural synaptic weights (Chao et al., 2005). Therefore, when animat behavior was successful (no PTS application), a random background stimulation was used between probes such that the plasticity accumulated from a series of PTSs was maintained. The goal of the simulated animat was now to learn to move within ±30 degree of a goal angle. Results |
MEART was first exibited in August 2002 at the Biennale of Electronic Arts Perth (BEAP). However, the precursor to MEART, Fish & Chips, was shown in 2001 at Ars Electronica in Austria. For this ground-breaking bio-art exibit, SymbioticA Research Group created MEART's drawing arm and used it as the embodiment of a semi-living artist. This was called Fish & Chips because an acute goldfish brain slice was maintained and electrically interfaced on a silicon chip, and used as the controlling “brain” of the arm. From the collaboration between SymbioticA in Perth and the Potter laboratory in Atlanta, MEART was born: the first robot controlled by a network of neurons in a culture dish, with a two-way interface via a MEA. To the existing drawing arm, we added a sensory system, where images from a CCD camera were translated into electrical stimuli for the cultured network. It was also the first neurally controlled robot whose brain lay far away from its body, with the internet in some ways serving as a very long nerve connecting brain to body. It was the first physical embodiment for a cultured network that remained continuously connected for extended periods of several days, creating numerous drawings during exhibitions. Early exhibitions were devoted to debugging the communication software and robot mechanics (Figure 5), and the most recent exhibitions allowed experimentation. We noticed early that continuous sensory input over the course of days tended to reduce the number of spontaneously occurring network-wide bursts. This led to a hypothesis that other types of bursting, such as epileptic seizures, might be treated by continuous multi-electrode stimulation. We quantified the short-term “quieting” effects of distributed multi-site stimulation on cortical cultures (Wagenaar et al., 2005b), and we are now pursuing the longer-term, or homeostatic effects of continuous stimulation that comes as a consequence of embodiment. For the data presented here, MEART's behavioral goal was to draw a solid 12 cm × 12 cm square within the center of its 30 cm × 30 cm workspace. The simulated animat was used to test training algorithms between MEART exhibitions in order to improve behavioral performance. The simulated animat's behavioral goal was to incrementally move within ±30 degree of a desired angle (note that this differed from MEART's goal behavior of producing pen markings, commanded by absolute location). For both MEART and the simulated animat, the relationship between changes in behavior and the decision whether or not to apply feedback training stimulation were identical, and thus results about plasticity and learning were comparable. Electrical stimulation can be an artificial inducer of neuronal plasticity, changing a network's input-output function. Bi and Poo found that for mono-synaptically connected cultured neurons firing within a few tens of milliseconds of each other, directional spike timing-dependent synaptic plasticity occurred (Bi and Poo, 1998). Repetitive stimulation of pairs of electrodes in a PTS could, therefore, cause plasticity in shared pathways of neural activation. For Meart, the transformation from visual sensation into the delivery of a PTS was fixed. For example, if previous movements occurred below the target area, the probe was paired with a predetermined electrode at the top of the MEA. Fetz and co-workers (Jackson et al., 2006) provided evidence in vivo of not only the induction of pathway plastic but of directional pathway plasticity: they repetitively stimulated a neuron in the primate motor cortex 5 milliseconds after the occurrence of an action potential on a different poly-synaptically connected neuron using a chronically implanted neural interface. After halting the stimulation, subsequent activity of the recorded neuron caused an increase in the firing rates in the vicinity of the stimulated neuron. In this manner, we hypothesized the PTS would lead to potentiation of the probe response in the vicinity of the second paired electrode. In other words, a directional plasticity could arise during application of PTS, potentiating the pathway from the neurons evoked near the probe electrode to the neurons later evoked at the second paired electrode. This would modify subsequent CAs and population vectors in response to probe stimuli such that arm movements would approach the target area. While successful behavior did not occur (Figure 7), neural plasticity did (Figures 7 and 8), suggesting training stimuli had the potential to modify behavior. Normalized plasticity was defined as the difference in distribution of movement-controlling output (the CAs) in a given 10-minute period (CAPost) to those of the first 10 minute (CAPre) as:  where  is a mean of CA vectors. A value of 1 indicates no change.
|
We concluded that since neurons at different electrodes are connected through multiple intermediate neurons and pathways, the effect of a given PTS cannot be predicted. By using feedback of behavioral performance to select and refine effective sequences of PTSs, instead of using MEART's fixed PTSs, the simulated animat could now achieve its goal-directed behavior (Figure 7). To demonstrate that the successful behavior was a consequence of the biological changes in the neural network and not an artifact of the algorithms, the desired movement angle was switched between three angles every 2 hours. Even though movement was commanded by absolute location for MEART and incremental movement for the animat, training was intended to produce the same effect on neural activity: shift the distribution of CAs (and in turn movement angles) toward a desired goal direction. The adaptive training algorithm allowed a search for “solutions” to achieve goal-directed behavior (Figures 7 and 8). Some PTSs may give desired neuronal plasticity while others may give the opposite or none. Furthermore, a neural network is continuously plastic, and the same PTS may have different effects at different times. The training algorithm commanded the application of a sequence of PTSs to produce the appropriate neural plasticity for successful adaptation. The learning curve in Figure 9 shows the percentage of successful movements in time; progressively fewer PTSs were needed to maintain the desired behavior, suggesting that the animat was learning the appropriate behavior.
|
|
||||||||||||
|
|||||||||||||||
Discussion |
|||||||||||||||
“To view Meart is to witness a collage of contradictions. It offers us the actual biological substance of the thinking brain yet out of its biological context and system of developmental ordering. What is visible to us as Meart in the space of public display is a visualization of and∕or window into ongoing experiments occurring thousands of miles away in a laboratory. The outcomes are neither pre-defined, nor are their meanings fully understood. Indeed, any of the aforementioned skeptical questions place us as viewers firmly in the midst of vigorous scientific debates—a fact underscored by the “real-time” nature of the Meart performance. Like a work of science fiction, Meart stimulates broad inquiry into our own lived contexts. However, unlike sci-fi, it is not simply a representational text, but also an operational one. It cannot be dismissed as a mere illustrative flight of fancy, but must be interrogated as a concrete example. Meart is an ‘operational fiction’—a cyborg of representation and reality, art and science, and of course flesh and transistor.” —Paul Vanouse, Excerpt from the Strange Attractors exhibition catalog, Zendai gallery, Shanghai, 2006.Gallery visitors were first captivated by the aesthetics of the kinetic sculpture. MEART's organic movement and the “breathing” sound of the pneumatic relay valves intermittently popping and hissing, not quite structured and not quite random, gave an intriguing sense of calm, maybe similar to watching trees sway in a gentle breeze. This hinted at the presence of an underlying natural process. A subtle curiosity to figure out what was happening turned into apprehension of the uniqueness of this semi-living artist, and then intense questions about the nature of the mind, the body, life, and about the artistic and scientific messages.
|
) of the CA into a population vector. The [X,Y] movement command was sent over the internet (yellow arrows) to the robotic arms every 4 seconds. Movement: The robotic drawing machine consisted of two perpendicular arms actuated by braided pneumatic artificial muscles, allowing independent retraction (R) or extension (E) of the left (EL∕RL) and right (ER∕RR) arms within approximately a 30 cm by 30 cm workspace. Similarly, smaller muscles pressed the pens to the paper when at the target location (T), or optionally to trace movement trajectories (M). The supply line from an air compressor was split between three pressure regulators (green circles, one for each arm and one for the pens). 24 V AC pneumatic valves (light blue rectangles) controlled muscle air pressure. Joint encoders (purple arrows; 10 k potentiometers) tracked arm location, and a BASIC Stamp microcontroller (BS2SX-IC) modulated the relay valves to provide accurate movement as commanded by the neurons' activity. Sensory feedback: A CCD camera located above the workspace captured an image of accumulating markings every 5 minutes. The images were pixelated into 8 bit grayscale values (isomorphic to the electrodes on the MEA) and sent back over the internet to command feedback stimulation of the neurons. Training: Animat behavior was compared to the goal behavior to control training stimulation. Feedback stimuli could change neuronal activity, in turn varying subsequent animat movement and sensory feedback, thus forming a closed-loop system. TCP∕IP sockets were used to communicate between the drawing robot and the neuronal network, which were often located on separate continents.
|
|||||||||||||||||||
|
|||||||||||||||||||
