XRDS: Thank you, Sougwen, for joining us. Before we get started, to people who are not familiar with you, how would you describe your work? Who or what are the key influences, and what core values and messages do you hope to convey?

Sougwen Chung: I'm an artist and researcher exploring sensory mixes of the future as it shapes what becomes of the human hand, the written word, and our collective rituals. In the past decade, I've been working with and developing collaborative AI systems, bio-sensors, and virtual reality environments to investigate the relational potential of emerging technology. I implicate my development as a creative and technical practitioner within the process, driving new curiosities and research directions.

I've come to understand that this passion for creating a hybrid philosophy of art/science/engineering/design has been influenced by my background as a Chinese-Canadian second-generation immigrant, and as the child of a computer programmer and a musician.

I believe where AI ends and We begin is one of the central questions of our time, one that requires both artistic- and research-based modes of inquiry. It's been fascinating to see how practices within the humanities like art, performance, and meditation can deepen our understanding of cybernetics and cognitive science, and by doing so can help shape the technologies that shape us.

XRDS: How did you become interested in human-machine collaboration? Could you give a little bit of background on how you started as an artist and researcher?

SC: When I began this direction of my work in 2014, the term "collaboration" in human-machine interaction wasn't in the Zeitgeist. When I decided to take my background in HCI to describe interactions as collaborations, it was a bit of a watershed moment as it was a term I felt hinted at the recursive feedback loops I was building and exploring through "Drawing Operations."^a When I was at the MIT Media Lab, I began tinkering with the design and construction of human-machine configurations as collaborations that could implicate both human and machine output as visual artifacts for aesthetic and investigative research.

XRDS: How do you view your machines in relation to you? Do you have similar/different roles in the creative process? And has your view and the roles changed or evolved over time?

SC: Obviously in the beginning, the engineering of disparate parts of the machine took precedence: microcontrollers, servo motors, and the code that defines the software that shape the construction of the tool. I view the machine systems as an evolving multi-generational configuration of collaborators that drive my understanding of the technologies of the day that simultaneously stimulate my understanding of a human process or ritual, which can result in a performance narrative—a type of embodied and computationally-mediated symbiosis.

My conception of machines has evolved from tool to collaborator, to push toward a passion for systems stewardship and human/machine/environmental couplings and symbiotic interrelations.

XRDS: Let's dive a little deeper into the computational aspect of your work. Could you give an overview of your workflow (e.g., from capturing your gestures to training the model)?

SC: My workflow is pretty porous in general. I try to leave as much space for possibility, experimentation, and adaptation in the work. Unfolding organically is an important part of the process. In the "D.O.U.G." project, each generation revolves around a theme with a corresponding dataset that drives the interaction model. In "Generation 1," the data was real-time camera input of my drawing, and I used simple color tracking to emulate a mimicry interaction. In "Generation 2," the Sisyphean task of digitizing, pruning, and classifying two decades of assorted drawings on paper and canvas garnered deep insights into the intensive human labor that goes into the compilation of a usable data-set. When my artistic style was digitized as positional mark-making and time-based positional sequences, the recurrent neural network (RNN) model in turn outputted the algorithmic bias of my own dataset as a kind of visual style. The digitized drawings as discrete files became the gestures for training the model, in which I had to adjust the amount of complexity of the drawings based on the limitations of the data-set, algorithmic interpretation, and the mechanical unit and the "feeling" of the robotic, embodied response during the performance. In each generation—whether we're working with data garnered from public cameras as in "Generation 3" or my own drawing data in "Generation 2"—the positions of urban movement and drawn lines transmute melodically, to take a synesthetic approach, building up layers on the canvas as marks, strokes, and stains.

XRDS: What were some surprising or favorite moments, and interesting constraints or challenges when working with the machines?

SC: In each robotic generation, there are always technical and conceptual challenges. Sometimes they're material—an excess of electrical current in one of the robotic joints resulting in a server error. Sometimes they are experiential—how to convey the sense of an expanse, extended place in the work? How to affect the audience's experience of time through performance, in the exhibited piece?

I'm always captivated by the beauty of the shared fallibility within human-machine systems, which as a performer, is something witnessed by the audience but also experienced by myself in the moment. It's a paradigm shift for how we are meant to think of machines, as computational systems to produce control. As the work continues to implicate the environment further, the metaphors by which we look at human and machine configurations expand.

XRDS: I love that you said, "Collaboration is an exercise in behavioral empathy."^b As the machine mimics and memorizes your movement, and draws in its own way, it feels like the empathy you have towards the machine is also in a sense towards yourself. You've also started to draw with your own EEG data. Has this process of looking inward (i.e., a reflection of your movement and internal state) made you learn something new about yourself or shaped your practice?

SC: In the work, the collaborations have become an enforcing mechanism for a type of practice, ritual, or experience, I'm trying to learn through and about. From drawing to meditation—the state during which I record my brainwaves—the feedback mechanism paints when the alpha state of my bio-feedback is higher, where elevations are indicative of a state of flow and a deeper state of meditation.

In all these experiments, it's about learning to draw or meditate or be different, but not in a way that is reliant on the technologies we're building. I think there's a larger metaphor to be gleaned about human-machine collaboration, one in which situates the machine as a creative catalyst. There is a potential to explore more broadly and radically, which makes me think we're just at the beginning.

XRDS: How do you think about the different ways to incorporate the human body, senses, and physiological signals in your practice?

SC: I was inspired by readings into Vedic meditation, and my continued research in robotics development and cybernetics theory.

In particular, I am inspired by the space of experience accessible through Vedic meditation and its links to quantum physics—the idea that experience is a precursor for matter. That there is a state of consciousness the practice of meditation facilitates access to, that operates at a sub-level of experience. It's suggested a level of conscious experience in Vedic meditation is somehow more true and more concrete, than the level of consciousness of our everyday lived experience. I wonder if it's why the quality of time changes, and the readings in one's brain, the activity of the alpha wave is enhanced. To be clear, my interest isn't in the quantification of the process through measurement, however interesting it is from the perspective of data science. Though by measuring my alpha brainwaves, I discovered in both meditation and in the flow state of drawing, the levels were elevated. The readings made it possible to construct a model of interaction—a human and machine configuration to experience that state of flow, of consciousness, in an alternative way. To engage with these abstractions in a form made spatial, made visible, made gestural.

XRDS: Your latest piece, Assembly Lines, pushes human-machine collaboration toward more of a ritual. Could you tell us a little more about the ceremonial aspect of the experience you are attempting to convey within the "Assembly Lines" live performance?

SC: The performance that activates "Assembly Lines" is a ritual. It begins with a two-minute meditation in which the audience is invited to sit, meditate, and reflect with me. It provides a space of dynamic contemplation and stillness, as the multi-sensory components of movement, drawing, and sound fill the space of the performance environment, and paint strews about the canvas. It is an unfolding of a quiet, daily, personal ritual, in the space of a kinetic installation.

I think of a ritual as a shared seeking of a collective experience of the endangered present.

I wonder if rituals today can be seen as a kind-of endangered experience—so many they've been disrupted by technological shifts. Rituals of communication, of gathering. In its debut iteration, "Assembly Lines" is in spatial dialogue with the Finnish forest in the wonderful EMMA Museum, the quality of light becomes a primary contemplative feature, strewn through the installation.

Experimenting with different styles of meditation through performance and research is a way to experience different internal states—it's a kind of virtuality. Being in the conversation of art and technology for as long as I have, there's so much talk about virtuality and embodiment in the medium of digital art. I think we sometimes forget that virtuality and embodiment are not medium-specific. Assembly Lines might be a reminder that the virtual exists within us, there are practices that engage with embodiments that are very old—they came before us and will outlive us. A reminder that while the future stands firm, we move in infinite space.

  Suggested Readings

Han, B. C. The Disappearance of Rituals: A Topology of the Present. Polity Press, 2020.

Kaku, M. Hyperspace: A Scientific Odyssey through Parallel Universes, Time Warps, and the Tenth Dimension. Oxford University Press, 1994.

Lourenço, I., Wahlberg, B., Ventura, R. Teaching robots to perceive time—A reinforcement learning approach [Extended version]. arXiv preprint. arXiv:1912.10113. 2019.

Wiener, N. The Human Use of Human Beings: Cybernetics and Society. Avon Books, 1973.

Lindsay, P. H., and Norman, D. A. Human Information Processing: An Introduction to Psychology. Academic Press, 1977.

  Authors

Sougwen Chung is a Chinese-Canadian artist and researcher, and is the founder and artistic director of Scilicet, a London-based studio exploring human and non-human collaboration. Chung is a former research fellow at MIT's Media Lab and is considered a pioneer in the field of human-machine collaboration — exploring the mark-made-by-hand and the mark-made-by-machine as an approach to understanding the dynamics of humans and systems. Her speculative critical practice spans performance, installation and drawings which have been featured in numerous exhibitions at museums and galleries around the world. In 2019, she was selected as the Woman of the Year in Monaco for achievement in the Arts & Sciences. In 2018 she was an inaugural E.A.T. Artist in Resident in partnership with New Museum and Bell Labs and was awarded a commission for her project Omnia per Omnia. In 2016, Chung received Japan Media Art's Excellence Award for her project, "Drawing Operations." She has been awarded Artist in Residence positions at Google, Eyebeam, Japan Media Arts, and Pier 9 Autodesk.

  Footnotes

a. "Drawing Operations" is an ongoing collaboration between artist Sougwen Chung and a robotic arm; https://sougwen.com/project/drawing-operations

b. https://sougwen.com/machinecollaboration

  Figures

Figure 1. Sougwen Chung and "Drawing Operations Unit: Generation (D.O.U.G) 1, 2, and 3."

Figure 2. "Assembly Lines" is a performative installation featuring a custom multi-robotic system driven by meditation and biofeedback.

 

Copyright held by Owner/Author.

The Digital Library is published by the Association for Computing Machinery. Copyright © 2023 ACM, Inc.