XRDS: Thank you, Patrick, for interviewing with us! Before we get started, for people who don't know your work, how would you describe your research interests and the field that you work in?

Patrick Chwalek: Hi, thanks for having me. So, my main interest is in understanding how environments impact the living beings that inhabit them, whether humans or wild animals. More broadly, I want to understand how different parts of the ecosystems interact with each other and how changes to the environment affect the species' behaviors. Typically, research happens inside a controlled environment, like a laboratory or a simulation chamber. My approach has been to develop tools and systems that can be deployed in the wild and at scale.

XRDS: Let's start with your projects around humans and the indoor environment. What were the problems that you were interested in and what systems did you build?

PC: One aspect I became interested in is human discomfort and, more specifically, our perception of indoor environmental quality and how it affects our health. This became especially important and on everyone's mind during the challenging times of the COVID pandemic, when most people remained indoors. The profound shift in our awareness occurred as people grew increasingly attuned to the quality of their surroundings. Yet amidst the clamor of sensationalized news stories, discerning precisely what mattered most to individuals was difficult.

Most commercial systems and research projects around indoor environment monitoring summarize a coarse description of a building or sections of a building. After reviewing the literature [1, 2], it occurred to me that a lot of an individual's comfort is what we experience around the face, such as the effect of the light intensity on drowsiness and the amount of C02 in the room on one's cognition. It became clear we need a system to sense around the face in order to obtain an accurate measure of some of these parameters where they matter for our own perception.

I set out to build distributed wearables designed for specific users that are personal. I designed AirSpecs [3], which is a pair of smart eyeglasses, equipped with various types of sensing capabilities, such as gas sensing around the nose and thorough human skin temperature measurements, that allowed us to sense environmental changes around the user and their physiological changes (see Figure 1). Smart glasses are great in that they are on our faces where we hear, see, smell, speak, and breathe. If you want to sense the world and how our perception correlates to it, it needs to be on or near the face. Practically speaking, smart glasses are the most ideal form factor and position to do such data collection.

Besides the quantitative data collected via the sensors on the glasses, we also wanted to investigate qualitatively how the user perceives comfort as a result of the smart glasses. In addition to the smart glasses, with my collaborator Sailin Zhong, a Ph.D. student from the University of Fribourg, we designed an accompanying iOS application (see Figure 2). We recorded data over weeks and collected objective and subjective metrics to better understand the variance of environments and user preferences. We also scaled this study beyond the United States to Switzerland and Singapore (see Figure 3).

---

The inherent distrust in technology is something out of our control but should be considered when designing the appearance and functionality of research systems

---

The implication is twofold: We now have a dataset of how people actually react to indoor environmental change and what matters to them about comfort. For example, users don't like to give up their agency in controlling and adjusting the indoor environment; thus, a semi-automated approach would be preferred. These insights can inform and enable building system designers or anyone who wishes to study human comfort to design comfort-related technologies for specific user needs [4, 5].

XRDS: Can you talk about the process and potential challenges with deploying this system outside of laboratory settings and across different geographical regions?

PC: I'm not going to lie; this was very difficult. There are a variety of things to consider when doing a global study. One would not only have to make your hardware system robust and easily serviceable but would also have to consider varying data privacy laws across the world. If cloud storage is for real-time data collection, one needs to pay attention to the countries' requirements for where that data can be stored and accessed.

Additionally, doing studies with users where we leverage custom hardware that we ask them to use during their waking hours is difficult, because they cannot be supervised the entire time. We implemented various strategies and redundancy mechanisms such that we can be alerted when a user's system is not working properly, either due to a user error or a system malfunction, and that allows us to apply a quick fix. A lot of time was put in by myself and Sailin on testing the robustness of the software and trying to predict all the ways our system would fail in the field.

XRDS: What are some interesting things you learned from both designing the system to working with users and getting feedback from them?

PC: When designing a wearable system that is very visible, one has to consider both the user's physical comfort and the social acceptance of wearable technology.

For physical comfort, we consulted with various manufacturers from Shenzhen and South Korea, taking into consideration aspects such as form factor, weight, and weight distribution. As a non-glasses wearer, one of the most surprising things I learned is that users who were not accustomed to wearing glasses were conditioned to wearing ours relatively quickly. In fact, they were surprised themselves that after some time they forgot they were wearing them. So that was encouraging for future smart eyeglass research. When considering social acceptance, we basically had to design a "commercial product" that the users would not feel self-conscious wearing. Although our glasses are far from perfect, we did receive mostly positive feedback on their appearance and comfort, both from the wearers and bystanders who interacted with users wearing them.

Another important aspect of ensuring the user would feel (socially) comfortable wearing the device is their privacy. One thing that we did not want to integrate is a camera since that leads to various surveillance problems. It also introduces an interesting dynamic for the wearer since people may think the glasses have an outwardly pointed camera, which impedes social interaction and leads to self-censorship. We did not want that to happen and wanted to preserve the natural experience. That being said, some of our sensors do share some resemblance to small cameras and so there were a few occurrences where outside bystanders did not believe the wearer when they said the system has no camera. The inherent distrust in technology is something out of our control but should be considered when designing the appearance and functionality of research systems.

And finally, we of course learned many interesting things about user's perception of comfort and how they act in response to changes in their indoor environment. One thing that stood out was that user preferences vary widely when it comes to monitoring the quality of their environments. Some users wanted to just get alerted when a problem was sensed while others wanted to see fine-resolution data on how their environments and physiology varied as a function of their physical activities (i.e., cooking, cleaning, opening a window).

XRDS: Now, in addition to humans and the indoor environment, you are also working with outdoor environments and, particularly, animals. How did this all start?

PC: In the summer of 2022, I did an internship with the National Geographic Society in Washington D.C., working with their Exploration Technology Lab on various tech for their explorers. I spent the summer working on various cameras and sensors that were used by their explorers. These systems include a rover-based camera system that takes closeup pictures of lions and hyenas in the wild, camera- and microphone-equipped systems that get mounted on whales to monitor their feeding habits, and much more. It was really fun, and a lot of my skillset fit perfectly in solving some of their problems. Afterward, I was able to partner with some explorers to spin up new projects and deploy them in interesting locations. Last year, I spent part of the summer in the Arctic Circle, collecting high-resolution acoustic data of wildlife on Svalbard, Norway, using custom microphone arrays I designed (see Figure 4) [6]. Right now, I am working with researchers to study endangered bees in Patagonia, creating new types of wearables to monitor African Wild Dogs in Botswana, and systems to support the monitoring of guardian frogs in the jungles of Borneo.

XRDS: Can you give an example of what are the challenges and problems when designing systems for wildlife?

---

When designing a wearable system that is very visible, one has to consider both the user's physical comfort and the social acceptance of wearable technology.

---

PC: I can dive a bit deeper into the bee project since I am in the process of deploying the system in Patagonia [7]. A problem in the Patagonia region is that the European honey bee is invasive and threatening the native Bombus dahlbomii, one of the world's largest bees and colloquially described as "flying mice." Traditionally, ecologists monitor these bees by scoping out specific areas and physically noting how many bees and what species they see. Today, researchers can do that with cameras, but they are limited by where the cameras are placed, battery power, storage space, and above all else, the elements. With all these constraints in mind, I am designing a new type of system that uses distributed microphones to hear the bees and approximately determine their location and species. These systems can also trigger external cameras based on certain cues. We also created an accompanying phone app that allows ecologists to easily synchronize their fieldnotes with the timestamps on the recorders, reducing any ambiguity when post-processing the data and labeling bee occurrences.

XRDS: How is it working with National Geographic Explorers and doing research in these unique settings?

PC: It is a unique experience and one I would not trade for the world. I never really fit as a traditional researcher in academia and am really driven by the prospect of seeing others use the devices that I make. The National Geographic Explorers I work with are very passionate about their focus areas, and there is a lot of synergy between the technology they need and what I can provide at a research level. Personally, this has broadened my horizons and taken me to some really interesting areas of the world that I likely would not have visited otherwise.

XRDS: How do the lessons learned from AirSpecs relate to your current projects for animals?

PC: Broadly, designing devices for applications where it's hard to predict what someone or something will do with it is difficult. One can argue that designing for humans might be harder than designing for animals since people are dynamic and do not always stick to a plan. But, as a human designing for fellow humans in the built environment, I can predict many of the use cases and failure modes, which is far more difficult to do for animal applications. For example, I do not know the hunting behavior of African wild dogs and how that may impact a worn electronic device. Similarly, it's hard to predict how the moisture and wildlife in the jungles of Borneo are going to impact the electronics we are putting in them. Luckily, I have some experience in designing tech for rough environments, especially with my time prior to my Ph.D. in designing outdoor surveillance equipment.

XRDS: What do you plan to do after you graduate?

PC: I would like to keep supporting some of these projects after I graduate since I have devoted quite a lot of time in conceiving and deploying them. I get a lot of personal fulfillment when I see others use them. Truthfully though, it is difficult to raise money for ecological applications, and much of that work is done through nonprofits and donors. What I would like to do is start a consultancy or start-up with a focus on these unique and specialized projects which can take my staff around the world and broaden their skillsets. I do feel that the work I do is giving me a unique skill set that allows me to design better systems, and I want to enable others to do the same.

XRDS: If people want to do what you do or get involved in studying different environments and their inhabitants, where should they start?

PC: My goal is to design systems and tools such that researchers or anyone interested can feel empowered to probe the environment and answer questions about the ecosystem. If you would like to use or build on top of my systems, whether it is AirSpecs or the systems I am building for explorers, feel free to get in touch with me. Even if you are not interested in building new tools, donating to non-profits would really help drive these types of ecological work further. The National Geographic Society is one example of a nonprofit that you can donate to and that funds explorers, but there are various others, especially at the local level.

  References

[1] Jayathissa, P., Quintana, M., Abdelrahman, M., and Miller, C. Humans-as-a-sensor for buildings: Intensive longitudinal indoor comfort models. Buildings 10, 10 (2020), 174.

[2] Jia, M., Choi, J., Liu, H., and Susman, G. Development of facial-skin temperature-driven thermal comfort and sensation modeling for a futuristic application. Building and Environment 207, 108479. (2022).

[3] Chwalek, P.C., Zhong, S., Ramsay, D., Perry, N., and Paradiso, J. AirSpec: A smart glasses platform, tailored for research in the built environment. In Adjunct Proceedings of the 2023 ACM International Joint Conference on Pervasive and Ubiquitous Computing & the 2023 ACM International Symposium on Wearable Computing (ISWC '23). ACM, New York, 2023, 204–204.

[4] Whitmore, N., Chan, S., Zhang, J., Chwalek, P., Chin, S., and Maes, P. Improving attention using wearables via haptic and multimodal rhythmic stimuli. PsyArXiv. February 22, 2024.

[5] Youn, H.J., Zhong, S., Shtarbanov, A., and Chwalek, P.C. NugiTex: An interactive, affective wearable that informs users of a plant's "comfort" level through haptic cues. In Companion Publication of the 2023 ACM Designing Interactive Systems Conference. ACM, New York, 2023, 251–255.

[6] Chwalek, P.C. SoundSHROOM: Sensor-equipped habitat recorder for outdoor omnidirectional monitoring. MIT Media Lab. https://www.media.mit.edu/projects/soundshroom/overview/

[7] Chwalek, P.C. BuzzCam. MIT Media Lab. https://www.media.mit.edu/projects/buzzcam/overview/

  Authors

Patrick Chwalek is a Ph.D. candidate in the Responsive Environments Group at the MIT Media Lab, where he develops and deploys sensors for real-world applications. He explores how environments influence people's physical and mental states, and how comfort varies with engagement. He is also actively creating devices to study wildlife in their natural habitats, such as Arctic birds, bees in Patagonia, and African wild dogs. Prior to joining the MIT Media Lab in 2018, Chwalek worked at MIT Lincoln Laboratory for three years, focusing on wearable embedded systems for physiological monitoring, remote surveillance systems, and machine learning algorithms for radar target classification.

  Figures

Figure 1. AirSpecs smart eyeglass device designed to measure environmental parameters and physiological parameters.

Figure 2. AirSpecs are fitted with a variety of sensors (top) and have supporting iOS (middle) and Apple Watch applications (bottom) to allow users to probe the data and offer subjective measurements.

Figure 3. Sailin Zhong (left), a Ph.D. candidate at the Human-IST Institute at the University of Fribourg, and Patrick Chwalek (right), a Ph.D. candidate at MIT Media Lab, wearing AirSpecs in Switzerland during a week-long workshop at the University of Fribourg and the Smart Living Lab, where they taught researchers how to use the platform.

Figure 4. The SoundSHROOM microphone array devices deployed on Svalbard, a Norwegian archipelago in the Arctic Ocean, during the summer of 2023.

 

Copyright held by owner/author.

The Digital Library is published by the Association for Computing Machinery. Copyright © 2024 ACM, Inc.