Digital fabrication machines, like 3D printers, laser cutters, and CNC mills, can be programmed to manipulate physical matter. Digital fabrication has become an increasingly popular topic in human-computer interaction (HCI) research. Before being used in homes and workplaces, digital fabrication evolved in the mid-20^th century for use in high-volume manufacturing environments. As a result, the way people program these machines today reflects computing constraints that are more than 50 years old: batch programming, separate applications for different parts of the fabrication pipeline, and limited ability to deviate from the norms.

To bring digital fabrication technology into the 21^st century, HCI has explored several conceptual approaches for people to use machines and their associated software in new and emerging domains. For example, interactive fabrication has explored how nontraditional input like voice data can control machines interactively [1]. Compositional fabrication gives users ways to add adjustments to a job while the machine is still working [2]. Personal fabrication calls for HCI researchers to build end-to-end pipelines that have domain knowledge built into the system, thus lowering the barrier for users without experience in that domain [3]. In particular, the personal fabrication lens has shaped a large body of work in HCI where researchers build end-to-end systems for non-experts in domains as diverse as hydraulics, meta-materials, and on-skin electronics.

  Emerging Cases of Exploratory Digital Fabrication

Despite this focus in HCI research, digital fabrication practitioners from science, art, and engineering prioritize safe experimentation, rather than end-to-end solutions. These unconventional applications might include 3D printing batteries at the micron scale, using cellular automata to make pottery, or using machine learning to test and design 3D printed lattices for withstanding impacts. In these cases, practitioners need to write their own code for custom machine control. Relying on researchers to build all-in-one tools for these tasks is not feasible.

In these contexts, end-user fabricators can neither rely on pre-made pipelines nor established computer-aided design (CAD) and computer-aided manufacturing (CAM) tools for getting the job done. In the case of established tools, a fabricator would typically have a CAD editor for producing geometry, a separate application for translating geometry into computer numerical control (CNC) code, perhaps some post-processing after that, and then another application for deploying the CNC code to the machine.

However, when deviating from the delineated steps in CAD and CAM software to make something that the software doesn't support, end-users have no choice but to program functionality themselves. We wish to highlight, rather than gloss over, emerging end-user programming of machines.

Most of the time, programming a digital fabrication machine from scratch means programming directly in CNC code, which is quite low-level. For example, in the popular G-Code language, the instruction G0 X100 Y100 Z20 F1000 moves the machine's tool to the location at 100mm on the x-axis, 100mm on the y-axis, and 20mm on the z-axis, all at a speed of 1000mm per minute. Having to think at this level of granularity makes programming a machine with CNC code similar to programming in assembly language: It works for low-level, well-defined tasks that are common in machine shops, but it is cumbersome to use for more unconventional fabrication.

For example, consider an artist who wants to replicate a 3D printing technique shown in WirePrint by Mueller et al. [4], where the authors printed a wireframe of a model rather than the entire object. The artist would need to do a great amount of experimentation to get the technique to work on their machine. They would need to try different rates of extruding filament from the print head, different machine speeds, and different model dimensions just to get the wireframe's triangles to stand up at all. All of these factors also vary based on the artist's current 3D printer, such as how it moves (machine kinematics) or the material behavior of the printing filament.

  Physical-Digital Programming

Given that scientists, artists, and engineers are constantly having to program new functionality, we should rethink how to best build tools for unconventional fabrication. Currently, conceptual approaches for digital fabrication in HCI largely focus on usability, or "lowering the floor," rather than on experimentation. While this is helpful in many cases, it also risks reinforcing the idea that digital fabrication must be like program execution, where the end user is hands-off and the system does all the work.

---

The way people program these machines today reflects computing constraints that are more than 50 years old.

---

However, this idea contrasts with existing observations where practitioners are constantly reshaping the process to their liking. Studies of exploratory digital fabrication in the wild like #PlotterTwitter by Twigg-Smith et al. have already shown us that, rather than shying away from programming, artists who work with drawing machines called plotters are already writing, borrowing, and remixing code themselves [5].

Perhaps this drive to program at the edge of the physical and digital worlds is not so surprising. In an article about the benefits of combining HCI and programming languages research, Chasins et al. stated "while both GUIs and languages…[make] it easy to say common things, a language empowers users to say uncommon things too" [6]. End-user fabricators are interested in uncommon tasks by definition, so we should consider how to view exploratory fabrication from a programming language lens.

On the other hand, while prepackaged, end-to-end pipelines are often too high-level and inflexible for experimental fabricators, G-Code's low-level control alone is not enough. The common practice of writing small snippets of G-Code lacks many features of programming languages that we take for granted, for example: functions, visualization, testing, previewing in-situ, interoperating with outside libraries, and adapting to different machines.

Here, we contemplate a third option: physical-digital programming. A physical digital programming environment would help practitioners quickly use custom code to generate toolpaths—which is where the machine's tool moves—then quickly visualize, experiment, and iterate on them just like a programmer would test and iterate on their code.

To motivate this new style of programming, we'll discuss an example, AESTUS, which is a series of wooden vases created by designer Oliver David Krieg [7]. The vases feature delicate grooves on their surface, which were made using a 7-axis milling arm. To design the grooves, Krieg wrote a custom program to control the robotic arm in a precise way. The grooves could not be 3D modeled and therefore required a different software solution than what was commercially available to Krieg. The only solution available to him was to write his own code in tandem with the behavior of the milling arm.

This constant creation of ad-hoc physical-digital code is commonplace across domains, but it is difficult for other designers, artists, scientists, and engineers to share, understand, and extend each other's code.

  Imprimer: A Computational Notebook Prototype for Physical-Digital Programming

One way to make physical-digital programming more accessible is through literate programming. With literate programming, a program comprises natural language, illustrations, and interactive elements interleaved with source code. Computational notebooks are a chief example of literate programming and are commonly used in scientific computing, machine learning, and related fields that deal with digital data. Properly extended, computational notebooks could provide a flexible programming environment for programs that cross into the physical world and back.

For instance, a designer inspired by Krieg might want to explore the possibilities of CNC milling in an open-ended way. With computational notebooks, it becomes possible for the artist to program toolpaths directly, therefore allowing them to engage with the aesthetic possibilities of machine behavior. Computational notebooks allow direct machine programming in a way that supports exploration of the machine's aesthetic language, rather than treating it as the executor of digital geometry. With Imprimer, the artist could easily document, extend, and share their code with a growing community of exploratory makers and artists.

To prototype literate programming for machines, we built Imprimer, where the end user works from a computational notebook to control a CNC mill with an AR projection guide (see Figure 1). In our implementation, we used the Observable Notebook tool and a large CNC mill for wood called a Shopbot whose cutting space measures 2.4 by 1.5 meters, or in the Shopbot's native units, 96 by 60 inches.

---

Scientists, artists, and engineers are constantly having to program new functionality, we should rethink how to best build tools for unconventional fabrication.

---

In particular, we chose to build Imprimer for CNC milling, rather than for other types of digital fabrication. This is because of the steep learning curve it presents to those without prior experience, which makes experimentation difficult. To explore the versatility of Imprimer, we implemented three demonstration notebooks: surface milling molds by sampling functions of two variables, sketch-based milling with augmented reality, and a parametric shelf generator with associated debugging tools.

For the first demonstration, a user can mill the surface of a material based on mathematical functions without having to do CAD first. This notebook (see Figure 2) includes a visualization for the surface of a function of two variables alongside a derived toolpath that can be dispatched to a CNC mill. For example, we implemented the two-variable unnormalized sinc function:

where f is the frequency or "waviness'" of the ripple that the function resembles, x and y are 2D coordinates on the milling surface, and z is the depth of the cut. Notebook users can use graphical input elements or edit code to experiment with the resulting machine toolpath in real time. We also provided a function that sums sine and cosine terms as well as the Goldstein-Price function commonly used to test optimization algorithms. To experiment with the milling topology, users can modify these functions or write their own functions in code as needed.

---

The physical-digital programming paradigm encourages rich visual and linguistic documentation alongside source code.

---

To support the library, we developed a network architecture to connect the Shopbot with the user's notebook, as well as an augmented reality overlay for visualizing movements on the Shopbot's physical bed. Code executed in the notebook directly affects both the Shopbot and the overlay. In the end, we explored a range of different functions and different parameters, milled the molds out of the insulation foam, and cast the tiles in plaster (see Figure 3).

Figures 4 and 5 showcase the second and third example notebooks, respectively.

  Experiments with Users

To better understand how users conceptualize literate programming as a technique for CNC milling, we conducted a user study with six participants who had varying levels of experience in CNC milling and in programming. Our initial experiments underscored the need for careful scaffolding of domain concepts if users were to understand and feel comfortable manipulating views or changing code.

In particular, one participant who was not familiar with CNC milling found it challenging to choose tooling parameters in the notebook cells; he was especially concerned with how precise measurements needed to be.

"Are you recording what we're doing now? Because that could be helpful for somebody installing the end mill. … So much information like 80%, 20% of the shaft in the collet—I'm not sure how bad it is if one thing is done wrong versus another."

Despite some tensions with using CNC notebooks for exploratory fabrication, participants expressed that the notebook lowered the barriers both to getting started with a machine as well as for reconfiguring existing workflows.

"The concept is neat, and I see a lot of applications. It seems if you can swap out the toolpath generation part to match an application—as long as you could write [code for] toolpath generation—then there's a lot of real-world use. Using a notebook feels like a more natural way to interact with a machine."

In particular, one participant from our user study, who is an engineering educator, highlighted Imprimer's potential as an educational tool to scaffold machine use.

"I really like that you can write code that creates buttons and sliders. This way you could limit the notebook to specific functionalities, like you can adjust and play with these numbers, but not these other ones that might be more dangerous to mill…most students would stick to the controls you give them."

  A Future with Physical-Digital Programming

Many existing digital fabrication systems orient makers toward getting things done, hiding complexity, and making design decisions that ensure manufacturability and ease of use. Others will offer many functionalities as well as in-app and external support to guide makers through the many features. With Imprimer, we contemplated an option that's the best of both: the code underlies each functionality from the start so makers can develop their own practice of machine control.

The user study made us aware of both the promises of Imprimer in terms of expanding the design space for users when it comes to novel digital fabrication workflows, as well as the potential challenges that deploying this system on a larger scale might involve. Every time a user engages in fabrication, their process will comprise a certain number of steps that might vary widely depending on materials, equipment, skill levels, access to resources, timeframe, among other constraints. Scaffolding the information in a way that was legible and clear for users proved to be one of the first design considerations with Imprimer. What was necessary to share and what was optional? How does the documentation of a workflow depend on machine conditions, level of expertise, and other factors which can be difficult to predict?

These questions quickly led us to the crux of Imprimer's design challenge, which is not to create the optimal computational notebook for future users, but rather to lay the foundations for other makers and programmers to build on. To this end, many aspects of the fabrication process remain rooted in physics and material behavior that is well-studied in other academic disciplines and could be shared in notebooks. This contrasts with a long-held goal of programmable matter through a seamless coupling of bits and atoms [8], which is not yet a feature of digital fabrication practice.

We argue for a different future: One where reusable code infrastructure helps end-user fabricators tackle new challenges that conventional tools haven't yet been designed for. Rather than trying to flatten the physical dimensions of fabrication into code, the physical-digital programming paradigm encourages rich visual and linguistic documentation alongside source code, which computational notebooks aptly support. Scientists, artists, and engineers are constantly developing their own solutions to work with machines in unconventional or novel ways. When they do not, they rely on a dispersed community of digital makers. Fabrication always involves ad-hoc processes—no two fabrication setups are the same, and no two makers come with the same abilities, resources, and sensibility.

---

Computational notebooks could provide a flexible programming environment for programs that cross into the physical world and back.

---

In addition to people already close to code, we should also consider people who do non-digital fabrication. We can see that while growth for software developers has increased sharply, opportunities for manual fabricators are decreasing (see Table 1). Worse, while the tools for learning software development are everywhere, tools for learning digital fabrication, and for connecting manual fabrication to the digital have stagnated.

Imprimer is a first step toward allowing a community of makers-as-programmers to grow, by providing an environment in which makers have access to source code, rich documentation, and graphics possibilities, as well as being able to easily share these processes with one another. With Imprimer, we envision a future in which making means developing both a material and a programming vocabulary; and where this growing vocabulary is extended and shared within a community.

The future of open-source fabrication will come not by abstracting machine operations for users, nor by relying on low-level machine CNC code alone, but by letting makers have better control and knowledge of their machines so that they can explore a fuller range of human-machine co-creations. Through physical-digital programming, we have the opportunity to unite diverse populations around a cross-disciplinary infrastructure of accessible machine operation, understanding, and collaboration.

  References

[1] Willis, K. D., Xu, C., Wu, K., Levin, G., Gross, M. D. Interactive fabrication: New interfaces for digital fabrication. In Proceedings of the Fifth International Conference on Tangible, Embedded, and Embodied Interaction. ACM, New York, 2010, 69–72.

[2] Kim, J., Zheng, C., Takahashi, H., Gross, M. D., Ashbrook, D., Yeh, T. Compositional 3D printing: expanding & supporting workflows towards continuous fabrication. In Proceedings of the 2^nd ACM Symposium on Computational Fabrication. ACM, New York, 2018, 5:1–5:10.

[3] Baudisch P. and Mueller S. Personal fabrication. Foundations and Trends in Human-Computer Interaction 10, 3–4 [2017], 165–293.

[4] Mueller, S., Im, S., Gurevich, S., Teibrich, A., Pfisterer, L., Guimbretière, F., and Baudisch, P. WirePrint: 3D printed previews for fast prototyping. In Proceedings of the 27^th Annual ACM Symposium on User interface Software and Technology. ACM, New York, 2014, 273–280.

[5] Twigg-Smith, H., Tran O'Leary, J. T., and Peek, N. Tools, tricks, and hacks: Exploring novel digital fabrication workflows on #PlotterTwitter. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. ACM, New York, 2021, 1–15.

[6] Chasins, S. E., Glassman, E. L., and Sunshine, J. PL and HCI: Better together. Communications of the ACM 64, 8 [2021], 98–106.

[7] Krieg, O. D. 2018. AESTUS. 2018; https://oliverdavidkrieg.com/?p=921

[8] Ishii, H., Lakatos, D., Bonanni, L., and Labrune, J. Radical atoms: Beyond tangible bits, toward transformable materials. Interactions 19, 1 [2012], 38–51.

  Authors

Jasper Tran O'Leary is a Ph.D. candidate in the Paul G. Allen School of Computer Science & Engineering at the University of Washington. He is interested in how innovation with programmable digital fabrication machines can feel more like actually programming—not just running code.

Gabrielle Benabdallah is a Ph.D. candidate in the Department of Human-Centered Design and Engineering at the University of Washington. With a focus on language technologies and digital fabrication, she designs and builds systems that question practices of sense-making with and through machines.

Nadya Peek is an assistant professor in the department of Human-Centered Design and Engineering at the University of Washington. Her lab's work focuses on the precision of machines for the creativity of individuals.

  Figures

Figure 1. Imprimer setup.

Figure 2. Surface milling notebook.

Figure 3. Tile molds milled with Imprimer.

Figure 4. Augmented reality sketching notebook.

Figure 5. Parametric shelf notebook.

  Tables

Table 1. U.S. Bureau of Labor Statistics. Occupational Outlook Handbook. 2022.

 

This work is licensed under a Creative Commons Attribution International 4.0 License.

The Digital Library is published by the Association for Computing Machinery. Copyright © 2023 ACM, Inc.