XRDS

Crossroads The ACM Magazine for Students

Sign In

Association for Computing Machinery

Magazine: Features
ChatGPT in Education: Partner or Pariah?

ChatGPT in Education: Partner or Pariah?

By ,

Full text also available in the ACM Digital Library as PDF | HTML | Digital Edition

Tags: Computer-assisted instruction, Natural language processing

back to top 

Prior to the invention of writing—and certainly before the creation of the printing press and the large-scale distribution of the written word—much of education was focused on memorization. As Justin E. H. Smith writes in The Internet Is Not What You Think It Is: "It is not that the technology of writing was unavailable to medieval scholars, but only that, for the most part, until the modern period true knowledge of an object of study involved internalizing that object by committing it to memory" [1]. Smith goes on to note that as the written word—enabled by new technologies like the printing press—became more widespread, early educators lamented the loss of emphasis on rote memorization and the negative impact that would have on future learners.

Of course, today that idea feels quaint altogether. Focusing too heavily on rote memorization has become a source of condemnation for certain subjects. The Woodrow Wilson National Fellowship Foundation's 2019 Reimagining American History Education criticizes history courses for relying too heavily on memorizing names and dates [2]. "Information is Not Knowledge" by Norton Tener and "How Well Aligned Are Textbooks to the Common Core Standards in Mathematics?" by Morgan Polikoff (among many others) criticize science classes for focusing on vocabulary definitions rather than the scientific process [3, 4]. Permanent external memory in the form of books and written notes reduced the importance of rote memorization, which in turn allowed teachers to focus on other skills that could not so easily be offloaded onto a new tool.

In many ways, this cycle characterizes the long and intertwined relationship between new technology and new teaching. Many new technological tools can be viewed through the lens of offloading cognitive tasks from humans to machines or devices (a view that forms the foundation of "Distributed Cognition" by Edwin Hutchins). When these tools first emerge, teachers are often troubled because the skills they have previously focused on teaching suddenly seem irrelevant; over time, however, curricula adapt to incorporate the new technology, allowing students to reach even higher heights than they could performing the task of the new tool themselves.

Take, for example, the emergence of relatively affordable scientific calculators. Released in 1990, the TI-81 was among the first scientific graphing calculators released to a wide market. The TI-81—as well as its successors—could perform numerous operations that students previously would have had to do by hand. Graphing a parabolic function, for example, was a long process that involved manually calculating the x- and y-intercepts, the vertex, and the focus; on a graphing calculator, however, students could just enter the equation to see the full graph. Those parameters could be derived from the graph rather than calculated as part of drawing the graph.

But drawing parabolic functions was a key part of mathematics education: If students could just plug the formula into a calculator, what was the point of that curriculum? So, as with many technological developments, the graphing calculator was initially prohibited from some math classes as rendering the curriculum irrelevant; the state of California, for example, prohibited calculators on state assessments.

Of course, with time, that changed. The curriculum adapted. Other states (such as Connecticut) went far in the opposite direction, requiring students to have calculators specifically because they would allow students to solve more advanced problems. "A Brief History of Calculators in the Classroom" by Audrey Watters provides a great rundown of the back-and-forth views on calculators in education [5]. Students continued to be taught how to do these calculations by hand—they were an important part of understanding what those parameters meant and how the graph related to the formula. Once they mastered drawing these graphs by hand, though, the graphing calculator joined the fray, allowing them to skip all those manual calculations. Equipped with this tool, they were then able to tackle more advanced problems because they could focus only on what made a particular problem new. Teachers' expectations for the kinds of problems students could solve rose significantly because they knew a calculator could complete the steps that would have taken even an expert several minutes to do by hand. Fast-forward to today, and we see a significant amount of modern mathematics curriculum is actually teaching students how to use the calculator correctly—a change that might annoy many traditionalists, but which is largely responsible for the advanced math students are now able to do as part of lower-level classes.

Such developments can be found throughout the long history of teaching with technology. New technologies constantly diminish the importance of skills previously prized by educators, but in exchange they create new skills to be taught and new levels of understanding and performance to be reached. The search engine diminished the need to navigate the Dewey decimal system and comb ancient tomes for information, but in return it created a new field of search savviness and made countless resources far more available to students. The computer keyboard lowered the importance of cultivating perfect penmanship, but allowed for non-linear construction of written essays and much greater revisability over time. Even the humble abacus reduced the necessity of performing complex mental arithmetic, but in turn created a new set of algorithms to be learned and provided a way for multiple individuals to more easily collaborate on a task through its external visibility of a previously purely mental process.

Viewed through this lens, the recent emergence of ChatGPT (and its competitors from other developers) is just another in a long line of technologies that offload what was previously a skill restricted only to humans and allows it to be performed by a machine. Alarmists see it as an existential threat to the very nature of education given just how much of our educational system is predicated on students' written answers. But instead, ChatGPT brings with it a new set of skills to be learned, and along with those skills the possibility to elevate our expectations for how well students fulfill our learning goals.

In the pushback against ChatGPT, detractors have been quick to point out its many flaws. Its responses carry little guarantee of accuracy, and in fact at times it can be a very convincing liar. It is regularly repetitive in its writing and reasoning style, frequently confusing details of different individuals and generating nearly-identical responses to superficially similar but deeply dissimilar questions. In one experiment I performed, ChatGPT attributed one accomplishment—the first video game music composer to have their game soundtrack played by a live symphony orchestra—to no fewer than six different composers. Tricking ChatGPT into giving a useful answer has proved to be such a prized and elusive skill that many have suggested "prompt engineer" as a career of the future, referring to the ability to generate the right input text to elicit the desired output from such an interactive agent.

All these observations form the foundation for a curriculum of the future on interacting with tools like ChatGPT. Just as students have been taught how to use search engines to identify trustworthy online sources and to integrate information from multiple such references, students should now be taught to evaluate the information generated by an AI agent, to understand the common mistakes such a tool may commit, and to properly correct them in their ultimate submission.

Critics of the role that ChatGPT may play in education are often quick to point out novices are poorly equipped to notice the mistakes that would jump out to experts. Using the technology risks exposing learners to inaccurate information before they are prepared to properly dismiss it. But identifying when a tool has made a mistake is part of teaching students to use any technology. Spell check, another invention often used by novices, regularly makes mistakes between homonyms or based on misunderstanding a sentence's subject and direct object. Even calculators are not infallible. True, calculators always give you exactly what you request, but minor errors in your query can generate major errors in the response. Understanding how to evaluate the output of a calculator for its likelihood to be correct to the original problem (rather than just the query you entered) is part of learning to use a calculator in math. The same way, understanding how to evaluate the output of ChatGPT for its likely correctness is part of learning to use ChatGPT for learning. A novice might not be able to immediately pick out a mistake in an automated response, but they can be taught to identify the types of errors the agent is likely to commit and how to follow-up and confirm the information they receive.

In return, what can we expect students to achieve when equipped with a tool like ChatGPT? For one, I would argue we can expect a clearer writing style to emerge from students' engagement with the tool. ChatGPT is very good at providing advice on syntax and word choice, the type of surface-level feedback students might expect from a peer or a cursory pass from a teacher. In this article, I have used ChatGPT when I struggled with repetitive phrasing and with some selected terminology. Five paragraphs ago, I was struggling to find a third way to say "diminished the need" and "lowered the importance." ChatGPT suggested "reduced the necessity" (plus "lessened the demand," "decreased the relevance," and "softened the significance," as well as 12 other phrases that were not very good alternatives). In the following paragraph, I was stuck on referring to alarmists as "chicken littles," a colloquialism likely lost on a large fraction of this article's readership. ChatGPT suggested "alarmists" along with several far less appropriate alternatives (with a reminder to avoid using colloquialisms in formal writing thrown into its response for good measure). And in the previous sentence, I could not think of a second way to say I was "stuck" on something; ChatGPT suggested "I was struggling," which grammatically made for an awkward fit into that sentence but fit nicely into the previous one. In all these instances, ChatGPT helped my writing, but I still had to apply my expertise to identify the one or two good options from the long list of poor alternatives that ChatGPT generated.

ChatGPT is also just one of many technologies that helped; the automated grammar check picked up on places where I forgot to switch "a" to "an" when changing "calculus" to "algebra." The keyboard and word processor allowed me to write the next paragraph before this one in line with how my mind built my understanding of this topic; a search engine supplied me with much-needed synonyms for novice (I chose "learner"), feedback (I chose "advice"), and source (I chose "reference").

Of course, the hysteria over ChatGPT comes from its potential to do far more than just generate an iterative improvement to students' writing style at scale. There are widespread accounts of instructors plugging their essay prompts into the tool and directly generating a decent-quality essay. Tools exist that promise to detect AI-generated text, but their accuracy rates are far too low to provide sufficiently convincing evidence to "convict" a student.


ChatGPT brings with it a new set of skills to be learned, and along with those skills the possibility to elevate our expectations for how well students fulfill our learning goals.


But what is far more important is that in most cases, the underlying learning goals for a class need not change based on the existence of ChatGPT, just as the underlying learning goals for an algebra class need not change based on the existence of a graphing calculator. In the latter, we still want students to understand the relationship between a quadratic function and its corresponding graph; we still want students to recognize instances of problems that are properly represented by a quadratic function; and we still want them to be able to use that graph to generate predictions or observations about the underlying problem. The existence of the calculator does not change what we want them to learn, though it may change how we assess their understanding. More importantly, it need not change that assessment through a collection of artificial rules and constraints designed to minimize how much they use the tool, like prohibiting the devices altogether and using digital proctoring to ensure remote students conform. Instead, access to calculators may change that assessment by deliberately expecting students to use the tool, by teaching them how to use it effectively, and by raising the expectations for what kinds of problems they ought to be able to solve when equipped with this new support.

It is only with the benefit of hindsight that we can recognize how calculators went from pariah to partner in mathematics education. Were we to transport ourselves back to 1991, we might find math teachers panicking because the calculator could generate an exact solution to their homework problems in seconds rather than the several minutes students would spend calculating them by hand, just as today we find teachers panicking because ChatGPT can generate a copyable essay that seemingly fits their assignment prompts. But mathematics education evolved not just to accommodate the calculator, but to embrace it as a tool for greater outcomes. Teaching has done the same for countless different earlier technologies, each time propelling human understanding and advancement forward as new abilities are unlocked by the partnership of teaching and technology. There remain challenges, of course—learning how to embrace new technologies in teaching is not easy, and ensuring equitable access to new tools among students from radically different backgrounds is essential. But educators have incorporated new technologies before, and we can do so again—to both our students' benefit and our own.

back to top  References

[1] Smith, J. E. The Internet Is Not What You Think It Is. Princeton University Press, Princeton, 2022.

[2] Lewer, B. Reimagining American History Education. The Woodrow Wilson National Fellowship Foundation, Princeton, 2019.

[3] Tener, N. Issues in education: Information is not knowledge. Childhood Education 72, 2 (1995), 100–100.

[4] Polikoff, M. S. How well aligned are textbooks to the common core standards in mathematics? American Educational Research Journal 52, 6 (2015), 1185–1211.

[5] Watters, A. A brief history of calculators in the classroom. Hack Education. March 12, 2015; https://hackeducation.com/2015/03/12/calculators

back to top  Author

David A. Joyner is executive director of OMSCS and online education in Georgia Tech's College of Computing. He is the author of The Distributed Classroom (MIT Press, 2023) co-authored with Charles Isbell and Teaching at Scale (Routledge, 2022). He is also the chair of the steering committee for the ACM Learning @ Scale conference. As part of his role, Joyner teaches several classes each semester totaling more than 7,500 for-credit students per year in addition to tens of thousands of students in his massive open online courses.

back to top 

xrds_ccby.gif This work is licensed under Creative Commons Attribution International 4.0.

The Digital Library is published by the Association for Computing Machinery. Copyright © 2023 ACM, Inc.