From course final projects that spark students' technical interests to project portfolios that help students negotiate into their dream jobs, student-driven projects let students encounter the freedom to shape their educational journey. Students choose what knowledge to pursue, which sources to consult, and what challenges are worth learning more about. Going off the well-beaten path, students also encounter real-world friction from their decisions and must make choices to handle that friction. But making decisions and dealing with the consequences can be uniquely frustrating and scary.

Philosopher Gert Biesta suggests three educational processes that encourage a student's appetite for staying with the difficulty [1]:

1. Interruption; an encounter with real-world friction, frustration, and doubt, opening one to the possibility that there is more to learn and understand.
2. Suspension; the time and space to reason about how one's current understanding meets real-world constraints without the pressure of making a quick or rash decision.
3. Sustenance; the support to seek out new information that lets one formulate and test a new plan.

In my research, I investigate how generative AI can help students encounter their ability to meaningfully personalize their path through formal education. In this article, I will share how I used GPT-3 to support interruption, suspension, and sustenance during my undergraduate internship, discuss early research on using generative AI to help students learn from materials of their choice without extra teacher effort, and outline my plan for how generative AI could help students access their agency in student-driven projects.

  AI as Interruption

As an undergraduate intern at Microsoft Research trying to decide what to do after graduating college, I had a lot of uncertainty about my future career path. Among other things, I wanted to know what machine learning papers to read to become a "real" researcher, what math classes to take, and whether it was worthwhile to start a Ph.D. after I graduated. As questions occurred to me, I would ask them to whichever researcher happened to be available and email the surplus to my mentor.

One day, my mentor responded to my email by saying he was very busy, so instead of writing back himself he forwarded GPT-3's answers and asked me what I thought of them.

GPT-3's answers were a quick and convenient starting point, but they weren't what I wanted. While GPT-3's advice did a solid job summarizing generic social norms—possibly lifted from hundreds of internet self-help blogs—I realized I wanted advice that went beyond the generics: stories of breaking norms, unexpected lessons learned, and the permission to make iterative adjustments to plans and dreams. Engaging with the unsatisfactory expertise of GPT-3 interrupted my process of bulk question-asking and created an opportunity for me to seriously reflect on my agency, purpose, and the resources available to me.

Yet, interruption, especially in such an unsatisfactory way, can be annoying or even invite feelings of hopelessness. Thus, successful implementations of interruption have to be followed by accessible support for suspension, meaning the time students need to reconcile their expectations with their experiences, and sustenance, which are the resources and guidance students need to create a new plan and execute it.

  AI as Suspension

Much of formal education assumes the goal of the student should be to emulate the thought processes of an expert [2, 3]. But sometimes, the student might see something that the expert doesn't. In some cases, the student might be savvy enough to explain their observation and improve upon the expert's under-standing. But in many cases, the novice might simply lack the vocabulary and skills to make a compelling case.

In my first-ever paper rebuttal, one of the reviewers had misunderstood a key aspect of our paper. My mentor and I disagreed on what the reviewer had misunderstood.

We went back and forth on a shared document. My mentor wrote his thoughts in beautiful prose, but I could only contribute sparse bullet points. In the face of my inexperience, the effort needed to incorporate my mentor's advice into my writing, and the looming deadline, I was ready to save everyone some time by giving up. Fortunately, my mentor saw that I was struggling and suggested I use GPT-3 to help me find the words.

So, we prompted GPT-3 for a list of potential interpretations, and I chose one that was good enough for me to use as a starting point. I criticized where the interpretation was lacking and GPT-3 created more sentences in response. The resulting block of text helped my mentor see what I wanted and start a critical conversation about it. We finished the rebuttal together, and both of our perspectives were included in the final submission.

Generative AI does not have the ability to reason about the world, but students do. The speed and fluency of AI tools open up new ways for students to articulate how their perspective meets the world, creating space to explore ideas that would otherwise be silenced.

  AI as Sustenance

I wrapped up my research internship with a paper describing some early results of using GPT-3 to simulate multiple humans in social science experiments [4]. We found GPT-3 could provide semi-realistic predictions for people's subjective judgments. Our prompting methodology replicated findings for people's tendency to punish unfair behavior (the ultimatum game), grammatical judgments (garden-path sentences), and obedience to a malevolent authority (the Milgram shock experiment).

However, in simulating people's responses to questions of knowledge and recall, we found GPT-3 had an unrealistic hyper-accuracy distortion. It predicted humans could perfectly recall precise values for obscure quantities (e.g., everyone knows that the melting point of aluminum is 660 degrees Celsius). This finding, which only became more pronounced for the larger models, has interesting implications for ways we can use language models to understand and augment student learning.

For instance, we could use large language models to proxy basic teacher behaviors and judgments (construct summaries of an article, set general learning goals, or generate sets of recall questions), but we may not expect it to properly represent aspects of the student learning experience such as mental exhaustion, rating the difficulty of recall-based questions, and simulations of whether new learning interventions improve understanding and recall.

Some students also struggle to understand what learning strategies improve their long-term understanding and recall [5]. If students and simulated subjects both lack an understanding of best practices for learning, we should not expect that giving large language models directly to students will automatically lead to better learning outcomes. Instead, we need to design tools that assist students in combining the strengths of language models, student agency, and well-established learning techniques to sustain students' learning in a way that's better than what any of the methods can accomplish alone.

  AI for Teacher-Independent Reading

Previous education tools were deeply tied to a specific text or a teacher's ability to author worked examples and branching rules. At best these tools could provide an adaptively paced learning experience, but the learning experience itself could not dramatically change in response to a student's interests or wells of deep knowledge outside of the covered domain. However new advances in general-purpose generative AI may be able to support open-ended student-driven learning, where a student decides what to learn and who to learn it from.

A student may choose to supplement their understanding by engaging with a technical reference text such as an online lecture or textbook. Such educational references are usually designed to flow logically and coherently to make facts and relationships evident to the student. But when the student tries to check their own understanding of the text, they might run into an issue. It is easy to mistake fluency, or the ability to follow the chain of logic outlined in the lesson materials, for deep understanding and retention. In classrooms, instructors avoid this problem by creating "active" learning aids such as note-taking guides, worksheets, and reading quizzes. However, building a successful aid is time-intensive and content-specific, and does not easily scale to all possible readings a student may choose to engage with. Modern AI does not have an instructor's expertise, but by guiding its generations with insights from learning science, we may be able to build aids that help students "stay with the difficulty" of learning and evaluate their knowledge on their own.

The learning science technique of "test-potentiated learning" interrupts a student's normal learning process by asking the student to attempt a set of comprehension exercises before seeing the reading [6]. This helps students prime their brains to pay more attention to specific gaps in their knowledge. By combining test-potentiated learning exercises with the benefits of online access, we create a learning environment that supports suspension: Each student can encounter questions, hints, and corrective feedback at their own pace, giving them the time and space to reconcile their pre-existing knowledge with new knowledge. One issue with test-potentiated learning is that students over attend to the facts covered by the exercise and give less attention to facts and complex interactions they are not primed to notice [7]. But now with modern generative AI, it may be significantly easier to author a high-coverage set of questions. We built an AI-supported experience that helped users retain facts and relationships from a biology textbook chapter by asking users to complete a set of fill-in-the-blank questions before starting the reading. These questions were generated by prompting a language model to role-play as a teacher, generate a summary of important facts from the chapter, and then judge the "most important" word to blank out for each fact.

During user testing, we encountered another issue arising from the test-then-teach lesson format—if students fail to retrieve the information needed to answer correctly, that practice is unlikely to improve memory [8] and understanding. Repeated failures due to an inability to remember or lack of prior knowledge can be so frustrating or perceived as unfair that students withdraw from the lesson. To give students the sustenance to stay with the difficulty, we consulted Slamucka and Fevreiski's theory on how generation failures interact with the testing effect and learning gain [9]. When a student struggles to correctly answer a question, it might be because they failed to process semantic features (the gist of the meaning or associated concepts) or because they failed to process lexical features (vocabulary terms or spelling). We used the language model to quickly generate a set of easy active learning exercises to target these failure modes (see Figure 1). Breaking vocabulary words into root parts could help students remember lexical features, matching a vocabulary word and a snappy catchphrase to definitions strengthens semantic recall strength, and spelling vocabulary words strengthens lexical recall strength. This tiered set of exercises gives novices a better chance at completing the fill-in-the-blank questions and absorbing more relationships and facts from the reading.

---

Generative AI does not have the ability to reason about the world, but students do.

---

A procedure for generating a variety of theoretically grounded active learning exercises without a teacher's direct effort has tremendous potential for empowering students to learn independently, rigorously, and from materials of their choosing. By leveraging this technology, students may be better equipped to personalize their educational journey or effectively engage in student-driven projects beyond the core classroom curriculum.

  AI for Student-Driven Projects

End-of-unit projects are common in many public high schools in the United States. These projects are seen as having lower stakes and higher engagement compared to tests. Instead of evaluating recall, student-driven projects let students showcase their technical skills, personal narratives, and problem-solving ability. A project provides the opportunity for students to connect skills learned in class to their out-of-class interests and real-world initiatives. High-quality projects can help students negotiate into jobs, internships, and college programs.

However, if projects are not properly scaffolded, they can turn into a "dessert."¹ Where iteratively developing new understanding and testing it against reality (the constraints of the physical world, the needs of external stakeholders, etc.) is secondary to creating a superficially appealing deliverable (learning from a textbook and displaying a poster on a classroom wall). Superficial "dessert" projects are common because high-school teachers struggle to mitigate students' organizational shortcomings without overshadowing their agency and motivation [10]. This problem is compounded by students' desire to explore ideas and emerging technology that high school teachers may not be familiar with.

As discussed in Kirchner et al., minimal guidance in discovery-based learning does not work [11]. Students who are not familiar with the domain they are trying to explore will struggle to search for information that could be useful for solving their problems. Traditional information-rich interaction modes like textbooks and search engines are not beginner friendly because they require the student to have a clear idea of the concept that they are looking for to search successfully and efficiently. Neuman found evidence that the linguistic and conceptual gaps between students' abstractions and those of systems designed for adults often prevented successful searching [12, 13].

Some students find mentors who can help them navigate the project-based learning landscape. However many students cannot access one-to-one personalized guidance at a time and place that works for their needs. The combination of metacognitive shortcomings, knowledge blind spots, and uncertainty can convince students to give up on their agency. My current research is focused on making student-driven project-based learning more feasible, equitable, and rewarding.

I am starting to build two interaction types: "Reflective Rubber Ducking" and "Filling in Knowledge Gaps." These interactions take students through the interruption, suspension, and sustenance learning processes, and will be intentionally designed to remind students they have the agency to make choices about what to learn, who to learn more from, and why.

"Reflective Rubber Ducking" is a conversational chatbot meant to help students define what they want from their projects by asking students project-specific, design-centric reflection questions (see Figure 2).

As an example, let's take a student with a deep interest in webcomics who has just been given an open-ended assignment to "visualize data" in their computer science class. Without reflecting, the student might aim for the low-hanging fruit: Download a dataset of webcomic transcripts or whatever is available from the internet and make some bar charts or "word clouds" to visualize patterns of the most frequent words. This type of project technically fulfills the goals of the assignment but has not helped the student see new possibilities of working and thinking.

In contrast, the "Reflective Rubber Ducking" conversational experience might interrupt a student's tendency to go for low-hanging fruit by asking the student to elaborate on why they chose webcomics as the focus of their project. If the student answers they find the webcomics personally meaningful or humorous, the chatbot might prompt them to think of ways they can design a data-driven user experience to highlight a meaningful or humorous aspect of the webcomic. The conversational chatbot supports suspension by coaching the student toward articulating the type of experience they want to create and how they want that change to be received by the real world. Then, as a student works to bring their student-driven project into the world, they will be more invested in not just learning the technical content to make their project successful, but also in investigating how that idea survives the constraints and demands of reality.

Large language models, while limited by their lack of lived experience, can still offer sustenance for students to build new understandings and plans. Their flexibility and fluency can boost students' ability to engage with the lived experiences and insights of other humans. The "Filling in Knowledge Gaps" interaction aims to help students elaborate on how their prior understanding and knowledge connect to new knowledge. Generative AI could reduce intimidation and increase curiosity by expressing dense technical facts and relationships as analogies (see Figure 3), silly poems, or active learning exercises (as mentioned previously).

  Concluding Thoughts

In their 2022 review of the art and practice of AI in education (AIED), Holmes and Tuomi criticized the limited support for student agency offered by the majority of AIED tools: "Personalisation, more broadly understood, is about subjectification and helping each individual student to achieve their own potential, to self-actualise, and to enhance their agency. This is something that few existing AIED tools do. Instead, while they provide adaptive pathways through the materials to be learned, most AIED tools have a tendency to drive the homogenisation of students. A critical interpretation of such AIED tools could suggest that they aim to ensure the students fit in the right box (pass their exams), prepared for their designated role in the world of work" [14]. While the previous crop of AIED tools aimed to democratize access to skills and practice problems, the next generation of AIED tools can be even more ambitious, empowering students to think critically about the goals of their education journey, the plans they make, and the freedoms they can encounter.

Inspired by my experiences with AI technology and one-on-one mentorship during my internship at Microsoft, I aim to build tools that support interruption, suspension, and sustenance in education. Supporting and evaluating new student-driven educational experiences at scale will present a unique set of risks and necessitate the development of new metrics, methodologies, and data sets to manage the complexity. However, I am optimistic that the growing ease of and enthusiasm for generative AI will foster community support for enhancing student-driven reflection and project-based learning. By leveraging these advancements, we can create a more personalized, meaningful, and empowering educational landscape for all students.

  References

[1] Biesta, G. Risking ourselves in education: Qualification, socialization, and subjectification revisited. Educational Theory 70, 1 (2020), 89-104.

[2] Bloom, B. S. Learning for mastery. Instruction and Curriculum. Regional Education Laboratory for the Carolinas and Virginia (RECLV). Topical Papers and Reprints, Number 1. Evaluation Comment 1, 2 (1968).

[3] Guskey, T. R. Lessons of mastery learning. Educational Leadership 68, 2 (2010), 52–57.

[4] Aher, G. V., Arriaga, R. I., and Kalai, A. T. Using large language models to simulate multiple humans and replicate human subject studies. In Proceedings of the 40^th International Conference on Machine Learning (ICML '23) Vol. 202. JMLR.org, 2023, 337–371.

[5] Schraw, G., Crippen, K. J., and Hartley, K. Promoting self-regulation in science education: Metacognition as part of a broader perspective on learning. Research in Science Education 36 (2006), 111–139.

[6] Arnold, K. M. and McDermott, K. B. Test-potentiated learning: Distinguishing between direct and indirect effects of tests. Journal of Experimental Psychology: Learning, Memory, and Cognition 39, 3 (2013), 940–945.

[7] Van Gog, T. and Sweller, J. Not new, but nearly forgotten: The testing effect decreases or even disappears as the complexity of learning materials increases. Educational Psychology Review 27 (2015), 247–264.

[8] Karpicke, J. D., Blunt, J. R., and Smith, M. A. Retrieval-based learning: positive effects of retrieval practice in elementary school children. Frontiers in Psychology 7 (2016).

[9] Slamucka, N. J. and Fevreiski, J. The generation effect when generation fails. Journal of Verbal Learning and Verbal Behavior 22, 2 (1983), 153-163.

[10] Barnes, J. L. and Bramley, S. A. Increasing high school student engagement in classroom activities by implementing real-world projects with choice, goals portfolios, and goals conferencing. (ED500846). ERIC, 2008; https://files.eric.ed.gov/fulltext/ED500846.pdf.

[11] Kirschner, P. A., Sweller, J., and Clark, R. E. Why minimal guidance during instruction does not work: An analysis of the failure of constructivist, discovery, problem-based, experiential, and inquiry-based teaching. Educational Psychologist 41, 2 (2006), 75–86.

[12] Neuman, D. Organizing information to facilitate its use: An exploratory study. Final Report to Montgomery County Public Schools. Rockville, MD: Montgomery County. 1991.

[13] Neuman, D. Designing databases as tools for higher-level learning: Insights from instructional systems design. Educational Technology Research and Development 41, 4 (1993), 25–46.

[14] Holmes, W., and Tuomi, I. State of the art and practice in AI in education. European Journal of Education 57, 4 (2022), 542–570.

  Author

Gati Aher is a Ph.D. candidate in the Machine Learning Department at Carnegie Mellon University, advised by Prof. Zachary Lipton. She has been developing student-driven project-based learning curricula since high school and at her undergraduate institution, Olin College of Engineering, which specializes in teaching engineering and computer science through project-based learning.

  Footnotes

1. https://www.pblworks.org/doing-project-vs-project-based-learning

  Figures

Figure 1. A set of test-potentiated learning exercises to prime students to understand and focus on the concept of "organelle" in a biology textbook reading.

Figure 2. Proof of concept: Lightly formatted excerpt from chatting with ChatGPT about a project idea, prompted with design-centric reflection and constructivist listening guidelines.

Figure 3. Proof of concept: Analogy generated using ChatGPT prompted with a multi-step prompt-chain.

 

This work is licensed under a Creative Commons Attribution International 4.0 License.

The Digital Library is published by the Association for Computing Machinery. Copyright © 2024 ACM, Inc.