The ability to understand, analyze, and effectively collaborate with artificial intelligence (AI) technology has become more important than ever before. In a society increasingly surrounded by AI technology, those educated to use these tools effectively and ethically will be more productive and better prepared for the future workforce [1]. AI education in K-12 settings has increasingly emphasized promoting AI competencies among children, broadening participation, and developing foundational skills and knowledge. However, a significant gap remains in understanding AI, specifically AI competencies among young learners [2]. This gap is especially noticeable among marginalized students, including those from low socioeconomic backgrounds, English language learners (ELLs), and students with special needs [3].

In our previous study with children from low-income Latino communities, where the majority were multilingual ELLs speaking Spanish as their primary language, we discovered these children had little to no experience with or understanding of AI (e.g., distinguishing AI technology from non-AI technology, and using AI in their daily lives). This highlighted the need to focus on teaching AI technology and developing AI literacy among these students. This informs our current study, which provides access to AI technology and emphasizes the importance of AI literacy. To promote AI literacy among marginalized youth, we integrate storytelling through creative writing activities, drawing on Han and Cai's [4] work on leveraging AI features and functions to enhance story creation and facilitate AI literacy through intelligent digital story writing (DSW).

  Enhance AI Literacy Through AI-Based Story Creation

Digital storytelling enhances interactive engagement through personalized learning experiences with multimedia support combining text, images, and videos [5]. Integrating language, visual, and digital representations is a potent cognitive development tool, enhancing students' ability to integrate images, sound, and editing techniques to convey causality and progression between narratives. Throughout the DSW experience, students can apply their prior knowledge, investigate, reflect on their daily experiences [6], and ultimately, explain and construct a complete narrative [6]. Thus, DSW tools have been incorporated into classrooms to help students learn new concepts [4]. In a 2022 study, Ng et al. leverage DSW as a pedagogy to introduce the concept of AI to primary school students [7]. Their findings revealed participating students could propose authentic narratives, apply their newly acquired AI knowledge, and develop meaningful AI-driven solutions in their digital stories. Overall, the study showed DSW as an inquiry-based approach effectively enhanced students' AI literacy, enabling students to utilize and apply AI knowledge to solve real-world problems, far beyond merely understanding related concepts. The authors proposed employing DSW as a pedagogical tool, which holds significant potential, to support and scaffold AI understanding, particularly among young children.

AI literacy encompasses a set of competencies that empower individuals to critically assess AI technologies, communicate and collaborate effectively with AI systems, and utilize AI as a functional tool [8]. Long and Magerko introduced a comprehensive list of AI literacy competencies and design guidelines specifically for AI education in K-12 settings [8]. Their study focused on making AI concepts accessible to non-technical learners by synthesizing a range of literature through a scoping review process. This includes teaching students to recognize AI and understand intelligence by distinguishing between technological artifacts that use AI and those that do not, critically analyzing the features that confer "intelligence" to an entity, and discussing the differences between human and machine intelligence.

---

The AI for K-12 guidelines will define what students in each grade level should know about AI, machine learning, and robotics.

---

Ng et al. highlighted that one of the primary objectives of AI literacy education in primary schools is to familiarize children with basic AI and computer science concepts [1]. They examined pedagogical approaches to AI literacy and developed a coding framework related to Bloom's taxonomy. According to Bloom's taxonomy, there are six levels of cognitive complexity, each requiring a higher degree of ordered thinking from students (i.e., know, understand, apply, analyze, evaluate, and create) [9]. The adoption of Bloom's taxonomy is justified by the novelty of AI literacy to educators and the lack of an established classification of cognitive processes in the context of AI learning [1, 10]. Touretzky et al. introduced AI for K-12 guidelines that outline what students in each grade should know about AI, machine learning, and robotics [10]. The five "big ideas" introduced were:

1. Computers perceive the world using sensors.
2. Agents maintain models/representations of the world and use them for reasoning.
3. Computers can learn from data.
4. Making agents interact comfortably with humans is a substantial challenge for AI developers.
5. AI applications can impact society in both positive and negative ways.

  Developing the Story-Bot

Our project utilized intelligent DSW activities to enhance engagement and broaden participation in AI education through multimedia engagement and AI methodologies (i.e., image, video, text-to-speech, speech-to-text, and translation). Utilizing generative AI (GenAI), we developed an intelligent bot designed to introduce fundamental AI concepts and facilitate story-writing activities with students. This approach aims to scaffold learning and simplify abstract AI concepts for students. Through storytelling, students explore new concepts in their own language (both reading and writing) by interacting with AI agents specifically crafted to introduce and create narratives around AI topics.

Through a series of prompts leveraging GenAI and GPT-3.5, the storybot was designed by creating a set of JSON-formatted prompts that combine rule-based and LLM-based interactions. We developed an AI lesson using rule-based interactions by providing educational resources (e.g., videos and text). The LLM-based interaction comes into play when students work on writing, with the bot offering feedback, suggestions, and ideas within the topic of AI through a student-AI collaborative writing activity.

The story-bot includes text-based chatbots that guide students through the writing process, including narrative writing about AI (see Figure 1). The current version introduces an AI concept through multimodal educational material (i.e., video and text-to-speech reading materials), once students engage with the lessons, students will build a story with an AI agent from topics around AI:

• Option 1. The Helpful Robot
• Option 2. The Singing Computer
• Option 3. The Magic Paintbrush
• Option 4. The Friendly Drone
• Option 5. The Talking Teddy Bear

Students will select a topic and create a story. After completing their story, they will explain AI concepts and their knowledge about AI to the bot. This process incorporates stealth assessment, as it requires students to teach the bot what they know about AI, positioning them as instructors to reinforce their understanding.

We scaffold the learning activity into four stages (see Figure 2). To start, students are introduced to the basic definition of AI through video tutorials and text descriptions. Then they read and synthesize the lessons, an AI bot will initiate a storytelling activity for students. Next, students build a story by collaborating with an AI bot on a topic of their choice. When students finalize their story, they are asked to answer questions driven by an AI bot (i.e., "Can you explain what AI is?"). As we are in the early stages of designing and developing the platform, we aim to optimize the balance between rule-based and LLM-based interactions by collaborating with educational stakeholders (e.g., teachers and students) to understand their unique needs and capacities.

  Methods

We conducted a pilot study at a community center in Southern California. Most participants were Latino students who were bilingual in Spanish and English. Over three days, the students interacted with the chatbot and engaged in story creation using the bot we developed. The study was part of a longer one-week "writing with AI" workshop at the community center's summer camp. For our study, students were asked to engage with AIstory-bot for two days, one hour per day. We conducted a user test study with 12 students ages 8 to 12 to understand AIstory-bot's applicability, affordances, and limitations in learning scenarios. Pre- and post-surveys were conducted to measure the students' efficiency, motivation, and AI knowledge. Additionally, we screen recorded students' writing processes, collecting writing outputs and student-AI utterances. From the data collection, we aimed to uncover the following research questions:

• In what ways does AIstory-bot help students enhance their efficacy and motivation over AI literacy?
• How do students perceive AIstory-bot for their learning?

We approached the first question to uncover the differences between before and after the AIstory-bot experiences, and we conducted semi-structured interviews to understand students' opinions about their experience with AIstory-bot.

On the first day, students conducted a pre-survey test, which took around 20 minutes to finish. The students used the AIstory-bot to learn AI concepts and write narratives about AI. The writing activity lasted around 30 minutes on average per student and there were 10-minute individual, semi-structured interviews for the remaining time; two on-site researchers recorded the interviews. On the second day, students focused on writing with AIstory-bot around AI concepts (see Figure 3). On the third day, students tried Story-AI to finalize their stories for the first 30 minutes and students took post-survey questionnaires for 20 minutes. We took the remaining 10 minutes to discuss what they liked and disliked about using AIstory-bot for their writing and how they would improve StoryAI. When students were finished writing, they spent their time drawing "how AIstory-bot might look like" on paper.

  Findings

The pre- and post-tests were designed to examine writing motivation that focused on AI literacy, specifically, how students' understanding (what is AI?), confidence (confidence in using AI), and motivation (I want to use AI for my writing) fared against their current knowledge and experience with AI (do you have any experience with AI on a daily basis?) In the post-test, we added questions regarding their experience in writing with AIstory-bot, and their opinion focused on the usability of their experiences with the platform.

---

We developed an intelligent bot designed to introduce fundamental AI concepts and facilitate story-writing activities with students.

---

Our preliminary findings from the study showed an increase in AI literacy among children, specifically AI awareness (i.e., what is AI), motivation (i.e., I'd like to learn more about AI), and confidence (i.e., I know AI better than before) from the pre-and post-survey around the students' perception toward their AI efficacy.

  Next Steps

We are currently analyzing the data collected from the summer camp. Our goal is to determine whether students' efficacy and motivation have changed through their experiences with the AIstory-bot. Additionally, we will assess the usability of the AIstory-bot, focusing on ease of use and enjoyment. We will also examine students' writing output and screen recordings to understand their strategies and learning outcomes from story writing and AI literacy perspectives. The video data will provide deeper insights into students' engagement and learning areas throughout their experiences.

We will also update the lesson stage in response to the current AIstorybot's lack of a comprehensive curriculum and instructional design. Our next steps involve developing lesson plans based on formative studies of existing K-12 and higher education AI practices and research. To address these goals, we will integrate formal curricula developed by researchers and educators—such as the Association for the Advancement of Artificial Intelligence (AAAI) and the Computer Science Teachers Association (CSTA). Inspired by CSTA's national standards for K-12 computing education, the AI for K-12 guidelines will define what students in each grade level should know about AI, machine learning, and robotics. Also, as a pedagogical strategy, we will incorporate Bloom's taxonomy into the curriculum along six levels of cognitive complexity: know, understand, apply, analyze, evaluate, and create. The adoption of Bloom's taxonomy can be beneficial with the emergence of AI literacy for K-12 classroom teachers and the lack of an established framework for categorizing cognitive processes in the context of AI literacy for K-12 education.

Utilizing insights from these formative studies, we will conduct participatory sessions with K-12 educators, particularly those specializing in computational thinking education, to co-create AI literacy lesson plans tailored to their students' unique capabilities. Designing and implementing new technologies in educational settings is a complex task that requires an in-depth understanding of the cultural, social, and pedagogical aspects of learning environments. Given that learning processes are complex—and education is deeply rooted in the quality of pedagogies, not just the technical innovations—we have found it is also important to collaborate with stakeholders in education (not only students) to examine how AI can impact students' experiences, efficacy, confidence, and motivation in real life. Specifically, we intend to include educators' insights that inform our understanding of the current educational context and help shape our approach to designing an AI platform that aligns with their goals and practices.

---

Our project utilized intelligent DSW activities to enhance engagement and broaden participation in AI education.

---

We will incorporate these lessons into the chatbot and subsequently conduct user studies to systematically evaluate the efficacy of the chatbot system in teaching AI concepts and enhancing participation in AI education. This evaluation will include pre- and post-tests to measure students' understanding of AI and monitor their learning progress. Additionally, we will perform randomized controlled trials comparing students' AI literacy development with and without the AIstory-bot module.

We will iteratively update the lessons, features, and functions through a feedback loop and user studies. This continuous improvement process aims to enhance the activities for teachers and students, thereby broadening participation and engagement in learning AI concepts through interactive digital storytelling with an AI agent.

  References

[1] Ng, D. T. et al. Conceptualizing ai literacy: An exploratory review. Computers and Education: Artificial Intelligence 2 (2021), 100041; :http://dx.doi.org/10.1016/j.caeai.2021.100041.

[2] Han, A. et al. Teachers, parents, and students' perspectives on integrating generative ai into elementary literacy education. In Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI '24). ACM, New York, 2024, 1–17;https://doi.org/10.1145/3613904.3642438.

[3] Song, Y. et al. A framework for inclusive AI learning design for diverse learners. Computers and Education: Artificial Intelligence 6 (2024), 100212; http://dx.doi.org/10.1016/j.caeai.2024.100212.

[4] Han. A. and Cai, Z. Design implications of generative AI systems for visual storytelling for young learners. In Proceedings of the 22^nd Annual ACM Interaction Design and Children Conference (IDC '23). ACM, New York, 2023, 470–474; https://doi.org/10.1145/3585088.3593867.

[5] Robin, B. R. Digital storytelling: A powerful technology tool for the 21^st century classroom. Theory Into Practice 47, 3 (2008), 220–228; http://dx.doi.org/10.1080/00405840802153916.

[6] Barber, J. F. Digital storytelling: New opportunities for humanities scholarship and pedagogy. Cogent Arts & Humanities 3, 1 (2016), 1181037;http://dx.doi.org/10.1080/23311983.2016.1181037.

[7] Ng, D. T. et al. Using digital story writing as a pedagogy to develop AI literacy among primary students. Computers and Education: Artificial Intelligence 3 (2022), 100054; http://dx.doi.org/10.1016/j.caeai.2022.100054.

[8] Long, D. and Magerko, B. What is AI literacy? Competencies and design considerations. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI '20). Association for Computing Machinery, New York, 2020, 1–16; https://doi.org/10.1145/3313831.3376727.

[9] Krathwohl, D. R. 2002. A revision of Bloom's Taxonomy: An overview. Theory Into Practice 41, 4 (2002), 212–218; http://dx.doi.org/10.1207/s15430421tip4104_2.

[10] Touretzky, D. et al. Envisioning AI for K-12: What should every child know about AI? Proceedings of the AAAI Conference on Artificial Intelligence 33, 01: AAAI-19, IAAI-19, EAAI-20 (2019); https://doi.org/10.1609/aaai.v33i01.33019795.

  Authors

Ariel Han is an incoming postdoctoral scholar at the University of Southern California. Han holds a doctorate degree in informatics from the University of California, Irvine. Her research interests focus on the intersection of AI, educational technology, learning sciences, and HCI. Han investigates fundamental questions on how people interact and collaborate with AI-powered tools and agents to empower teaching and learning. She is an advocate of playful learning and passionate about equitable and accessible digital space for young people to explore their creative potential.

Shenshen Han is a Ph.D. student in informatics at UC Irvine. He is an experienced software developer who has been working as a system developer with various platforms and frameworks such as web, mobile, embedded and XR. Han has 10-plus years' experience of developing professional and academic projects with Java, Python, JavaScript and Arduino C. His research interest is in computing education.

  Figures

Figure 1. Student_01 (Steve, a pseudonym) was asked to create a story using AIstory-bot. This screenshot shows the student's interaction with AIstory-bot exploring narrative writing.

Figure 2. The process of AIstory-bot experiences.

Figure 3. Student_02 (Emma, a pseudonym) was asked to create a story using AIstory-bot. This screenshot shows the student's interaction with AIstory-bot to write a story around AI concepts.

 

This work is licensed under a Creative Commons Attribution International 4.0 License.

The Digital Library is published by the Association for Computing Machinery. Copyright © 2024 ACM, Inc.