Exploring How AI can Support Peer Assessment of Collaborative Learning

William R. Penuel

So many applications of AI in education focus on personalization and individualization of learning, but what if AI could help with one of the most important aspects of disciplinary learning today, namely helping students learn how to build knowledge effectively and equitably together? Standards like the Next Generation Science Standards call for students to be able to work together to pose and answer questions about phenomena and solve problems using science and engineering practices. These require that students work together to elicit the expertise, perspectives, and experiences of group members and construct arguments that build on the ideas of peers. While students may benefit from developing these capabilities with targeted individual support, these capabilities cannot be learned solely from interacting with a single AI assistant.

Sharing their ideas and building knowledge with others requires students to take social risks, risks that may be higher among students from marginalized groups and communities, that is, students from Black, Indigenous, and People of Color communities, students with disabilities, and non-binary gender, gender creative, and gender expansive students. It may be especially hard in classrooms where English is the dominant, if not exclusive language, and where many members of the classroom community are emergent multilingual students. Conversely, where young people do feel a sense of belonging, they are more likely to engage in academically productive talk.

CoBi: An AI Partner for Classroom Collaboration

In this session, two researchers from the Institute for Student AI-Teaming (iSAT), presented their work to develop “the Community Builder” (CoBi), an AI partner that supports more effective and equitable collaboration in STEM classrooms. This NSF-funded initiative is involved in a five-year, multidisciplinary effort to design, test, and implement an AI partner that can interact with real students in classrooms and give feedback to students on how well they are collaborating. 

Participants in the session (state science leaders and professional development leaders from across the US) shared their views on some key challenges to facilitating small group participation. They named a number of issues teachers face when orchestrating collaborative learning that the iSAT team is taking up: students being hesitant to contribute; one student dominating the group; managing multiple groups; students with low social status’ ideas being ignored, even when they can be useful. 

The participants heard how young people contributed ideas to the development of CoBi.  iSAT worked with young people over a series of long workshops to elicit their ideas for collaboration and how to improve it in schools. These Learning Futures Workshops asked the question: What kinds of features of collaboration could help you learn, and what might an AI tool do to help you collaborate more effectively? Through these Learning Futures Workshops, the iSAT team got the idea for an AI partner that could function a lot like a community builder within one of the housing co-ops they visited as part of one of the workshops. The researchers also learned a number of valuable lessons about how to support students in designing for forms of collaboration they’d never actually experienced in schools.

CoBi is not a “standalone AI” tool, but is intended to help students and teachers reflect on the quality of their collaboration. Whenever students start using CoBi or look at data from CoBI, they are either establishing or revisiting community agreements. There are specific community agreement teaching routines used with CoBi–a set of instructional practices that iSAT developed with partner teachers to establish and revisit these agreements throughout the implementation of iSAT curriculum units–which include a unit that teaches how AI is used in moderating online gaming communities.

In the establishing agreements routine, students brainstorm ideas for how to collaborate with one another in expansive and applied ways. Then the teacher facilitates a discussion where they come up with agreed upon classwide agreements to be presented with CoBi.  Throughout the curriculum, CoBi helps the teacher revisit these agreements through observation, supporting reflection, and presenting noticings from students. There are even opportunities for students to think critically about how CoBi works and to apply AI-learning in ways that allow them to be critical thinkers about this classroom tool. 

Classroom and lab studies are ongoing with CoBi, but CoBi offers a different vision for how AI can support peer and self-assessment and collaboration from typical AI tools, one oriented toward supporting learner flourishing rather than optimization and efficiency. 

Discussion Prompt:

What other possibilities for AI can you imagine, after hearing about CoBi?

How else might AI support peer assessment of collaborative learning in STEM?