A recent Hackathon, hosted by /dev/color, brought together selected organizations to explore Stanford Ideal Learning Lab’s Science Tutor for Effective Problem Solving (STEPS), an AI-driven pedagogic agent designed to help undergraduate STEM students strengthen their problem-solving processes.

We represented the ETS Research Institute to examine how measurement science can be integrated into AI tutoring environments. Because STEPS already functions as a formative assessment platform, the hackathon offered us a timely opportunity to examine how evidence of student learning can be used in real time to personalize AI tutoring.

Our primary deliverable was a comprehensive architectural mapping of an end-to-end, closed-loop learner model pipeline that could be embedded within the STEPS platform. We developed this architecture through collaboration between our ETS team and /dev/color. In our proposed design, we leverage multiple sources of evidence, including measures from students' use of STEPS, diagnostic assessment results, and self-reports to inform personalized learning. These data streams are then synthesized into learner profiles that can guide the tutoring experience. Specifically, we translated quantitative measures, such as scores or patterns of student-chatbot interactions, into pedagogical instructions for the AI tutor. These determine how much support to provide and when to prompt reflection. These pedagogical instructions are incorporated into the prompts used by the generative AI tutoring system within STEPS, allowing the tutor to adapt its support based on evidence about each learner’s needs to create a more personalized learning experience. Our collaboration with /dev/color helped make sure the approach was grounded in research and practical to implement.

Based on the positive feedback we received from the director of Stanford’s Ideal Learning Lab, Dr. Shima Salehi, and other hackathon participants, we identified several opportunities for ETS:

• Measurement as an enabler of adaptive AI: As AI tutoring systems mature, their creators are actively looking for systematic, evidence-based methods to model student learning progressions, non-cognitive traits and durable skills. ETS is uniquely positioned to fill this gap.
• The value of our tools: Integrating ETS measurement research and capabilities into AI tutoring systems demonstrates a clear pathway for our research to power next-generation learning technologies.
• A model for future partnerships: The hackathon provides a strong conceptual blueprint for how ETS can partner with top-tier academic research labs to provide the underlying measurement infrastructure for their educational platforms.

By combining our expertise at ETS with emerging AI technology, we can move toward a more responsive, understanding system of support for every learner.