Taking Feedback and Avoiding AI Bias: An Interview with Carlo Schmidt (Part 3)

Welcome back to Formative’s four-part interview series with Director of Product Carlo Schmidt on how Formative is integrating AI into the platform.

Part 2 looked at Formative’s philosophy with regard to how AI can - and should! - benefit educators and learners. In Part 3, Carlo talks about the role user feedback played in shaping AI integration, and he addresses some of the recent controversies involving AI’s role in contemporary culture.

This interview has been edited and condensed for length and clarity.

Formative Team: You and the Product Team were working with a user-feedback group. How has that experience shaped the AI work?

Carlo Schmidt: Talking to our teachers and students: that dictates 99% of our work at Formative. I was at a school last week, and I sat down with a science teacher. The AI was open on my computer screen, and I said, “Let's work together to see if we can get this thing to generate something that's meaningful for you.” He was looking to just basically create a quick "Do Now" for his class: like three quick questions on a concept they covered yesterday. I sat with him for about forty-five minutes, and we were just exploring: we're having a good time. We had ChatGPT open. We asked it, “All right, generate three questions on this concept.” And the teacher said, “Wow, those question are really good; they’d be even better if it said this.” So we're slowly figuring out how to ask the AI the question in a way that yields the best results for   teachers. Anytime we've shown it to teachers or students, typically the response has been, “This is incredibly powerful.” There's this “Aha!” moment.

Formative Team: That sort of reaction must be incredibly motivating for the work you're doing.

Carlo Schmidt: Well, we're moving into a new phase of how you can work with students in the classroom. For so long, teachers have just spent so much time and been so focused on content creation or grading papers that they see the AI doing that, and they think, ”70% of my job is gone. What am I gonna do?” It's understandable, but it's also somewhat of a bummer to hear them say that.

But imagine them spending 70% more of their time working with [students] in class. That's the mental shift. We're trying to unlock that potential of AI and EdTech so that the teacher is free to focus more on the individual student.

Formative Team: It feels like the conversation around AI has come to some sort of fever pitch. Like the Bing Sydney conversation that worried everyone.

Carlo Schmidt: Where it sounded sentient? And said it had a dark self?

Formative Team: Yes, and it started confessing its love to the interviewer! Or even these reports of people asking the AI for questions and getting biased responses that tend to reflect a certain limited worldview. How does talking about these problems shape the work you're doing?

Carlo Schmidt: I love that people are talking about that. Talking about bias within the context of AI feels as if we've reached a point in the country where it's okay to talk about those things. It may be the stepping stone of saying. “Yeah, it's biased. It's biased because we're biased. It learned from us. So how do we talk about it? And how can we all do better at removing our own biases from our classrooms?” 

My hope is, maybe it'll be a turning point. It's a lifelong endeavor to understand how you are biased in the work you're doing, and to identify those things for ourselves and sort of filter them - or vet them before we give them to our students - so that our students are getting as content that's as unbiased as humanly possible.

At Formative, we know we can't do all the vetting on our own. But we've spent a lot of time setting up guardrails on the technology. That way, we can minimize more of the exposure to bias that teachers and students will encounter when they're using AI on the platform.

Join us next week for Part 4 of this interview, where Carlo will speculate on what the future holds for Formative’s AI integrations.