This technology is incredible. Would it be possible to use this technology in something like more advanced math classes and alert the teacher that many of the students are showing signs of confusion/ disengagement? I could see real value in teachers getting notified (especially if it was possible) in real-time that they've lost the class, and need to stop and answer/questions or clarify a concept before continuing a lesson.

Thanks for your question! The system we trained is specific to pre-school classrooms. However, the same machine learning methods could likely be applied for a wide range of classroom contexts and target variables (including confusion and disengagement). What would be needed is a dataset of hundreds of classrooms and associated target values, which is the same order of magnitude of training data we had for the pre-school classes.

I love the idea of using technology to help guide teachers in producing better student outcomes!  How have you planned to recruit teachers to use this?  

Do you have plans to integrate this technology to grades beyond pre-school classrooms?

For the past several months we have been developing an experimental protocol to explore how automated classroom feedback, as provided by our machine learning-based analysis system, can help teachers to become more perceptive of classroom interpersonal dynamics. We will begin the experiment with classroom teachers this summer. After a pilot run of our study, we will recruit participants more broadly. Since the experiment will be held purely online, we hope to cast a wide net for recruitment. Thanks for your interest!

So far we have focused mostly on pre-school classrooms, but we have also conducted machine learning analyses on elementary and middle-school students. While each new population of students requires its own dataset for training, the general machine learning architecture that we use can likely generalize to a wide variety of students and learning contexts.

This is a fascinating tool for teachers! I'd be interested in learning about the findings from the classroom intervention.  The possibilities for expanding this beyond preschools is noted above. I think there is also potential for applying this tool with learning of students from underrepresented groups in STEM. Great work!

Really a fascinating cyberlearning project! 

Love it! I can't wait to see how this plays out and where it leads you all with the research and implementation.

Wonderful video and project.  It's very intriguing and inspiring. What have you found to be your biggest challenges?

One of the fun and challenging aspects of the project has been bridging between our disciplines: education, learning science, and computer science. On the artificial intelligence and machine learning side, there is no dearth to computational challenges of the automatic perception of school classrooms! These include background noise, overlapping speech, uncontrolled lighting, people moving in and out of the scene...not to mention the semantically very high-level construct we are trying to estimate with our system.

This is a really interesting and exciting project! I'm curious to hear more about classroom teachers' reactions to this technology. Do they tend to be intrigued and interested by the idea of receiving feedback from an AI system? Or skeptical? I think it will be fascinating to see how the system's perception of classroom dynamics supports, or differs from, that of participating teachers. Can't wait to hear more!

I wondered if you feel you have achieved the goals you set out to with your project.

Fantastic... your data can give strong examples and evidence for the range of interpersonal actions teachers demonstrate.. both consciously and subconsciously. I am glad to hear that new teacher training is in development.  

What great work.  These real-time feedback classroom detection systems have the potential to be great dashboard tools for teachers.  In our work with the JSMF Teaches as Learners program, we have funded work that may be of interest to this project:
Please check out Amy Ogan's lab at CMU and the work they are doing too: https://www.amyogan.com