We would be happy to share the pre/post survey questions with you if you'd like to share your email. 

Qualitatively? We asked teachers and coders to think of one student who "stood out" to them. We got back AMAZING and touching responses regarding the shy students, students with major or multiple disabilities, students who had major behavior issues that largely disappeared while working on these lessons, and those that established a positive classroom identity (either socially or as a source of help and expertise) with their peers (or even teacher) for the first time.

Quantitatively and qualitatively?
The lessons come with exit tickets completed by students that give educators and coders a chance to see if kids are particularly excited by (or confused) by that day's content. In future implementations, it will be helpful to encourage adults to review those exit tickets as they can be quite helpful to jumping into the next lesson! We are analyzing those now to report as outcomes of the randomized control study we conducted.

As Heidi alluded, we also administered student (and adult!) surveys targeting knowledge, interest, and self-efficacy regarding problem solving, collaboration, coding, robotics and computer science and student pre & post tests to reflect changes in knowledge and persistence. Analyses are underway, but we are excited to see that we are definitely opening many kids' eyes to content and careers they hadn't considered or were scared of. We're also inspired that MANY students want to do more coding (Gigabots and other types) and that the students who participated showed small, but statistically significant improvements in knowledge and self-efficacy after just six lessons.

This was a district partnership grant where external measures collected by the district were important and required. We hope that somehow we've effected positive change in those very distal measures. In the meantime, we are happy to move the needle on the more proximal measures.

As we are all adjusting to the affects of the pandemic, have you thought about using virtual visits to solve you challenge of having coders visit the classroom once a week? If so, can you share your plan for implementation, I'm sure many of us are grappling with the same issues.

We are working our way through so much data, it will be fascinating to delve into questions like these. Thanks for raising now and maybe setting us off on that path! (In other words, we don't know yet!)

An important thing to note is that we asked teachers many questions about their enthusiasm, expectations, and confidence in regards to the program and found positive change from pre to post. Their responses revealed the program has promise for sustainability and expansion/adaptation. Both coders and teachers were enthusiastic about recommending the CIR program to colleagues in and out of their organization/district and felt comfortable with repeating the experience; most teachers even said they were comfortable teaching a couple lessons on their own (though it was clear from their surveys that they greatly appreciated, felt reassured by and leaned heavily on the visiting coders).

Teachers’ ratings of how the lessons affected student perceptions and the students themselves increased dramatically from pre to post. I think teachers who are in the room as students engage for the first time, or get to be the expert for once, or express newfound confidence can't help but be more committed to and excited about the program. (Indeed, many asked to do it again and, unprompted, indicated they'd try to incorporate more coding and collaborative project-based learning into their classes).

It would be so interesting if you could follow some of your teachers for a couple of years to see whether and how they incorporated some of what they gained from their experiences into their instruction. Perhaps a purposefully varied subset of teachers based on characteristics that seem predictive of particular outcomes. It sounds like you're already awash in data so more isn't always better, but good questions always raise more questions!

I think this is a great idea! We'll try to incorporate it into our plans! Thanks.

I'd love to hear what you find out, Mari.

Thanks, Alison --
Coders were recruited through personal networks, posts to websites, and word-of-mouth. Some were repeat coders from the pilot project. For scheduling reasons / distance reasons, some coders worked with multiple classes. (Again, a large argument for incorporating virtual coding visits!) Lane Educational Service District has a number of really powerful community outreach and "inreach" initiatives that are trying to connect schools and industry, fill the STEM pipeline, provide really strong career/technical experiences, etc. This program was a way to bring new companies, organizations, and self-employed folks to the table.

One of the coders helped develop and revise the lessons and train teachers and coders. The program is designed to be co-taught, but coder/teacher pairs took every type of approach imaginable due to confidence, personalities, time, etc. Though we haven't yet analyzed that data, we did ask pairs to document who taught each lesson to help us better quantify who was teaching, modeling, assisting students, etc. In the ideal world, we'd have much more time for interaction, training and practice before the pairs work with students (remember, these coders are not classroom teachers just as much as most teachers are not computer science experts!).

Thank you! It's definitely a great place to start. The software that the LEGO robots use is mostly drag-and-drop commands, so kids can get a sense of how code is executed at a basic level.