NSF Awards: 1516541, 1515788, 1515771
In our Research in Service to Practice project, we investigate ways that the design of activities and family conversations can make tinkering engineering-rich. We find that there is more talk about engineering when families are engaged in a design challenge for tinkering, such as to make something that rolls, or make something that flies. We also study conversations in which children and their caregivers reflect on their experiences during and after tinkering. We are learning that the design of tinkering activities can boost the amount of engineering and other STEM-related talk among children and their caregivers as they reflect on their experiences, and that this is important in advancing engineering learning opportunities for children.
Catherine Haden
Professor
In keeping with the theme for STEM for All 2020, we are pleased to share what we are Learning from Research and Practice. Our project is a collaboration between researchers from Loyola University Chicago and Northwestern University and practitioners from Chicago Children's Museum. Together, we focus on how encouraging children and their caregivers to engage in reflection during and after tinkering experiences can deepen young children’s informal engineering learning.
When do you see reflection during STEM activities? How might you encourage children to engage in reflection after STEM-related experiences?
Our results show that the design of programs and exhibit spaces, and strategies for facilitation by museum staff can encourage engagement in engineering, and support STEM-rich reflection during and after tinkering.
Sasha Palmquist
Ateng' Ogwel
Marion Usselman
Thanks for the interesting video. Families are so important in promoting STEM learning, and promoting reflection in the museum environment is a powerful approach. Have you looked at any cross-cultural differences in the nature and degree of family engagement and reflection?
Catherine Haden
Professor
Thanks for this great question!
There is a growing literature examining cultural differences in parent-child conversational reflection about past experiences (i.e., reminiscing) - not specific to informal learning or STEM experiences - that shows parent-child reminiscing/reflection is less frequent in East Asian families than European Am families, and that the content of talk may vary across different cultural communities. Nevertheless, in the prior studies on reminiscing, and also in informal STEM education research, there is a paucity of studies of STEM engagement and reflection among Latinx families –the cultural community we have been focusing on in our project at Chicago Children’s Museum.
We are learning that Latinx families talk less about engineering during tinkering when the parents had fewer than 12 years of schooling and received the orientation. In contrast, Latinx parents with higher levels of schooling who also received a facilitated orientation from a museum staff talked the most about engineering during tinkering. The children's engineering talk when reflecting about the experience immediately afterward did not differ as a function of whether or not they received the facilitated orientation about engineering.
We are digging in deeper to understand these initial findings. We added to our sample of Latinx families this past summer, and will be able to say more soon about the ways that facilitation and reflection can support engineering learning through tinkering for Latinx families.
Sasha Palmquist
Diley Hernandez
Hi Catherine, I wonder if you have you looked at the role of language as a mediator in this relationship. Have you compared families speaking in Spanish versus English? I imagine that if the language used by the staff conducting the orientation was English this might impact the engagement of Latinx families in which Spanish is the home language.
Diana Acosta
Doctoral Student
Hi Diley, thank you for your question!
Most of our sample of Latinx families were bilingual. We made sure to have English-Spanish bilingual museum staff available to provide the facilitated orientations (also, all the exhibit signage is in English and Spanish). Families chose which language they preferred for the orientation. Anecdotally, we saw some families pick one language but then ask the museum facilitator to switch to the other language. For instance, sometimes the orientation began in Spanish but then switched to English at the request of the parent because their child understood English better.
Jennifer Kidd
Hi Catherine,
I've come to check out your project after you checked out mine. Clever how they designed this! I love the insights you've shared about reflection - the idea that families reflect more when given guidance, when they have their object in front of them, and when they can test their designs. You asked about refection in our project. We do ask our college students to reflect (in writing) at a number of places in their projects (e.g. after a "dress rehearsal" of their lesson, after the deliver their lesson). Our students and the elementary students also reflect orally as they engage in the engineering design process, which is I believe what you are investigating. I have found it challenging to figure out how to structure the test, reflect, redesign, re-test process. When students design and test only to discover their artifact does not work, they immediately want to redesign and retest. They have motivation and inertia to continue the EDP. Pausing to ask them to reflection as a whole group can be tricky. They are distracted by desire to "fix" their solutions right away. On the other hand, when students are successful on their first attempt, they often feel like they are "done" and are also not as motivated to reflect. I believe we can leverage group dynamics here a bit. If students design independently and then test together, they might be more motivated to reflect together and then redesign as a team. In any case, I think this is a very rich area of research and I'm curious to hear what you have learned.
Ateng' Ogwel
Sasha Palmquist
Catherine Haden
Professor
Thanks for your thoughts. In our work, reflection is often part of a collaborative process. For example, the child tries something out and announces it doesn't work, or didn't work the way they wanted it to, and the caregiver might ask "why do you think it isn't working?" and that launches them into a reflective conversation. I can see ways to "stop the action" after testing to reflect being challenging, but potentially important in your setting. One thing we have tried to do to avoid the "one and done" is to have a secondary challenge at the ready. For example, if they are making a slide, adding a complication after the first version is completed, to have to make the slide go around an obstacle.
Zenon Borys
I love the presentation! and completely agree that tinkering coupled with reflection is a key step in STEM learning. The connections I'm making to this video are that your findings seem to span beyond engineering. We had similar goals for a course we taught this semester focused on integrating technology and STEM. We often used the design cycle to engage our students in this tinker, reflect, refine, repeat process and it led to richer ideas and discussions. I also try to point out to the math teachers I work with that whenever we have students "guess and check" we're having them tinker. And for some reason, the language supporting tinkering opens more doors than the way we often talk about it in math ed. It makes me wonder (and hope) that if having rich experiences as the students you showcased tinkering better equips them to tinker in other domains.
Ateng' Ogwel
Catherine Haden
Professor
Really appreciate your thoughts! I think the thinking skills involved in tinkering have the potential to be domain general learning mechanisms - as you suggest! The ideas of trying something, evaluating progress toward a goal, and seeking more information - all are ways to engage in a deep dive into learning anything. I like the idea that people who are, or get comfortable with tinkering can excel across domains with this approach to learning.
Eric Hamilton
Catherine and all,
It is nice to see a project from the land of the Ramblers :) where I held my first faculty position in some century other than the current one. I think one thing that is helpful about the video - the focus on reflection - is something we should promote a bit more in our IC4 project also appearing in this video town hall. It is a little trickier for us, since everything is virtual. Our students interact only over the web (even pre-Covid) (with some in-place exceptions), so I think we need to be a little more purposeful of the reflections over their collaboration periods. Do you have a way to assess or measure the quality of the reflective process?
Catherine Haden
Professor
Hi Eric - nice that we have the Rambler connection! Yes! I've thought a lot about the quality of reflection, both in my earlier work on parent-child reminiscing about past events, and now as we look at reflection as part of the STEM learning process. We look at quality in terms of "elaborativeness" - how detailed the reflection is, and in parent-child reflections, how collaborative and joint the reflection is. For example, when the parent or child asks a question in the reflection, how is that question responded to by the conversational partner. More joint, collaborative reflections can lead to richer conversations that can advance learning. We also look at the content of the reflections. In this work on engineering learning we code for the extent to which families are talking about different parts of the engineering design process in their reflections. I'm also very interested in ways we can boost the quality of the reflections by providing some tidbits of information to families that can increase their understanding of the engineering involved in their tinkering.
Sasha Palmquist
Eric Schearer
Nice work. I think higher ed can learn a lot from what you are doing at the museum. I think the idea of reflection is overlooked in engineering education at the college level (maybe other levels too). I want especially to make reflection a bigger part of our undergraduate research projects. I'm doing it ad hoc right now.
I will certainly check out your work, but can you point me in the direction of a critical review of reflection in engineering education?
Nickolay Hristov
Monica VanDieren
Catherine Haden
Professor
A couple of resources that might be of use to you come to mind. One is the resources here: http://cpree.uw.edu/ The other is a paper (that is also available on that site) I've just found helpful as a background for thinking about the work we do with young children and families: https://depts.washington.edu/cpreeuw/wordpress/wp-content/uploads/2015/07/Integrating-Reflection-ASEE-2014.pdf
Eric Schearer
Thanks Catherine. This is really helpful.
Ateng' Ogwel
Catherine,
Am touched by the experiences shared in the video, and the fact that even the young children are involved in the project. Are you having these tinkering activities in a museum? makerspaces? or in community locations? Science is fun at whatever age, and early experiences might have impact in shaping career choices among many children
Catherine Haden
Catherine Haden
Professor
Thanks for your response! The work featured in the video took place in a children's museum - the Tinkering Lab exhibit at Chicago Children's Museum. Other work we have been doing involving tinkering, engineering learning and reflection has also been hosted by a local library - and the activities for families have happened in area community centers. In some of our new work Chicago Children's Museum is creating videos for families to be guided to tinkering and storytelling at home.
Ateng' Ogwel
Ateng' Ogwel
The response is useful, at least for me as I envisage more of informal STEM learning and promotion of indigenous STEM knowledge. Schools are not the only place where learning takes place, and communities need to have enhanced responsibility in creating opportunities for learners to 'tinker' and explore! Thanks for the quick reply
Catherine Haden
Brian Kruse
Thank you for sharing your project! Giving learners the ability to manipulate materials to explore phenomena is a wonderful way to learn. How explicit are you in focusing on the phenomena they are exploring, or is the entire focus on the engineering aspects?
Do you work with classroom groups who come to the museum on field trips? If so, how do you and the teachers contextualize the field trip so it integrates with classroom activities, and is more than a novel, stand-alone experience?
Ateng' Ogwel
Kim Koin
Director of Art and Tinkering Lab Studios
Hi Brian! Thank you for your questions.
Tinkering Lab is a place where exploration of tools, materials and physical phenomena are at the forefront of the exhibit experience. From a museum facilitation standpoint, we use a modified engineering design process (make it, test it, fix it) which is used to assist children and their caregivers in moving towards a function-focused yet open-ended goal such as "make it fly" or "make it roll." We are continually encouraging children to look closely and verbalize what is happening with their material or project- from articulating how exactly their vehicle rolled down the ramp, to noticing how much saw dust is generated when they saw at different angle.
Field trips at Chicago Children's Museum have access to the whole museum, so we are interacting with school groups in the same way we work with our general visitors. It's a stand-alone experience. Within research, we are now looking to work with more families that have engaged in our research in Tinkering Lab to Tinker at Home. It's an exciting new integration!
Ateng' Ogwel
Monica VanDieren
Thanks for sharing your project. This is a great reminder of the importance of reflection in learning for all subjects at all stages of education. I've recently been exploring using reflective activities more in my multivariable calculus class and in my honors thesis design and development course. I'm curious about what kinds of structured reflective prompts you provided the families?
Jill Rhoden
Catherine Haden
Professor
Hi Monica, in addition to what Tsivia has said below, we also sometimes ask children to reflect on their experiences with a researcher - which can make it a bit less structured than the support a parent might provide having experienced the activity with the child. Those prompts are very open ended, such as "Tell me about what you did today" "Tell me about this creation" "What did you do ti make it?" "What did you learn?" We will also asks some specific prompts, such as "Did you test your creation? Tell me about testing it." And "Did you have to do anything to fix your creation?" Some of the younger children need the more specific prompts; older children generally come up with that information with open-ended prompting.
Ateng' Ogwel
Sasha Palmquist
Ateng' Ogwel
Thanks for these further insights. Such questions and probing remarks have a way of encouraging children to be more observant and curious, and thus attend to critical aspects of experience. Ultimately, it trains them to see patterns, disimilarities and commonalities. That is the true power of purposeful reflection. Developing such scientific attitudes and habits at foundational levels is a great investment, and hope that children get such opportunities as they progress, be it at school or home
Tsivia Cohen
AVP Play & Learning Initiatives
Hi Monica. Many of the reflections we analyzed came (with permission) from families who'd visited the Story Hub exhibit (at Chicago Children's Museum)--a booth where visitors are invited to make a video featuring their conversation about any exhibit. For that reason, the prompts (in English or Spanish) are very general: "Find an exhibit you visited today. Pick one you'd like to talk about." If (for instance) they picked Tinkering Lab, they heard: "Tinkering Lab is a place to try new things. Talk about what you did. Adults, you can help by asking questions and adding your own memories." Just a note: We also have a construction exhibit that takes time-lapsed pictures of visitors' buildings and asks more specific prompts (Talk about how you got started? What problems did you solve?), but in the current research we've been interested in how families structure their own reflections.
Ateng' Ogwel
Sasha Palmquist
Preeti Gupta
Director of Youth Learning and Research
What advice do you have for practitioners who have tinkering experiences at their sites to increase reflection? What are pitfalls to avoid?
Sasha Palmquist
Tsivia Cohen
AVP Play & Learning Initiatives
Hi Preeti. Great question! For the most part, children (typically 5 to 10 years) were very willing to reflect during and/or after their tinkering experience. Being accepting and attentive goes a long way, but a few other strategies seemed to help: Taking a picture of the child's project-in-progress seemed to stop time and give children a chance to reflect on what had happened so far. When they were done tinkering, it also helped to have their project right there to talk about. In both cases we often started with "Tell me about what you made,"and followed up (as relevant) by asking how it worked, if they'd tested it (on a ramp for instance), what happened, what they tried or wanted to try..... As for pitfalls, it helped not to be too scripted in what we asked, so we weren't asking irrelevant questions or ones they'd already answered. It was sometimes tricky to find the right moment to ask children about their project; they needed to get to a point when they'd engaged in some problem solving or had tried testing their project. A question that seemed difficult for some children to answer (on the spot) was "what did you learn?" Maybe that one takes more time to consider. Hope that helps.
Sasha Palmquist
Michael Horn
Love this project and everything that you're finding! Thanks so much for the video
Catherine Haden
Professor
Thanks! Love the work you are doing with music and coding, illustrating the many routes to engaging students in STEM!
Nickolay Hristov
Ph.D.
I very much echo Eric Schearer’s point above that reflection and tinkering are activities that higher ed faculty want in our classrooms as well. The references to CPREE and ASEE paper are very helpful (Catherine, please note that the link does not point to the paper as is and requires some tinkering with the address string to make work). I was successful in this form:
https://depts.washington.edu/cpreeuw/wordpress/...
… but I would be eager to learn about cases and applications in formal learning settings beyond engineering. The bibliography of the ASEE paper points to a few but the citations are impressively focused on engineering. Please share if you know of any.
Catherine Haden
Professor
Thank you! I hope I have fixed the link.
I'm hoping others can chime in about your question about cases and applications in formal learning settings. Reflection is almost certainly a domain general learning mechanism. But figuring out when and how to encourage it seems to be an area ripe for further study.
Ateng' Ogwel
Martin Storksdieck
Director
Great project: thanks for sharing. A few reflections and then a few questions.
Reflections:
a) Remy Dou at FIU has an AISL-funded project to look at science talk in Latinx families. It might be useful to connect with him.
b) I was wondering how you distinguish tinkering from making, and what you see as the connection (if you see any) to engineering design. We worked on a making project that had kids reflect on design before building and tried to have plan the next step, alas, the kids (and their parents) were much more interested and motivated to do what I would consider "tinkering": playing around in exploratory fashion to see what works, and afterwards (maybe) ask why it worked.
(c) Is your work tied to "reflective practice" in general? I.e., how is it different, if at all?
My question:
To what degree do tinkerers want to reflect? And conversely: why would they not want to?
Ateng' Ogwel
Catherine Haden
Professor
Hi Martin, Thank you for your reflections!
(a) I will definitely look for Remy Dou's work.
(b) The distinction between tinkering and making (and maybe even engineering) is somewhat challenging in our work with young children, and we have found it useful to focus on what are the characteristics they share that can lead to a deeper understanding of STEM related practices and STEM learning. These characteristics include learner driven exploration with real tools and materials, minimal didactic instruction, varied experiences and outcomes, creative problem solving, playing, testing, chance taking, and the potential importance of both hands-on activity and social collaborative interactions for what is learned (e.g., Brahms & Wardrip, 2014).
We have played around with encouraging reflection at different points in the activity - e.g., museum staff asking for reflections on how things are going, what's working, what needs more work, at times when families seem to have initially tested their creations. We are still working on how to encourage reflection in the natural stream of activity. Making opportunities for testing very salient seems key.
(c) I see our work as a variation on reflective practice, one that emphasizes how social interactions with adults (museum staff, caregivers) that encourage reflection can mediate children's engagement in learning.
Great question - Sometimes I think tinkerers see reflection and tinkering as part of the same (borrowed from a learner quoted in a book on creativity) "you have to reflect on your tinkering and tinker with your reflections. This way, you can see what works, what doesn't, and how to use experiences to improve your invention."
Ateng' Ogwel
Leigh Peake
I will echo the love for this project and for the combination of "tinkering" and "reflection." I appreciate the suggestion above about broadening how we think about the act of tinkering and the lessons that can thereby be learned. While our projects are very different, I'm intrigued by what we can learn from you. The word "tinkering" has been applied (in this Showcase!) to how kids in our informal program work with data -- they tinker with data. We also likewise settled on how essential reflection is. Periodically through the 2.5 hr informal experience students are asked to pause & reflect and to record a video of their small group reflection in their digital Field Notebook. We struggled (and continue to mess around with) the wording of prompts for those reflections. The scene at about 0:53 in your video looks like your young learners also record videos? I'm curious how that's structured? Is it just an open-ended invitation for families to reminisce? I also appreciated the note in your video about families talking more when they had the object in front of them -- we learned that the hard way and revised the Field Notebook to present students the digital assets they created during the experience (e.g., maps of sea surface temperature) as a reference point for the reflections. I wonder if we could adapt the coding you used to determine the increase in STEM talk that you saw as families worked with their objects. Anyway, no need to answer every question but I really enjoyed your video and the opportunity to think out loud "with" you about it!
Catherine Haden
Professor
Thanks for all the great thoughts here!
We do have two ways of eliciting children's reflections after tinkering. One is in Chicago Children's Museum's Story Hub multimedia component. In Story Hub families can record their reflections on any of the exhibits in the museum. Families record their videos, responding to very open-ended prompts, and then have the option to email their video to themselves, and to grant permission for us to look at it for research purposes. We have a paper out (published in Science Education) that summarizes what we have learned from analyzing those narrative reflections that I'm happy to share (just email me at chaden@luc.edu); it is in that paper that we found that having the creation present engendered more STEM talk.
With other families we have been asking children to reflect immediately after their tinkering experience with a researcher who asks open ended questions. That's what you see at 1:54 in the video.
I am very happy to share the coding manual with you. Again, just reach out to my via email. Cheers!
Preeti Gupta
Director of Youth Learning and Research
Hello Presenters...sorry I couldn't put attention to this discussion thread for 2 days but glad to read all that is being said. Thank you Tsivia for addressing my questions. I agree...this question about "what did you learn" never gets us anywhere! Your project has prompted three thoughts for me.
1) Although your project is based in tinkering and engineering, how can we apply your findings to on-floor facilitation in settings that are not so hands-on such as collections-based learning spaces like natural history museums and botanical gardens?
2) On that note, with the pandemic completely changing the nature of what interactions will look like and the increase on online learning, how can we think about facilitation of museum objects online and bringing best practices to bear even when working in that online environment?
3) What ways can the findings of your project be brought to schools and teachers where the time for reflection is not that much but also the need for reflection is the most?
Tsivia Cohen
AVP Play & Learning Initiatives
Hello again, Preeti. You ask some terrific questions. We have so much more to learn--from both research and from our own observations as practitioners. "What did you learn?" is a question I've seen children struggle with (that take-away is admittedly anecdotal), while data continues to give us a more nuanced picture of what prompts are most productive. Probably situational, so with your questions about how to elicit reflections about museum objects or in settings that don't provide as many hands-on experiences, I suspect the prompts could be very different. Would asking people to tell a story about an object before they learn more about it be a useful prompt? (A story--including correct and incorrect assumptions--might provide connections to prior and subsequent knowledge.) Our model has been conversational, so teachers and schools might need to think about how to ensure that all learners have an opportunity to reflect with an accepting, and interactive "listener." We continue to be very interested in how the stories people tell impact their understanding of concepts. In recent weeks we have been been creating on-line videos, including one that invites children to create a playground ride for a small toy and then post a story. We're also reaching out to some families more directly--to learn more about their reflections. Hope that begins to address some of what you've asked. Lots more to reflect on and explore.
Jill Rhoden
Interesting project and great information! Thank you for sharing. It has reminded me the importance of having our teachers reflect during our upcoming professional development. We traditionally have the teachers participate in activities through the lens of the student and then reflect through the lens of a teacher but I like the idea of using some of your prompts while they are still in 'student mode' as a model for how they can encourage their students to reflect on their projects on a regular basis.
Catherine Haden
Tsivia Cohen
Catherine Haden
Professor
Yes, our emphasis on reflection extends to museum staff as well, including encouraging them to share their reflections during and after prototyping tinkering programs (there is an example in the video of staff trying out a program before we share it with visitors).
Emily Timmel
This research is important for all involved with education for the young. Cutting edge partnership that has informed so much great work and understanding of family education, inquiry-driven, child-centered education.
Catherine Haden
Tsivia Cohen
Tsivia Cohen
AVP Play & Learning Initiatives
Thank you, Emily! So much to learn from from partnerships between museums and universities--as you know.
Doug Scott
I enjoyed watching how experiential learning through tinkering is a valid process that benefits many students. I have always thought through my hands. Well done. The kids are truly enjoying their work and thoughtfully discussing it. You have captured a learning experience that benefits many types of learners.
Catherine Haden
Catherine Haden
Professor
Thanks for watching and commenting. We are delighted by your response.
Sue Allen
This is a great project, and a perfect example of a powerful long-term research-practice collaboration. Thank you!