NSF Awards: 1813254, 1814275
Effective teaching requires understanding that goes beyond knowing the subject matter and includes professional knowledge that teachers draw upon as they engage in the work of teaching within a specific discipline, which we call content knowledge for teaching (CKT). Supporting the development of teachers’ CKT requires valid and reliable assessments to measure teachers’ CKT and instructional materials to build teachers’ CKT. This video will highlight the ongoing work from a collaborative research project across two institutions – Educational Testing Service and Western Washington University – to develop and examine how CKT science assessment items and instructional tasks targeting matter and its interactions at the elementary (K-5) level can be utilized in elementary teacher education settings to evaluate and support elementary science teachers’ ability to apply their content knowledge to the work of teaching science. We will describe the summative and formative CKT assessment tools that we have developed for use within elementary science method and content courses and explain how we envision their productive use within these settings. Overall this video will offer an insider’s look at a set of tools that can be used to positively impact preservice elementary teachers’ CKT in one critically important science content area and detail the learning opportunities these tools afford for both preservice elementary teachers and the teacher educators that work with them.
Jamie Mikeska
Research Scientist
Welcome to our video! Thank you for taking the time to watch our video and comment on the research and development work we’ve been doing to support elementary science teacher educators and the preservice teachers they work with. We are excited to share with you the work that we’ve been doing to help build and assess elementary preservice teachers’ content knowledge for teaching (CKT) in one science content area: matter and its interactions. During the first two years of the project, we’ve been busy working to develop a set of instructional materials – what we call CKT Packets – for elementary teacher educators to use as part of the elementary science content and methods courses and a CKT matter summative assessment instrument to measure the preservice teachers’ CKT over time. We look forward to hearing from you with any questions, comments, or ideas about how the field might use these types of CKT tools in teacher education and professional development settings.
Stefan Irby
Amy Wagler
Hannah Sevian
I'm curious how you are building in ways to support preservice elementary teachers' ability to apply ideas about matter and its interactions in the context of different crosscutting concepts, such as cause and effect, mechanism and explanation, or patterns, or systems and system models.
Dante Cisterna
Jamie Mikeska
Research Scientist
Good question! For each of the CKT matter items we developed, we identified the crosscutting concept (in addition to the disciplinary core idea and scientific practice) that is most closely aligned to what is being assessed within that item. Each CKT matter item starts with a description of an instructional scenario that situates the pre-service elementary teacher into a specific aspect of the work of teaching science. Sometimes the items require the teachers to more explicitly apply matter ideas in the context of the crosscutting concepts, and other times the connection is more implicit. At a recent conference, I talked with another researcher about this and we agreed that strengthening these connections and making them more explicit would be another way to broaden the impact of this research and development work. Our partners at Western Washington University have also developed CKT matter packets and some of the reading pages and portions of the lesson plans they designed for teacher educators also supports these connections.
Debi Hanuscin
Profeesor
Hi, Hannah-
The CKT Packets include explicit connections to the CCCs--for example, the Packet focused on the content idea of conservation of matter is linked to Energy and Matter. Each Packet has a section that outlines connections to the standards and the expanded lesson plans help teacher educators elaborate on those connections.
Susan Kowalski
I enjoyed your video. It sounds like the packets are a great resource for pre-service teachers to deepen their understanding of important science concepts and how to teach them. Have you thought about making these packets available to in-service teachers? I'm wondering if teachers might be able to use the packets in a professional learning community.
Dante Cisterna
Judi Fusco
Josie Melton
Post Doctoral Research Associate and Senior Instructor
Hi Susan,
Yes, packets can also be used for in-service teachers to develop their CKT. Packets can be accessed at https://cktscience.org/ . Note that you must register in order to access materials on the website. Some packets are still in development, but will be made available once they are ready. Thanks for your interest!
Judi Fusco
Billy Jackson
I am curious as to how you see the materials being used. Unlike in math, in many places, preservice elementary teachers often do not take content courses specifically designed at understanding elementary science ideas as a much deeper and fundamental level required to teach it. This is probably why starting elementary report that they feel the least prepared to teach science among all of the subjects they are responsible for.
I like having the assessment particularly in that it seems to a goal of the project to validate the instrument specifically for preservice teachers, as other instruments have focused on inservice teachers mostly in their validation efforts. I wander if the assessment might be used as a basis for future efforts among policymakers to try and advocate for change in how the future teachers are prepared in science to mirror what is happening in math.
Dante Cisterna
Josie Melton
Post Doctoral Research Associate and Senior Instructor
At Western Washington University, where we are developing the instructional materials, the packets have been piloted in elementary science methods courses and science content courses offered specifically for future elementary teachers. We have found that the packets deepen the content focus in methods courses and provide more opportunities to highlight teaching practices in the science content courses. Another potential use is for in-service professional development. Thanks for the question!
Billy Jackson
I am curious as to how you see the materials being used. Unlike in math, in many places, preservice elementary teachers often do not take content courses specifically designed at understanding elementary science ideas as a much deeper and fundamental level required to teach it. This is probably why starting elementary report that they feel the least prepared to teach science among all of the subjects they are responsible for.
I like having the assessment particularly in that it seems to a goal of the project to validate the instrument specifically for preservice teachers, as other instruments have focused on inservice teachers mostly in their validation efforts. I wander if the assessment might be used as a basis for future efforts among policymakers to try and advocate for change in how the future teachers are prepared in science to mirror what is happening in math.
Alison Heimowitz
School and Teacher Liaison
Like Billy Jackson, I am interested in learning more about how you see these materials being used given that pre-service elementary teachers generally lack a deep understanding of science concepts and principles. Because not all students learn science the same way, I'm also wondering in what ways you were able to incorporate culturally responsive teaching into the packets?
Dante Cisterna
Jamie Mikeska
Research Scientist
In the CKT packets, there is definitely a focus on attending to the variation and complexity of scientific ideas that students bring with them and understanding the various sources for those ideas. This focus provides a way for teacher educators to help their pre-service teachers learn about science concepts and principles but in the context of considering student ideas. We also think it will support the pre-service teachers in bringing an equity-focused lens to their future science instruction -- as learning how to elicit and attend carefully to students and what they bring with them is vitally important!
Debi Hanuscin
Profeesor
I'll add that we designed the CKT packets to be flexibly used either in content or pedagogy courses-- there are several teacher/preservice teacher content-focused curricula (e.g., PET, Physics by Inquiry, Making Sense of Science) that these would work in concert with.
Albert Byers, Ph.D.
What a wonderful project, and as I see colleagues such as Deborah H and Sean S at Horizon Research, with ETS, you all have assembled a wonderful team of deep thinkers. I look forward to looking at these assessment tools for elementary pre-service teachers in more detail!
As an aside, and excuse the simplicity of this question before I am able to look at the tasks in more depth for pre-service teachers: How are these tools similar or in part draw upon the notion of Krajcik's "educative instructional materials" for teachers (providing teacher background support in subject matter and pedagogical implementation) or the work of Shulman (year's ago) looking at pedagogical content knowledge within specific disciplines. I see some synergy, and it may be as simple as it is an "assessment task" for pre-service teachers!
Very exciting and needed project! Thank you for sharing.
Dante Cisterna
Jamie Mikeska
Research Scientist
Thank you for your kind words Albert. If you want to take a look at the current CKT packets and example CKT matter assessment items, you can do so at our project's website at cktscience.org. You'll need to register for a free account to view the packets and sample items.
In response to your questions, in the design of the packets we did take up the idea of educative instructional materials -- but for teacher educators! That is what we hope is part of the contribution of this work to the larger field. On the cktscience.org website, you can read more about the various design heuristics and educative features we've incorporated into the CKT packets. In addition, many of the CKT matter items require teachers to leverage their pedagogical content knowledge -- particularly their knowledge of students' ideas and knowledge of instructional strategies -- in this content area. Those are great connections to the work that we are doing on this project.
Albert Byers, Ph.D.
Thank you Jamie. I really appreciate the work and what this contribution will make. I definitely will dig deeper and seek to learn more! Years ago, I led an effort developing little online nuggets of just-enough, just-in-time, just-for-me science content and PCK modules called Science Objects/SciPacks at NSTA (with a GREAT team of internal folks and external experts) and we spent significant time thinking along these lines (drawing from the latest research at the time). This is impressive work, building on the prior body of knowledge and will have meaningful impact. Kudos! You've got my vote.
Debi Hanuscin
Profeesor
Thank YOU, Al-- One of our faculty workshop participants told us that the packets were like "Page Keeley's books but for faculty!"-- we tried to base them on tasks that could be implemented easily, while also supplying the supplemental materials and information to help develop preservice teachers' ideas more deeply.
Judi Fusco
Senior Researcher Stem Teaching and Learning
I enjoyed your video and think the work you are doing with pre-service teachers is so needed. So the focus is on content, but it seems like, since these are situated in practice, there would be some pedagogical knowledge benefits, too? What do you see? (I know that's not the focus, but I got curious as I watched, especially as your advisor Sean Smith brought it up, too.)
Great job!
Dante Cisterna
Josie Melton
Post Doctoral Research Associate and Senior Instructor
Just to add to Jamie's response, the packets are structured so that they target one content sub-topic about matter and its interactions, and one teaching practice. So, as preservice teachers engage in the lesson provided in the packet, they are deepening their content knowledge while learning about instructional decisions that teachers make when teaching about that content. If you are interested, feel free to check out some packets on our website by signing up to access the tools: https://cktscience.org/
Judi Fusco
Judi Fusco
Senior Researcher Stem Teaching and Learning
Thanks, Josie and Jamie for your replies. I will check out the packets. C and P are certainly interrelated. Very interesting work you have!
Jamie Mikeska
Research Scientist
Our work is focused at the intersection of content and practice. To build and measure the preservice teachers' CKT in this area, we focus on how they leverage their content knowledge to engage in science teaching practices (e.g., eliciting student ideas; selecting instructional resources to address specific learning goals; etc.)-- or what we refer to as the 'Work of Teaching Science.' As part of the CKT matter assessment development process, we conducted cognitive interviews with almost 80 preservice teachers to examine how these items functioned and to determine the knowledge that teachers used to respond to each item. Overwhelmingly we found that the teachers had to both attend to and draw upon aspects of the content and aspects of content teaching in order to successfully respond to the CKT matter items, which suggests these item types measure more than only content knowledge alone and having teachers interact with these types of items within the CKT Packets would also have some pedagogical benefits as well.
Judi Fusco
Kenne Dibner
Senior Program Officer
Hi Jamie! Nice to see you in this virtual context - hope you and your family are doing well!
Great video, as I would expect. This is super interesting work. I'm wondering about the evaluation component - did you happen to notice any differences in teacher performance? We know so little about how in service PD matters for student performance down the road - I'd be interested in knowing if you could account for any differences in teachers' uptake of the materials.
Great seeing your face here!
Dante Cisterna
Jamie Mikeska
Research Scientist
Hi Kenne! It is great to see you too. Thanks so much for taking a look at our video. Right now we are in the second year of this four year research project. The first two years have been focused on developing these tools so we do not have much evaluation data to share at this point (although we do have a lot of data from our field test, pilots, and cognitive interviews for the CKT matter summative instrument). However, we are gearing up to integrate the tools within elementary science content and method courses starting this fall and plan to examine the extent to which using the CKT Packets helps to improve the preservice teachers' CKT about matter. As part of that work, we will be able to learn more about how both teacher educators and their preservice teachers use, understand, and interact with these materials. The connection to student performance down the road is a good one -- we are not able to examine that connection yet, but hope to in future work!
Judi Fusco
Kenne Dibner
Senior Program Officer
It's so hard to connect student performance to teacher PD, but having teacher use data would be helpful. Exciting!
Kenne Dibner
Senior Program Officer
Also, I meant to tell you to look out for the new NASEM study on Science in PK-5! First committee meeting was this week!
Dante Cisterna
Michael I. Swart
Thank you for sharing this work. For the assessment, do you have a resource site? Curious about the types of items, scoring, validity, reliability?
In the video, was mentioned that CKT has a metacognitive component to connect pedagogical knowledge with content knowledge. Can you expound on the approach of the CKT framework and any theoretical constructs that undergird its operationalization?
Katherine Castellano
Senior Research Scientist
Hi Michael,
Thanks for your interest in our work. We have our CKT packets and sample items available at our website: https://cktscience.org/. You just need to register to access them. We are still in the midst of finalizing the scaling of our assessment so those technical details are not posted yet.
We relied on the "Work of Teaching Science" framework in our theorization of the CKT construct. We provide more information about this on our "CKT" tab of our website: https://cktscience.org/ckt/.