R324A150078

In this video, we show our Institute of Education Sciences-funded research related to improving mathematical word-problem outcomes for elementary students experiencing difficulty with mathematics. We designed Pirate Math Equation Quest to help students become strong word-problem solvers. We show how we implemented the intervention with embedded activities related to understanding the equal sign as relational and using equations to set up and solve word problems. Students participating in Pirate Math Equation Quest demonstrated significant word-problem improvement over students receiving regular classroom instruction (ES = 2.44). In our video, we show several examples of students using the Pirate Math Equation Quest strategies.

## Sarah Powell

Lead PresenterAssociate Professor

Gwen Stovall

## Jacqueline Genovesi

FacilitatorVice President

Sarah, what a worthy research project. I'm wondering if it was difficult for students to learn the different steps in the schemas? What supports did you give students who struggled to remember the steps?

## Sarah Powell

Lead PresenterAssociate Professor

What a good question! I wouldn't say it is difficult but I would say it takes a long time. We spend 6-10 lessons on each schema. This involves introducing a schema with pictures and stories then working on different variations of unknown information within a schema. Each lesson lasts 25-30 minutes.

We have created a poster for each of the schemas. We have these posters placed on the table while we're working with the student or group of students.

## Jacqueline Genovesi

FacilitatorVice President

Thank you for explaining. I'm wondering if you included both typical learning students and non-typical learning students in the project. If so, did you see any difference in the effectiveness of the schemas? In particular many students with dyslexia have difficulty with word problems. However, they also have difficulty in processing/remembering multiple-step methods and using mnemonics to remember steps. Just wondering what your experience was with this subgroup.

Hilary Kreisberg

## Sarah Powell

Lead PresenterAssociate Professor

Our focus (in this project) were students experiencing mathematics difficulty. All the students scored below the ~15th percentile on a pretest of solving addition and subtraction word problems. We had some students who also experienced reading difficulty - we did not see pretest reading scores act as a moderator. We also had a large percentage of dual-language learners. In three different analyses, we have noted no differential response to intervention based on language status. I find this really interesting and very important. A sustained intervention with a focus on word-problem language and lots of discussion between the teacher and student seems to be important for all students experiencing mathematics difficulty.

Deni Basaraba

Jacqueline Genovesi

## Deni Basaraba

I agree that the lack of differential response to the intervention based on language status is interesting - and important! I wonder if including measures of language proficiency (overall, English language proficiency and/or mathematical language) would shed any light on why no differential response was observed? In other words, were the language proficiencies of participating monolingual and DLLs comparable? Or could it be that providing systematic instruction about the schema of mathematics word problems can help mitigate challenges that are often attributed to language-learner status? What an exciting project!

## Sarah Powell

Lead PresenterAssociate Professor

Hi Deni. I would say it's several things.

1. Our dual-language learners were in third grade. Most had been in U.S. schools since kindergarten. Therefore, the English proficiency of these students was quite strong. We have their TELPAS (Texas version of English proficiency test) scores, and most students scored on the higher end of the scoring scale.

2. We collected reading (word reading, passage reading) and passage comprehension scores, as well as oral vocabulary. In our moderator analyses, none of these moderated treatment effects. In many ways, that's a good thing. Students - regardless of initial reading or vocabulary - responded similarly to the intervention. The intervention is very language intensive with repetitive language and vocabulary used within each lesson, but it's helping all students understand word-problem schemas.

## Deni Basaraba

Thank you, Sarah. I wasn't trying to advocate for differential response by any means - it's definitely a great finding that Pirate Math has been effective for all learners!

Thank you for sharing this project!

## Abigail Levy

FacilitatorDistinguished Scholar

These findings are interesting and important, as you say. And consistent with what studies have found in science learning - time to follow investigations with discussion, reflection, and critiquing are needed to generate deep understanding that can then be applied to new learning. I wonder if you're seeing the impacts of your work with students in math extend to other subjects, e.g., ELA, or science?

Thank you for sharing this project.

## Sarah Powell

Lead PresenterAssociate Professor

Hi Abigail. Our project focuses only on mathematics, but we have anecdotes from both the classroom teachers and our project tutors about how much more students talk about mathematics after doing our 16-week intervention. We haven't measured this in a quantifiable way - but we should in the future. We also need to see if/how such intense intervention work does lead to improvements in language arts or science.

## Abigail Levy

FacilitatorDistinguished Scholar

Hi Sarah, this would be such a terrific next study. We need to know more about whether/how building skills in one discipline can strengthen skills in other disciplines. Understanding the connections better will enable us to see learning opportunities that we didn't know existed, and avoid trying to make connections that won't be fruitful. So much to do!

## Sarah Powell

Lead PresenterAssociate Professor

Agreed! And exploring those connections - especially early connections between mathematics and science - may lead to teachers to teach across topics. I'd love to see "math word problems" taught more explicitly in science and reading and history and such.

## Hilary Kreisberg

Hi Sarah! Thank you for sharing your work! Your video is so well made and put together! I have several questions and can't wait to learn more.

1. I, like Jacqueline, was just wondering about the use of mnemonics to remember multiple steps and the associated challenges I've seen with students focusing on understanding the context of the story, the question being asked, and the associated acronyms to help them solve. I am wondering what the letters stand for in the mnemonics (does GR x N = P stand for GR = Number of groups, N = Size of Group, P = Product?) and how you determined what the acronyms should be?

2. I am also curious about the language and how it might interfere with the mathematics. Step 5 of the chart shown for Equal Groups says Find X. Have you seen students confused by that given there is no variable X in your sentence, or confuse the multiplication sign with the X? Though in the video it did look like the student changed the letter N from the acronym to an X, perhaps recognizing that that was the missing part to find? I also notice that the use of the word "Find" varies from steps 2,3 to step 5 - going from looking for the terms in the problem to solving. Has that caused any students in your study issues?

3. I'm also curious about how students will use these acronyms as they age out of some of the mathematics (i.e. I notice the subtraction mnemonic wouldn't work if I am subtracting, say, 2-5)? How do these work with fractions or when we use multiplication as scaling rather than groupings?

4. How did you teach students to approach the various problem-types? I see the last part of RUN is naming the schema. How do students learn to do that? Is it by doing the equations without context first to see where the unknown is (as shown in the video) and then apply it to story problems? I appreciate that the RUN strategy highlights the quantities by focusing on labels and not just numbers (typical C for circle the numbers in process strategies) and that students have to focus on the context of the story to truly understand what type of problem it is.

5. Lastly, what age groups have you used this with? Have you found it challenging in Kinder/Gr 1 to use with the vast amount of letters to know?

Thanks for your work and interested to learn more about the studies!

## Sarah Powell

Lead PresenterAssociate Professor

Hi Hilary!

1. We teach students a meta-cognitive strategy (RUN - Read the problem, Underline the label, and Name the problem type) that they use for every word problem. Then, they learn about each of the 3 or 4 schemas (depending upon which intervention). We have a mini-poster for each schema with a schema equation (e.g., GR x N = P - you're right about the naming). Students follow the mini-poster steps for a while and then seem to move away from it. They'll start to recognize the unknown and mark that first then fill in the rest of the information. We provide the supports (i.e., steps) but then they start to do what makes sense for them. The schema equations are:

P1 + P2 = T (part 1 + part 2 (+ part 3) = total ...sometimes with three or four parts

G - L = D (greater - lesser = difference)

ST +/- C = E (start +/- change (+/- change) = end) ...sometimes with multiple changes

GR x N = P (groups x times in each group = product)

We have a comparison/set equation (S x T = P) but haven't tested it yet.

2. Our program - Pirate Math - is all about solving for X. Students learn of this as a variable and really like being a math pirate. We use X in addition and subtraction equations first, and students do well with this. We were pretty worried that students would find X confusing when we started the equal groups schema, but none of them blinked an eye. They had such a strong interpretation "that's X" versus "the multiplication symbol" that we never saw any confusion. If we had started with multiplicative problems in the intervention first, however, I wonder if we might have seen a different interpretation. At the end of the intervention, we do talk about X as being any other letter or blank so that students transfer solving for X to other situations.

3. These actually do continue to work, it's pretty cool. For example, "The highest temperature in Austin was 115. The lowest temperature in Fairbanks was -55. What's the difference between these temperatures?" G - L = D. 115 - (-55) = X. Our study occurred at third grade, so we didn't include problems with fractions. We have a new project starting at fourth grade in which we'll use fractions with the schema equations.

4. In the "Name the problem type" of RUN, we had students ask themselves a few questions:

Are parts put together for a total?

Are amounts compared for a difference?

Does an amount increase or decrease to a new amount?

Are there groups with an equal number in each group?

These questions - accompanied with gestures - helped students focus on the schema. At first, we (tutors) asked the questions with the gestures. Then, students start to ask them on their own.

And yes, we moved away from typical attack strategies (e.g., CUBES) because we wanted the attack strategy to be less about circling things and more about thinking about the problem. RUN is from Lynn Fuchs at Vanderbilt (as is a lot of the foundation of Pirate Math).

5. This was all done in 3rd grade. We worked with ~150 students a year for 3 years. In the 4th year, we worked with ~90 students. Each year, ~2/3 of students participated in the intervention.

We've got a few papers published on preliminary parts of the project. The primary paper just got accepted and should be out soon. We're working on a few other analyses now. For example, we see intervention effects on the word-problem outcome in 4th grade - 8 to 10 months after intervention ceased. Pretty neat stuff!

## Hilary Kreisberg

Thanks for the info. I like the idea of being a pirate and finding "a treasure," per se, and finding math adventurous and fun- like detective work. I'm happy to hear students did not find X confusing and it sounds like it was all the teacher intervention and upfront exposure. I wonder if teachers who aren't trained under this program would have the same success?

When you say your questions are accompanied by gestures, are you talking like total physical response? So, for total, kids might cup hands together to show two parts coming together? Interested to learn more about this!

I am still curious about the subtraction part. Your example still puts the larger number as the subtrahend. What about in 7th grade when I have a problem such as one from Illustrative Mathematics: What is the difference in height between 30 m up a cliff and 87 m up a cliff? What is the distance between these positions? In this question, they are trying to help students see the "difference" between difference and distance, and here, the difference actually requires one to subtract the larger number from the smaller (30 - 87 = -57). Curious as to how your strategy would work there?

Warmly,

Hilary

## Sarah Powell

Lead PresenterAssociate Professor

Hey Hilary,

We work really closely with our tutors and provide lots of feedback to them. I would love to do a study in which we see how the program is used under less-controlled settings.

We used hand gestures to represent each of the schemas.

Total: (1) hands apart (2) bring hands together

Difference: (1) hold one hand higher (2) hold other hand low (3) talk about difference between the hands

Change: (1) hold one hand in front (2) raise hand - for increase (3) lower hand - for decrease

Equal Groups: (1) hold hand up like a cap (2) place fingers of other hand in the cup

In our video, you see one of our students using her version of the gestures.

To solve the problem from 7th grade, I'd interpret this as...Think of the cliff as a number line. 87m is greater than 30m. So, 87m is the greater amount. 30m is the lesser amount. 87 - 30 = 57m. There are likely other ways to interpret this problem, so I'll think about it.

Sarah

## Alison Billman

FacilitatorDirector of Early Elementary Curriculum

What an interesting project and it is exciting to hear about the impact it is having for students. They must be so excited. Some of Lynn Fuchs's earlier work focused on helping students become strategic thinkers when reading (Peer Assisted Learning Strategies or PALS). From the video, Pirate Math seems like it might be a teacher-intensive intervention. Have you considered how it might be expanded to have students work with each other as partners?

## Sarah Powell

Lead PresenterAssociate Professor

Hey Alison,

I worked with Lynn on some of the earlier versions of Pirate Math. In this version, we were really diving into the algebraic reasoning necessary to do well with solving word problems.

We did two variations of Pirate Math Equation Quest. In one, we worked with students individually. In the other, we worked with groups of 4 students. In those groups, we paired students together to work on word problems and discuss word problems. It was a little less formal than some of the pair structures in PALS, but it was a great way to use the intervention in small groups. In one of Lynn's previous Pirate Math studies, they did something similar in classrooms of 2nd-grade students. After receiving teacher instruction, students worked in pairs to solve the word problems by schema. The results from that project were very strong as well!

## Mitchell Nathan

This is such an important project for struggling students. I really like how you have given them key steps to focus on that will let them get started. I think that can be such a barrier for many students. It's wonderful to see you scaling this to large student populations. Do you also find that this helps students across achievement levels?

## Sarah Powell

Lead PresenterAssociate Professor

Hi Mitchell,

We've been really focused on helping students experiencing difficulty in mathematics, so we didn't work across achievement levels. In some of the work of Lynn Fuchs (which I worked on while at Vanderbilt), we used an earlier version of Pirate Math in classrooms of second-grade students. The results from that study demonstrated that all students benefitted similarly from the whole-class Pirate Math intervention.

We'd like to expand the reach of Pirate Math Equation Quest - we're heading to fourth grade next year!

## Leanne Ketterlin Geller

Hi Sarah! Thank you for sharing information about this important project. Your research team has created an impressive suite of materials to support student learning.

## Sarah Powell

Lead PresenterAssociate Professor

Thanks, Leanne! We're very excited to finally be able to share all the materials with teachers across the U.S. We hope to help many kids become strong problem solvers!

## Candace Walkington

I really enjoyed your video - the students' explanations were awesome! Is it typical with the intervention for students to explain their problem-solving processes out loud like that?

## Sarah Powell

Lead PresenterAssociate Professor

Not a first. As the kids become more comfortable with problem solving, we ask them to pretend to be a "professor" and teach the word problem to someone else. The kids love doing that!

## Jenny Yun-Chen Chan

I love the metacognitive strategies and the gestures for the problem type!!!! They make the problem solving process very explicit and intentional. I wonder how these ideas (explicit metacognitive strategies and incorporating gestures) can be extended beyond word problems. I am also interested in learning more about how English Language Learners may benefit from the intervention. At first glance, the intervention seems to be very language heavy, but maybe the gestures help ELLs make connections between the math language and ideas?

I can't wait to learn more, meet the team at conferences (when the world opens up again), and continue the conversation :D

## Sarah Powell

Lead PresenterAssociate Professor

Hi Jenny! I believe that the gestures would translate to other areas of mathematics - similar to the research of Martha Alibali, Candace Walkington, and others. In our study, approximately 65% of the students categorized as dual-language learners. In our analyses, we did not see any differences in the response to the intervention based on dual-language status. I believe we're supporting all students in the language of mathematics and word problems, and that's important to do.

And yes, hope to see you around!

Jenny Yun-Chen Chan

## Lynn Selking

Hello Sarah!

Thank you for sharing this work, both through this STEM for All Video Showcase and through the free materials on the website! I noticed there are teacher and student materials both for small group and individual instruction and it seems that the materials for individual instruction "stretch" some of the small group lessons out into 2 separate lessons. Can you share how in the study it was decided when to use the individual versus small group lessons?

## Sarah Powell

Lead PresenterAssociate Professor

Hi Lynn!

First, we developed the individual intervention. We tested the efficacy of this intervention across three cohorts of students. Once we determined the program worked well and led to large improvements in word-problem solving, we designed the small-group intervention. At this point, based on our conversations with teachers and a review of 3rd-grade standards, we added in the equal groups schema. We tested the small-group intervention over one year by providing the intervention to students in groups of 4. This program showed similar results to the individual intervention, and we believe many educators would be more likely to use the small-group intervention rather than individual because individual intervention is difficult to schedule.

To post to this discussion go to