NSF Awards: 1712312

Calculus is a cornerstone of college-level mathematics programs and is an integral part of STEM programs. Less than half of all college students who intend to pursue a major in STEM fields succeed in doing so, and educational researchers have found that calculus presents a significant hurdle for these students.

Research has demonstrated the effectiveness of active learning strategies and has suggested the use of “flipped” classrooms as one way to support the incorporation of these strategies into class. However, relatively little is known about how students watch and learn from instructional videos.

The goal of the Calculus Videos Project is to design instructional videos and conduct research on their effectiveness. We use research on student thinking about calculus concepts to design instructional videos; for each topic we also design a website with tasks to promote students’ thinking and to measure their understanding of the concept. We then investigate:

- The ways students interact with the videos;
- The aspects of the videos students attend to as they watch;
- The ways students make sense of and learn from the videos;
- How various ways of structuring the video-watching experience can influence each of these aspects.

Over the past three years, our videos have been used by thousands of students at numerous post-secondary institutions. Although we are in the initial stages of data analysis, initial results show that our videos help students learn calculus concepts, and using these videos provides instructors with time and opportunities to incorporate active learning strategies into their teaching.

## Aaron Weinberg

Lead PresenterProfessor

Greetings from the Calcvids team! We hope you enjoyed the video.

The goals of the Calculus Videos Project are to design, create, and evaluate the effectiveness of instructional videos for first-semester calculus. We have developed videos and supporting materials for over 30 topics, all of which are posted on our project website calcvids.org. We also evaluate the effectiveness of our videos—and of various ways of structuring the video-watching process—by conducting interviews with students and collecting data from students at numerous colleges and universities. We are in the early stages of data analysis.

We look forward to your questions!

## Jeremy Pina

I strongly support what I understand to be the central goal of your work -- to introduce a system of calculus videos into undergraduate classrooms as a means of lifting some of the cognitive burden of teaching calculus topics so as to provide more time and "thinking space" for discourse -- but I'm curious about the context in which your project relates to students' informal reliance on calculus videos hosted by, for example, YouTube and Khan Academy.

In my own (admittedly limited!) experience, I've found that novice calculus students tend to share with one another a common pool of resources of this type. Are Calcvids complementary to this informal instruction, are they intended to replace it, or is there some other contextual relationship you're prefiguring?

## Matt Thomas

Co-PresenterAssociate Professor

Thanks Jeremy! While folks are likely using video resources in all kinds of ways (especially now), I think our videos focus on conceptual understanding in a way that a lot of other places (like YouTube and Khan Academy) don't. We focus a lot less on "how to solve this type of problem" and a lot more on "how to think about this concept." I think this means that they could be compatible in a classroom. Though I think there's a bit of a "but it's good for you" element too - students are likely going to go search out videos to help them solve a problem like the one on their homework more than a "here's how to think about this" video.

Matt

Sasha Palmquist

Jeremy Pina

## Aaron Weinberg

Lead PresenterProfessor

In addition to what Matt described, I think there are two important ways in which the Calcvids videos are different from most other videos.

First, our videos are mathematically correct. For example, Kahn Academy explains the concept of a derivative in terms of "the rate of change of a vertical variable over the rate of change of a horizontal variable"; these sorts of informal descriptions sound fine at first glance, but, when you think about what they're saying, they're simply incorrect. I think if you look at most of the Khan Academy videos—and many others—you'll find that they're rife with mathematical inaccuracies.

Second, our videos target particular ways of reasoning that the research literature has found to be productive for students to really understand the underlying concepts. So, for example, we ground all of the discussion of derivatives in terms of constant rates of change, and the ways "constant rate of change" is introduced, described, and depicted, are grounded in the idea of coordinating amounts of change between two varying quantities—a way of thinking that research has suggested as important for understanding calculus concepts.

Sasha Palmquist

Jeremy Pina

## Michael Tallman

Co-PresenterAssistant Professor

Quantitative and covariational reasoning are the particular ways of thinking we designed our videos to support. If you're interested in reading about the theoretical design principles of our videos, feel free to check out the Resources page of our website (https://calcvids.org/instructorinfo/).

Sasha Palmquist

Jeremy Pina

## Monica VanDieren

These are great videos, and maybe even more relevant now that most of us are remote teaching! I will share them with my colleagues, who have been searching for high quality materials to supplement their abrupt switch to online instruction.

Your work and videos are also inspiring me to think of additional ways that our project (CalcPlot3D) can be disseminated and new ways that it can be used in instruction.

## Matt Thomas

Co-PresenterAssociate Professor

Thanks Monica! I hope they're helpful in folks with all the online instruction. And thanks for mentioning your project too - it looks very cool, and dissemination is definitely a tricky piece!

Matt

## Michael Tallman

Co-PresenterAssistant Professor

Thank you, Monica. If you or your colleagues have suggestions for how we might improve the videos, we would love your feedback.

## Overtoun Jenda

FacilitatorAssistant Provost and Professor of Mathematics

Thanks for doing this. Are there plans to scale it up to other courses in mathematics beyond calculus?

## Aaron Weinberg

Lead PresenterProfessor

We'd love to scale up to other classes. Our funding ends this year, so we wouldn't be able to do it in the current project. But if there is enough interest in the community, we're hoping to apply for additional funding in the future.

## Deborah Moore-Russo

I've used these videos - they are well received by students.

Matt Thomas

Sasha Palmquist

## David Touretzky

Interesting video! I watched some of the calculus tutorial videos. They show students manually plotting functions in an approximate way to get an idea of the shape of a function and its derivative. What happens if you give students access to convenient plotting software like Mathematica or CalcPlot3D? Are they more eager to try plotting functions than if they have to do it manually? Or do they respond by trying more things blindly rather than thinking through what's going on?

## Aaron Weinberg

Lead PresenterProfessor

That's a great question! In our videos, we generally tried to not tie them to any particular software, so that they could be used by instructors without requiring that their students have experience with particular technology. But we strongly encourage instructors to utilize mathematical software that will let their students do things like adjust graphing windows and make numerous approximations so that they can more easily explore the underlying mathematical concepts.

Sasha Palmquist

## Paul Seeburger

I appreciate the thought that has gone into the creation of this series of calculus videos. I had not seen them before, but just went to your site and viewed video #3 on the Fundamental Thm of Calculus Part 2. I enjoyed it, and see that the approach taken in it fits quite well with the intuitive approach I have used in teaching this concept, but which I have not seen so well presented in most of the calculus textbooks.

Good work! I look forward to viewing more of your fairly extensive collection for first semester Calculus. I may be able to integrate some of them into my fall semester Calculus I Honors class!

Sasha Palmquist

## Matt Thomas

Co-PresenterAssociate Professor

Thanks Paul! We've tried to walk that line of "intuitiveness" - which is pretty tricky sometimes. One setting that we wanted to really explore in the videos was the idea of constant rate of change and average rate of change. Students seem to often have an idea that these ideas are intuitive, until you start picking apart the ideas. Average rate of change is great for this - what is it an average of? Without just a computation, what does it mean to say you're traveling at an average rate of 45 mph? Building on intuition seems to be key here while also inspecting that intuition!

Sasha Palmquist

## Michael Tallman

Co-PresenterAssistant Professor

Thank you, Paul. If you do use the videos in your Calculus I Honors class, our project team would be interested to hear your suggestions about how we can improve the videos.

Sasha Palmquist

## Hong Liu

Excellent work, I shared with my department chair and a few other chairs of math department. Hope that your videos get more attention than those of Khan Academy. We will let you know the feedback from our teachers.

## Matt Thomas

Co-PresenterAssociate Professor

Thanks Hong! We'd love to hear any feedback!

## Kathryn Kozak

These videos look incredible. I teach my calculus class using inquiry based materials (mostly group work). Do you know of anyone who is using group work and uses these videos also? Also, can these be used in an online class as the primary way to deliver the material even if there is no group work component? I am teach calc I in the fall and am worried about the course needing to be online. I do non't have a plan for doing group work online, and these videos may be a way to create the content for the course. Also a friend is teaching calc I online this summer, and is looking to create videos. Are these videos enough information where she could use them instead of creating her own?

## Aaron Weinberg

Lead PresenterProfessor

Our videos are designed to be used in many ways—we've had instructors use them as part of IBL classes, as part of flipped classes that incorporate group work, and as complements to more traditional lecture. The videos can definitely be used as the primary way to deliver material, and they should hit all of the topics that are addressed in a standard first-semester calculus class. On our website (calcvids.org) we also provide the Powerpoint files that were used to create the videos, so instructors can modify them and create their own versions of the videos, or use some of the resources in other ways in their classes.

Sasha Palmquist

## Matt Thomas

Co-PresenterAssociate Professor

Thanks Kathryn! To piggy-back off Aaron, I think the place that you might add to the videos (or add additional videos if you wanted) might be in the pure computation side. Because we focused a lot on the concepts, I've used more time in class when I've used the videos on working problems in groups. We certainly have examples in the more procedural sections (e.g. product rule, chain rule), and group work might be a good place to build on those to help the students use those ideas in practice.

## Sasha Palmquist

FacilitatorSenior Manager of Community

Thank you for creating such a flexible tool for learning! I particularly appreciate the thoughtfulness that was invested in making the videos and associated powerpoint files "portable" to a range of learning contexts. I am curious whether the core project team or any of the members of your user testing community have implemented these videos in informal settings (e.g. a museum exhibition or outreach programs)?

## Aaron Weinberg

Lead PresenterProfessor

Thank you! We created these videos with the curriculum of a standard first-semester calculus class in mind, so we haven't yet thought about whether and how they might be used in other contexts. But we'd love to hear about any ideas for other venues in which they might be useful, and to think about how the videos might be adapted for those settings.

## Jeanne Reis

FacilitatorDirector

What a great resource! I'll be sharing this with my network of math educators for sure. I went through the Constant Rate of Change lesson, and was impressed by the contextualized lesson. The graphics and animations definitely supported understanding of the concepts too.

In the reflection page, participants are asked to answer the following question: What were the 2 or 3 moments or aspects in the videos that you found the most confusing or needed additional explanation?

How are you using the data you get in response to that question? Have you revised any of your lessons based on input from your users?

Have you considered designing the UI to sequence the lessons by level of difficulty, topic, or grade?

And I wonder if you've considered adding some support for vocabulary learning, such as a glossary, or mini-lessons on terms or concepts.

Thanks for sharing your work!

## Aaron Weinberg

Lead PresenterProfessor

Thank you!

The questions about the students' confusion are for our research - to help us understand how students watch and learn from the videos. Specifically, we've hypothesized that many students don't recognize moments when they don't understand something, and we're interested in learning more about the students and moments who do experience confusion; then we plan on looking at how these students interact with and learn from the videos.

We have made some significant changes to our videos over the past three years based on feedback from students and instructors. And we'd love to get more ideas about how they could be improved!

Revising the UI and adding vocabulary is an interesting idea! We haven't yet thought about it, but we should definitely do some brainstorming. Most recently, we've been thinking about additional ways to make these materials available to instructors (e.g., ways to integrate the videos and assessments with a LMS), so we should think about UI/etc. as we do this!

Jeanne Reis

## Mitchell Nathan

I love this as a way to promote active learning in calculus! I also really appreciate the network of colleges you have assembled to create a real community of learners. I wanted to know if you thought about whether this helps students develop the visualization skills that so many of us had to do before such simulations were available. I am thinking that this could be part of a pathway where scaffolding/fading the simulations could provide the easier access early on and then foster visualization skills as students' develop more calculus understanding. Thanks for sharing this wonderful project!

Jeanne Reis

## Aaron Weinberg

Lead PresenterProfessor

Thank you!

We've been thinking a lot about the visualizations. We're hoping that they'll help make clear many of the things that textbooks and lecturers are trying to do when they describe dynamic imagery. When we dig through our data, we'll be looking to see whether students reference some of the animations in their descriptions.

However, I think that it would be even better to develop many of these visualizations as stand-alone GeoGebra applets, where students can experiment with manipulating values and controlling the animations themselves. I suspect that this interactivity (not quite embodiment, but closer to it?) would be more impactful than just watching the animations in the videos.

Jeanne Reis