1. Larry Bell
  2. Sr. Advisor for Strategic Initiatives
  3. ChemAttitudes: Using Research to Support Interest, Relevance, and Self-Efficacy
  4. https://www.nisenet.org/chemattitudes
  5. Museum of Science, Boston
  1. Kayla Berry
  2. Assistant Project Manager
  3. ChemAttitudes: Using Research to Support Interest, Relevance, and Self-Efficacy
  4. https://www.nisenet.org/chemattitudes
  5. Museum of Science, Boston
  1. Brad Herring
  2. Director, National STEM Networks
  3. ChemAttitudes: Using Research to Support Interest, Relevance, and Self-Efficacy
  4. https://www.nisenet.org/chemattitudes
  5. NISE Net
  1. Emily Hostetler
  2. Education Associate II
  3. ChemAttitudes: Using Research to Support Interest, Relevance, and Self-Efficacy
  4. https://www.nisenet.org/chemattitudes
  5. Museum of Science, Boston
  1. Elizabeth Kollmann
  2. Manager, Research and Evaluation
  3. ChemAttitudes: Using Research to Support Interest, Relevance, and Self-Efficacy
  4. https://www.nisenet.org/chemattitudes
  5. Museum of Science, Boston
  1. Catherine McCarthy
  2. https://www.nisenet.org
  3. Project Leader - NISE Network
  4. ChemAttitudes: Using Research to Support Interest, Relevance, and Self-Efficacy
  5. https://www.nisenet.org/chemattitudes
  6. NISE Net
  1. Lily Raines
  2. Manager of Science Outreach
  3. ChemAttitudes: Using Research to Support Interest, Relevance, and Self-Efficacy
  4. https://www.nisenet.org/chemattitudes
  5. American Chemical Society
  1. David Sittenfeld
  2. Manager, Forums And National Collaborations
  3. ChemAttitudes: Using Research to Support Interest, Relevance, and Self-Efficacy
  4. https://www.nisenet.org/chemattitudes
  5. Museum of Science, Boston
Public Discussion
  • Icon for: Larry Bell

    Larry Bell

    Lead Presenter
    Sr. Advisor for Strategic Initiatives
    May 4, 2020 | 03:33 p.m.

    Thank you for coming to look at our video.  In this NSF-funded ChemAttitudes project, museum educators and researchers worked together using a process called “design-based research” to simultaneously revise a theoretical framework and a set of hands-on activities that embody strategies for increasing public interest, sense of relevance, and feelings of self-efficacy toward learning about chemistry. You can find links to all of the outputs of the project at https://www.nisenet.org/chemattitudes, including a digital version of the kit of materials distributed in 2018 and a Let’s Do Chemistry framework and strategies guide. Beyond all the material posted online, the project team has presented sessions and workshops at the Association of Science-Technology Centers and American Chemical Society professional meetings. Now we are working on a Let’s Do Chemistry Train-the-Trainer Workshop in order to spread the findings of the project throughout the chemistry community so that they can be put to use by chemists who engage with public audiences at National Chemistry Week and all kinds of other events. We were originally planning to run one-day hands-on workshops at a variety of professional conferences, but we are now developing a virtual workshop with four one-hour sessions and prep work to do before each.  We’d love your thoughts related to this current work:

    • What are the best ways of reaching practitioners of informal education and public outreach broadly throughout a field like chemistry?
    • Has anyone revised hands-on in-person workshops for the online virtual environment? 
    • What has worked well for you, either as a presenter or a participant in a virtual workshop?
    • What strategies have helped the workshop facilitate learning by doing, rather than just listening to presentations?

    We, of course, have our own ideas about this but would love to learn from you.

     
    4
    Discussion is closed. Upvoting is no longer available

    Catherine McCarthy
    Elizabeth Kollmann
    Leigh Peake
    David Sittenfeld
  • Icon for: Jack Broering

    Jack Broering

    Program Coordinator
    May 5, 2020 | 07:31 a.m.

    Your video was very informative and your approach to educating students via hands-on experiences is certainly an excellent way to pique the interest of students.  A program at the University of Cincinnati targeted this approach by educating teachers on how to develop "hands-on" lessons for their classrooms.  As an extension of this work, STEMucation Academy was formed.  You can access more hands-on units of instruction at our website and more specifically at the following link (http://stemucationacademy.com/units/).  We have modified our training to included both in-person and online courses.  What has worked well for us?  We assign a personal coach to each teacher who takes our course to help guide them through our training program.  This is an added expense for our program but have found it to be effective way to deliver on-line training while still providing access to an expert via email, phone and texting. 

     
    3
    Discussion is closed. Upvoting is no longer available

    Holly Morin
    Catherine McCarthy
    Kayla Berry
  • Icon for: David Sittenfeld

    David Sittenfeld

    Co-Presenter
    May 5, 2020 | 08:39 a.m.

    Thanks, Jack. That's a fascinating approach and I can see how the coach could be really helpful in these physically disconnected times.

     
    1
    Discussion is closed. Upvoting is no longer available

    Catherine McCarthy
  • Icon for: Marianne Dunne

    Marianne Dunne

    K-12 Administrator
    May 5, 2020 | 09:24 a.m.

    Hi Larry and David! Loved hearing about this project and am interested in hearing more about the development of the professional learning experience for teachers you are working on. The ACCT project that I am working with in the Boston Public Schools has recently shifted to virtual online PD due to the Covid crisis. You can find more information about the ACCT project  on the American Chemical Society’s ChemEd Xchange, www.chemedx.org/acct, and our team would be happy to speak more with you about the research and activities with teachers. The first few years we developed and offered face to face year long PD and next year we were planning to move to a hybrid model of online and face to face. However we decided to move to the online model for SY 2020-2021. We have a video featured  here as well--check out --Formative Assessment with a Bang!  http://stemforall2020.videohall.com/p/1809. Again great to see the work you are doing!

    Can you tell me more about the kits and instructional materials you sent out to Museums?  Marianne

     

     
    1
    Discussion is closed. Upvoting is no longer available

    Catherine McCarthy
  • Icon for: David Sittenfeld

    David Sittenfeld

    Co-Presenter
    May 5, 2020 | 09:30 a.m.

    Hi Marianne!  I was just telling Sharon the other day about your project and wondering whether you were going to have a video.  Now I know!  I will check out your video later today.  It would be really interesting to think with your team about the interactions between what you have learned about formal ed interactions and what we've done with informal chemistry learning.

    You can view all of the digital kit materials here, including the research-to-practice guide.  https://www.nisenet.org/chemistry-kit. Each activity contains training videos including example interactions/facilitations, along with activity and facilitation guides.  There are also a number of PD materials mentioned above.

    Or here on the ACS site: https://www.acs.org/content/acs/en/education/outreach/explore-science-lets-do-chemistry.html

     

    Thanks and hope you're all doing well during this crazy time!

     
    2
    Discussion is closed. Upvoting is no longer available

    Ariel O'Brien
    Catherine McCarthy
  • Icon for: George Hein

    George Hein

    Professor Emeritus
    May 5, 2020 | 01:25 p.m.

    Larry et al.

    Nice to see your project.  As a former chemist, I appreciate this.  I think Joe Griffith did a similar chemistry project (for older students and adults?) at one of the Smithsonian Museums quite a while ago in the 1980s

    George Hein

     
    1
    Discussion is closed. Upvoting is no longer available

    Catherine McCarthy
  • Icon for: Larry Bell

    Larry Bell

    Lead Presenter
    Sr. Advisor for Strategic Initiatives
    May 5, 2020 | 01:41 p.m.

    George,

    Nice to run into you virtually as we currently have no operas or concerts at which to bump into each other.

    All the best,

    Larry

     
    1
    Discussion is closed. Upvoting is no longer available

    Catherine McCarthy
  • Icon for: Susan Kowalski

    Susan Kowalski

    Researcher
    May 5, 2020 | 01:40 p.m.

     I love how you have taken on some of the more challenging aspects of chemistry experiences (materials, mess, and safety) to help informal education leaders across the country help students get excited about chemistry! I'm interested in your ideas for virtual presentation. Have you considered using short video clips of what students would do hands-on, but stopping the videos at critical moments to ask students in an interactive online setting, "What happens next? What should we do next? What are your predictions?" If you've tried this or something like it, I'd love to hear how it went.

     
    2
    Discussion is closed. Upvoting is no longer available

    Holly Morin
    Catherine McCarthy
  • Icon for: Emily Hostetler

    Emily Hostetler

    Co-Presenter
    May 5, 2020 | 01:51 p.m.

    Hi Susan, I'm so happy to hear you appreciate our process!

    While we as a project team are moving forward with our upcoming Train-the-Trainer workshops to help chemistry professionals learn how to implement the LDC framework and train others to facilitate hands-on activities, the Museum of Science has been moving full speed ahead with "Science in Action" livestreams where we do just what you are asking! Check out https://www.mos.org/mos-at-home to find some of our videos and livestreams using zoom webinar. David and I actually hosted a live chemistry demo where I asked viewers about ocean acidification and it was so wonderful to ask questions and have viewers respond in the chat. It seems to be working really well for us live. I have not been a part of programs that try this pre-recorded yet. Although I am sure someone will be trying that soon in this Covid-19 world!

    Best,
    Emily

     
    2
    Discussion is closed. Upvoting is no longer available

    Holly Morin
    Catherine McCarthy
  • Icon for: David Sittenfeld

    David Sittenfeld

    Co-Presenter
    May 5, 2020 | 02:02 p.m.

    In addition to what Emily is saying about the MOS Live moments for predictions which is very relevant, we did a daylong in-person pilot workshop at the Association of Science and Technology Centers conference last year. During that workshop we showed videos from on-the-floor facilitation of hands-on activities and asked the workshop participants to think about what strategies they wanted to call out as seeming particularly effective.  I like the idea of freezing the videos at given intervals as you are suggesting, we didn't do that and I think it could be helpful...but I also know that things can get a little tricky doing too much of that online and watching videos via screenshare is problematic in and of itself.

     
    3
    Discussion is closed. Upvoting is no longer available

    Holly Morin
    Preeti Gupta
    Catherine McCarthy
  • Icon for: Shirley Lin

    Shirley Lin

    Higher Ed Faculty
    May 5, 2020 | 02:16 p.m.

    Thank you for this video on building positive attitudes towards chemistry. Do you have any research or thoughts upon how to change negative attitudes that may have developed for students exposed to a poor chemistry course in high school? At the college level, it frequently seems impossible to change these impressions once they are formed.

     
    1
    Discussion is closed. Upvoting is no longer available

    Catherine McCarthy
  • Icon for: Elizabeth Kollmann

    Elizabeth Kollmann

    Co-Presenter
    May 5, 2020 | 03:41 p.m.

    Shirley, thank you for your questions. This project was not focused on formal education, but we did learn some things through our research that might be applicable to that audience. As it says in our summary, we were focused on increasing museum visitors' interest in chemistry, understanding of its relevance, and feelings of self-efficacy (that they are able to do or talk about chemistry). Our thought was that focusing on these three areas would improve individuals' attitudes towards chemistry, which may also mean an improvement in attitudes about school chemistry. 

    Our research focused understanding the content and format strategies that can be included in hands-on activities that lead to increases in interest, relevance, and self-efficacy. I think that these strategies could also be used as a part of formal education activities. We found that to increase interest in chemistry, it is most important to focus on both the content and format of your activities. In particular, interest can be increased if content includes basic chemistry concepts and connections to everyday life, and if the format is interactive for the participants and allows for observation of phenomena. To increase understandings of relevance, the content that is included is particularly important. In this case, it is important to include content related to everyday life as well as applications and uses of chemistry. Finally, to increase feelings of self-efficacy, it is important to focus on the format of an activity. Formats that feel particularly important are ensuring that the activity is interactive and allows for use of tools and materials. 

    We are in the process of writing some journal articles about our findings, but in the meantime, you can learn more through our research to practice guide which can be found here: https://www.nisenet.org/sites/default/files/chem_ldc_framework_and_strategies_guide.pdf

     
    3
    Discussion is closed. Upvoting is no longer available

    Catherine McCarthy
    Kayla Berry
    David Sittenfeld
  • Icon for: Larry Bell

    Larry Bell

    Lead Presenter
    Sr. Advisor for Strategic Initiatives
    May 5, 2020 | 02:32 p.m.

    This is definitely an issue that was referred to in the National Academies 2016 report on Effective Chemistry Communication in Informal Environments. I will toss the question to my co-presenters who I think will have some thoughts about it.

     
    1
    Discussion is closed. Upvoting is no longer available

    Catherine McCarthy
  • May 5, 2020 | 06:18 p.m.

    Such terrific work! I am wondering how you assess "what works" at different points in your DBR process. I'm specifically thinking about measuring your progress toward the goals of stimulating interest, a sense of relevance, and feelings of self-efficacy.

     
    1
    Discussion is closed. Upvoting is no longer available

    Catherine McCarthy
  • Icon for: Elizabeth Kollmann

    Elizabeth Kollmann

    Co-Presenter
    May 6, 2020 | 09:30 a.m.

    Catherine, we had 15 activities that we modified based on our theoretical framework and then tested iteratively. After the educators made their modifications, we tested the activity with about 10 visitors groups. These groups were asked to rate whether the activity decreased or increased their interest in chemistry, understanding of relevance, and feelings of self-efficacy. We also asked visitors to tell us what about the activity made them feel this way. This helped us understand whether we were achieving our visitor goals and the design and content criteria that were most useful in impacting interest, relevance, and self-efficacy. The researchers then took the findings to the educators and had discussions about what about the activity needed to be tweaked to improve achievement of the goals. Each activity was tested and modified at least twice, but some activities were modified three times. This process helped us to improve our activities as well as the theoretical framework that we were creating.

  • Icon for: Hannah Sevian

    Hannah Sevian

    Higher Ed Faculty
    May 6, 2020 | 08:38 a.m.

    I'm so excited to see how far you have taken this amazing project! Seven years ago, I was trying to find a keynote speaker, when I chaired a Gordon conference on chemistry education, who could talk about chemistry activities and exhibits in museums, and that was when I learned what the NAS report you mention in your video also said -- that chemistry is extremely underrepresented in science museums. It is SO important that you are tackling this problem! The only active chemistry exhibits I could find were one at the Museum of Science and Technology in Chicago, and one at the Museum of Science in Boston. So I invited the person who developed the MoS exhibit, MJ Morse, who has since retired from MoS (it turns out that my now-collaborator, Marianne Dunne in our project, who also commented above, had worked with MJ when she worked at MoS -- small world we have in Boston!). At the Gordon conference, MJ presented findings about her "Chemistry of Smells" exhibit and associated PD for teachers at MoS, and the design of how she created it, which involved a lot of learning about and interviewing experts on the chemistry of smell. It was so impressive to learn about all the behind-the-scenes design that went into how she developed that excellent program! Which brings me to a wondering question about your project -- What are the ways that you are engaging with chemistry experts as you develop the outstanding informal science chemistry activities that you are sharing with museums and other informal science groups?

  • Icon for: Larry Bell

    Larry Bell

    Lead Presenter
    Sr. Advisor for Strategic Initiatives
    May 6, 2020 | 09:59 a.m.

    Hannah,

    Glad to hear about your past interaction with MJ. As for the current project, there are several ways in which we interacted with chemistry experts. From the beginning the project was conceived in collaboration with staff of the American Chemical Society. Mary Kirchhoff (who appears in the video) and I served on the National Academies committee that worked on the Effective Chemistry Communication report noted in the project description above on this page. After that report was completed we started to talk about this potential project. Eventually four or five members of the ACS staff, mostly those involved with education and public outreach, participated in the project in a variety of ways, serving on the leadership team and on several of the working groups. Some of them participated in weekly calls and all in periodic project team meetings. The project's official advisory committee, members of which participated in annual project meetings, involved several working chemists who were recommended by our ACS staff colleagues. The advisors also included a number of folks who were trained in chemistry but now worked outside the chemistry field, in fields like informal science education or social sciences. They participated in selected and analyzing the initial candidate activities for our design-based research testing, reviewing project progress from time to time, and helping us as we moved on to next stages of the project. And then we involved chemists as activities were tested on the floors of the Museum. One of the ways we did that was in connection with annual National Chemistry Week and Chemists Celebrate Earth Week events, during which a large number of chemists served as facilitators for hands-on activities in the Museum. And now we are designing an online train-the-trainer workshop and have 15 chemists who will be the participants in our pilot workshop. They will go through the experience and give us feedback to help us improve the workshop for later presentation to the broader chemistry community.  There's more but I have to run. One of my colleagues or I will finish the story when we can get back online.

  • Icon for: David Sittenfeld

    David Sittenfeld

    Co-Presenter
    May 6, 2020 | 10:09 a.m.

    Hi Hannah, in addition to all of the great stuff that Larry has shared - ACS also brought in a group of chemists who have particular focus and expertise on outreach using hands-on activities who serve on their Committee on Community Activities as external reviewers before the final step of our development process.  These people gave us very helpful feedback and advice on the activities and training materials in a way similar to what happens with a peer-reviewed paper.  We also got external review on our safety training guide and certain other materials.  

     
    1
    Discussion is closed. Upvoting is no longer available

    Holly Morin
  • Icon for: Leigh Peake

    Leigh Peake

    Informal Educator
    May 7, 2020 | 06:33 a.m.

    I always love seeing the great work going on at MOS -- thank you for sharing!

     
    1
    Discussion is closed. Upvoting is no longer available

    Elizabeth Kollmann
  • May 7, 2020 | 03:00 p.m.

    Great to see such outreach and interests spurring interests in Chemistry. Can you expound on the constructs and mechanisms of the theoretical framework and how it is connected by the interventions/programs and its relation to the interest, relevance, self-efficacy constructs?

     
    1
    Discussion is closed. Upvoting is no longer available

    Catherine McCarthy
  • Icon for: Larry Bell

    Larry Bell

    Lead Presenter
    Sr. Advisor for Strategic Initiatives
    May 7, 2020 | 03:36 p.m.

    Michael,  I think that everything you are looking for, and perhaps more, is laid out in this LDC framework and strategies guide.

    https://www.nisenet.org/sites/default/files/chem_ldc_framework_and_strategies_guide.pdf

    The project researchers are currently working on some journal articles that may fill in some of the details missing from the guide, and we are also working on some new pd resources. But for the moment, I will refer you to the guide and ask if that expounds on the matters you've asked about.  Thanks for your interest.

     
    1
    Discussion is closed. Upvoting is no longer available

    Kayla Berry
  • Small default profile

    Dianne Clark

    May 7, 2020 | 03:07 p.m.

    I would like to find out how Sci-Port Discovery Center in Shreveport, LA can be included in these discussions.  Louisiana did not have much involement in this study and we need to be involved!     Dianne Clark, Executuive Director of Sci-Port

  • Small default profile

    Dianne Clark

    May 7, 2020 | 03:12 p.m.

    Please pardon my errors!  I was watching a webinar and trying to type.  

  • Icon for: Larry Bell

    Larry Bell

    Lead Presenter
    Sr. Advisor for Strategic Initiatives
    May 7, 2020 | 03:55 p.m.

    Diane, thanks for your interest. I see that two locations in Louisiana got physical Let's Do Chemistry kits, but that Sci-Port was not one of them. For the immediate future, the digital kit is available online and has lots of resources developed in the project including a publication laying out the work of the project and both the framework and ideas about how to use it. You can find links to all of the components of the digital kit here: https://www.nisenet.org/chemistry-kit. We have done sessions and a workshop at ASTC conference and sent 250 kits out mostly to science and children's museums, so our current PD is focused on chemists and especially chemists that reach audiences under-represented in STEM.  We are just at the point of developing a workshop and then sharing the tools with others for conducting similar workshops. We'll keep your interest in mind as we move forward. In the mean time perhaps the guide that is part of the kit could be helpful   https://www.nisenet.org/sites/default/files/chem_ldc_framework_and_strategies_guide.pdf. There are lots of other training resources included in the digital kit as well. I hope they can be helpful as we work through the current phase of the project.

     
    1
    Discussion is closed. Upvoting is no longer available

    Catherine McCarthy
  • Icon for: Martin Storksdieck

    Martin Storksdieck

    Facilitator
    May 7, 2020 | 08:03 p.m.

    This is such a wonderful project, and I love the fact that you frame this as a follow-up to the NRC study!  Congrats to what seems like a successful projects across all sorts of criteria.  Two quick questions:

    How do professional chemists react to these opportunities?  Did you get a sense that they feel empowered to do outreach with these resources?  Have you been able to convince some to try who would otherwise not?

    Likewise: how are informal educators doing with them? 

     
    2
    Discussion is closed. Upvoting is no longer available

    Catherine McCarthy
    Holly Morin
  • Icon for: David Sittenfeld

    David Sittenfeld

    Co-Presenter
    May 8, 2020 | 10:12 a.m.

    Hi Martin!

    Our colleagues from ACS's Office of Science Outreach invited me to talk about the framework as part of their online OTP course for chemists earlier this year. There were dozens of participants (I can't remember how many exactly, but it was definitely a lot!) and from the comments these folks seemed very excited to learn about this empirically derived framework and and apply it to their outreach work.  

    With respect to informal educators -- we have heard that the kits are used on the floor regularly (both for daily programming and for special programs) in a number of institutions (or were being used that way until we hit our new normal).  I'm looking forward to seeing if/how partners are using them in this virtual world we're in. Emily and I actually adapted one of the activities to present a program at home for ACS's Chemists Celebrate Earth Week event a couple of weeks.  But that's all different from whether the ISE community is applying the framework itself to the design and facilitation of hands-on activities more broadly.  We did a pre-conference workshop at ASTC last year and felt like we learned a lot from it, and also will be working to share the findings in ways that Liz talked about in a different answer. But as Larry said above, this work with the workshops that target the chemistry community specifically beyond the timeline for our project.  All of that said, there's really no reason that these workshops could not be adapted for the ISE community going forward, we'd just want to apply learning from these workshops to such an effort.

     
    1
    Discussion is closed. Upvoting is no longer available

    Catherine McCarthy
  • Icon for: Larry Bell

    Larry Bell

    Lead Presenter
    Sr. Advisor for Strategic Initiatives
    May 8, 2020 | 11:53 a.m.

    Martin, let me add a little more to what David has already said. Project findings have mostly been rolled out through the distribution of 250 Let’s Do Chemistry kits late in 2018. The majority of those kits went to science or children’s museums, although some went to universities with chemistry outreach activities. In online reports we ask kit recipients to complete, the open-ended response boxes have provided us with a lot of different impacts on the organizations that received kits. I am including some of those comments here as they said some pretty interesting things.

    We heard the following kinds of things from university-based chemistry professionals:

    “The emphasis on increasing INTEREST, RELEVANCE and SELF-EFFICACY rather than expertise in facts is empowering to freshman college students in presenting activities to youngsters.”

    “The Let's Do Chemistry kits were great inspiration for me to develop new chemistry programming.”

    “The Explore Science: Let's Do Chemistry kit was ultimately an invaluable tool in helping the [our] Student ACS chapter to reach out beyond the confines of the chemistry department.

    “The Let's Do Chemistry kit has also helped to enhance our undergraduate STEM instruction in several ways. We use outreach events to develop public communication of science skills in our undergraduate STEM majors and the NISENet kits and training materials have played a huge role in enhancing the quality of our instruction in this area. “

    “We now have an established collaboration with [a science center] and already have many plans in the works to grow our relationship and provide more chemistry outreach activities to the community. This would not be possible without the NISE kits!”

    Museum professionals commented on how this project helped them add chemistry content to their offerings, train staff and volunteers in facilitation techniques, provided models for how they will do other work within their organizations, and even changed their institutional roles within their communities!

    “These kits have provided [our museum] with valuable chemistry resources that we will continue to use for years.”

    “I think the most profound impact the kit has had on our organization is just thinking about chemistry more. As an institution that focuses on younger children, we have struggled with how to relate chemistry to our audience and present it in a way that's engaging and applicable to them”

     “Overall the Let's Do Chemistry Kit has been immensely helpful because we did not have the knowledge or the skill set on staff to create chemistry specific programs.”

     “The training videos, helped them understand the processes of facilitation, as well as see how they would be done by someone else. Many of the staff members do not have a lot of experience communicating science effectively, so having this exposure is invaluable to our staff.“

    “The introduction of Chemistry concepts using the kits has allowed volunteers to comprehend and teach topics and improve their confidence in facilitation techniques. These skills will transfer to other activities and programs at our museum.”

    “The toolkit has contributed modification in staff's facilitation techniques, activity development techniques, and has been incorporated into various Museum offerings.”

    “I have now started to create facilitator guides, modeled after yours, for every activity we create! They are wonderful and so useful to pass on so others can use the activities.”

    “Our event was the first of its kind bringing science into our Children's Museum.”

    “The Let's Do Chemistry kit has allowed us to reach children of an older age and give confidence to our staff to be able to conduct activities that will hold their audience's attention…allowing [our children’s museum] to start to be known for creating learning experiences for a broader age range.”

    Most of these comments come from the museum staff rather than the broader chemistry community. The ACS staff who have been involved in this project have been very supportive and enthusiastic, and have been using project materials and techniques in their own work. The real roll out into the chemistry community is only now just beginning with the train-the-trainer workshops we are implementing with a pilot starting today. A year from now, we will have more data to support an answer to your question.

     
    2
    Discussion is closed. Upvoting is no longer available

    Catherine McCarthy
    Holly Morin
  • Icon for: Martin Storksdieck

    Martin Storksdieck

    Facilitator
    May 8, 2020 | 12:13 p.m.

    This is great feedback, thanks for sharing, Larry.  Not to put you on the spot, but since there might have been reasons why chemistry was underrepresented as a discipline despite its salience: What did you learn about barriers or challenges to doing hands-on chemistry as outreach or museum programming? 

     
    2
    Discussion is closed. Upvoting is no longer available

    Alex DeCiccio
    Holly Morin
  • Icon for: Larry Bell

    Larry Bell

    Lead Presenter
    Sr. Advisor for Strategic Initiatives
    May 8, 2020 | 01:21 p.m.

    Martin, I can't say that we systematically studied that question. We thought that the National Academies report laid what past studies had found to be barriers, such as the general nature of the field, conceptual challenges associated with the scale of molecules, messiness, potential danger, bad personal experiences with the topic, etc. The National Academies report made a number of references to NISE Net, which had the original challenge of engaging public audience in learning about nanotechnology. That topic has a number of the conceptual and practical challenges that chemistry has. So we adopted the NISE Net approach to rolling ideas and materials out to the field supported by the design-based research work. One of the key ideas behind the NISE Net approach is to make the materials as easy to use as possible. As you read in some of the comments I posted above, museum staff at many places seem to lack particular experience or expertise in chemistry. For those who were daring enough to try expanding their offerings into chemistry, the outcomes have not always worked well. Here's a comment from one of the kit recipients who lays out the problem: 

    "It's always a challenge to come up with new exciting floor kits, and even if you find one that sounds good, it can be difficult to gather the right materials for testing.  If you do manage to do that, it is frustrating if the idea doesn't work out. Having  a pre-made kit handed to us eliminates part of the difficulties. This is especially true when we come across an activity that sounds good in theory, but difficult in practice. A perfect example is the Sublimation Bubbles kit. It's hard to predict how our visitors will react to something like dry ice, and if it will be safe to do with our youngest visitors. But to have something that has been tried and tested handed to us, with a demo video, makes it a lot easier to understand how the activity actually works, and helps us envision how to cater it to our audience.” (from a children’s museum)

    So I think the answer to your question that I can draw from the comments and anecdotes is perhaps that chemistry is a little harder than other topics in a number of different ways. There are conceptual challenges, communication challenges, and practical challenges. Taken together perhaps these add up to inhibit the presentation of hands-on chemistry in science and children's museums. But it does appear that if someone does the hard work, and makes things really easy to use, and supplies people with everything that they need, then the field embraces the topic and asks for more.

    Now there are two really big questions moving forward. Several of the comments suggest that kit recipients learned some things from using the kit that they are going to apply to their own work. Will that actually translate to an increased level of chemistry content going forward? The museums may take what they learned from this project and make new engaging physics or biology activities, which is great. We are focusing these upcoming train-the-trainer workshops on members of the chemistry community because they are the ones with the long-term commitment to chemistry as a topic. Chemists, or chemistry students, working with museum staff might be able to increase effective chemistry engagement in informal settings in an ongoing way. We've already seen that in the ongoing work of the ACS staff. We'll know more about how that is going in a year.

    The second big question is whether we can apply what we have learned to exhibits. It's definitely more challenging to do wet chemistry in exhibits than in demonstrations or facilitated hands-on activities. The advantage of exhibits is that they are accessible to all visitors whenever the museum is open, instead of only when a staff member or volunteer is out with an activity cart. But exhibits can end up making the chemistry more abstract and remote. So the format has its own additional challenges.

    As the research team writes up final reports and publishes papers about the project work, perhaps we will have more solidly supported answers to your question, but this is the way it looks to me from the things participants have told us so far.

     
    3
    Discussion is closed. Upvoting is no longer available

    Catherine McCarthy
    Alex DeCiccio
    Kayla Berry
  • Icon for: Lily Raines

    Lily Raines

    Co-Presenter
    May 8, 2020 | 01:27 p.m.

    I'd like to second all the above! And to also add that I'm really looking forward to the feedback we get specifically from the chemistry community on our train-the-trainer workshops.

    While not directly reflecting opinions of the framework per se, the ACS Outreach Training Program (www.acs.org/otp - which is free to everyone!) highlights the framework in the context of activity facilitation. 84 people have completed that course online and 20 in person, and reviews are overwhelmingly positive.

     
    1
    Discussion is closed. Upvoting is no longer available

    Kayla Berry
  • Icon for: David Sittenfeld

    David Sittenfeld

    Co-Presenter
    May 9, 2020 | 11:53 a.m.

    Thank you Lily!  I went back and watched the section of the online OTP on hands-on activities (section 4) that you, Patti, and David H. kindly invited me to present on.  As part of that online workshop, Patti conducted a poll to ask the participants who had heard of the Let's Do Chemistry Kit.  Out of the people who watched that day, 67% had never heard of it.  So that gave us good confidence that we're bringing this framework to a group of people who will do great things with it, and points to some nice potential for these online workshops we're hosting together.

  • Icon for: Martin Storksdieck

    Martin Storksdieck

    Facilitator
    May 7, 2020 | 08:09 p.m.

    In-person vs online professional development workshop for teaching hands-on skills: in 2004, the Astronomical Society of the Pacific, with a bunch of other partners, conducted AFGU = Astronomy from the Ground Up, which taught hands-on outreach activities to informal educators. We used an in-person model (weekend) and compared it with an equivalent partially asynchronous online workshop (3 weeks). It was fascinating to think about equivalency here, but here a few nuggets I remember: Both worked, but only because we constructed them both from a clear conceptual framework. We knew precisely why we did what. And probably not a surprise: Since in the online version educators practiced at their home institution with real audiences, rather than with colleagues at some remote workshop, transfer was better. They used it more.

     
    2
    Discussion is closed. Upvoting is no longer available

    Alex DeCiccio
    David Sittenfeld
  • Icon for: Larry Bell

    Larry Bell

    Lead Presenter
    Sr. Advisor for Strategic Initiatives
    May 7, 2020 | 11:07 p.m.

    Martin, that's a very interesting outcome. As we began to shift over from in-person workshops to online workshops, we began to feel that the resources would are creating for the online versions may be more useful, or easier to use, for the workshop participants. But really it was the current environment that pushed us in that direction. Your note suggests that the online version may indeed provide better results. It's helpful to have such a positive prospect.

     
    3
    Discussion is closed. Upvoting is no longer available

    Alex DeCiccio
    Holly Morin
    David Sittenfeld
  • Icon for: Nickolay Hristov

    Nickolay Hristov

    Facilitator
    May 9, 2020 | 08:50 a.m.

    I find these findings about online vs. in-person professional development intriguing as well as intuition and practice to date seem to suggest otherwise . Caution is due against overgeneralizing what might be a specific case and yet, it is good to bring up the topic for discussion and study further.  I would like to stay in the conversation and pick up similar snippets of the picture from different conversations and videos/projects.  Please share additional insight or references if you have any. 

     
    1
    Discussion is closed. Upvoting is no longer available

    David Sittenfeld
  • Icon for: David Sittenfeld

    David Sittenfeld

    Co-Presenter
    May 9, 2020 | 09:02 a.m.

    Martin's case study, along with your good caveat, makes me think that an interesting next idea to test would be a series of online workshops with practitioners from around the country, pairing up local informal educators and chemists and allowing them to optimize their activity and trainings, test their adaptations out at their local science center, and share what they learn with the larger national cohort.  That would require a dedicated project, but I think it could be an interesting model for exploration here based on this conversation.

     
    2
    Discussion is closed. Upvoting is no longer available

    Holly Morin
    Martin Storksdieck
  • Icon for: Larry Bell

    Larry Bell

    Lead Presenter
    Sr. Advisor for Strategic Initiatives
    May 9, 2020 | 10:10 a.m.

    Those interested in this topic, using online platforms for professional development, might like seeing Sue Allen's video and the related discussion.  She says:

    "mastering the medium turns out to be much more complex than one might think at first glance - after years of doing this our coaches have become extremely skilled at the nuances of supporting social interaction and trouble-shooting, while simultaneously providing the PD activities. It's hugely challenging for someone new to it."

    And she cites a publication about that work. Her insights could be helpful for learning how to do it well.

     
    3
    Discussion is closed. Upvoting is no longer available

    Catherine McCarthy
    Holly Morin
    David Sittenfeld
  • Icon for: Martin Storksdieck

    Martin Storksdieck

    Facilitator
    May 9, 2020 | 11:36 a.m.

    Agreed, Larry: we can learn a lot from their project.

  • Icon for: Preeti Gupta

    Preeti Gupta

    Facilitator
    May 7, 2020 | 08:18 p.m.

    Thank you for such rich and comprehensive conversation. You have done so much to develop and facilitate activities that do make a difference to visitor attitudes about chemistry. Have you thought about following up with visitors after a few weeks or months? How is this project informing your next inquiry? 

     
    1
    Discussion is closed. Upvoting is no longer available

    Catherine McCarthy
  • Icon for: Elizabeth Kollmann

    Elizabeth Kollmann

    Co-Presenter
    May 8, 2020 | 09:11 a.m.

    Preeti, given the scope of our study, we were not able to do follow-up with participants, but I would love to do that. Maybe that can be our next project. 

    I am continuing to apply the theoretical framework produced through this project to new work, thinking in particular about the content and format strategies that should applied to new exhibits and programs if they have goals of interest, relevance, or self-efficacy. We have now seen very similar findings about relevance as a part of a NISE Net study and this ChemAttitudes study which make me think that findings about this goal are likely to apply to many STEM content areas.

  • May 8, 2020 | 07:51 p.m.

    Thank you for developing these resources.  I use them extensively as part of our visits to elementary schools and for public events! https://www.instagram.com/fresnochemclub/

     
    2
    Discussion is closed. Upvoting is no longer available

    Catherine McCarthy
    Elizabeth Kollmann
  • Icon for: Larry Bell

    Larry Bell

    Lead Presenter
    Sr. Advisor for Strategic Initiatives
    May 9, 2020 | 10:15 a.m.

    Dermot, I am glad that they are so useful for you.  For those who haven't found them yet, the digital version of all of the resources developed in this project can be found at https://www.nisenet.org/chemistry-kit.

  • May 9, 2020 | 05:13 p.m.

    Love this beautifully filmed video. This would get anyone and everyone interested in chemistry. We're doing work on STEM stereotypes that dissuade young girls from fully engaging on STEM topics. Are you finding that your resources and approaches can also help regarding gender stereotype issues that plague our society?

     
    1
    Discussion is closed. Upvoting is no longer available

    Catherine McCarthy
  • Icon for: Elizabeth Kollmann

    Elizabeth Kollmann

    Co-Presenter
    May 9, 2020 | 06:00 p.m.

    Andrew, we were not specifically looking to counteract gender stereotypes with this project. However, 64% of our focus subjects for this study were women and girls. Therefore, I believe that creating activities using the criteria from our theoretical framework would increase many females' interest in chemistry, understandings of its relevance, and feelings that they are able to do and talk about chemistry.

  • Icon for: Larry Bell

    Larry Bell

    Lead Presenter
    Sr. Advisor for Strategic Initiatives
    May 9, 2020 | 05:39 p.m.

    Andrew, I suppose you can say that simply because the project design focused (in part) on self-efficacy, and the groups with which the activities were tested included both boys and girls mixed, the activities were developed in a way that would support girls perceptions that chemistry is something they could do. I'll let others on the project talk more about how that played out in the way the research was conducted. But I will share a few short anecdotes that recipients of a Let's Do Chemistry kit reported to us:

    • We had a young girl attend the event dressed in a lab coat. Her mother was thrilled by the event and shared with us how few science events are available for young girls. At the end of the evening, one of our Dymax Volunteers came up to me and was in tears. She said that her night was made when she met her younger self. She was referring to the young girl in the lab coat.
    • A student was intrigued with the notion that what she experienced was actually chemistry. She thought only lab coat-clad older men did chemistry. But now she can claim that she is as chemist too!
    • The most memorable moment from our Explore Science: Let's Do Chemistry  event has to be when a group of Girl Scouts were participating in the Sublimation Bubbles station. They were having a blast learning all about the bubbles, dry ice, and the chemistry behind it all when one of them turned to their troop leader and said, "We're like real scientists now!"
     
    1
    Discussion is closed. Upvoting is no longer available

    David Sittenfeld
  • May 9, 2020 | 07:09 p.m.

    Thank you Elizabeth and Larry: 64% is a big number and "We are like real scientists now!" is priceless. 

  • Icon for: Holly Morin

    Holly Morin

    Marine Research Associate
    May 10, 2020 | 10:57 a.m.

    Your content rich video is great and tells the story of your project quite well--nicely done! As an informal educator/marine scientist, I really appreciate the iterative approach you took to designing your program's activities. About how long do you feel like that design process took? Any tips for others who might be looking to implement a similar approach to developing their STEM outreach activities?

    I look forward to exploring the resources linked above and thinking about how things might be integrated into our chemical oceanography camps and after school programs in the future!

     
    1
    Discussion is closed. Upvoting is no longer available

    Alex DeCiccio
  • Icon for: Elizabeth Kollmann

    Elizabeth Kollmann

    Co-Presenter
    May 10, 2020 | 02:24 p.m.

    Holly, we used a design-based research process. This allowed for iterative refinement of activities and a theoretical framework at the same time. This can be quite a long process, but it is worth it. I feel our activities and research findings were better for having used this process. We tested 15 activities which were modified using the content and design criteria from the framework. These activities were then further modified, based on our data, two or three more times. This allowed us to create activities that were more likely to achieve project goals as well as better understand the design criteria that led to these outcomes. I believe that the entire data collection process took about a year, but we took even longer to do a thorough analysis of all of the data that we collected. 

     
    2
    Discussion is closed. Upvoting is no longer available

    Alex DeCiccio
    Holly Morin
  • Icon for: Holly Morin

    Holly Morin

    Marine Research Associate
    May 10, 2020 | 02:50 p.m.

    Wow, a full year (or longer)! Fantastic to take the time to go through that process, especially for 15 activities. It's clear the effort was worth it.  Thank you!

  • Icon for: Larry Bell

    Larry Bell

    Lead Presenter
    Sr. Advisor for Strategic Initiatives
    May 10, 2020 | 04:27 p.m.

    Holly and Liz, if I remember right, we did the 15 activities in batches of 5 at a time, and this wasn't just a formative evaluation process, which we use all the time. Along with improving each batch of activities, we were also trying simultaneously to build a generalizable theoretical framework supported by the project data. I think that resulted in added steps and time in the DBR process as compared to the steps and time needed for more traditional formative evaluation. Does that seem true to you, Liz? But in the end you don't just have the finished activities, you have the framework, which can be applied to all sorts of future activities.

  • Icon for: Elizabeth Kollmann

    Elizabeth Kollmann

    Co-Presenter
    May 11, 2020 | 02:16 p.m.

    Larry, that's right. We did slightly different data collection than we typically do for traditional formative evaluation. The data collection focused not only on goal achievement but also on design strategies that participants felt helped them to achieve the goals. We did some quick analysis first to help educators understand how their activities were doing and how they could be improved. We then did some additional analysis to inform the theoretical framework. Some of these data also helped us think about additional criteria or activities to test. Because educators and researchers work so closely together as a part of design-based research, and because of the need for additional focus on the framework, DBR is a slightly longer process than formative evaluation. The research team even mentioned that they wished they had even more time while we were collecting and analyzing data.

  • Icon for: Alex DeCiccio

    Alex DeCiccio

    Media and Production Specialist
    May 10, 2020 | 11:05 p.m.

    Your work is wonderful and I enjoyed seeing how a this group identified a need and took a novel approach to find ways to help close the gap. 

    • What are the best ways of reaching practitioners of informal education and public outreach broadly throughout a field like chemistry?

    in response to this prompt from the first post, also learning from comments and concepts embedded in this showcase, are there ways you can tie the design based research approach to identity? Would elements of storytelling techniques such as place and emotion connected to a summation of personal events that prompted change or a dramatic event help drive these concepts home in the settings you’ve described?

  • Icon for: Larry Bell

    Larry Bell

    Lead Presenter
    Sr. Advisor for Strategic Initiatives
    May 11, 2020 | 08:34 a.m.

    Alex, thanks for your comment and question. In a way, including the girl in the video who says at the end "I wasn't expecting that," and a number of the stories that kit recipients sent us in their reports ("We're like real scientists now!") connect the interest, relevance, and self-efficacy piece with identity and, I think, show the power of storytelling. That together with the research work and outcomes, I hope will be compelling. We are just starting a pilot workshop with members of the chemistry community and we will see what they think in the next few weeks.

     
    1
    Discussion is closed. Upvoting is no longer available

    Alex DeCiccio
  • Icon for: David Sittenfeld

    David Sittenfeld

    Co-Presenter
    May 11, 2020 | 10:20 a.m.

    To add to what Larry is saying, I could see some of this in the "deepen understanding" piece of the facilitation framework component.

    In that deepen understanding piece, the activity facilitator/educator is advised to consider strategies that 

    "encourage participants to draw from their own experience, make observations, and test their answers", "encourage them to make connections to everyday life and societal issues", and "contribute ideas and information from your {the facilitator's} own experience."

    I can see clear connections between identity, narrative, and these facilitation strategies.

     
    1
    Discussion is closed. Upvoting is no longer available

    Alex DeCiccio
  • Icon for: Alex DeCiccio

    Alex DeCiccio

    Media and Production Specialist
    May 11, 2020 | 07:41 p.m.

    Thanks Larry and David. One more follow up, is there room to tie subject matter exports to engage your audiences remotely in the current context? I see related concepts discussed in the thread above but wonder how we can engage deeply , with a sense of community, when sitting in front of a screen. What are your thoughts? 

  • Icon for: David Sittenfeld

    David Sittenfeld

    Co-Presenter
    May 11, 2020 | 07:58 p.m.

    Alex, your question is fascinating, and one of the interesting things here to me is to consider what "subject matter experts" actually means in this context. I think perhaps it could mean people who are really committed to an skilled at chemistry outreach. There a number of ways that I imagine these kinds of people could be involved in building community and sharing the learning that is embodied in our framework. The most direct is the train-the-trainer model we are employing. We have designed the workshops that participants who complete it will go back and train others that they know and are in a position of facilitating hands-on chemistry activities. In addition to this train-the-training, though, I see that these workshops are in and of themselves creating networks of subject matter experts in the realm of hands-on chemistry outreach and that these folks can serve as mentors for one another and build a collective community and a shared sense of self-efficacy.

     
    1
    Discussion is closed. Upvoting is no longer available

    Alex DeCiccio
  • Icon for: Alex DeCiccio

    Alex DeCiccio

    Media and Production Specialist
    May 12, 2020 | 07:07 a.m.

    Thanks David and I agree with your context specific consideration of a "subject matter expert" as someone who is committed and skilled in chemistry outreach. I also like how that is a pretty inclusive way of looking at the role. That's really great.

    The example you provide with the train-the-trainer model is awesome. There must be a visible "ripple" effect. Creating more synergies and committed people around the work while helping to close the gap you've identified in the video and overall project around outreach activities with chemistry.

    You are helping me understand not only your work and project more clearly but also sparking ideas for how we do our work as well. Thank you.

  • May 12, 2020 | 09:44 a.m.

    Thank you Larry - you always tackle the hard challenges! It's great to see that you've made so much progress on the activity front - I wasn't at all sure that there would be rich kits of chemistry activities that could address the challenges of "safety and mess" but you've done it, and with a strong theoretical framework too. Congratulations!

    One thing I wonder is whether you've looked at the role of parents/caregivers in the work? I'm thinking that as we all pivot more toward families spending time doing joint educational activities at home, it would be helpful to know what kinds of curiosity, concerns, experiences, and tolerances parents have for chemistry in particular. And maybe you already captured some of that in the family work you've done in the museum context... sorry if I missed this in the earlier discussion. Thanks.

     
    1
    Discussion is closed. Upvoting is no longer available

    Catherine McCarthy
  • Icon for: Larry Bell

    Larry Bell

    Lead Presenter
    Sr. Advisor for Strategic Initiatives
    May 12, 2020 | 10:12 a.m.

    Thanks, Sue. We didn't really focus on parents as facilitators in this project. Our model was based on museum educators and volunteers, and chemists and chemistry students doing National Chemistry Week activities, as the facilitators of the hands-on activities. The kits were distributed in 2018 and the prospect of a closing of everything was nowhere on our minds. That said, the guidelines for facilitation, which drew heavily from the Exploratorium's guidance on facilitation, can be applied by parents as well as by museum volunteers or chemists doing outreach. But your question, I think, goes more toward parent's attitudes and predispositions concerning chemistry. And I don't think I really have an answer about that.  Dietram Scheufele's group at UW-Madison did work as a part of this project with public audiences online in the absences of any of the hands on material, and their findings might provide some leads for research that specifically focuses on parents. Similarly, the research teams at the Museum of Science and the Science Museum of Minnesota observed and interviewed groups interacting with museum facilitators, and perhaps comments by the adults in the groups might also point to some further questions related to what you are asking. David and his team have done some online programing in the last couple of weeks and Liz's group is working on evaluation. Maybe they have something to contribute to this discussion.

  • Icon for: David Sittenfeld

    David Sittenfeld

    Co-Presenter
    May 12, 2020 | 10:38 a.m.

    Larry's discussion about the research work led by Liz's and Dietram's groups is spot-on but let me just add in something more anecdotal from my perspective. I have observed (again, anecdotally) that parents and adults may bring their children to large-scale wow-style chemistry demonstrations but need to be calibrated with relevance from the beginning - and throughout -- or else there is the risk of tuning out.  This doesn't seem to happen the same way with hands-on activities, when they are well-facilitated. Perhaps it's because of concerns that parents have about kids either messing things up or being unsafe - I don't know.  Obviously, this is part and parcel with the format, so it's not really revolutionary to say this, but there seems to be something that may inherently prime the adults for participation in relevance discussions with hands-on activities that may need more establishment with more demo-type interactions.  We didn't study this at all, but I am bringing it up because obviously our new normal is more like demos in that even hands-on activities can be modeled but it's harder to actually get people doing them online in real time. So something to think about - perhaps the relevance piece can be imparted to adults in training them how to do these activities with their kids, rather than just doing the activities demo-style, so that the adults get more out of the experience?  This obviously would need evidence to support it since I am just spinning my wheels here.

     
    1
    Discussion is closed. Upvoting is no longer available

    Elizabeth Kollmann
  • Icon for: Elizabeth Kollmann

    Elizabeth Kollmann

    Co-Presenter
    May 12, 2020 | 12:01 p.m.

    Sue, as Larry said, we really focused on the facilitation that educators gave to visitors as opposed to parent-child or within group interactions. We based this work on facilitation strategies established by the Exploratorium's Tinkering Studio, but gave them our own spin for our context and project. We focused on three phases of facilitation, invite participation, support exploration, and deepen understanding. We found that facilitators spent most of their time supporting exploration during our activities. Not too surprisingly, it seemed that supporting exploration facilitation helped visitors' self-efficacy, and that deepening understanding moves appeared to support relevance. 

  • May 12, 2020 | 01:53 p.m.

    Great project to bring more chemistry instruction to informal learning environments! 

  • Icon for: Larry Bell

    Larry Bell

    Lead Presenter
    Sr. Advisor for Strategic Initiatives
    May 12, 2020 | 04:59 p.m.

     Thanks for your comment, Christine.

  • Icon for: David Sittenfeld

    David Sittenfeld

    Co-Presenter
    May 12, 2020 | 07:20 p.m.

    I wanted to thank everyone for coming to watch our video and for the thought-provoking comments and questions.  This discussion has sparked me to envision some potential extensions of this work that I likely would not have thought of otherwise.

  • Further posting is closed as the showcase has ended.