The following summative report describes overarching evaluation findings from the evaluation of Leap into Science, including future considerations for the Leap into Science program team (program team). The report aims to summarize takeaways from 2018-2023 and report on overall insights pertaining to the core evaluation questions of interest.
In fall 2019, the Bell Museum received funding via a NASA TEAM II grant to create Mars: The Ultimate Voyage, a full-dome planetarium show and accompanying hands-on activities that focus on the interdisciplinary roles that will be needed to send humans to Mars. This report from Catalyst Consulting Group presents the findings from the summative evaluation completed in March–May 2023.
The Children’s Museum will collaborate with six Hartford Public Library branches, three Hartford Family Centers, and the Connecticut Children’s Medical Center to provide hands-on Science, Technology, Engineering, Arts, and Mathematics (STEAM) - based programs to over 1,000 local 3 to 14-year old children and their care givers. Program design and development will include planning for field trips to the museum. All participants will be given age-specific, supplemental STEAM materials to continue their learning activities at home, and families can attend more than one week of library programs, or more than three Saturdays of family center programs. The goal will be to help urban Hartford youths find new pathways toward responsible citizenry and fiscal stability.
The Aldrich Contemporary Art Museum will amplify its partnership with Hart Magnet School, a Title 1 elementary school in urban Stamford, Connecticut, by increasing exposure and access to the arts for first-fifth graders, their families, and educators. A new program model, leveraging the museum's artist exhibitions, will focus on technology and an inquiry-based approach to science. Students, educators, and families will be encouraged to see and think in new ways through on-site STEAM tours at the museum, artist-led workshops at Hart, teacher professional development, and afterschool family activities. Outside evaluators will work with the project team to develop goals and associated metrics to measure how the model of museum-school partnership can enhance student achievement, engage families more deeply in their child's school experience and community, and contribute to teacher professional development. The evaluator will also train museum staff on best practices for program assessment.
We present results of an experimental study of an urban, museum-based science teacher PD programme. A total of 125 teachers and 1676 of their students in grades 4–8 were tested at the beginning and end of the school year in which the PD programme took place. Teachers and students were assessed on subject content knowledge and attitudes towards science, along with teacher classroom behaviour. Subject content questions were mostly taken from standardised state tests and literature, with an ‘Explain:’ prompt added to some items. Teachers in the treatment group showed a 7% gain in subject content
As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program funds innovative research, approaches, and resources for use in a variety of settings. This project will develop a national infrastructure of state and regional partnerships to scale up The Franklin Institute's proven model of Leap into Science, an outreach program that builds the capacity of children (ages 3-10) and families from underserved communities to participate in science where they live. Leap into Science combines children's science-themed books with hands-on science activities to promote life-long interest and knowledge of science, and does so through partnerships with informal educators at libraries, museums, and other out-of-school time providers. Already field-tested and implemented in 12 cities, Leap into Science will be expanded to 90 new rural and urban communities in 15 states, and it is estimated that this expansion will reach more than 500,000 children and adults as well as 2,700 informal educators over four years. The inclusion of marginalized rural communities will provide new opportunities to evaluate and adapt the program to the unique assets and needs of rural families and communities.
The project will include evaluation and learning research activities. Evaluation will focus on: 1) the formative issues that may arise and modifications that may enhance implementation; and 2) the overall effectiveness and impact of the Leap into Science program as it is scaled across more sites and partners. Learning research will be used to investigate questions organized around how family science interest emerges and develops among 36 participating families across six sites (3 rural, 3 urban). Qualitative methods, including data synthesis and cross-case analysis using constant comparison, will be used to develop multiple case studies that provide insights into the processes and outcomes of interest development as families engage with Leap into Science and a conceptual framework that guides future research. This project involves a partnership between The Franklin Institute (Philadelphia, PA), the National Girls Collaborative Project (Seattle, WA), Education Development Center (Waltham, MA), and the Institute for Learning Innovation (Corvallis, OR).
In this case study, we highlight the work of the Bay Area STEM Ecosystem, which aims to increase equity and access to STEM learning opportunities in underserved communities. First, we lay out the problems they are trying to solve and give a high level overview of the Bay Area STEM Ecosystem’s approach to addressing them. Then, based on field observations and interviews, we highlight both the successes and some missed opportunities from the first collaborative program of this Ecosystem. Both the successes of The Bay Area STEM Ecosystem--as well as the partners’ willingness to share and examine
Chemistry is an important and widely relevant field of science. However, when compared with other STEM content areas, chemistry is under-represented in U.S. science museums and other informal educational environments. This project will build, and build knowledge about, innovative approaches to delivering informal science learning activities in chemistry. The project will not only increase public interest and understanding of chemistry but also increase public perception of chemistry's relevance and increase the public's self-efficacy with respect to chemistry. This project outcomes will include a guide for practitioners along with activity materials that will be packaged into a kit, distributed, and replicated for use by informal science educators, chemists, and chemistry students at 250 sites across the U.S. The project team will reach out to organizations that serve diverse audiences and diverse geographic locations, including organizations in rural and inner-city areas. The kits will provide guidance on engaging girls, people with various abilities, Spanish speakers, and other diverse audiences, and include materials in Spanish. Written guides, training videos, and training slides will be included to support training in science communication in general, as well as chemistry in particular. This project is supported by the Advancing Informal STEM Learning (AISL) program funds research and innovative resources for use in a variety of settings, as a part of its overall strategy to enhance learning in informal environments.
This project will take an innovative approach to develop informal educational activities and materials about chemistry. Rather than starting with content goals, the project will start with a theoretical framework drawn from research about affecting attitudes about science related to interest, relevance, and self-efficacy. A design-based research approach (DBR) will be used to apply that framework to the development of hands-on educational activities about chemistry, while also testing and modifying the framework itself. (DBR blends empirical educational research with the theory-driven design of learning environments.) Existing or new educational activities that appear to embody key characteristics defined in the framework will be tested with public audiences for their impact on visitors. Researchers and educators will determine how different characteristics of the educational activities defined in the framework affect the outcomes. The activities will be modified and tested iteratively until the investigators achieve close alignment between framework and impacts.. The project team will continue the design-based research approach both to examine groups of activities in which synergies can have impacts beyond single interactions as well as to examine varied ways of training facilitators who can also significantly affect outcomes. In this way, the project will generate knowledge about how kits of hands-on informal learning activities can stimulate attitudes of interest, relevance, and self-efficacy with respect to the neglected field of chemistry. The project teams will broadly disseminate project outcomes within the educational research, science and informal Science, Technology, Engineering and Mathematics (STEM) education communities. While this project will focus on chemistry, the strategies it will develop and test through a design-based research process will provide valuable insight into effective approaches for informal STEM education more broadly.
Our Sky is a series of Out-of-School Time and museum educational programs that inspire an appreciation and understanding of Earth and Space Science (ESS) in a diverse population of children ages 3-10 and their parents, caregivers and educators. All resources are developed through a partnership between Boston Children’s Museum and Smithsonian Astrophysical Observatory in Boston and Cambridge, Massachusetts. The vision for Our Sky is that children ages 3-10 and their adult caregivers will gain an appreciation for celestial objects and phenomena as a foundation for understanding of Earth and Space Science. All resources and activities will be designed to realize this vision, and to:
Serve a diverse range of audiences, with an emphasis on urban and low-income children and families;
Increase appreciation among diverse adults and children of the sky as an accessible science learning resource;
Share NASA resources with, and help develop foundational STEM skills in children;
Encourage adults to engage in and guide ESS learning experiences with children;
Inspire practical application of STEM skills by children and adults as they explore celestial objects together; and
Expand the capacity of museum staff and afterschool educators to engage families in learning STEM skills through ESS exploration.
Our Sky activities will result in:
A series of museum-based programs that incorporate NASA resources and ESS activities across a range of content areas; and
A series of new ESS-focused activities for the award-winning “Beyond the Chalkboard” afterschool curriculum that include NASA resources and will reach hundreds of thousands of children around the world.
How do informal learning organizations work with schools as part of a broader educational ecology? We examined this question through a comparative case study of two collaborative efforts whereby informal arts education organizations, a children's museum and a community-based organization, worked with an urban school district to redefine the provision of educational services for children and youth. Grounded conceptually in organizational theory, our study identified factors that enable and constrain collaboration across the formal-informal divide. We argue that examining the dynamics of cross
The article assesses professional development in the field of science through curricular and instructive methods in the museum. The National Science Education Standards, along with independent researchers, confirm and stress the importance of quality professional development for elementary science educators which can be provided by museum services and models. The study involved participants from two different elementary schools within the same urban district serving a Latino student body, which were recognized as bottom tiers in the academic performance index of California's Department of
With the Museum's increasing interest in urban biodiversity, we have started looking at all types of wildlife in our highly modified industrial, suburban, and urban habitats. One thing that quickly struck us was that in our own backyard, Exposition Park, nobody had documented any lizards since 1988. This seemed strange, as lizards are common in other parts of Los Angeles, and it led to the question, "Why are there no lizards here?" We hope to answer this question with the LLOLA (pronouced "lola") project. LLOLA aims to do two things: 1) Confirm the presence or absence of lizards in Exposition park. (After all, nobody has looked extensively for them! 2) Find out where lizards DO occur in the Los Angeles Basin, and start to hypothesize why they can survive there.