This document summarizes lessons learned from implementing Leap into Science: Cultivating a National Network for Informal Science and Literacy (Leap into Science) from 2017-2023.
The following summative report describes overarching evaluation findings from the evaluation of Leap into Science, including future considerations for the Leap into Science program team (program team). The report aims to summarize takeaways from 2018-2023 and report on overall insights pertaining to the core evaluation questions of interest.
In fall 2019, the Bell Museum received funding via a NASA TEAM II grant to create Mars: The Ultimate Voyage, a full-dome planetarium show and accompanying hands-on activities that focus on the interdisciplinary roles that will be needed to send humans to Mars. This report from Catalyst Consulting Group presents the findings from the summative evaluation completed in March–May 2023.
This project is expanding an effective mobile making program to achieve sustainable, widespread impact among underserved youth. Making is a design-based, participant-driven endeavor that is based on a learning by doing pedagogy. For nearly a decade, California State University San Marcos has operated out-of-school making programs for bringing both equipment and university student facilitators to the sites in under-served communities. In collaboration with four other CSU campuses, this project will expand along four dimensions: (a) adding community sites in addition to school sites (b) adding rural contexts in addition to urban/suburban, (c) adding hybrid and online options in addition to in-person), and (d) including future teachers as facilitators in addition to STEM undergraduates. The program uses design thinking as a framework to engage participants in addressing real-world problems that are personally and socially meaningful. Participants will use low- and high-tech tools, such as circuity, coding, and robotics to engage in activities that respond to design challenges. A diverse group of university students will lead weekly, 90-minute activities and serve as near-peer mentors, providing a connection to the university for the youth participants, many of whom will be first-generation college students. The project will significantly expand the Mobile Making program from 12 sites in North San Diego County to 48 sites across California, with nearly 2,000 university facilitators providing 12 hours of programming each year to over 10,000 underserved youth (grades 4th through 8th) during the five-year timeline.
The project research will examine whether the additional sites and program variations result in positive youth and university student outcomes. For youth in grades 4 through 8, the project will evaluate impacts including sustained interest in making and STEM, increased self-efficacy in making and STEM, and a greater sense that making and STEM are relevant to their lives. For university student facilitators, the project will investigate impacts including broadened technical skills, increased leadership and 21st century skills, and increased lifelong interest in STEM outreach/informal science education. Multiple sources of data will be used to research the expanded Mobile Making program's impact on youth and undergraduate participants, compare implementation sites, and understand the program's efficacy when across different communities with diverse learner populations. A mixed methods approach that leverages extant data (attendance numbers, student artifacts), surveys, focus groups, making session feedback forms, observations, and field notes will together be used to assess youth and university student participant outcomes. The project will disaggregate data based on gender, race/ethnicity, grade level, and site to understand the Mobile Making program's impact on youth participants at multiple levels across contexts. The project will further compare findings from different types of implementation sites (e.g., school vs. library), learner groups, (e.g., middle vs. upper elementary students), and facilitator groups (e.g., STEM majors vs. future teachers). This will enable the project to conduct cross-case comparisons between CSU campuses. Project research will also compare findings from urban and rural school sites as well as based on the modality of teaching and learning (e.g., in-person vs. online). The mobile making program activities, project research, and a toolkit for implementing a Mobile maker program will be widely disseminated to researchers, educators, and out-of-school programs.
The Harvard Museums of Science and Culture will improve the ability of middle school teachers to use museum-based digital resources to support classroom instruction aligned with state and national science standards. Working with advisory teachers from five collaborating school districts, the museum will co-create classroom activities, based on digital resources from its collections, along with associated teacher professional development programs at three sites across urban and rural Massachusetts. The project will provide schools with access to classroom-ready resources that successfully support student learning. Teachers will learn how to use these materials, integrate them into their teaching, and enhance their skills to teach science content and practice. External evaluators will assess the project's effectiveness by measuring teacher implementation of the digital resources in the classroom, requests for information and assistance, and changes in teachers' confidence and comfort levels.
This project will scale up fully virtual or face-to-face STEM professional development to afterschool educators in both urban and rural settings. Given that many afterschool educators have little or no background in STEM education, there is demand for professional development that is effective, inexpensive, and accessible. This project will build national capacity in STEM education by developing the STEM skills of over 1,500 educators across multiple states and will ultimately impact over 31,000 under-represented youth in these areas. The project will also deliver robust materials through a free open-source mechanism, for use by educators anywhere and anytime. The project will broaden participation in STEM by engaging community educators in the rural parts of the nation, a critically under-represented group in STEM. It will also reach educators from low-income urban communities across three states and seven cities, targeted through strategic networks and partnerships, including organizations such as the YMCA, 4-H, and the National Afterschool Association.
This collaborative project is scaling the ACRES model (Afterschool Coaching for Reflective Educators in STEM). The model humanizes the virtual experience, making it social and engaging, and allows educators to learn, share, and practice essential STEM facilitation skills with a focus on making STEM relevant and introducing STEM careers to youth. In addition to enhancing the professional STEM skills of rural and urban educators, the project will create a national cohort of coaches with deep expertise in (i) converting in-person activities for youth into a highly engaging, choice-rich online format, (ii) engaging isolated informal educators in supportive professional learning communities, and (iii) coaching foundational research-based STEM facilitation skills that ensure these activities are pedagogically sound. A key part of this broad implementation project involves studying how to integrate an effective professional development program into afterschool organizations, including the ways afterschool programs adapt the materials to be culturally responsive to their local communities. The researchers will also study factors contributing to the longer-term sustainability of the program. The research will use surveys, interviews, direct observations, and case studies of participants to provide the field with valuable insights into scaling a program in the afterschool world.
This project is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for extending access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants.
Described by Wohlwend, Peppler, Keune and Thompson (2017) as “a range of activities that blend design and technology, including textile crafts, robotics, electronics, digital fabrication, mechanical repair or creation, tinkering with everyday appliances, digital storytelling, arts and crafts—in short, fabricating with new technologies to create almost anything” (p. 445), making can open new possibilities for applied, interdisciplinary learning in science, technology, engineering and mathematics (Martin, 2015), in ways that decenter and democratize access to ideas, and promote the construction
DATE:
TEAM MEMBERS:
Jill CastekMichelle Schira HagermanRebecca Woodland
Nature-based playgrounds—known as playscapes—offer numerous opportunities for young children to learn about nature. In the current study, we focus on teacher talk on playscapes, namely to capture the spontaneous utterances teachers offer when engaging with young children during playscape visits. Two different playscapes were contrasted, both of which featured loose parts, native plants, and running water. The difference in playscape was whether it featured ecosystems: While the rural playscape had a natural forest and a wetland, the urban playscape had a man-made stream and a garden. Ten
DATE:
TEAM MEMBERS:
Heidi KloosCatherine MaltbieRhonda BrownVictoria Carr
Parents and adult caregivers play a significant role in young children's understanding of (and participation in) science, technology, engineering, and mathematics (STEM). Research suggests that early engagement with STEM can have a profound impact on children's use of STEM process skills such as exploration, observation, and problem-solving, as well as future academic success. An immediate yet ongoing challenge facing informal STEM learning providers is to understand how limited resources can be used to support effective STEM learning opportunities and experiences for all children and families. Through a collaboration between researchers, Head Start, two science centers (one rural, one urban), and educators, this project aims to foster STEM access and engagement with specific attention to young children and their caregivers. This project is funded by the Advancing Informal STEM Learning (AISL) program which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments.
This Pilot and Feasibility study will apply an experimental, mixed-methods design to examine parent/caregiver and child (ages 4-5) interactions before, during, and after informal STEM experiences to identify which factors influence children's transfer of learning STEM process skills across multiple informal contexts. Research results will lay the foundation for a future longitudinal study. The project team will ask: (1) What types of parent/caregiver-child engagement at the science center are most predictive of children's application of STEM process skills in subsequent problem-solving tasks and school readiness? (2) How do variations in parent/caregiver-child conversational strategies during the science center visit influence children's memory and learning? and (3) How can informal educators best support Head Start family engagement and children's emerging STEM knowledge? This study will collect data on 240, 4-5-year-old children, with their caregivers, in two different science centers that serve a largely rural and largely urban population. Data sources will include video/audio of caregiver-child interactions at the science centers and at home, as well as children's recall, engagement with a problem-solving task, and school readiness scores. Coding and analysis of the tasks during and after the science center visit will detail mechanisms underlying children's memory, learning, and application of STEM process skills that transfer to the problem-solving task. The project will be implemented by a research-practice partnership, leveraging the expertise of project partners and communities to ensure the use of culturally responsive research practices. This research has the potential to strategically impact how science centers and Head Start grantees work together on Family Engagement programming to achieve equitable STEM learning opportunities, broadening participation for low-income young children and their families.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
The goal of the National Science Foundation?s Research Coordination Network (RCN) program is to advance a field or create new directions in research or education by supporting groups of investigators to communicate and coordinate their research, training and educational activities across disciplinary, organizational, geographic and international boundaries. This RCN will bring together scholars and practitioners working at the intersection of equity and interdisciplinary making in STEM education. Making is a culture that emphasizes interest-driven learning by doing within an informal, peer-led and creative social environment. Hundreds of maker spaces and maker-oriented classroom pedagogies have developed across the country. Maker spaces often include digital technologies such as computer design, 3-D printers, and laser cutters, but may also include traditional crafts or a variety of artist-driven creations. The driving purpose of the project is to collectively broaden STEM-focused maker participation in the United States through pursuing common research questions, sharing resources, and incubating emergent inquiry and knowledge across multiple working sites of practice. The network aims to build capacity for research and knowledge, building in consequential and far-reaching mechanisms to leverage combined efforts of a core group of scholars, practitioners, and an extended network of formal and informal education partners in urban and rural sites serving people from groups underrepresented in STEM. Maker learning spaces can be particularly fruitful spaces for STEM learning toward equity because they foster interest-driven, collective, and community-oriented learning in making for social and community change. The network will be led by a team of multi-institutional and multi-disciplinary researchers from different geographic regions of the United States and guided by a steering committee of prominent researchers and practitioners in making and equity will convene to facilitate network activities.
Equitable processes are rooted in a commitment to understand and build on the skills, practices, values, and knowledge of communities marginalized in STEM. The research network aims to fill in gaps in current understandings about making and equity, including the many ways different projects define equity and STEM in making. The project will survey the existing research terrain to develop a dynamic and cohesive understanding of making that connects to learners' STEM ideas, communities, and historical ways of making. Additionally, the network will collaboratively develop central research questions for network partners. The network will create a repository for ethical and promising practices in community-based research and aggregate data across sites, among other activities. The network will support collaboration across a multiplicity of making spaces, research institutions, and community organizations throughout the country to share data, methodologies, ways of connecting to local communities and approaches to robust integration of STEM skills and practices. Project impacts will include new research partnerships, a dissemination hub for research related to making and equity, professional development for researchers and practitioners, and leveraging collective research findings about making values and practices to improve approaches to STEM-rich making integration in informal learning environments. The project is funded by NSF's Advancing Informal STEM Learning (AISL) program, which supports innovative research, approaches, and resources for use in a variety of settings. As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program funds innovative research, approaches, and resources for use in a variety of settings.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
The independent evaluation firm Knight Williams, Inc. conducted a formative evaluation during Year 2 of the SciGirls CONNECT2 program in order to gather information about the partner educators’ use of, reflections on, and recommendations relating to the draft updated SciGirls Strategies. The evaluation aimed for two educators from each of 14 partner organizations – specifically the program leader and one educator who was familiar with the original SciGirls Seven – to provide reflections on their use of the draft SciGirls Strategies in their programs through an online survey and follow-up