This document summarizes lessons learned from implementing Leap into Science: Cultivating a National Network for Informal Science and Literacy (Leap into Science) from 2017-2023.
In fall 2019, the Bell Museum received funding via a NASA TEAM II grant to create Mars: The Ultimate Voyage, a full-dome planetarium show and accompanying hands-on activities that focus on the interdisciplinary roles that will be needed to send humans to Mars. This report from Catalyst Consulting Group presents the findings from the summative evaluation completed in March–May 2023.
This project is expanding an effective mobile making program to achieve sustainable, widespread impact among underserved youth. Making is a design-based, participant-driven endeavor that is based on a learning by doing pedagogy. For nearly a decade, California State University San Marcos has operated out-of-school making programs for bringing both equipment and university student facilitators to the sites in under-served communities. In collaboration with four other CSU campuses, this project will expand along four dimensions: (a) adding community sites in addition to school sites (b) adding rural contexts in addition to urban/suburban, (c) adding hybrid and online options in addition to in-person), and (d) including future teachers as facilitators in addition to STEM undergraduates. The program uses design thinking as a framework to engage participants in addressing real-world problems that are personally and socially meaningful. Participants will use low- and high-tech tools, such as circuity, coding, and robotics to engage in activities that respond to design challenges. A diverse group of university students will lead weekly, 90-minute activities and serve as near-peer mentors, providing a connection to the university for the youth participants, many of whom will be first-generation college students. The project will significantly expand the Mobile Making program from 12 sites in North San Diego County to 48 sites across California, with nearly 2,000 university facilitators providing 12 hours of programming each year to over 10,000 underserved youth (grades 4th through 8th) during the five-year timeline.
The project research will examine whether the additional sites and program variations result in positive youth and university student outcomes. For youth in grades 4 through 8, the project will evaluate impacts including sustained interest in making and STEM, increased self-efficacy in making and STEM, and a greater sense that making and STEM are relevant to their lives. For university student facilitators, the project will investigate impacts including broadened technical skills, increased leadership and 21st century skills, and increased lifelong interest in STEM outreach/informal science education. Multiple sources of data will be used to research the expanded Mobile Making program's impact on youth and undergraduate participants, compare implementation sites, and understand the program's efficacy when across different communities with diverse learner populations. A mixed methods approach that leverages extant data (attendance numbers, student artifacts), surveys, focus groups, making session feedback forms, observations, and field notes will together be used to assess youth and university student participant outcomes. The project will disaggregate data based on gender, race/ethnicity, grade level, and site to understand the Mobile Making program's impact on youth participants at multiple levels across contexts. The project will further compare findings from different types of implementation sites (e.g., school vs. library), learner groups, (e.g., middle vs. upper elementary students), and facilitator groups (e.g., STEM majors vs. future teachers). This will enable the project to conduct cross-case comparisons between CSU campuses. Project research will also compare findings from urban and rural school sites as well as based on the modality of teaching and learning (e.g., in-person vs. online). The mobile making program activities, project research, and a toolkit for implementing a Mobile maker program will be widely disseminated to researchers, educators, and out-of-school programs.
The Harvard Museums of Science and Culture will improve the ability of middle school teachers to use museum-based digital resources to support classroom instruction aligned with state and national science standards. Working with advisory teachers from five collaborating school districts, the museum will co-create classroom activities, based on digital resources from its collections, along with associated teacher professional development programs at three sites across urban and rural Massachusetts. The project will provide schools with access to classroom-ready resources that successfully support student learning. Teachers will learn how to use these materials, integrate them into their teaching, and enhance their skills to teach science content and practice. External evaluators will assess the project's effectiveness by measuring teacher implementation of the digital resources in the classroom, requests for information and assistance, and changes in teachers' confidence and comfort levels.
Many Black youth in both urban and rural areas lack engaging opportunities to learn mathematics in a manner that leads to full participation in STEM. The Young People’s Project (YPP), the Baltimore Algebra Project (BAP), and the Education for Liberation Network (EdLib) each have over two decades of experience working on this issue. In the city of Baltimore, where 90% of youth in poverty are Black, and only 5% of these students meet or exceed expectations in math, BAP, a youth led organization, develops and employs high school and college age youth to provide after-school tutoring in Algebra 1, and to advocate for a more just education for themselves and their peers. YPP works in urban or rural low income communities that span the country developing Math Literacy Worker programs that employ young people ages 14-22 to create spaces to help their younger peers learn math. Building on these deep and rich experiences, this Innovations in Development project studies how Black students see themselves as mathematicians in the context of paid peer-to-peer math teaching--a combined social, pedagogical, and economic strategy. Focusing primarily in Baltimore, the project studies how young people grow into new self-definitions through their work in informal, student-determined math learning spaces, structured collaboratively with adults who are experts in both mathematics and youth development. The project seeks to demonstrate the benefits of investing in young people as learners, teachers, and educational collaborators as part of a core strategy to improve math learning outcomes for all students.
The project uses a mixed methods approach to describe how mathematical identity develops over time in young people employed in a Youth-Directed Mathematics Collaboratory. 60 high school aged students with varying mathematical backgrounds (first in Baltimore and later in Boston) will learn how to develop peer- and near-peer led math activities with local young people in informal settings, after-school programs, camps, and community centers, reaching approximately 600 youth/children. The high school aged youth employed in this project will develop their own math skills and their own pedagogical skills through the already existing YPP and BAP structures, made up largely of peers and near-peers just like themselves. They will also participate in on-going conversations within the Collaboratory and with the community about the cultural significance of doing mathematics, which for YPP and BAP is a part of the ongoing Civil Rights/Human Rights movement. Mathematical identity will be studied along four dimensions: (a) students’ sequencing and interpretation of past mathematical experiences (autobiographical identity); (b) other people’s talk to them and their talk about themselves as learners, doers, and teachers of mathematics (discoursal identity); (c) the development of their own voices in descriptions and uses of mathematical knowledge and ideas (authorial identity); and (d) their acceptance or rejection of available selfhoods (socio-culturally available identity). Intended outcomes from the project include a clear description of how mathematical identity develops in paid peer-teaching contexts, and growing recognition from both local communities and policy-makers that young people have a key role to play, not only as learners, but also as teachers and as co-researchers of mathematics education.
This Innovations in Development project is funded by the Advancing Informal STEM Learning (AISL) program.
DATE:
-
TEAM MEMBERS:
Jay GillenMaisha MosesThomas NikundiweNaama LewisAlice Cook
This project will scale up fully virtual or face-to-face STEM professional development to afterschool educators in both urban and rural settings. Given that many afterschool educators have little or no background in STEM education, there is demand for professional development that is effective, inexpensive, and accessible. This project will build national capacity in STEM education by developing the STEM skills of over 1,500 educators across multiple states and will ultimately impact over 31,000 under-represented youth in these areas. The project will also deliver robust materials through a free open-source mechanism, for use by educators anywhere and anytime. The project will broaden participation in STEM by engaging community educators in the rural parts of the nation, a critically under-represented group in STEM. It will also reach educators from low-income urban communities across three states and seven cities, targeted through strategic networks and partnerships, including organizations such as the YMCA, 4-H, and the National Afterschool Association.
This collaborative project is scaling the ACRES model (Afterschool Coaching for Reflective Educators in STEM). The model humanizes the virtual experience, making it social and engaging, and allows educators to learn, share, and practice essential STEM facilitation skills with a focus on making STEM relevant and introducing STEM careers to youth. In addition to enhancing the professional STEM skills of rural and urban educators, the project will create a national cohort of coaches with deep expertise in (i) converting in-person activities for youth into a highly engaging, choice-rich online format, (ii) engaging isolated informal educators in supportive professional learning communities, and (iii) coaching foundational research-based STEM facilitation skills that ensure these activities are pedagogically sound. A key part of this broad implementation project involves studying how to integrate an effective professional development program into afterschool organizations, including the ways afterschool programs adapt the materials to be culturally responsive to their local communities. The researchers will also study factors contributing to the longer-term sustainability of the program. The research will use surveys, interviews, direct observations, and case studies of participants to provide the field with valuable insights into scaling a program in the afterschool world.
This project is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for extending access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants.
Described by Wohlwend, Peppler, Keune and Thompson (2017) as “a range of activities that blend design and technology, including textile crafts, robotics, electronics, digital fabrication, mechanical repair or creation, tinkering with everyday appliances, digital storytelling, arts and crafts—in short, fabricating with new technologies to create almost anything” (p. 445), making can open new possibilities for applied, interdisciplinary learning in science, technology, engineering and mathematics (Martin, 2015), in ways that decenter and democratize access to ideas, and promote the construction
DATE:
TEAM MEMBERS:
Jill CastekMichelle Schira HagermanRebecca Woodland
Nature-based playgrounds—known as playscapes—offer numerous opportunities for young children to learn about nature. In the current study, we focus on teacher talk on playscapes, namely to capture the spontaneous utterances teachers offer when engaging with young children during playscape visits. Two different playscapes were contrasted, both of which featured loose parts, native plants, and running water. The difference in playscape was whether it featured ecosystems: While the rural playscape had a natural forest and a wetland, the urban playscape had a man-made stream and a garden. Ten
DATE:
TEAM MEMBERS:
Heidi KloosCatherine MaltbieRhonda BrownVictoria Carr
The U.S. Fish and Wildlife Service estimates that over 41 million people connect to nature through birding. Learning about birds in their natural environments offers opportunities for informal engagement in STEM by a broad range of individuals and groups. Birders often engage in scientific data gathering and analyses, geolocation and remote sensing, and phenology. They also become aware of ecological changes in bird habitats and migratory patterns due to rising temperatures and climate-related events like sea level rise, droughts, fires, and extreme weather. As such, the birding community is an ideal network to better understand and communicate the impacts of climatological changes on bird populations to the public. With this Innovations in Development project, the National Audubon Society will develop a new avian-focused, conservation and climate science community science curriculum for its Nature Centers, and test the effectiveness of the curriculum in educating the public about avian-focused conservation and climatological changes through guided nature experiences. Birding can serve as a pivotal entrée for young people into STEM fields and careers. Through its programs and partnerships, Audubon will leverage its national network to ensure that through this project a more diverse group of voices, particularly young adults and young adults of color, become involved in asking critical questions and developing solutions to address important environmental issues of the future. If successful, the broader impacts of this project on capacity building and public engagement could be far-reaching and long-lasting.
Over the three-year project duration, Audubon will bring educators from its nationwide network of thirty-four Nature Centers (including urban, suburban, and rural sites), together with over 510 young adults (ages 18-25) from its network of college campus chapters. An evidence-based curriculum and community science activities will be created and tested, relying heavily on a team of experts in ornithology, climate science research, STEM curriculum design, diversity, and informal science education. College students will advise on the design of content and activities to effectively interest and engage young adults. These students will be recruited from the new Audubon Campus Chapters Program, which includes 111 college and university campuses, among them, 19 Historically Black Colleges and Universities (HBCUs) and other Minority Serving Institutions (MSIs). The target population will be surveyed to also understand their current and likely participation in guided nature experiences and knowledge base in climate science. Current best practices in guided nature experiences will be gathered from across the Audubon network. The implementation efforts will result in a national STEM model, with train-the-trainer guides and workshops for informal science educators and public engagement opportunities focused on improving the state and condition of avian habitats and communities through climate science research. An external evaluation will be conducted and will include data collection methods such as retrospective pre and post surveys, semi-structured interviews, focus groups, and an embedded assessment to determine impact. The findings will be used to iteratively refine the evidence-based curriculum and measure STEM learning outcomes for the guided nature experience participants. The evaluation will address four areas: (1) fidelity of program implementation to promote accountability; (2) formative evaluation to understand needs and interests of young adults (ages 18-25), and subsequently inform program design; (3) outcomes for Center educators, to inform iterative improvement; and (4) outcomes for program participants, to contribute to the growing knowledge base on effective practices for STEM learning in informal settings.
This Innovations in Development project is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
DATE:
-
TEAM MEMBERS:
Loren SmithMark ScallionHeather Starck
Parents and adult caregivers play a significant role in young children's understanding of (and participation in) science, technology, engineering, and mathematics (STEM). Research suggests that early engagement with STEM can have a profound impact on children's use of STEM process skills such as exploration, observation, and problem-solving, as well as future academic success. An immediate yet ongoing challenge facing informal STEM learning providers is to understand how limited resources can be used to support effective STEM learning opportunities and experiences for all children and families. Through a collaboration between researchers, Head Start, two science centers (one rural, one urban), and educators, this project aims to foster STEM access and engagement with specific attention to young children and their caregivers. This project is funded by the Advancing Informal STEM Learning (AISL) program which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments.
This Pilot and Feasibility study will apply an experimental, mixed-methods design to examine parent/caregiver and child (ages 4-5) interactions before, during, and after informal STEM experiences to identify which factors influence children's transfer of learning STEM process skills across multiple informal contexts. Research results will lay the foundation for a future longitudinal study. The project team will ask: (1) What types of parent/caregiver-child engagement at the science center are most predictive of children's application of STEM process skills in subsequent problem-solving tasks and school readiness? (2) How do variations in parent/caregiver-child conversational strategies during the science center visit influence children's memory and learning? and (3) How can informal educators best support Head Start family engagement and children's emerging STEM knowledge? This study will collect data on 240, 4-5-year-old children, with their caregivers, in two different science centers that serve a largely rural and largely urban population. Data sources will include video/audio of caregiver-child interactions at the science centers and at home, as well as children's recall, engagement with a problem-solving task, and school readiness scores. Coding and analysis of the tasks during and after the science center visit will detail mechanisms underlying children's memory, learning, and application of STEM process skills that transfer to the problem-solving task. The project will be implemented by a research-practice partnership, leveraging the expertise of project partners and communities to ensure the use of culturally responsive research practices. This research has the potential to strategically impact how science centers and Head Start grantees work together on Family Engagement programming to achieve equitable STEM learning opportunities, broadening participation for low-income young children and their families.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.