In fall 2019, the Bell Museum received funding via a NASA TEAM II grant to create Mars: The Ultimate Voyage, a full-dome planetarium show and accompanying hands-on activities that focus on the interdisciplinary roles that will be needed to send humans to Mars. This report from Catalyst Consulting Group presents the findings from the summative evaluation completed in March–May 2023.
The Kaulele Kapa Exhibit was created to explore the effectiveness of a Hawaiian culture-based framework and approach in increasing learner engagement and depth of knowledge in STEM among Native Hawaiian/Pacific Islander (NHPI) learners. The exhibit utilized hands-on and interactive activities, coupled with scientific and cultural information, to create relevant learning experiences for these communities. To determine the effectiveness, exhibit attendees were invited to complete a survey that asked about how the exhibit influenced their interest and understanding of STEM and Hawaiian culture
This report provides an overview of the rationale and design of the STEM Excellence Project, the key findings and takeaways from the research, and future directions for raising the aspirations of rural students for success in STEM fields.
This document provides information about the quantitative and qualitative data collected and analyzed for this project with hopes that it informs future research and evaluation efforts for STEM education in rural contexts.
The project team published a research synopsis article with Futurum Science Careers in Feb 2023 called “How Can Place Attachment Improve Scientific Literacy?”
The Discovery Center at Murfree Spring, in partnership with six science centers and museums, will promote and invest in science education in rural communities with limited museum access. This coalition will work with two cohorts of rural school communities (12 total) and focus on engaging, learning from, and supporting rural school districts, teachers, families, and communities through relationship building, asset mapping, and the collaborative integration and implementation of museum resources. Additional activities include the production of publications, virtual presentations, and a virtual tool kit. The project will illustrate the ways in which museums can collaborate to support STEM and literacy at the K-2 level, enhance teacher self-efficacy, attitudes and beliefs, and engage family and community, strengthening services for Americans who live in the most rural areas.
The University of Montana spectrUM Discovery Area will implement “Making Across Montana” —a project to engage K–12 students and teachers in rural and tribal communities with making and tinkering. In collaboration with K–12 education partners in the rural Bitterroot Valley and on the Flathead Indian Reservation, the museum will develop a mobile making and tinkering exhibition and education program. The exhibition will be able to travel to K–12 schools statewide. The project team will develop a K–12 teacher professional development workshop, along with accompanying curriculum resources and supplies. The traveling program and related materials will build schools’ capacity to incorporate making and tinkering—and informal STEM experiences more broadly—into their teaching.
This project is expanding an effective mobile making program to achieve sustainable, widespread impact among underserved youth. Making is a design-based, participant-driven endeavor that is based on a learning by doing pedagogy. For nearly a decade, California State University San Marcos has operated out-of-school making programs for bringing both equipment and university student facilitators to the sites in under-served communities. In collaboration with four other CSU campuses, this project will expand along four dimensions: (a) adding community sites in addition to school sites (b) adding rural contexts in addition to urban/suburban, (c) adding hybrid and online options in addition to in-person), and (d) including future teachers as facilitators in addition to STEM undergraduates. The program uses design thinking as a framework to engage participants in addressing real-world problems that are personally and socially meaningful. Participants will use low- and high-tech tools, such as circuity, coding, and robotics to engage in activities that respond to design challenges. A diverse group of university students will lead weekly, 90-minute activities and serve as near-peer mentors, providing a connection to the university for the youth participants, many of whom will be first-generation college students. The project will significantly expand the Mobile Making program from 12 sites in North San Diego County to 48 sites across California, with nearly 2,000 university facilitators providing 12 hours of programming each year to over 10,000 underserved youth (grades 4th through 8th) during the five-year timeline.
The project research will examine whether the additional sites and program variations result in positive youth and university student outcomes. For youth in grades 4 through 8, the project will evaluate impacts including sustained interest in making and STEM, increased self-efficacy in making and STEM, and a greater sense that making and STEM are relevant to their lives. For university student facilitators, the project will investigate impacts including broadened technical skills, increased leadership and 21st century skills, and increased lifelong interest in STEM outreach/informal science education. Multiple sources of data will be used to research the expanded Mobile Making program's impact on youth and undergraduate participants, compare implementation sites, and understand the program's efficacy when across different communities with diverse learner populations. A mixed methods approach that leverages extant data (attendance numbers, student artifacts), surveys, focus groups, making session feedback forms, observations, and field notes will together be used to assess youth and university student participant outcomes. The project will disaggregate data based on gender, race/ethnicity, grade level, and site to understand the Mobile Making program's impact on youth participants at multiple levels across contexts. The project will further compare findings from different types of implementation sites (e.g., school vs. library), learner groups, (e.g., middle vs. upper elementary students), and facilitator groups (e.g., STEM majors vs. future teachers). This will enable the project to conduct cross-case comparisons between CSU campuses. Project research will also compare findings from urban and rural school sites as well as based on the modality of teaching and learning (e.g., in-person vs. online). The mobile making program activities, project research, and a toolkit for implementing a Mobile maker program will be widely disseminated to researchers, educators, and out-of-school programs.
ECHO, Leahy Center for Lake Champlain will increase its capacity to serve rural schools through programming opportunities under its STEM in Motion 2.0 program. In partnership with rural schools, they will conduct two year-long teacher institutes blending in-person and virtual professional development. They plan to develop a total of 270 in-person and virtual classroom STEM programs and produce 18 classroom curriculum kits and standard-activity aligned guides. As a result of STEM in Motion 2.0’s activities, the museum anticipate that 54 teachers will have additional capacity to deliver high-quality STEM learning experiences to K–5th grade students in underserved, rural communities.
The Harvard Museums of Science and Culture will improve the ability of middle school teachers to use museum-based digital resources to support classroom instruction aligned with state and national science standards. Working with advisory teachers from five collaborating school districts, the museum will co-create classroom activities, based on digital resources from its collections, along with associated teacher professional development programs at three sites across urban and rural Massachusetts. The project will provide schools with access to classroom-ready resources that successfully support student learning. Teachers will learn how to use these materials, integrate them into their teaching, and enhance their skills to teach science content and practice. External evaluators will assess the project's effectiveness by measuring teacher implementation of the digital resources in the classroom, requests for information and assistance, and changes in teachers' confidence and comfort levels.
This project addresses a longstanding problem in informal science education: how to increase the likelihood of consequential science, technology, engineering, and mathematics (STEM) learning from short duration experiences such as field trips. Although informal learning experiences can greatly contribute to interest in and knowledge of science, there is a shared concern among educators and researchers that students may have difficulty recalling and using scientific information and practices emphasized during these experiences, even though doing so would further their science learning. Nonetheless, science learning is rarely, if ever, a "one-shot deal." Children acquire knowledge about science cumulatively across different contexts and activities. Therefore, it is important that informal science learning institutions identify effective practices that support the consolidation of learning and memory from exhibit experiences to foster portable, usable knowledge across contexts, such as from informal science learning institutions, to classrooms, and homes. To this end, this Research in Service to Practice project seeks to harness the power and potential of visual representations (e.g., graphs, drawings, charts, maps, etc.) for enhancing learning and encouraging effective reflection during and after science learning experiences. The project promises to increase learning for the 9,000+ 5th and 6th grade students from across the rurality and growing diversity of the state of Maine who annually participate in LabVenture, a 2.5-hour exploration of the Gulf of Maine ecosystem at Gulf of Maine Research Institute. The research will provide new and actionable informal science learning practices that promote engagement with visual representations and reflection, and science understandings that can be applied broadly by informal science institutions. This project is funded by the Advancing Informal STEM Learning (AISL) and the Discovery Research PreK-12 (DRK-12) programs. It supports the AISL program goals to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. It supports the DRK-12 program goal of enhancing the learning and teaching of STEM by preK-12 students and teachers.
The project is grounded in the idea that visual representations, including drawings, can both enhance science learning and encourage reflection on doing science that can support extension of that learning beyond a singular informal science experience. The project uses design-based research to address the following research questions: (1) Does reflection during an informal science learning experience promote students’ retention and subsequent use of science information and practices that are part of the experience? (2) Does interpreting and constructing visual representations, such as drawings, improve students’ understanding and retention of information, and if so, how and when? and (3) Does combining visual representations and narrative reflections confer benefits on students’ science learning and engagement in science practices both during the informal learning experience, and later in their classrooms and at home? These questions will be pursued in collaboration with practitioners (both informal educators and classroom teachers) and a diverse team of graduate and undergraduate student researchers. Approximately 600 student groups (roughly 3000 individual students) will be observed during the LabVenture experience, with further data collection involving a portion of these students at school and at home. The project will yield resources and video demonstrations of field-tested, empirically based practices that promote engagement with visual representations and reflection, and science understandings that can travel within students' learning ecosystem. In support of broadening participation, the undergraduate/graduate student researchers will gain wide understanding and experience connecting research to practice and communicating science to academic and nonacademic audiences.
For thousands of years, Native Hawaiian/Pacific Islander (NHPI) seafarers have successfully utilized systemic observation of their environment to traverse vast expanses of open ocean and thrive on the most remote islands on earth. Developing NHPI trust in the scientific enterprise requires building connections that bridge the values and concepts of 'ike kupuna (traditional knowledge) with scientific knowledge systems and contemporary technology. This project will develop and research a pop-up science exhibit that connects indigenous Hawaiian knowledge with contemporary Western science concepts. The exhibit will show how community knowledge (that is consistent with underlying scientific principles and natural laws) has informed innovation by indigenous peoples. This community-initiated and developed project will begin with a single pop-up exhibit designed to incorporate several hands-on culture-based STEM activities that integrate traditional and modern technologies. For example, the exhibit may cover indigenous systems of star navigation for ocean voyaging, systems of netting for food and water containers, or systems of home design with local and natural materials. This project seeks to develop preliminary evidence of the effectiveness of such an approach for supporting rural Hawaiian youths' STEM engagement, understanding, and personal connections to Native Hawaiian STEM knowledge. Findings from this pilot and feasibility study will inform the development of a larger pop-up science center grounded in indigenous Hawaiian STEM knowledge, and advance intellectual knowledge around culturally sustaining pedagogy by helping informal STEM education practitioners understand community initiated and developed STEM exhibits.
This pop-up science center pilot will be led by a local Hawaiian community organization, INPEACE, in collaboration with several local community members and other community-based organizations. The preliminary research will iteratively explore whether and how an existing Hawaiian culture-based framework can be used to design hands-on STEM exhibits to enhance rural learner engagement, depth of STEM knowledge, and connection to Native Hawaiian STEM knowledge. Research efforts led by Kamehameha Schools, which has a long history of conducting research from an indigenous worldview, will engage 120 learners from various rural communities across Hawaii, from which 40 will be pre-selected middle-school youth, and 80 individuals will be from public audiences of learners ages 12 and up. Through a series of observations, interviews, pre and post surveys with validated instruments, and focus groups, the research will probe: (1) The learners' thoughts on the science practice and its relevance to old and new Hawaii and modern society. (2) The level at which related STEM topics have been understood, and (3) The learners' perceptions about their connection to Native Hawaiian STEM knowledge. Results from this pilot study will inform a future pop-up science center development project, and add to the scarce literature on community-driven, culturally sustaining exhibition development.
This Pilots and Feasibility Studies project is funded by the NSF Advancing Informal STEM Learning program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.