In fall 2019, the Bell Museum received funding via a NASA TEAM II grant to create Mars: The Ultimate Voyage, a full-dome planetarium show and accompanying hands-on activities that focus on the interdisciplinary roles that will be needed to send humans to Mars. This report from Catalyst Consulting Group presents the findings from the summative evaluation completed in March–May 2023.
The SciGirls in Space: Exploring the Moon, Mars and NASA Careers Implementation Evaluation Report focuses on the STEM outreach component of the project for 15 girl-serving organizations who used research-based gender equitable and culturally responsive instructional strategies,NASA-aligned media and STEM activities with a focus on integrating of NASA women role models.
Few people realize that the largest part of our planet’s biosphere remains virtually unexplored and unknown. This enormous habitat, accounting for an area of 116 million square miles or the equivalent size of roughly 30 times the area of the United States, is the abyssal zone of the deep ocean. The abyssal sea floor, at about 6000 ft., contains more than four times as much habitat for animal life as all of the dry mountains, forests, deserts, plains and jungles combined. Microscopic larvae in the deep ocean, are essential for the renewal and replenishment of life and they repopulate areas damaged by human activities such as mining and trawling, and they make marine protected areas both feasible and important. The National Science Foundation has funded intensive studies of oceanography related to larval recruitment for decades. However, findings from this large NSF investment of personnel, technology and funding have never been widely presented to the public. This project proposes to remedy this by developing a 40 minute giant screen film to be shown in science centers across the country, supported by virtual reality and augmented reality learning tools. The film will cover select deep ocean science expeditions using the deep-sea vehicles Alvin and ROV Jason. Content will include elements of the research process, activities related to the design and operation of deep-sea vehicles as well as interviews with scientists and technologists. The companion activities, Deep-Ocean Pilot (a VR-360° viewing station) and Plankton Quest (an AR biology treasure hunt) will extend the audience experience of the deep ocean out of the giant screen theater and into the surrounding museum environment. The website and social media will extend awareness and resources into homes. The project will be appropriate for a broad general audience, with particular appeal for the target audience of women and girls (ages 7-20). The larval biologist team is led by the PI at the University of Oregon, in collaboration with scientists from North Carolina State University, Western Washington University and the University of Rhode Island. Several young women scientists will be featured in the film providing role models. The production company, Stephen Low Productions, Inc. will use the latest technology on the Alvin and other cinematic tools to capture the visual images in the abyss. Collaborating museums will participate in the development and implementation of the Virtual and Augmented Reality learning tools as well as showing the film in their theaters.
Broader impact project goals include 1) Advancing public awareness of the abyssal ocean, the role of microscopic larvae, and what scientists are learning from expeditions that use deep submergence technologies; 2) Introducing public audiences and young women specifically to the wide range of STEM-related occupations encompassed in the field of ocean exploration and research; and 3) Advancing STEM learning research and practice in the area of immersive media in conveying STEM concepts and enhancing audience identification with STEM. Oregon State University’s STEM Research Center will build new knowledge by conducting formative and summative evaluation of the film and its associated support products (e.g., Virtual and augmented reality activities, website resources), addressing the following evaluation questions: 1)What do audiences take away from their experience in terms of fascination/interest, awareness and understanding related to ocean science exploration? 2) To what degree does the film alone or in combination with supplemental experiences trigger career awareness in girls and young women, and youth of racial/ethnic backgrounds? 3) To what degree do immersive experiences (a sense of “being there”) contribute to learning from the film? 4) How enduring are outcomes with audiences past the onsite immediate experience? Formative evaluation will be designed as ongoing improvement informed by empirical evidence in which evaluators work with team members to answer decision-relevant questions in a timely and project-focused way. The summative evaluation will be structured as an effectiveness study using mixed methods and ascertaining whether key programmatic outcomes have been reached and the degree to which particular program elements will have contributed to the results.
DATE:
-
TEAM MEMBERS:
Craig YoungAlexander LowStephen LowGeorge von DassowTrish Mace
This project is expanding an effective mobile making program to achieve sustainable, widespread impact among underserved youth. Making is a design-based, participant-driven endeavor that is based on a learning by doing pedagogy. For nearly a decade, California State University San Marcos has operated out-of-school making programs for bringing both equipment and university student facilitators to the sites in under-served communities. In collaboration with four other CSU campuses, this project will expand along four dimensions: (a) adding community sites in addition to school sites (b) adding rural contexts in addition to urban/suburban, (c) adding hybrid and online options in addition to in-person), and (d) including future teachers as facilitators in addition to STEM undergraduates. The program uses design thinking as a framework to engage participants in addressing real-world problems that are personally and socially meaningful. Participants will use low- and high-tech tools, such as circuity, coding, and robotics to engage in activities that respond to design challenges. A diverse group of university students will lead weekly, 90-minute activities and serve as near-peer mentors, providing a connection to the university for the youth participants, many of whom will be first-generation college students. The project will significantly expand the Mobile Making program from 12 sites in North San Diego County to 48 sites across California, with nearly 2,000 university facilitators providing 12 hours of programming each year to over 10,000 underserved youth (grades 4th through 8th) during the five-year timeline.
The project research will examine whether the additional sites and program variations result in positive youth and university student outcomes. For youth in grades 4 through 8, the project will evaluate impacts including sustained interest in making and STEM, increased self-efficacy in making and STEM, and a greater sense that making and STEM are relevant to their lives. For university student facilitators, the project will investigate impacts including broadened technical skills, increased leadership and 21st century skills, and increased lifelong interest in STEM outreach/informal science education. Multiple sources of data will be used to research the expanded Mobile Making program's impact on youth and undergraduate participants, compare implementation sites, and understand the program's efficacy when across different communities with diverse learner populations. A mixed methods approach that leverages extant data (attendance numbers, student artifacts), surveys, focus groups, making session feedback forms, observations, and field notes will together be used to assess youth and university student participant outcomes. The project will disaggregate data based on gender, race/ethnicity, grade level, and site to understand the Mobile Making program's impact on youth participants at multiple levels across contexts. The project will further compare findings from different types of implementation sites (e.g., school vs. library), learner groups, (e.g., middle vs. upper elementary students), and facilitator groups (e.g., STEM majors vs. future teachers). This will enable the project to conduct cross-case comparisons between CSU campuses. Project research will also compare findings from urban and rural school sites as well as based on the modality of teaching and learning (e.g., in-person vs. online). The mobile making program activities, project research, and a toolkit for implementing a Mobile maker program will be widely disseminated to researchers, educators, and out-of-school programs.
The Field Museum of Natural History will present “Changing Face of Science,” an exhibition series targeting pre-teens and teenagers and featuring Field Museum scientists and science educators who are women or people of color. Over three years, the museum will mount six exhibitions that highlight the experiences and work of museum scientists from diverse backgrounds in a range of disciplines. Programming will include on-site field trips and virtual events during which students and educators will interact with featured researchers. By presenting the stories of individuals from groups traditionally underrepresented in scientific fields, the museum will provide role models who will show that science is accessible and inspire a diverse group of future scientists.
RK&A conducted prototype testing for the Game Changers exhibition with three audiences: adults, female youth 10-17 years old, and male youth 13-14 years old. The Game Changers exhibition is being developed by the Lemelson Center for the Study of Invention and Innovation at the National Museum of American History.
The Cincinnati Museum Center will develop a permanent exhibition to showcase its invertebrate paleontology collection and develop related educational programming that builds on a strong commitment to gender equity. Using focus groups, prototypes, surveys, and feedback from existing programs, the museum will incorporate community input from key audiences into the design of the 4,800 square-foot immersive gallery, which will blend science, history, and technology. The museum will engage external designers to create schematic and final exhibit designs. The museum will develop and test related educational programs for families and students, with a special focus on engaging girls ages 7 to 14 in STEM activities.
Sciencenter will develop a touring exhibition, Engineer.Design.Build, to spark interest and build confidence in STEM by providing learning opportunities about the broad impact engineers have on the environment and society. The museum will partner with Cornell University's College of Engineering to develop scientific content which will be reviewed by an advisory board of representatives from the academic, business, and informal science education sectors. Partners from informal learning institutions will provide expertise on the educational content to ensure that it is accessible and engaging for the target audience of 5-11 year olds. Through a combination of focus groups, youth/guest feedback during exhibition development, and experts in girls' engagement in STEM on the advisory board, the museum will ensure that the exhibition and programming are designed to appeal to girls, and accessible to all learners. The project will include front-end, formative, and summative evaluation through observations and mediated interviews, collecting data from youth, families, and school groups.
The Da Vinci Science Center will expand its Women in Science and Engineering Network by partnering with community organizations, colleges, and universities to enhance the STEM learning and support ecosystem for women and girls in the Lehigh Valley and surrounding communities in eastern Pennsylvania. The museum will assess the needs of K-12 girls, undergraduate women, and women in STEM employment, and map opportunities for cross-sector collaborations to support them. The project team will identify marketing and recruitment messages that encourage STEM-interested girls and women to participate in programs and follow developmental pathways within a STEM learning ecosystem. Based on identified needs and messages, the museum will pilot and evaluate new STEM programs for girls and women, and train educators and mentors to sustain this work.
Empathy is a critical part of the engineering design process. It allows engineers to more deeply understand their clients’ perspectives and design solutions that meet the needs of diverse stakeholders. Studies also show that reframing engineering education to prioritize empathy for others can counteract stereotypes of engineering as impersonal and invite a wider range of identities into the field. This approach can help to address persistent gender disparities in engineering, which reflect a need for engineering education to increase its efforts to include girls’ perspectives. Informal