This document is the final evaluation report for the project, which focuses both on formative evaluation of the collaborative+interdisciplinary presentation creation process and summative evaluation of audience learning outcomes.
PocketMacro is a mobile app designed by The Human-Computer Interaction Institute at Carnegie Mellon University to help users better identify benthic macroinvertebrates commonly found in streams and other waterways. In Summer 2021, the app and other supplemental materials were highlighted during trainings for educators held at the Stroud Water Research Center.
The evaluation team from Rockman et al Cooperative (REA) surveyed and interviewed educators who participated in the summer trainings to determine what they took away from the experience and to gather feedback about the PocketMacro app
The Polar Literacy (PL) project explores the development and implementation of Out of School Time (OST) learning opportunities focused on polar literacy concepts and authentic data with middle school aged youth. This poster was presented at the 2021 NSF AISL Awardee Meeting.
In July 2020, Dr. Brigid Barron and her team at Stanford University’s Graduate School of Education and the Joan Ganz Cooney Center convened a virtual workshop to mobilize a community of investigators to explore innovative methods for studying family and community learning during the pandemic. Participants included NSF RAPID-COVID grantees from Stanford University, University of Washington, and the University of Michigan.
This report summarizes the strategies and insights generated at this workshop so that they may be shared among a wider network of researchers, practitioners, funders, and
This award is funded in whole or in part under the American Rescue Plan Act of 2021 (Public Law 117-2).
Despite the rich scientific evidence of adaptations and their evolutionary basis, there are concerning public misconceptions about evolution, processes of natural selection, and adaptations in the biodiverse world. Such misconceptions begin early. Younger elementary school children are often resistant to the idea that one “kind” of animal could descend from a completely different kind of animal, and they see features as having always existed. Other misconceptions lead to an inaccurate belief that changes in individual organisms acquired in a lifetime are passed directly on to offspring or that entire populations transform as a whole. These cognitive biases and "intuitive” misunderstandings can persist into adulthood. This Innovations in Development project will counter that narrative through an informal science project focusing on the blue whale one of nature’s most spectacular stories of adaptation. It is a species that lives life at extremes: a long-distance migrator, a deep diver, an extravagant eater, the largest animal to ever exist. With its awe-inspiring size and rich mosaic of anatomical, physiological, and behavioral specializations, it serves as a bridge to an enriched understanding of universal concepts in elementary biology and can begin to dispel the deeply rooted misconceptions. The project deliverables include a giant screen film documenting the field work of research scientists studying the blue whales in the Indian Ocean and Gulf of Mexico; multi-platform educational modules and programs that will build on the blue whale content from the film for use in science center programs and rural libraries; and professional development webinars that will offer content utilization and presentation skills for ISE facilitators. Project partners include California Science Center, STAR Library Education Network, HHMI Tangled Bank Studios and SK Films.
The external evaluation studies will gather data from 20 participating rural libraries and 6 science museums. A formative evaluation of the film will be conducted in a giant screen theater setting with 75 families. After viewing a fine-cut version of the film they will complete age-appropriate post-viewing surveys on the film’s engagement, storytelling, content appeal and clarity, and learning value in communicating key science concepts. An external summative evaluation will include three studies. Study 1 will assess the implementation of the project at the 26 organizations, addressing the question: To what extent is the project implemented as envisioned in the libraries and science center settings? Baseline information will be collected, and later partners will complete post-grant surveys to report on their actual implementation of the project elements. In addition, the study will examine outcomes relating to professional development. Study 2 will be an evaluation of the film as experienced by 400 youth and parents in science centers and examining the question: To what extent does experiencing the film engage youth and parents and affect their interest, curiosity, and knowledge of blue whales, adaptations, and the scientific process? Study 3 will examine: To what extent and how does experiencing an educational module (virtual field trips, hands on activities, augmented reality) affect youth and parents’ interest, curiosity, and knowledge of adaptations and scientific process?
This project will scale up fully virtual or face-to-face STEM professional development to afterschool educators in both urban and rural settings. Given that many afterschool educators have little or no background in STEM education, there is demand for professional development that is effective, inexpensive, and accessible. This project will build national capacity in STEM education by developing the STEM skills of over 1,500 educators across multiple states and will ultimately impact over 31,000 under-represented youth in these areas. The project will also deliver robust materials through a free open-source mechanism, for use by educators anywhere and anytime. The project will broaden participation in STEM by engaging community educators in the rural parts of the nation, a critically under-represented group in STEM. It will also reach educators from low-income urban communities across three states and seven cities, targeted through strategic networks and partnerships, including organizations such as the YMCA, 4-H, and the National Afterschool Association.
This collaborative project is scaling the ACRES model (Afterschool Coaching for Reflective Educators in STEM). The model humanizes the virtual experience, making it social and engaging, and allows educators to learn, share, and practice essential STEM facilitation skills with a focus on making STEM relevant and introducing STEM careers to youth. In addition to enhancing the professional STEM skills of rural and urban educators, the project will create a national cohort of coaches with deep expertise in (i) converting in-person activities for youth into a highly engaging, choice-rich online format, (ii) engaging isolated informal educators in supportive professional learning communities, and (iii) coaching foundational research-based STEM facilitation skills that ensure these activities are pedagogically sound. A key part of this broad implementation project involves studying how to integrate an effective professional development program into afterschool organizations, including the ways afterschool programs adapt the materials to be culturally responsive to their local communities. The researchers will also study factors contributing to the longer-term sustainability of the program. The research will use surveys, interviews, direct observations, and case studies of participants to provide the field with valuable insights into scaling a program in the afterschool world.
This project is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for extending access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants.
Black girls display high interest, confidence and ability in STEM but face multiple barriers including racial, ethnic and gender stereotypes, low exposure to STEM role models, low awareness of diverse STEM fields and financial obstacles to STEM education. It is critical to infuse STEM education with specific and intentional culturally responsive and anti-racist strategies to attract and retain Black girls in STEM. Through this combination of media, role modeling and outreach, Black SciGirls will help increase access to STEM education for Black girls, preparing them for future workforce participation. This project will study the impacts on elementary/middle school Black girls’ exposure to early career Black female STEM professionals as role models. Deliverables include 1) professional development for STEM educators and Black STEM professional women to prepare them to lead STEM programs for girls 2) a PBS series of role model videos of early-career Black STEM professionals and 3) a research study that examines how/if in person and media-based STEM role models increase Black girls’ interest and confidence in STEM, motivation to pursue future STEM studies, and STEM identity. While women make up 47% of the U.S. workforce, they are underrepresented in STEM and only 1.6% are Black women.
The research study will examine how educators’ use of role models addresses a critical barrier for Black girls, seeing women in STEM who look like them. The research study questions are: How and in what ways do Black STEM women role models influence Black girls’ interest in STEM? How and to what extent do role models report changes in their confidence and ability to engage girls in STEM as a result of training in best practices in role modelling? and, How and to what extent are parents engaged in supporting girls’ involvement in STEM, as a result of the participation of role models? The research team will visit participating local SciGirls programs to collect qualitative data, including observations of program activities, interviews, and focus groups. To ensure reliable outcomes and utilize robust theoretical underpinnings, the research will combine pre/post survey data and an in-depth cross-case studies employing qualitative and quantitative data collection. This mixed-methods approach will enable gathering data that comprehensively offers insight into Black girls’ STEM experiences and those of the Black STEM professional women role models and parents who support them. Qualitative data that centers girls’, role models’, and parents’ perspectives will contribute to this identity-centered study. A culturally responsive evaluation will determine the extent to which the project builds educators’ ability to integrate equitable and anti-racist practices to build Black girls’ interest and confidence in STEM studies.
This Research in Service to Practice project is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to (a) advance new approaches to and evidence-based understanding of the design and development of STEM learning in informal environments; (b) provide multiple pathways for broadening access to and engagement in STEM learning experiences; (c) advance innovative research on and assessment of STEM learning in informal environments; and (d) engage the public of all ages in learning STEM in informal environments.
DATE:
-
TEAM MEMBERS:
Rita KarlAngel Miles NashRonda BullockAdrienne StephensonLataisia Jones
Wireless radio communications, such as Wi-Fi, transmit public and private data from one device to another, including cell phones, computers, medical equipment, satellites, space rockets, and air traffic control. Despite their critical role and prevalence, many people are unfamiliar with radio waves, how they are generated and interact with their surroundings, and why they are the basis of modern communication and navigation. This topic is not only increasingly relevant to the technological lives of today’s youth and public, it is critical to the National Science Foundation’s Industries of the Future activities, particularly in advancing wireless education and workforce development. In this project, STEM professionals from academia, industry and informal education will join forces to design, evaluate, and launch digital apps, a craft-based toolkit, activity guides, and mobile online professional learning, all of which will be easily accessed and flexibly adapted by informal educators to engage youth and the public about radio frequency communications. Experiences will include embodied activities, such as physically linking arms to create and explore longitudinal and transverse waves; mobile experiences, such as augmented reality explorations of Wi-Fi signals or collaborative signal jamming simulations; and technological exploration, such as sending and receiving encrypted messages.
BSCS Science Learning, Georgia Tech, and the Children’s Creativity Museum (CCM) with National Informal STEM Education Network (NISE Net) museum partners will create pedagogical activity designs, digital apps, and a mobile online professional learning platform. The project features a rigorous and multipronged research and development approach that builds on prior learning sciences studies to advance a learning design framework for nimble, mobile informal education, while incorporating the best aspects of hands-on learning. This project is testing two related hypotheses: 1) a mobile strategy can be effective for supporting just-in-time informal education of a highly technical, scientific topic, and 2) a mobile suite of resources, including professional learning, can be used to teach informal educators, youth, and the general public about radio frequency communications. Data sources include pre- and post- surveys, interviews, and focus groups with a wide array of educators and learners.
A front-end study will identify gaps in public understanding and perceptions specific to radio frequency communications, and serve as a baseline for components of the summative research. Iterative formative evaluation will incorporate participatory co-design processes with youth and informal educators. These processes will support materials that are age-appropriate and culturally responsive to not only youth, with an emphasis on Latinx youth, but also informal educators and the broader public. Summative evaluation will examine the impact of the mobile suite of resources on informal educators’ learning, facilitation confidence and intentions to continue to incorporate the project resources into their practice. The preparation of educators in supporting public understanding of highly technological STEM topics can be an effective way for supporting just-in-time public engagement and interests in related careers. Data from youth and museum visitors will examine changes to interest, science self-efficacy, content knowledge, and STEM-related career interest. If successful, this design approach may influence how mobile resources are designed and organized effectively to impact future informal education on similarly important technology-rich topics. All materials will be released under Creative Commons licenses allowing for widespread sharing and remixing; research and design findings will be published in academic, industry, and practitioner journals.
This project is co-funded by two NSF programs: The Advancing Informal STEM Learning program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants. The Innovative Technology Experiences for Students and Teachers (ITEST) program, which supports projects that build understandings of practices, program elements, contexts and processes contributing to increasing students' knowledge and interest in science, technology, engineering, and mathematics (STEM) and information and communication technology (ICT) careers.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.