This document is the final evaluation report for the project, which focuses both on formative evaluation of the collaborative+interdisciplinary presentation creation process and summative evaluation of audience learning outcomes.
The Joseph Moore Museum at Earlham College will revise its interpreter training and educational programs to reflect current best practices in participatory STEM education. This project will include strengthening their programs to better prepare undergraduate educators, as well as updating the delivery of their top three requested programs to ensure learner-centered experiences. The project will include the development of a training program modeled on a combination of principles set out by the National Association of Interpretation and the Reflections on Practice program. Undergraduate educators will undergo systematic training in the fundamentals of educational theory and practice and benefit from a program of sustained evaluation and mentorship.
The U.S. Fish and Wildlife Service estimates that over 41 million people connect to nature through birding. Learning about birds in their natural environments offers opportunities for informal engagement in STEM by a broad range of individuals and groups. Birders often engage in scientific data gathering and analyses, geolocation and remote sensing, and phenology. They also become aware of ecological changes in bird habitats and migratory patterns due to rising temperatures and climate-related events like sea level rise, droughts, fires, and extreme weather. As such, the birding community is an ideal network to better understand and communicate the impacts of climatological changes on bird populations to the public. With this Innovations in Development project, the National Audubon Society will develop a new avian-focused, conservation and climate science community science curriculum for its Nature Centers, and test the effectiveness of the curriculum in educating the public about avian-focused conservation and climatological changes through guided nature experiences. Birding can serve as a pivotal entrée for young people into STEM fields and careers. Through its programs and partnerships, Audubon will leverage its national network to ensure that through this project a more diverse group of voices, particularly young adults and young adults of color, become involved in asking critical questions and developing solutions to address important environmental issues of the future. If successful, the broader impacts of this project on capacity building and public engagement could be far-reaching and long-lasting.
Over the three-year project duration, Audubon will bring educators from its nationwide network of thirty-four Nature Centers (including urban, suburban, and rural sites), together with over 510 young adults (ages 18-25) from its network of college campus chapters. An evidence-based curriculum and community science activities will be created and tested, relying heavily on a team of experts in ornithology, climate science research, STEM curriculum design, diversity, and informal science education. College students will advise on the design of content and activities to effectively interest and engage young adults. These students will be recruited from the new Audubon Campus Chapters Program, which includes 111 college and university campuses, among them, 19 Historically Black Colleges and Universities (HBCUs) and other Minority Serving Institutions (MSIs). The target population will be surveyed to also understand their current and likely participation in guided nature experiences and knowledge base in climate science. Current best practices in guided nature experiences will be gathered from across the Audubon network. The implementation efforts will result in a national STEM model, with train-the-trainer guides and workshops for informal science educators and public engagement opportunities focused on improving the state and condition of avian habitats and communities through climate science research. An external evaluation will be conducted and will include data collection methods such as retrospective pre and post surveys, semi-structured interviews, focus groups, and an embedded assessment to determine impact. The findings will be used to iteratively refine the evidence-based curriculum and measure STEM learning outcomes for the guided nature experience participants. The evaluation will address four areas: (1) fidelity of program implementation to promote accountability; (2) formative evaluation to understand needs and interests of young adults (ages 18-25), and subsequently inform program design; (3) outcomes for Center educators, to inform iterative improvement; and (4) outcomes for program participants, to contribute to the growing knowledge base on effective practices for STEM learning in informal settings.
This Innovations in Development project is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
DATE:
-
TEAM MEMBERS:
Loren SmithMark ScallionHeather Starck
The employment demands in STEM fields grew twice as fast as employment in non-STEM fields in the last decade, making it a matter of national importance to educate the next generation about science, engineering and the scientific process. The need to educate students about STEM is particularly pronounced in low-income, rural communities where: i) students may perceive that STEM learning has little relevance to their lives; ii) there are little, if any, STEM-related resources and infrastructure available at their schools or in their immediate areas; and iii) STEM teachers, usually one per school, often teach out of their area expertise, and lack a network from which they can learn and with which they can share experiences. Through the proposed project, middle school teachers in low-income, rural communities will partner with Dartmouth faculty and graduate students and professional science educators at the Montshire Museum of Science to develop sustainable STEM curricular units for their schools. These crosscutting units will include a series of hands-on, investigative, active learning, and standards-aligned lessons based in part on engineering design principles that may be used annually for the betterment of student learning. Once developed and tested in a classroom setting in our four pilot schools, the units will be made available to other partner schools in NH and VT and finally to any school wishing to adopt them. In addition, A STEM rural educator network, through which crosscutting units may be disseminated and teachers may share and support each other, will be created to enhance the teachers’ ability to network, seek advice, share information, etc.
This application requests support to enable a team of experienced science educators and biomedical and behavioral health network scientists to develop and implement the Worlds of Connections curriculum. Most middle school students are familiar with patient care-related health careers (e.g., nurses, dentists, surgeons), but few know about emerging careers in network science that can be leveraged to improve population health. This innovative and research-based science program is strategically designed to increase awareness of, understanding of, and interest in the important role of network science for health. This project will design learning activities that incite interest in network science applications to biomedical and public health research. The long- term goal is to enhance the diversity of the bio-behavioral and biomedical workforce by increasing interest in network science among members of underrepresented minority communities and to promote public understanding of the benefits of NIH-funded research for public health. The goal of this application is to identify and create resources that will overcome barriers to network science uptake among underserved minority middle school youth. The central hypothesis is that the technology-rich field of network science will attract segments of today’s youth who remain uninterested in conventional, bio-centric health fields. Project activities are designed to improve understanding of how informal STEM experiences with network science in health research can increase STEM identities, STEM possible selves, and STEM career aspirations among youth from groups historically underrepresented in STEM disciplines at the center of health science research (Aim 1) and create emerging media resources via augmented reality technologies to stimulate broad interest in and understanding of the role of network science in biomedical and public health research (Aim 2). A team led by University of Nebraska-Lincoln sociologists will partner with the University of Nebraska at Omaha; state museums; centers for math, science, and emerging media arts; NIH-funded network scientists; educators; community learning centers at local public schools; learning researchers; undergraduates; software professionals; artists; augmented reality professionals; storytellers; and evaluation experts to accomplish these goals and ensure out of school learning will reinforce Next Generation Science Standards. The Worlds of Connections project is expected to impact 35,250 youth and 20,570 educators in Lincoln and Omaha, Nebraska by: adding network science modules to ongoing 6th-8th-grade afterschool STEM clubs in community learning centers; adding network science for health resources to a summer graduate course on “activating youth STEM identities” for sixth to twelfth grade STEM teachers; connecting teachers with local network scientists; creating free, downloadable, high-quality emerging media arts-enhanced stories; and publishing peer-reviewed research on the potential of network science to attract youth to health careers. Coupled with the dissemination plan, the project design and activities will be replicable, allowing this project to serve as a model to guide other projects in STEM communication.
PUBLIC HEALTH RELEVANCE:
The lack of public understanding about the role of network science in the basic biological and social health sciences limits career options and support for historically underrepresented groups whose diverse viewpoints and questions will be needed to solve the next generation of health problems. The Worlds of Connections project will combine network science, social science, learning research, biology, computer science, mathematics, emerging media arts, and informal science learning expertise to build a series of monitored and evaluated dissemination experiments for middle school science education in high poverty schools. Broad dissemination of the curriculum and project impacts will employ virtual reality technologies to bring new and younger publics into health-related STEM careers.
Consideration of the needs of individuals with a wide range of disabilities is not always considered in the early design stages of an informal STEM learning (ISL) activity or program. The primary access approach for people with disabilities becomes the provision of accommodations once the ISL product or environment is created. In contrast, the Universal Design approach considers users with a wide range of characteristics throughout the design process and works to create products and environments that are accessible, usable, and inclusive. This project, called AccessISL, led by the University of Washington's DO-IT (Disabilities, Opportunities, Internetworking and Technology) Center and Museology Program, includes an academic museology program and local ISL sites, representing museums, zoos, aquariums, makerspaces, science centers, and other sites of informal STEM learning. Insights will be gained through the engagement of people with disabilities, museology graduate students and faculty, and ISL practitioners. The AccessISL project model, composed of a set of approaches and interventions, builds on existing research and theory in the fields of education science, change management, effective ISL practices, and inclusive design processes. The project will collect evidence of policies and practices (or lack thereof) that improve the inclusiveness of ISL with respect to a wide range of disabilities and considers approaches for the design and development of new strategies; explores what stakeholders need to make change happen; uncovers challenges to the adoption of inclusive practices in public ISL settings and explores ways to overcome them; and proposes relevant content that might be included in museology curriculum. This project is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants.
This project addresses the following two objectives:
For ISL personnel and museology faculty: to increase knowledge, skills, and actions to make ISL programs, facilities, courses, and resources more welcoming and accessible to participants with disabilities and embed relevant practices within their work.
For postsecondary STEM students with disabilities and museology students: to increase knowledge and skills in advocating for ISL offerings that are welcoming and accessible to everyone, including those with a wide variety of disabilities, and to encourage individuals with disabilities to pursue careers in ISL.
The project employs a student-centered approach and a set of practices that embrace the social model of disability, social justice education, disability as a diversity issue, intersectionality, and Universal Design. A leadership team of interns--each member a STEM student with a disability or a museology graduate student--along with project staff will engage with the University of Washington's Museology Program to identify and implement strategies for making ISL activities and courses more welcoming and accessible to individuals with disabilities. An online community of practice will be developed from project partners and others nationwide. A one-day capacity building institute will be held to include presentations, student/personnel panels for sharing project and related experiences, and group discussions to explore issues and further identify systemic changes to make ISL programs more welcoming and accessible to individuals with disabilities. As prototypes of the AccessISL Model are developed, evaluation activities will primarily be formative (looking for strengths and weaknesses) and remedial (identifying/implementing changes that could be made to improve the model). The model will continue to be fine-tuned through formative evaluation. Evaluation of the model components will focus on the experience of a range of stakeholders in the project. Specifically, quantitative data collected will include levels and quality of engagement, accessibility recommendations and products developed, and delivery of ISL services. Qualitative data will be collected through observations, surveys, focus groups, interviews, and case studies.
AccessISL project products will include proceedings of an end-of-project capacity building institute, promising practices, case studies, a video, and other online resources to help ISL practitioners and museology faculty that will result in making future ISL opportunities more inclusive of people with disabilities. AccessISL will advance knowledge and ensure long-term impact using multiple strategies:
broadening the STEM participation of people with disabilities as well as women, racial/ethnic minorities, and other underrepresented groups through the application of universal design
strengthening associations and creating synergy and durable relationships among stakeholders,
encouraging teaching about disability, accessibility, and universal design in museology courses,
empowering students with disabilities and current and future ISL practitioners to advocate for accessible ISL and develops an infrastructure to promote accessible ISL programs nationwide, and
contributing to the body of promising practices with products that will (a) enhance understanding of issues related to the inclusion of people with disabilities in ISL programs and (b) promote inclusive practices.
Outcomes will benefit society by making STEM opportunities available to more people and enhancing STEM fields with the talents and perspectives of people with disabilities.
This project is funded by the National Science Foundation's (NSF's) Advancing Informal STEM Learning (AISL) program, which supports innovative research, approaches, and resources for use in a variety of learning settings.
This workshop series, convened by the Kavli, Rita Allen, Packard and Moore Foundations, was intended to view the entire system of people who support scientists’ engagement and communication efforts in order to explore how this system can be most effective and sustainable. The discussions examined where this system is thriving, the limits people within the system face and what can be done to ensure their efforts are commensurate with the demand for quality communication and engagement support.
Conducted over four closely scheduled workshops in late 2017 and early 2018, the convenings brought
This award was provided as part of NSF's Social, Behavioral and Economic Sciences Postdoctoral Research Fellowships (SPRF) program and is supported by SBE's Developmental Sciences program and the Directorate for Education and Human Resources' (EHR) Advancing Informal STEM Learning program. The goal of the SPRF program is to prepare promising, early career doctoral-level scientists for scientific careers in academia, industry or private sector, and government. SPRF awards involve two years of training under the sponsorship of established scientists and encourage Postdoctoral Fellows to perform independent research. NSF seeks to promote the participation of scientists from all segments of the scientific community, including those from underrepresented groups, in its research programs and activities; the postdoctoral period is considered to be an important level of professional development in attaining this goal. Each Postdoctoral Fellow must address important scientific questions that advance their respective disciplinary fields. Under the sponsorship of Dr. Sandra D. Simpkins at the University of California, Irvine, this postdoctoral fellowship award supports an early career scientist exploring high-quality and culturally responsive, math afterschool program (ASP) practices for under-represented minority (URM) youth. Mathematical proficiency is the foundation of youth's STEM pursuits. Yet today, far too many youth do not pursue STEM based on a perception that they are "not good at math". Students need to engage in contexts that spark their interest and their continued mastery and growth. ASPs are settings for such dynamic opportunities, particularly for URM students such as Latinos who attend lower quality schools and do not feel supported. In college, URM students often struggle with uninspiring and culturally incongruent STEM learning environments. The intergenerational nature of university-based STEM ASPs, whereby younger students are paired with undergraduate (UG) mentors, are opportunities to support both K-12 and UG students' motivational beliefs in math and STEM more broadly. This project will examine these intergenerational developmental processes in the context of a math enrichment ASP located at a Hispanic-Serving Institution. By studying how ASPs can serve as an important lever for promoting URM students' access and success in STEM, this project seeks to meaningfully inform efforts to broaden the participation of underrepresented groups in these fields.
This project seeks to understand how participating in a math enrichment ASP supports both youth participants' and UG mentors' motivational beliefs in math; to describe high-quality and culturally responsive practices; and to understand how to support the effectiveness of youth-staff relationships. To accomplish these research objectives, data will be collected from both youth participants and UG mentors through multiple methods including surveys, in-depth interviews, participant-observations, and video observations of youth-staff interactions. This project will add to our understanding of university-ASP partnerships. Further, the knowledge gained from this study will impact the larger landscape of practice and research on STEM ASPs by 1) addressing critical gaps in the current literature on high-quality and culturally responsive STEM ASP practices and 2) informing ASP staff development training. Overall, this mixed methods project will provide critical and rich information on the ways that ASPs can effectively deliver on its promise of promoting positive development for all youth, especially URM youth who may need and benefit from these spaces the most. The invaluable insight garnered from this study will be disseminated to traditional academic audiences to advance knowledge, as well as to local, state, and national organizations to inform the larger landscape of practice in STEM ASPs.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
Many scientists want to connect with the public, but their efforts to do so are not always easy or effective. Visionary programs and institutions are leading the way identifying the support needed to enable scientists’ connections with the public. However, the current appetite by -- and demand for -- scientists to do this exceeds the capacity of those who facilitate quality communication and engagement efforts. More can be done to ensure that those who support scientists are networked, sharing best practices, and supported by a reliable infrastructure.
This workshop series, convened by the Kavli, Rita Allen, Packard and Moore Foundations, was intended to view the entire system of people who support scientists’ engagement and communication efforts in order to explore how this system can be most effective and sustainable. The discussions examined where this system is thriving, the limits people within the system face and what can be done to ensure their efforts are commensurate with the demand for quality communication and engagement support.
Conducted over four closely scheduled workshops in late 2017 and early 2018, the convenings brought together leaders in different parts of the field who bridge scientists and the public and led to the emergence of a number of key priority areas. While the initial intention was to also hold a plenary event to provide a more holistic view of scientists’ support system in order to collectively discern directions to advance the field, we feel a more efficient way forward right now is to focus our efforts and resources on building community and advancing these priority areas.
Our invitation-only workshops brought together scientists, academic leaders, engagement professionals, researchers, communication trainers, and foundation leaders. For each workshop, we also commissioned a “landscape overview”, to better understand the high-level state of each community. Workshops included:
Workshop I: Communication and engagement training programs - Dec. 4-5, 2017 at SUNY Global Center/Alan Alda Center for Communicating Science in New York
Workshop II: Associations, societies and other professional organizations - Feb. 28 - March 1, 2018 at the Howard Hughes Medical Institute in Chevy Chase, MD
Workshop III: Academic institutions - March 27-28, 2018 at UC San Diego
Workshop IV: Science engagement facilitators (museums, science festivals, connectors) - May 2-3, 2018 at Monterey Bay Aquarium
TBD - Workshop V: Plenary event
The goal of the workshops was to explore how to ensure scientists’ communication and engagement support is effective and sustained. In doing so, we hoped to 1) deepen our understanding of how scientists are currently supported in these areas, 2) map the broader support system to expose the opportunities and obstacles that play a role in achieving this goal, and 3) identify strategic and practical next steps that move us closer to this goal. This initiative also aimed to forge and strengthen networks across communities and institutions – and in so doing, take a view of the entire system to explore how everyone can better ensure their efforts are impactful, mutually supportive, and connected to a greater whole.
Included in the links below are summaries from each workshop.
This is an introduction to four essays in a Commentary that examine contributions of universities to science communication’s development but also challenges in
consolidating those efforts.
As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants. This project will develop and test intergenerational science media resources for parents that are participating in adult education programs and their young children. The materials will build on the research-based and successful children's television program, Fetch with Ruff Ruffman. The target audience includes parents enrolled in adult education programs who lack a high school diploma or are in English as a Second Language classes. These resources will support parents' engagement in science activities with their children both in the adult education settings as well as at home. Adult and family educators will receive professional development resources and training to support their integration of the parent/child activities. Project partners include the National Center for Families Learning, Kentucky Educational Television, and Alabama Public Television,
The goals of the Ruff Family Science project are to: (1) investigate adult education settings that feature an intergenerational learning model, in order to learn about the unique characteristics of adults and families who are enrolled in these programs; (2) examine the institutional circumstances and educator practices that support joint parent/child engagement in science; (3) iteratively develop new prototype resources meet the priorities and needs of families and educators involved in intergenerational education settings; and (4) develop the knowledge needed to create a fuller set of materials in the future that will motivate and support diverse, low-income parents to investigate science with their children. The research strategy is comprised of three main components: Phase 1: Needs Assessment: Determine key motivations and behaviors common to adult education students who are also parents; surface obstacles and assets inherent in these parents' current practices; and examine the needs and available resources for supplementing parents' current engagement in family science learning. Phase 2: Prototype Development: Iteratively develop two prototype Activity Sets, along with related educator supports and training materials, designed to promote joint parent-child engagement with English and Spanish-speaking families around physical science concepts. Phase 3: Prototype Field Test: Test how the two refined prototype Activity Sets work in different educational settings (adult education, parent education, and parent and child together time). Explore factors that support or impede effective implementation. Sources of data for the study include observations of adult and parent education classes using an expert interview protocol, focus groups, adult and family educator interviews, and parent surveys.
This poster was presented at the 2010 Association of Science-Technology Centers Annual Conference. The Saint Louis Science Center is a partner in Washington University's Cognitive, Computational, and Systems Neuroscience interdisciplinary graduate program funded by the NSF-IGERT (Integrative Graduate Education and Research Traineeship) flagship training program for PhD scientists and engineers.