This is the final evaluation report from RMC Research Corp. for the PES@LTERs project. Appendix includes instruments.
RMC Research designed evaluation activities to provide formative and summative feedback to Harvard Forest and the Hubbard Brook Research Foundation (Hubbard Brook) on their plan to embed public engagement with science (PES) into the cultures and practices of Long-Term Ecological Research Sites (LTERs) in the northeastern US. The purpose of this project was to build PES mechanisms into long-term ecosystem studies that create on-going, open exchanges between scientists and
This is a poster created for the 2019 annual meeting of the American Geophysical Union. Below is the accepted abstract of the poster:
The Long Term Ecological Research (LTER) network spans 28 sites across North America and beyond. Here we share work from two LTER sites in the northeastern US—the Harvard Forest, MA and the Hubbard Brook Experimental Forest, NH—where boundary organizations have been developing a framework for embedding effective public engagement and knowledge co-production into these long-term studies. Here we share this framework and how it has been developed and applied in
The U.S. Fish and Wildlife Service estimates that over 41 million people connect to nature through birding. Learning about birds in their natural environments offers opportunities for informal engagement in STEM by a broad range of individuals and groups. Birders often engage in scientific data gathering and analyses, geolocation and remote sensing, and phenology. They also become aware of ecological changes in bird habitats and migratory patterns due to rising temperatures and climate-related events like sea level rise, droughts, fires, and extreme weather. As such, the birding community is an ideal network to better understand and communicate the impacts of climatological changes on bird populations to the public. With this Innovations in Development project, the National Audubon Society will develop a new avian-focused, conservation and climate science community science curriculum for its Nature Centers, and test the effectiveness of the curriculum in educating the public about avian-focused conservation and climatological changes through guided nature experiences. Birding can serve as a pivotal entrée for young people into STEM fields and careers. Through its programs and partnerships, Audubon will leverage its national network to ensure that through this project a more diverse group of voices, particularly young adults and young adults of color, become involved in asking critical questions and developing solutions to address important environmental issues of the future. If successful, the broader impacts of this project on capacity building and public engagement could be far-reaching and long-lasting.
Over the three-year project duration, Audubon will bring educators from its nationwide network of thirty-four Nature Centers (including urban, suburban, and rural sites), together with over 510 young adults (ages 18-25) from its network of college campus chapters. An evidence-based curriculum and community science activities will be created and tested, relying heavily on a team of experts in ornithology, climate science research, STEM curriculum design, diversity, and informal science education. College students will advise on the design of content and activities to effectively interest and engage young adults. These students will be recruited from the new Audubon Campus Chapters Program, which includes 111 college and university campuses, among them, 19 Historically Black Colleges and Universities (HBCUs) and other Minority Serving Institutions (MSIs). The target population will be surveyed to also understand their current and likely participation in guided nature experiences and knowledge base in climate science. Current best practices in guided nature experiences will be gathered from across the Audubon network. The implementation efforts will result in a national STEM model, with train-the-trainer guides and workshops for informal science educators and public engagement opportunities focused on improving the state and condition of avian habitats and communities through climate science research. An external evaluation will be conducted and will include data collection methods such as retrospective pre and post surveys, semi-structured interviews, focus groups, and an embedded assessment to determine impact. The findings will be used to iteratively refine the evidence-based curriculum and measure STEM learning outcomes for the guided nature experience participants. The evaluation will address four areas: (1) fidelity of program implementation to promote accountability; (2) formative evaluation to understand needs and interests of young adults (ages 18-25), and subsequently inform program design; (3) outcomes for Center educators, to inform iterative improvement; and (4) outcomes for program participants, to contribute to the growing knowledge base on effective practices for STEM learning in informal settings.
This Innovations in Development project is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
DATE:
-
TEAM MEMBERS:
Loren SmithMark ScallionHeather Starck
resourceprojectProfessional Development, Conferences, and Networks
Implicit bias and inequities limit the quality, effectiveness, and widespread acceptance of the outdoor and environmental science education field. The field continues to struggle to find resonance with the most tenacious concerns and challenges of communities of color, especially with regards to environmental issues that disproportionately impact the health and economies of these communities. In a time of calls to action to dismantle systemic racism and bias, there is a renewed interest in solution driven approaches to address matters of equity, inclusion, and cultural relevance in education and within organizational change frameworks. This Innovations in Development project will develop and test a model to build individual and organizational capacities to create and sustain equitable, inclusive, culturally relevant workplaces and learning environments, and support professionals of color currently in positions in outdoor science programs and may be at risk of leaving the field. With capacity building support for systems change, the model will help organizations to lead with equity as they plan for the future. The need for this work could not be timelier. If successful, the knowledge gleaned from the Working Toward Equitable Organizations model could inform future efforts to transform the field through institutional changes that result in a more diverse STEM workforce at all levels of leadership and inclusive programs and practices that support STEM learning and engagement in outdoor and environmental science education programs.
Over the three-year project duration, the project will be centered on two strands: (1) Support for Organizational Systems and (2) Professionals of Color Engagement. For the first strand, two cohorts of outdoor science program leaders will engage in intensive reflection, professional learning, and development. They will consider all aspects of their work through an equity lens, develop action plans, and make necessary adjustments to curricula, guiding documents, and practices. For the second strand, a cohort model will be used to create professional learning and engagement communities for professionals of color in outdoor science programs as they navigate the challenges associated with being in the minority in a predominately white-dominated field. In addition, a rigorous research study will be conducted to examine how the professional learning model contributes to changes in organizations to create more inclusive and equitable career paths for professionals of color and will describe under what conditions outdoor science programs are able to make institutional change. A culturally responsive evaluation will inform the design and development of the model and assess its effectiveness. Together, the evaluation and research will identify promising aspects of the work and directions for future scaling. The project will develop and document a scalable model for program leaders and professionals of color that builds the capacity of organizations to promote equity, inclusion, and cultural relevance. It is poised to impact twenty organizations nationwide, 60 professionals of color and ultimately, 200,000 students annually.
This Innovations in Development project is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
DATE:
-
TEAM MEMBERS:
Craig StrangValeria RomeroJedda Foreman
Most researchers are keenly interested in disseminating their work beyond traditional publication routes. With an eye to increasing broader impacts, scientists can benefit from partnerships with informal educators who interact daily with the public and see their role as translating science to increase the public’s intellectual and emotional connections with the natural world. Typically, researchers give a one-time lunch hour talk, generally a modified version of a presentation aimed at scientific peers. Talks during which scientists show slides and interpreters mainly listen are a missed
The University of Guam (UOG) NSF INCLUDES Launch Pilot project, GROWING STEM, addresses the grand challenge of increasing Native Pacific Islander representation in the nation's STEM enterprise, particularly in environmental sciences. The project addresses culturally-relevant and place-based research as the framework to attract, engage, and retain Native Pacific Islander students in STEM disciplines. The full science, technology, engineering and mathematics (STEM) pathway will be addressed from K-12 to graduate studies with partnerships that include the Guam Department of Education, Humatak Community Foundation, Pacific Post-Secondary Education Council, the Guam Science and Discovery Society, the Society for the Advancement of Chicanos/Hispanics and Native Americans in Science (SACNAS) and the University of Alaska-Fairbaanks. As the project progresses, the project anticipates further partnerships with the current NSF INCLUDES Launch Pilot project at the University of the Virgin Islands.
Pilot activities include summer internships for high school students, undergraduate and graduate research opportunities through UOG's Plant Nursery and the Humatak Community Foundation Heritage House. STEM professional development activities will be offered through conference participation and student research presentations in venues such as the Guam Science and Discovery Society's Guam Island-wide Science Fair and SACNAS. Faculty will be recruited to develop a mentoring protocol for the project participants. Community outreach and extension services will expand public understanding in environmental sciences from the GROW STEM project. Project metrics will include monitoring the diversity of partners, increases in community engagement, Native Pacific Islander participation in STEM activities, the number of students who desire to attain terminal STEM degrees and the number of community members reached by pilot STEM extension and outreach activities. Dissemination of the GROWING STEM pilot project results will occur through the NSF INCLUDES National Network, partner annual conferences, and local, regional and national STEM conferences.
DATE:
-
TEAM MEMBERS:
John PetersonCheryl SanguezaElse DemeulenaereAustin Shelton
As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants. This project will embed public engagement with science (PES) into the cultures and practices of two Long-Term Ecological Research (LTER) sites: the Hubbard Brook Experimental Forest in New Hampshire and the Harvard Forest in Massachusetts. The goals are 1) to build knowledge about the mutual learning between scientists and adult stakeholders in face-to-face engagement setting and 2) to develop evidence-based practices in the content of place-based ecosystem research. This is a collaborative project of 3 universities (Michigan State University, Harvard, and CUNY) and the two LTERs. Two primary research questions guide this work. First, how willing are participating scientists to take part in PES? What are their attitudes and beliefs about whether engagement can be effective and whether they have the necessary skills? Second, how willing are participating scientists to build relationships with stakeholders using normed tactics? Both qualitative and quantitative methods will be used to collect evidence including semi-structured interviews and surveys. A general set of hypothesis include that there will be positive changes in LTER scientists willingness to participate in PES, attitudes, and efficacy beliefs.
DATE:
-
TEAM MEMBERS:
John BesleySarah GarlickPeter GroffmanPamela TemplerKathleen Lambert
As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program supports new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This project will meet this goal through rigorous research and the broad implementation of an environmental science literacy professional development and learning program for informal educators and youth engaged in outdoor science programs (OSP). With growing support from the literature and the Next Generation Science Standards (NGSS), much attention has been placed on creating and leveraging interdisciplinary science learning opportunities beyond science classrooms. As such, an estimated 300 residential OSPs currently exist in the United States. Unfortunately, the informal educators often charged with facilitating these deep and impactful science learning experiences often lack robust formal training in evidenced-based, age-appropriate environmental science content knowledge and pedagogy specific for the youth in their programs. This issue is often more pronounced in under-resourced and under-served programs and communities. This project will directly address these pervasive challenges in the field by not only providing much needed science focused professional development and resources to informal educators but also by specifically targeting and training informal leaders and educators serving youth in predominately rural areas, low-income communities, and underrepresented communities.
Approximately 200 OSP leaders at 100 OSPs around the country will participate in a week-long, intensive training in the professional development model at one of five regional residential leadership institutes. OSP leaders will then redeliver the training to the approximately 1,500 OSP educators/field instructors in their home institutions. The OSP educators/field instructors will then use what they learn through the professional development to facilitate the environmental science learning program (i.e., curriculum, field experiences, resources, pedagogy) to over 1 million youth (grades 3-8) enrolled in their residential outdoor science programs. In addition, a rigorous implementation study, efficacy study and evaluation will be conducted. The implementation study will investigate: (a) Which of the professional learning model practices were implemented and (b) What successes and challenges the programs faced implementing the model. The mixed methods efficacy study will explore: (a) if outdoor science programs contribute to the development of science learning activation and environmental literacy? and (b) what are the features of these experiences that are correlated with increases in science learning activation and environmental literacy. Approximately 25-35 youth will be randomly selected from each of 50 randomly selected sites to participate in the efficacy study. The data and findings from the research and evaluation produced by this project will contribute to a relatively sparse knowledge and research base specific to youth efficacy and implementation processes and practices across nearly 1/3 of the estimated 300 existing residential outdoor science programs in the United States.
This poster was presented at the 2016 Advancing Informal STEM Learning (AISL) PI Meeting held in Bethesda, MD on February 29-March 2. Earth Partnership: Indigenous Arts and Sciences (IAS) refines a model for integrating Indigenous and Western STEM education utilizing a 10-step framework for ecological restoration, project-based learning, and professional development. Through community dialogues and a collaborative design process with Native Nations of Wisconsin, Earth Partnership is developing an Indigenous Arts and Sciences approach that has allowed Native participants to voice their insights
In recent years, there has been much concern over the decline of biologists who actually identify themselves to be naturalists, which negatively impacts the field of conservation and the study of biology as a whole. This could result in a decrease in individuals who participate in naturalist-like activities, such as informal environmental education and environmental volunteerism. The purpose of my study was to determine what discourse identities were held by naturalist development program participants, how these discourse identities related to their volunteer motives in environmental settings