This is the final evaluation report from RMC Research Corp. for the PES@LTERs project. Appendix includes instruments.
RMC Research designed evaluation activities to provide formative and summative feedback to Harvard Forest and the Hubbard Brook Research Foundation (Hubbard Brook) on their plan to embed public engagement with science (PES) into the cultures and practices of Long-Term Ecological Research Sites (LTERs) in the northeastern US. The purpose of this project was to build PES mechanisms into long-term ecosystem studies that create on-going, open exchanges between scientists and
Northern ecosystems are rapidly changing; so too are the learning and information needs of Arctic and sub-Arctic communities who depend on these ecosystems for wild harvested foods. Public Participation in Scientific Research (PPSR) presents a possible method to increase flow of scientific and local knowledge, enhance STEM-based problem solving skills, and co-create new knowledge about phenology at local and regional or larger scales. However, there remain some key challenges that the field of PPSR research must address to achieve this goal. The proposed research will make substantial contributions to two of these issues by: 1) advancing theory on the interactions between PPSR and resilience in social-ecological systems, and 2) advancing our understanding of strategies to increase the engagement of youth and adults historically underrepresented in STEM, including Alaska Native and indigenous youth and their families who play an essential role in the sustainability of environmental monitoring in the high latitudes and rural locations throughout the globe. In particular, our project results will assist practitioners in choosing and investing in design elements of PPSR projects to better navigate the trade-offs between large-scale scientific outcomes and local cultural relevance. The data collected across the citizen science network will also advance scientific knowledge on the effects of phenological changes on berry availability to people and other animals.
The Arctic Harvest research goals are to 1) critically examine the relationship between PPSR learning outcomes in informal science environments and attributes of social-ecological resilience and 2) assess the impact of two program design elements (level of support and interaction with mentors and scientists, and an innovative story-based delivery method) on the engagement of underserved audiences. In partnership with afterschool clubs in urban and rural Alaska, we will assess the impact of participation in Winterberry, a new PPSR project that investigates the effect of changes in the timing of the seasons on subsistence berry resources. We propose to investigate individual and community-level learning outcomes expected to influence the ability for communities to adapt to climate change impacts, including attributes of engagement, higher-order thinking skills, and their influence on the level of civic action and interest in berry resource stewardship by the youth groups. Using both quantitative and qualitative approaches, we compare these outcomes with the same citizen science program delivered through two alternate methods: 1) a highly supported delivery method with increased in-person interaction with program mentors and scientists, and 2) an innovative method that weaves in storytelling based on elder experiences, youth observations, and citizen science data at all stages of the program learning cycle. This project is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants. The project also has support from the Office of Polar Programs.
DATE:
-
TEAM MEMBERS:
Katie SpellmanElena SparrowChrista MulderDeb Jones
As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program funds innovative research, approaches and resources for use in a variety of settings. The project will conduct research designed to deepen our fundamental knowledge about culture, experience, and ecosystems cognition and to develop innovative practices and approaches to support learning about changing ecological systems and environmental decision making. Work on cultural differences in the production of complex systems knowledge is severely lacking. This gap in knowledge may contribute to the continued reproduction of inequities in science education. More broadly findings from this project will have clear implications for theories of cognitive development, especially those pertaining to how knowledge is shaped by culture and experience. Focusing on ecosystems may represent an opportunity to not only increase engagement and achievement in science among non-dominant communities and Native youth specifically, but also advance effective learning for all communities. The primary deliverables for the project are conference presentations and research publications. However, the project will also develop additional resources freely available to researchers, educators, and the general public. These will include summer curricular materials and teaching tools, professional development workshops, practitioner briefs about research findings that can be used in professional development workshops and shared share more broadly, and evaluation reports.
A deeper understanding of cultural influences on conceptions of the natural world can serve to advance the educational needs of children, including children from diverse linguistic and cultural backgrounds. Project research will include two interrelated series of studies designed to expand knowledge about human cognition of complex ecosystems and the affordances of informal STEM learning environments in developing and supporting the critical 21st century skill of ecological systems level reasoning. The first consists of a series of experiments focused on ecological cognition and the role of humans in nature. The second consists of design-based research interventions in informal settings, summer workshops for youth and the communities, focused on ecological systems level thinking and socio-environmental decision making. The project will recruit and engage both child and adult participants from two broad cultural communities, Native Americans and European Americans living in urban and suburban communities, in part because it affords a sharp test of human-nature relations. Sampling from two different urban communities will avoid simple Native-non-Native comparative binaries and to conduct Native-to-Native comparative analysis. Based on results from this, the project will result in: 1) foundational knowledge about human learning and reasoning and ecosystems and environmental decision making, 2) culturally responsive models of learning and practice about complex ecosystems for indoors and outdoors informal learning environments, and 3) insights about research-practice-community partnerships. One important objective of the research is to broaden participation and close opportunity gaps for under-represented groups in STEM fields broadly and more specifically for Indigenous people. Members of Indigenous communities, who provide strong role models for other aspiring scholars, will be involved as postdoctoral fellows, research assistants and graduate fellows.
As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants. This project will embed public engagement with science (PES) into the cultures and practices of two Long-Term Ecological Research (LTER) sites: the Hubbard Brook Experimental Forest in New Hampshire and the Harvard Forest in Massachusetts. The goals are 1) to build knowledge about the mutual learning between scientists and adult stakeholders in face-to-face engagement setting and 2) to develop evidence-based practices in the content of place-based ecosystem research. This is a collaborative project of 3 universities (Michigan State University, Harvard, and CUNY) and the two LTERs. Two primary research questions guide this work. First, how willing are participating scientists to take part in PES? What are their attitudes and beliefs about whether engagement can be effective and whether they have the necessary skills? Second, how willing are participating scientists to build relationships with stakeholders using normed tactics? Both qualitative and quantitative methods will be used to collect evidence including semi-structured interviews and surveys. A general set of hypothesis include that there will be positive changes in LTER scientists willingness to participate in PES, attitudes, and efficacy beliefs.
DATE:
-
TEAM MEMBERS:
John BesleySarah GarlickPeter GroffmanPamela TemplerKathleen Lambert
The Water for Life project has been an effective, and in some cases an essential vehicle for addressing issues around water quality and retention in island settings where water security is an on-going challenge. The focus on local partnerships was a highly valued attribute of the WfL project, and the informal science and conservation education resources produced and disseminated by the project have had a significant impact on these populations
During March 2017, Kathy Burke and Kari Nelson conducted a study of visitors to the special exhibition, The Power of Poison. The objective of this study was to discover how visitors moved through the exhibition and interacted with it, and how visitors perceived the intended message and related the exhibition to the overall goals of the NHMU. To achieve the objectives of this study, two evaluation methods were used: unobtrusive tracking and timing and cued questionnaires.
The Power of Poison is among the most thoroughly used exhibitions to be hosted in the NHMU special exhibitions gallery
The Extreme Plants Traveling Sideshow is a theatre piece performed at the Natural History Museum of Utah in relation to the special exhibition, The Power of Poison. While NHMU has a history of Museum Theatre performances, this was the first to be performed since its move to the Rio Tinto Center.
The goal of this evaluation was to understand the visitor experience with museum theater and museum theatre’s value in communicating science content, as well as to inform future productions at NHMU. With this in mind, we sought answer the following questions:
1. Did guests feel the performance
Learn how to create opportunities for young people from low-income, ethnically diverse communities to learn about growing food, doing science, and how science can help them contribute to their community in positive ways. The authors developed a program that integrates hydroponics (a method of growing plants indoors without soil) into both in-school and out-of-school educational settings.
As part of the National Science Foundation (NSF) funding for the In Defense of Food project directed by Kikim Media, the independent evaluation firm Knight Williams Inc.1 conducted a summative evaluation of the project’s key deliverables, which included: a PBS television broadcast program, an outreach effort, and an educational curriculum. This report (Study 2 of 3) focuses on the outreach effort.
As part of the National Science Foundation (NSF) funding for the In Defense of Food project directed by Kikim Media, the independent evaluation firm Knight Williams Inc. conducted a summative evaluation of the project’s key deliverables, which included: a PBS television broadcast program, an outreach effort, and an educational curriculum. This report (Study 1 of 3) considers the film’s overall appeal, clarity, learning value, and motivational impact among viewers matching the film’s target audience, and focuses on the following six questions:
1) Did viewers find the film appealing, engaging
This innovative research project promotes the progress of science, enhances the national STEM workforce, and benefits society by helping to overcome the challenge of broadening participation of those who are underrepresented in STEM fields. Although many programs designed to broaden participation exist, few individuals in "STEM-disenfranchised" populations -- individuals who feel alienated, marginalized, or incapable of participating in STEM -- choose to make use of these opportunities, due mainly to their own self-identities. This project's focus is on three STEM-disenfranchised groups: 1) adults who have been recently released from incarceration; 2) youth who have been released from juvenile custody; and 3) refugee youth, and builds on existing science education programs. The research team will establish the "Alliance to Strengthen the STEM Tapestry (ASSiST)" -- with members from academia, workforce agencies, NGOs, and government agencies -- to explore how individuals who have an identity prematurely tied to failure in science might benefit from novel interventions that promote a shift of self-identity to becoming science learners, which will then lead them to explore STEM education and job training resources that already exist. Three novel interventions will involve drama activities, story- telling, and ecological restoration projects. This bold approach is designed to help these populations interweave their diverse ways of knowing with STEM workforce, higher education, and to become science-aware citizens, which will enhance U.S. leadership in STEM. ASSisT will create a strategic plan that can be interwoven with those of other NSF INCLUDES Alliances, and identify pathways to distribute outcomes to a national level. This work will provide pathways to bring other groups that are disenfranchised and who -- if motivated and directed -- could strengthen the STEM workforce and education tapestry.
Investments to broaden participation in science in the USA have supported abundant programs and resources, but few individual in "STEM-disenfranchised" populations -- individuals who feel alienated, marginalized, or incapable of participating in STEM -- choose to make use of these opportunities, due most significantly to their own self-identities. The proposed "Alliance to Strengthen the STEM Tapestry (ASSisT)" will carry out research on novel interventions that are designed to lead these individuals to avail themselves of the science education and training resources that already exist. The initial focus is on: 1) adults who have been recently released from incarceration; 2) youth who have been released from juvenile custody; and 3) refugee youth. Using a collective impact approach, ASSisT will carry out early-exploratory research to investigate how the project's novel interventions -- 1) ecological restoration, 2) story-telling/autoethnography, and 3) devised theater -- might shift participants towards self-identification and subsequent involvement with the STEM community. The Intellectual Merit of our approach is grounded in social science research, specifically, identity theory, social cognitive theory, and resilience theory. Using a one-group pretest-posttest design, qualitative research techniques will identify which elements are most critical to foster change, e.g., perceived competence in STEM subjects, congruence of self-perception with those in STEM, mastery of STEM workforce skills, and/or the importance of being a STEM-aware citizen. Broader impacts relate directly to NSF's call for greater STEM participation of women and underrepresented ethnic and socioeconomic minorities with impacts on the initial 30 cohort members for this pilot project. ASSisT will: create a common agenda; recruit cohorts of each STEM-disenfranchised group; design and implement research to test novel interventions; populate a STEM opportunities map; evaluate and analyze outcomes; articulate a strategic plan that can be interwoven with those of other NSF INCLUDES Alliances; and identify pathways to disseminate outcomes and benchmarks to a national level.
DATE:
-
TEAM MEMBERS:
Nalini NadkarniJordan GertonDiane PatakiSydney Cheek-O'DonnellRussell Isabella