This is the final evaluation report from RMC Research Corp. for the PES@LTERs project. Appendix includes instruments.
RMC Research designed evaluation activities to provide formative and summative feedback to Harvard Forest and the Hubbard Brook Research Foundation (Hubbard Brook) on their plan to embed public engagement with science (PES) into the cultures and practices of Long-Term Ecological Research Sites (LTERs) in the northeastern US. The purpose of this project was to build PES mechanisms into long-term ecosystem studies that create on-going, open exchanges between scientists and
This Smart and Connected Community (SCC) project will partner with two rural communities to develop STEMports, an innovative Science, Technology, Engineering and Mathematics (STEM) learning game for workforce development. The game's activities will take players on localized Augmented Reality (AR) missions to both engage in STEM learning challenges and discover emerging STEM careers in their community, specifically highlighting innovations in the fields of sustainable agriculture and aquaculture, forest products, and renewable energy. Community Advisory Teams (CATs) and co-design teams, including youth, representatives from the targeted emerging STEM economies, and decision-makers will partner with project staff to co-design STEMports that reflect the interests, cultural contexts, and envisioned STEM industries of the future for each community.
The project will: (a) design and pilot an AR game for community STEM workforce development; (b) develop and adapt a community engagement process that optimizes community networking for co-designing the gaming application and online community; and (c) advance a scalable process for wider applications of STEMports. This project is a collaboration between the Maine Mathematics and Science Alliance and the Field Day Lab at the University of Wisconsin-Madison to both build and research the co-designing of a SCC based within an AR environment. The project will contribute knowledge to the informal STEM learning, community development, and education technology fields in four major ways:
Deepening the understanding of how innovative technological tools support rural community STEM knowledge building as well as STEM identity and workforce interest.
Identifying design principles for co-designing the STEMports community related to the technological design process.
Developing social network approaches and analytics to better understand the social dimensions and community connections fostered by the STEMport community.
Understanding how participants' online and offline interactions with individuals and experiences builds networks and knowledge within a SCC.
With the scaling of use by an ever-growing community of players, STEMports will provide a new AR-based genre of public participation in STEM and collective decision making. The research findings will add to the emerging literature on community-wide education, innovative education technologies, informal STEM learning (especially place-based learning and STEM ecosystems), and participatory design research.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
Over the seven years prior to this award, the principal investigator from George Mason University and a national team of scientists, professional societies, science communication researchers, and broadcast meteorologists have been engaged in an effort to include in TV and other weather broadcasts information about current research on the interactions of climate and weather. A Climate Matters network has been established that involves 350 weathercasters at 218 stations, in 119 media markets, nationwide. A particular focus of the initiative has been to help the public become more familiar with the science behind how their local weather and its trends are related to the dynamics of the climate. Many communities nationwide are engaged in deliberations about how to understand, plan for, and adapt to the potential impacts of changes in their weather on important factors pertaining to their economy and well-being, such as natural resources, natural disasters, agriculture, industry, and health. The goal of this continuing project is to expand the quantity and nature of the coverage of such information into the news segments of local news media. By stimulating local reporting on climate impacts and their relationships to personal and community-wide decision-making, this project will potentially help millions of Americans better understand and respond to critical factors that are affecting their lives. This project is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants.
The project involves five inter-related, complementary activities: (1) Knowledge building through formative research and process evaluation, specifically in-depth interviews and random sample surveys of journalists in each of the participating journalism professional societies; (2) Recruiting 400 news directors, producers, reporters and additional weathercasters into the Climate Matters network; (3) Providing climate reporting training and professional development to members of the network; (4) Producing and distributing Climate Matters reporting packages to all members of the network on a near-weekly basis; and (5) Evaluating the impacts of the climate reporting on public understanding of science.
DATE:
-
TEAM MEMBERS:
Edward MaibachSusan HassolBernadette PlackyRichard CraigTeresa Myers
As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants. This project will embed public engagement with science (PES) into the cultures and practices of two Long-Term Ecological Research (LTER) sites: the Hubbard Brook Experimental Forest in New Hampshire and the Harvard Forest in Massachusetts. The goals are 1) to build knowledge about the mutual learning between scientists and adult stakeholders in face-to-face engagement setting and 2) to develop evidence-based practices in the content of place-based ecosystem research. This is a collaborative project of 3 universities (Michigan State University, Harvard, and CUNY) and the two LTERs. Two primary research questions guide this work. First, how willing are participating scientists to take part in PES? What are their attitudes and beliefs about whether engagement can be effective and whether they have the necessary skills? Second, how willing are participating scientists to build relationships with stakeholders using normed tactics? Both qualitative and quantitative methods will be used to collect evidence including semi-structured interviews and surveys. A general set of hypothesis include that there will be positive changes in LTER scientists willingness to participate in PES, attitudes, and efficacy beliefs.
DATE:
-
TEAM MEMBERS:
John BesleySarah GarlickPeter GroffmanPamela TemplerKathleen Lambert
"Strengthening Networks, Sparking Change: Museums and Libraries as Community Catalysts" combines findings from a literature scan and input from the library, museum and community revitalization fields with case studies about the experiences and vision of museums and libraries working to spur change in their communities. It describes the complementary conceptual frameworks of social wellbeing and collective impact and explains how libraries and museums can use these concepts to partner with community-based organizations, government agencies and other cultural or educational organizations. It
NatureStart Network brought together early childhood educators and environmental educators to support nature play, exploration, and inquiry for young children and their families within urban environments. Project partners included the Forest Preserves of Cook County and two established Head Start programs in the Chicago area, Mary Crane Center and El Valor. The foundation of the project was a series of three two-day professional learning sessions that took place over an eighteen month period. Through hands-on, collaborative learning and reflection activities, the participating educators
DATE:
TEAM MEMBERS:
David Becker
resourceprojectProfessional Development, Conferences, and Networks
The National Writing Project (NWP) is collaborating with the Association of Science-Technology Centers (ASTC) on a four-year, full-scale development project that is designed to integrate science and literacy. Partnerships will be formed between NWP sites and ASTC member science centers and museums to develop, test, and refine innovative programs for educators and youth, resulting in the creation of a unique learning network. The project highlights the critical need for the integration of science and literacy and builds on recommendations in the Common Core State Standards and the National Research Council's publication, "A Framework for K-12 Science Education: Practices, Crosscutting Concepts, and Core Ideas." The content focus includes current topics in science and technology such as environmental science, sustainability, synthetic biology, geoengineering, and other subjects which align with science center research and exhibits. The project design is supported by a framework that incorporates a constructivist/inquiry-based approach that capitalizes on the synergy between rigorous science learning and robust literacy practices. Project deliverables include a set of 10 local partnership sites, professional development for network members, a project website, and an evaluation report highlighting lessons learned. Partnership sites will be selected based on interest, proximity, history, and expertise. Two geographically and demographically diverse cohorts, consisting of five partnerships each will be identified in Years 2 and 3. Each set of partners will be charged with creating a comprehensive two-year plan for science literacy activities and products to be implemented at local sites. It is anticipated that the pilot programs may result in the creation of new programs that merge science and writing, integrate writing into existing museum science programs, or integrate science activities into existing NWP programs. Interest-driven youth projects such as citizen science and science journalism activities are examples of programmatic approaches that may be adopted. The partners will convene periodically for planning and professional development focused on the integration of science and literacy for public and professional audiences, provided in part by national practitioners and research experts. A network Design Team that includes leadership representatives from NWP, ASTC, and the project evaluator, Inverness Research, Inc., will oversee project efforts in conjunction with a national advisory board, while a Partnership Coordinator will provide support for the local sites. Inverness Research will conduct a multi-level evaluation to address the following questions: -What is the nature and quality of the local partner arrangements, and the larger network as a whole? -What is the nature and quality of the local science literacy programs that local partners initiate, and how do they engage local participants, and develop their sense of inquiry and communication skills? First, a Designed-Based Implementation Research approach will be used for the developmental evaluation to assess the implementation process. Next, the documentation and portrayal phase will assess the benefits to youth, educators, institutions, and the field using surveys, interviews, observations of educators, and reviews of science communication efforts created by youth. Finally, the summative evaluation includes a comprehensive portfolio of evidence to document the audience impacts and an independent assessment of the project model by an Evaluation Review Board. This project will result in the creation of a robust learning community while contributing knowledge and lessons learned to the field about networks and innovative partnerships. It is anticipated that formal and informal educators will gain increased knowledge about science and literacy programs and develop skills to provide effective programs, while youth will demonstrate increased understanding of key science concepts and the ability to communicate science. Programs created by the local partnerships will serve approximately 650 educators (450 informal educators and 200 K-12 teachers) and 500 youth ages 9-18. Plans for dissemination, expansion, and sustainability will be undertaken by the sub-networks of the collaborating national organizations drawing on the 350 ASTC member institutions and nearly 200 NWP sites at colleges and universities.
Expanding on the encouraging outcomes of an NSF-funded conference, this three-year project led by the National Center for Science and Civic Engagement at Harrisburg University of Science and Technology, in collaboration with the Koshland Science Museum of the National Academy of Sciences, will explore and evaluate ways to support new collaborations between professionals in institutions of higher education and informal STEM education around areas of common interest. The primary goal is to develop the educational infrastructure to grow and efficiently sustain multiple cross-organizational partnership activities at the intersection of learning about science, society and civic engagement around such possible topics as energy, environment, genetics, earth resources, computers and ethics, nanotechnology, etc. The initiative is: 1) creating a joint organizing "secretariat" to provide communications and support through low-cost shared services for at least six partnerships around the country; 2) providing partnership support and technical assistance to seed the six national partnerships, and 3) sharing evaluation and analysis services across all the partnerships. The outcomes of the work pertain to improvements in professional knowledge and practice in higher education and informal science education, as well as the improvement of learning by undergraduates and by the general public.
The Global Soundscapes! Big Data, Big Screens, Open Ears Project uses the new science of soundscape ecology to design a variety of informal science learning experiences that engage participants through acoustic discovery Soundscape ecology is an interdisciplinary science that studies how humans relate to place through sound and how humans influence the environment through the alteration of natural sound composition. The project includes: (1) an interface to the NSF-funded Global Sustainable Soundscapes Network, which includes 12 universities around the world; (2) sound-based learning experiences targeting middle-school students (grades 5-8), visually impaired and urban students, and the general public; and (3) professional development for informal science educators. Project educational components include: the first interactive, sound-based digital theater experience; hands-on Your Ecosystem Listening Labs (YELLS), a 1-2 day program for school classes and out-of school groups; a soundscape database that will assist researchers in developing a soundscape Big Database; and iListen, a virtual online portal for learning and discovery about soundscape. The project team includes Purdue-based researchers involved in soundscape and other ecological research; Foxfire Interactive, an award-winning educational media company; science museum partners with digital theaters; the National Audubon Society and its national network of field stations; the Perkins School for the Blind; and Multimedia Research (as the external evaluator).
In December 2008, the Monterey Bay Aquarium, in partnership with the New England Aquarium and the National Aquarium in Baltimore, hosted a summit of leaders from aquariums across North America and beyond. The intent was to empower aquarium professionals to elevate their collective focus on the relationship between climate change and the ocean, and to develop collaborative strategies for communicating climate change and inspiring solution-‐oriented actions among our institutions and audiences. The summit represented the birth of a community. The three‐day event succeeded in mobilizing 34