This document presents the final evaluation report for the NSF-funded AISL project: "Multimodal Visitor Analytics: Investigating Naturalistic Engagement with Interactive Tabletop Science Exhibits."
Through the T523: Formative Evaluation for Educational Product Development course, our team conducted a semester-long formative evaluation for the Museum of Science, Boston (MoS) Gaia Exhibit. The Gaia Exhibit (Gaia) is a new, temporary art installation located in the MoS’s Blue Wing exhibition hall. Gaia that strives to inspire appreciation for the earth and climate change awareness. The exhibit displays imagery of the Earth’s surface on a twenty feet diameter, three-dimensional globe. Additional exhibit elements include projected questions on the floor to prompt reflection and exhibit-
This NOVA multiplatform media initiative consisted of a 2-hour nationally broadcast PBS documentary, Polar Extremes; a 10-part original digital series, Antarctic Extremes; an interactive game, Polar Lab; accompanying polar-themed digital shorts, radio stories, text reporting, and social media content; a collection of educational resources on PBS LearningMedia; and community screening events and virtual field trips for science classrooms. Across multiple media platforms the project’s video content had nearly 13 million views.
The research explored the potential for informal STEM learning
This handout was prepared for the Climate Change Showcase at the 2019 ASTC Conference in Toronto, Ontario. It highlights resources available on InformalScience.org related to the topic of climate change.
RK&A conducted a summative evaluation of five climate change and resiliency programs at the Science Museum of Virginia (SMV). The summative evaluation was the culmination of a three-year grant from the National Oceanic and Atmospheric Administration (NOAA)—front-end and formative evaluations were completed in the two years prior. The goal of this evaluation was to test the extent to which the programs achieved the intended visitor outcomes defined in SMV’s Impact Framework.
How did we approach this study?
The summative evaluation focused on five programs—the Digital Dome theater’s
‘6 Degrees of Connection’ is an informative and memorable program that combines a one-hour Science On a Sphere® interactive presentation with a follow-up creative art activity – each aspect of which encourages middle school students to think of interconnections among phenomena on the planet, including the natural world and human activity.
The ‘6 Degrees of Connection’ program was developed by the Nurture Nature Center (Easton PA) in collaboration with the Maryland Science Center (Baltimore MD) through an extensive process of developing the concept, prototypes, and final program. Lehigh
Explore the Science of Spring: A Live Media Event is an Innovations in Development project produced by the signature PBS series Nature. The new primetime series Spring LIVE (working title) will break the frame of a traditional documentary, letting viewers themselves explore the dramatic seasonal changes of spring through the immediacy of live television. On-camera hosts, scientists and naturalists in locations across the U.S., and scores of citizen scientists will use observation and scientific inquiry to explore the workings of nature during this season of rebirth. The unfolding stories of seasonal change will illuminate larger scientific insights--into the biodiversity of species in habitats, the interconnectedness of plants and animals in diverse ecosystems, the global phenomenon of species migration, and how spring "green-up" can be affected by environmental change--while inspiring appreciation for species conservation and habitat preservation. Spring LIVE is conceived as an ongoing series, with this inaugural season composed of three one-hour programs broadcast live on three consecutive nights, along with real-time interactions via Facebook. Reaching long-standing Nature viewers (2.5 million per episode), Spring LIVE will seek to turn mature adults and diverse families into citizen science doers, and leverage younger Nature online audiences through social media and community engagement in partnership with citizen science projects.
Spring LIVE will build public knowledge of and engagement in phenology and citizen science. The project will also conduct knowledge-building research on the effectiveness of Facebook as a science learning tool. It will experiment with eliciting audience participation via Facebook within the live shows to generate synchronous, second-screen thought and discussion. An exploratory study by Multimedia Research will look at the impact of this feature, addressing the question: To what extent and how does Facebook interactivity within live science shows impact adult engagement, learning and motivation? Spring LIVE will also engage multiple partners to expand reach and impact and build capacity in their fields. National partners include the National Park Service and Next Avenue; citizen science partners include Celebrate Urban Birds, National Phenology Network, Monarch Blitz, and SciStarter, among others. PBS stations will work with these organizations to involve diverse, intergenerational audiences in observation of nature and seasonal change. Project evaluation, implemented by Knight Williams Research Communications, will focus on the impact of live television on science learning, and the success of the integration of citizen science projects on air, online, and in communities. This project is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
Digital Observation Technology Skills (DOTS) is a framework for integrating modern, mobile technology into outdoor, experiential science education. DOTS addresses longstanding tensions between modern technology and classical outdoor education by carefully selecting appropriate digital technology for educational purposes and by situating these tools in classical experiential pedagogy.
DATE:
TEAM MEMBERS:
R. Justin HoughamMarc NutterCaitlin Graham
Glaciers around the world are undergoing dramatic changes. Many people, however, have a limited understanding of the scope of these changes because they are geographically distant and difficult to visualize. Although both digital learning tools and online scientific data repositories have greatly expanded over the last decade, there is currently no interface that brings the two together in a way that allows the public to explore these rapidly changing glacial environments. Therefore, to both improve public understanding and provide greater access to already existing resources, the project team will develop the Virtual Ice Explorer to encourage informal learning about glacial environments. This web application will feature an immersive virtual environment and display a suite of environmental data for an array of Earth's glacial systems. An interactive globe will allow users to select from a collection of sites ranging from polar regions to tropical latitudes. Each featured site will offer users an opportunity to interact with (1) a 3D rendering of the landscape; (2) a local map of the site; (3) historical and contemporary photographs of the site; (4) background information text describing the location, past research, and climate impacts; and (5) available environmental data. One of the most original features of the application will be its realistic, immersive 3D rendering of glacial landscapes by combining very high-resolution digital elevation models and satellite imagery with the application's built-in capabilities for immersive virtual environments. Although immersive environments often require expensive equipment, we are maximizing accessibility by developing the Virtual Ice Explorer to run in a web browser and function across various devices. Thus, the application will be available to anyone with internet access, and they can explore at their own pace.
As part of the successful development of Virtual Ice Explorer, the project team will create a platform for digital elevation models to be visualized and explored in 3D by users within the web application; curate digital elevation models, maps, images, text, and environmental data for inclusion in the web application for up to 11 geographically diverse glaciers/glacial landscapes; iteratively user-test the web application with project partners; and design the architecture of the system to readily scale to a larger collection of glaciers/glacial landscapes. To extend dissemination of the final products, the team has partnered with the U.S. Geologic Survey to showcase four benchmark glaciers in their long-term Glaciers and Climate project. In addition to improving understanding of glacier systems in informal learning environments, the project team will explore applications for spatial learning, employment of 3D environments for educational interventions, and use of Virtual Ice Explorer in formal learning environments. This project is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants. The project also has support from the Office of Polar Programs.
DATE:
-
TEAM MEMBERS:
Jason CervenecJesse FoxJulien Nicolas
Over the seven years prior to this award, the principal investigator from George Mason University and a national team of scientists, professional societies, science communication researchers, and broadcast meteorologists have been engaged in an effort to include in TV and other weather broadcasts information about current research on the interactions of climate and weather. A Climate Matters network has been established that involves 350 weathercasters at 218 stations, in 119 media markets, nationwide. A particular focus of the initiative has been to help the public become more familiar with the science behind how their local weather and its trends are related to the dynamics of the climate. Many communities nationwide are engaged in deliberations about how to understand, plan for, and adapt to the potential impacts of changes in their weather on important factors pertaining to their economy and well-being, such as natural resources, natural disasters, agriculture, industry, and health. The goal of this continuing project is to expand the quantity and nature of the coverage of such information into the news segments of local news media. By stimulating local reporting on climate impacts and their relationships to personal and community-wide decision-making, this project will potentially help millions of Americans better understand and respond to critical factors that are affecting their lives. This project is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants.
The project involves five inter-related, complementary activities: (1) Knowledge building through formative research and process evaluation, specifically in-depth interviews and random sample surveys of journalists in each of the participating journalism professional societies; (2) Recruiting 400 news directors, producers, reporters and additional weathercasters into the Climate Matters network; (3) Providing climate reporting training and professional development to members of the network; (4) Producing and distributing Climate Matters reporting packages to all members of the network on a near-weekly basis; and (5) Evaluating the impacts of the climate reporting on public understanding of science.
DATE:
-
TEAM MEMBERS:
Edward MaibachSusan HassolBernadette PlackyRichard CraigTeresa Myers
Polar Extremes: Enhancing Experiential Digital Learning is an integrated media and research project produced by the PBS science series, NOVA, that will bring polar science to informal learners through traditional storytelling and experiential, digital learning environments. Stark, cold, and seemingly frozen in time, the top and bottom of the Earth feel other-worldly, completely removed from our everyday existence. Yet, nothing could be further from the truth. The Arctic and Antarctic exert profound influence over our entire planet. Disturbances in these icy realms can send transformative ripples around the globe, altering the circulation of the atmosphere and oceans, and affecting every form of life. And although the poles might seem constant and everlasting, they--like our planet--are always changing, with a deep and complex past. NOVA will provide informal science learners access to specialized research happening at the ends of the earth, introducing them to today's scientists exploring the major drivers of the climate, uncovering the deep history of past paleoclimates, or perfecting climate and weather models. The project includes: a 2-hour nationally broadcast PBS documentary (working title Polar Extremes); a NOVA Polar Lab, an experiential interactive learning platform on polar science; and a Polar Exploration Initiative consisting of a 10-part YouTube series, a collection of 360 videos, virtual field trips, and social media reporting "on location" from Antarctica, along with other polar-themed video, radio and digital journalism. It also includes a research program conducted in collaboration with the University of California, Santa Barbara (UCSB) to study how narrative-driven and experiential learning can foster informal learning in polar science across a diverse array of audiences. NOVA, the most popular science program on television, with a robust digital presence, will bring current polar science to millions. NOVA will use a range of media to transport viewers to remote polar locations, to interact with polar scientists, manipulate polar data, or vicariously explore the frozen tundra--using a mix of learning approaches. This project will develop and test the impact of two forms of informal learning: traditional narrative-driven storytelling and active, experiential learning. Both components will be developed through audience research, formative evaluation or pilot testing, and experiments. The overarching goal is to determine the best way to combine and leverage traditional and interactive media technologies to educate the public about polar science. How can these modes enhance learning outcomes? The study uses the Informal Science Learning "strand framework" developed by the National Research Council in Learning Science in Informal Environments: People, Places, and Pursuits (2009). Because different age groups and socioeconomic backgrounds may engage differently with different types of learning materials and platforms, the project components are designed to test a variety of different learning approaches, with different audiences. This study will be one of the first to address the relative efficacy of various forms of experiential education and whether active versus vicarious experiential learning depends on the characteristics of the learners. As engagement technologies continue to evolve, this project will help inform how to best design and apply them effectively. The project will apply these new lessons specifically to present polar research to the public and to offer audiences an opportunity to explore and learn about these remote regions in new ways that bring them to life, make them relevant, and enhance learning outcomes. This project is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants. The project has co-funding support from the Office of Polar Programs (OPP).
DATE:
-
TEAM MEMBERS:
Paula ApsellLisa LeombruniJulia CortHunter Gehlbach
As part of its overall effort to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program, seeks to advance new approaches to, and evidence-based understanding of, the design and development of science, technology, engineering, and mathematics (STEM) learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants. In alignment with these aims, the STEM + Digital Literacies (STEM+L) project will investigate science fiction as an effective mechanism to attract and immerse adolescents (ages 10-13) from diverse cultural backgrounds in environmental and human health content and socio-scientific issues. This work is particularly novel, as the current knowledge base is limited, and largely addresses the high school level. Therefore, the results of the proposed effort could yield important findings regarding the feasibility of this activity as an effective platform for science learning and engagement for younger students. As such, STEM+L would not only advance knowledge in the field but would also contribute to a growing AISL portfolio on digital literacy and learning.
STEM+L is an early stage Innovations in Development project that will engage thirty middle school students in out of school time experiences. Over a twenty-four-week period, students will work collaboratively in groups in-person and online with their peers and field experts to design, develop, and produce STEM content rich, multimedia science fictions. The in-person learning experiences will take place on the University of Miami campus during the summer and academic year. Culminating activities include student presentations online and at a local Science Fiction Festival. The research component will employ an iterative, design-based approach. Four research questions will be explored: (a) How do students learn science concepts and multimodal digital literacies through participating in the STEM+L Academy? (b) How do students change their views in STEM related subject matter and in pursuing STEM related careers? (c) How do students participate in the STEM+L Academy? (d) How do we best support students' participation and learning of STEM+L in face-to-face and online environments? Data collection methods include video records, student-generated artifacts, online surveys, embedded assessments, interviews, and multimodal reflections. Comparative case analysis and a mixed methods approach will be employed. A rigorous evaluation will be conducted by a critical external review board. Inclusive and innovative dissemination strategies will ensure that the results of the research and program reach a broad range of audiences including both informal and formal STEM and literacy educators and researchers, learning scientists, local communities, and policy makers through national and international conference presentations, journal publications, Web2.0 resources, and community outreach activities.