Recent advances in multimodal learning analytics show significant promise for addressing these challenges by combining multi-channel data streams from fully-instrumented exhibit spaces with multimodal machine learning techniques to model patterns in visitor experience data. We describe initial work on the creation of a multimodal learning analytics framework for investigating visitor engagement with a game-based interactive surface exhibit for science museums called Future Worlds.
DATE:
TEAM MEMBERS:
Jonathan RoweWookhee MinSeung LeeBradford MottJames Lester
resourceresearchMuseum and Science Center Exhibits
Multimodal models often utilize video data to capture learner behavior, but video cameras are not always feasible, or even desirable, to use in museums. To address this issue while still harnessing the predictive capacities of multimodal models, we investigate adversarial discriminative domain adaptation for generating modality-invariant representations of both unimodal and multimodal data captured from museum visitors as they engage with interactive science museum exhibits.
DATE:
TEAM MEMBERS:
Nathan HendersonWookhee MinAndrew EmersonJonathan RoweSeung LeeJames MinogueJames Lester
resourceresearchMuseum and Science Center Exhibits
Recent years have seen a growing interest in investigating visitor engagement in science museums with multimodal learning analytics. Visitor engagement is a multidimensional process that unfolds temporally over the course of a museum visit. In this paper, we introduce a multimodal trajectory analysis framework for modeling visitor engagement with an interactive science exhibit for environmental sustainability.
DATE:
TEAM MEMBERS:
Andrew EmersonNathan HendersonWookhee MinJonathan RoweJames MinogueJames Lester
resourceresearchMuseum and Science Center Exhibits
In this paper, we introduce a Bayesian hierarchical modeling framework for predicting learner engagement with Future Worlds, a tabletop science exhibit for environmental sustainability.
DATE:
TEAM MEMBERS:
Andrew EmersonNathan HendersonJonathan RoweWookhee MinSeung LeeJames MinogueJames Lester
resourceevaluationMuseum and Science Center Exhibits
This document presents the final evaluation report for the NSF-funded AISL project: "Multimodal Visitor Analytics: Investigating Naturalistic Engagement with Interactive Tabletop Science Exhibits."
In this paper, we investigate bias detection and mitigation techniques to address issues of
algorithmic fairness in multimodal models of museum visitor visual attention.
DATE:
TEAM MEMBERS:
Halim AcostaNathan HendersonJonathan RoweWookhee MinJames MinogueJames Lester
resourceresearchMuseum and Science Center Exhibits
This NOVA multiplatform media initiative consisted of a 2-hour nationally broadcast PBS documentary, Polar Extremes; a 10-part original digital series, Antarctic Extremes; an interactive game, Polar Lab; accompanying polar-themed digital shorts, radio stories, text reporting, and social media content; a collection of educational resources on PBS LearningMedia; and community screening events and virtual field trips for science classrooms. Across multiple media platforms the project’s video content had nearly 13 million views.
The research explored the potential for informal STEM learning
Glaciers around the world are undergoing dramatic changes. Many people, however, have a limited understanding of the scope of these changes because they are geographically distant and difficult to visualize. Although both digital learning tools and online scientific data repositories have greatly expanded over the last decade, there is currently no interface that brings the two together in a way that allows the public to explore these rapidly changing glacial environments. Therefore, to both improve public understanding and provide greater access to already existing resources, the project team will develop the Virtual Ice Explorer to encourage informal learning about glacial environments. This web application will feature an immersive virtual environment and display a suite of environmental data for an array of Earth's glacial systems. An interactive globe will allow users to select from a collection of sites ranging from polar regions to tropical latitudes. Each featured site will offer users an opportunity to interact with (1) a 3D rendering of the landscape; (2) a local map of the site; (3) historical and contemporary photographs of the site; (4) background information text describing the location, past research, and climate impacts; and (5) available environmental data. One of the most original features of the application will be its realistic, immersive 3D rendering of glacial landscapes by combining very high-resolution digital elevation models and satellite imagery with the application's built-in capabilities for immersive virtual environments. Although immersive environments often require expensive equipment, we are maximizing accessibility by developing the Virtual Ice Explorer to run in a web browser and function across various devices. Thus, the application will be available to anyone with internet access, and they can explore at their own pace.
As part of the successful development of Virtual Ice Explorer, the project team will create a platform for digital elevation models to be visualized and explored in 3D by users within the web application; curate digital elevation models, maps, images, text, and environmental data for inclusion in the web application for up to 11 geographically diverse glaciers/glacial landscapes; iteratively user-test the web application with project partners; and design the architecture of the system to readily scale to a larger collection of glaciers/glacial landscapes. To extend dissemination of the final products, the team has partnered with the U.S. Geologic Survey to showcase four benchmark glaciers in their long-term Glaciers and Climate project. In addition to improving understanding of glacier systems in informal learning environments, the project team will explore applications for spatial learning, employment of 3D environments for educational interventions, and use of Virtual Ice Explorer in formal learning environments. This project is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants. The project also has support from the Office of Polar Programs.
DATE:
-
TEAM MEMBERS:
Jason CervenecJesse FoxJulien Nicolas
Polar Extremes: Enhancing Experiential Digital Learning is an integrated media and research project produced by the PBS science series, NOVA, that will bring polar science to informal learners through traditional storytelling and experiential, digital learning environments. Stark, cold, and seemingly frozen in time, the top and bottom of the Earth feel other-worldly, completely removed from our everyday existence. Yet, nothing could be further from the truth. The Arctic and Antarctic exert profound influence over our entire planet. Disturbances in these icy realms can send transformative ripples around the globe, altering the circulation of the atmosphere and oceans, and affecting every form of life. And although the poles might seem constant and everlasting, they--like our planet--are always changing, with a deep and complex past. NOVA will provide informal science learners access to specialized research happening at the ends of the earth, introducing them to today's scientists exploring the major drivers of the climate, uncovering the deep history of past paleoclimates, or perfecting climate and weather models. The project includes: a 2-hour nationally broadcast PBS documentary (working title Polar Extremes); a NOVA Polar Lab, an experiential interactive learning platform on polar science; and a Polar Exploration Initiative consisting of a 10-part YouTube series, a collection of 360 videos, virtual field trips, and social media reporting "on location" from Antarctica, along with other polar-themed video, radio and digital journalism. It also includes a research program conducted in collaboration with the University of California, Santa Barbara (UCSB) to study how narrative-driven and experiential learning can foster informal learning in polar science across a diverse array of audiences. NOVA, the most popular science program on television, with a robust digital presence, will bring current polar science to millions. NOVA will use a range of media to transport viewers to remote polar locations, to interact with polar scientists, manipulate polar data, or vicariously explore the frozen tundra--using a mix of learning approaches. This project will develop and test the impact of two forms of informal learning: traditional narrative-driven storytelling and active, experiential learning. Both components will be developed through audience research, formative evaluation or pilot testing, and experiments. The overarching goal is to determine the best way to combine and leverage traditional and interactive media technologies to educate the public about polar science. How can these modes enhance learning outcomes? The study uses the Informal Science Learning "strand framework" developed by the National Research Council in Learning Science in Informal Environments: People, Places, and Pursuits (2009). Because different age groups and socioeconomic backgrounds may engage differently with different types of learning materials and platforms, the project components are designed to test a variety of different learning approaches, with different audiences. This study will be one of the first to address the relative efficacy of various forms of experiential education and whether active versus vicarious experiential learning depends on the characteristics of the learners. As engagement technologies continue to evolve, this project will help inform how to best design and apply them effectively. The project will apply these new lessons specifically to present polar research to the public and to offer audiences an opportunity to explore and learn about these remote regions in new ways that bring them to life, make them relevant, and enhance learning outcomes. This project is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants. The project has co-funding support from the Office of Polar Programs (OPP).
DATE:
-
TEAM MEMBERS:
Paula ApsellLisa LeombruniJulia CortHunter Gehlbach
In this project, education researchers, environmental scientists, and educators will develop a computer tool to let STEM educators and curriculum developers build local environmental science models. The system will use data about land use to automatically construct map-based simulations of any area in the United States. Users will be able to choose from a range of environmental and economic issues to include in these models. The system will create simulations that ask students to change to patterns of land use -- for example, increasing land zoned for housing, or open land, or industrial development -- to try to meet environmental and social goals. As a result, students will be able to learn about the interaction of environmental and economic issues relevant to their own city, town, neighborhood, or region. These map-based simulations will be incorporated into an existing science, technology, engineering, and mathematics (STEM) education tool, Land Science, in which learners work in a fictional planning office to study how zoning affects economic and environmental issues in a community. Research has shown that Land Science is mode effective when learners are exploring issues in an area near their home, and the current study will investigate how and why local simulations improve environmental science learning. This project is funded by the Advancing Informal STEM Learning (AISL) program which supports work to enhance learning in informal environments by funding innovative research, approaches, and resources for use in a variety of settings.
In this project, the research team will build, test, and deploy a toolkit that will allow informal STEM educators and developers of informal STEM programming to easily adapt an existing environmental science learning environment, which consists of a place-based virtual internship in urban planning and ecology, to their local contexts, learning objectives, and learner populations. Land Science is a virtual internship in which young people explore the environmental and socio-economic impacts of land-use decisions. To do so, they play the role of interns at an urban planning firm developing a new land-use proposal for the city of Lowell, Massachusetts: they read reports, virtually visit sites, determine stakeholder priorities, and use a geographic information system (GIS) model to evaluate the socio-economic and environmental impacts of land-use choices. No one plan can satisfy all stakeholders, so learners must compromise to create an effective plan and justify their decisions. Land Science has been shown to improve civic engagement, interest in eco-social issues, and understanding of scientific models, but it is most effective when the location of the virtual internship is in or near the learners' home town. To improve the accessibility and impact of this effective learning intervention, the interdisciplinary research team, which includes learning scientists, land-use experts, and informal STEM educators, will develop a Local Environmental Modeling toolkit, which will allow educators to change the location of the simulation and the stakeholder groups, zoning codes, and environmental and socio-economic indicators included in the land-use model. The system will ensure that the model produced is functional, realistic, and appropriately complex. The localized versions of Land Science produced by informal STEM educators will be used in a range of contexts and locations, allowing the research team to study the effects of an online, place-based learning intervention on environmental science learning, STEM interest and motivation, and civic engagement.
DATE:
-
TEAM MEMBERS:
David ShafferKristen ScopinichHolly GibbsJeffrey Linderoth