This is a compilation of front-end, formative, and a partial summative evaluations, and an exploratory study using the xMacroscope, a data visualization technology developed for generating data from an exhibit using data captured from visitor actions.
Few people realize that the largest part of our planet’s biosphere remains virtually unexplored and unknown. This enormous habitat, accounting for an area of 116 million square miles or the equivalent size of roughly 30 times the area of the United States, is the abyssal zone of the deep ocean. The abyssal sea floor, at about 6000 ft., contains more than four times as much habitat for animal life as all of the dry mountains, forests, deserts, plains and jungles combined. Microscopic larvae in the deep ocean, are essential for the renewal and replenishment of life and they repopulate areas damaged by human activities such as mining and trawling, and they make marine protected areas both feasible and important. The National Science Foundation has funded intensive studies of oceanography related to larval recruitment for decades. However, findings from this large NSF investment of personnel, technology and funding have never been widely presented to the public. This project proposes to remedy this by developing a 40 minute giant screen film to be shown in science centers across the country, supported by virtual reality and augmented reality learning tools. The film will cover select deep ocean science expeditions using the deep-sea vehicles Alvin and ROV Jason. Content will include elements of the research process, activities related to the design and operation of deep-sea vehicles as well as interviews with scientists and technologists. The companion activities, Deep-Ocean Pilot (a VR-360° viewing station) and Plankton Quest (an AR biology treasure hunt) will extend the audience experience of the deep ocean out of the giant screen theater and into the surrounding museum environment. The website and social media will extend awareness and resources into homes. The project will be appropriate for a broad general audience, with particular appeal for the target audience of women and girls (ages 7-20). The larval biologist team is led by the PI at the University of Oregon, in collaboration with scientists from North Carolina State University, Western Washington University and the University of Rhode Island. Several young women scientists will be featured in the film providing role models. The production company, Stephen Low Productions, Inc. will use the latest technology on the Alvin and other cinematic tools to capture the visual images in the abyss. Collaborating museums will participate in the development and implementation of the Virtual and Augmented Reality learning tools as well as showing the film in their theaters.
Broader impact project goals include 1) Advancing public awareness of the abyssal ocean, the role of microscopic larvae, and what scientists are learning from expeditions that use deep submergence technologies; 2) Introducing public audiences and young women specifically to the wide range of STEM-related occupations encompassed in the field of ocean exploration and research; and 3) Advancing STEM learning research and practice in the area of immersive media in conveying STEM concepts and enhancing audience identification with STEM. Oregon State University’s STEM Research Center will build new knowledge by conducting formative and summative evaluation of the film and its associated support products (e.g., Virtual and augmented reality activities, website resources), addressing the following evaluation questions: 1)What do audiences take away from their experience in terms of fascination/interest, awareness and understanding related to ocean science exploration? 2) To what degree does the film alone or in combination with supplemental experiences trigger career awareness in girls and young women, and youth of racial/ethnic backgrounds? 3) To what degree do immersive experiences (a sense of “being there”) contribute to learning from the film? 4) How enduring are outcomes with audiences past the onsite immediate experience? Formative evaluation will be designed as ongoing improvement informed by empirical evidence in which evaluators work with team members to answer decision-relevant questions in a timely and project-focused way. The summative evaluation will be structured as an effectiveness study using mixed methods and ascertaining whether key programmatic outcomes have been reached and the degree to which particular program elements will have contributed to the results.
DATE:
-
TEAM MEMBERS:
Craig YoungAlexander LowStephen LowGeorge von DassowTrish Mace
Data is increasingly important in all aspects of people’s lives, from the day-to-day, to careers and to civic engagement. Preparing youth to use data to answer questions and solve problems empowers them to participate in society as informed citizens and opens doors to 21st century career opportunities. Ensuring equitable representation in data literacy and data science careers is critical. For many girls underrepresented in STEM, developing a "data science identity" requires personally meaningful experiences working with data. This project aims to promote middle school-aged girls’ interest and aspirations in data science through an identity-aligned, social game-based learning approach. The goals are to create a more diverse and inclusive generation of data scientists who see data as a resource and who are equipped with the skills and dispositions necessary to work with data in order to solve practical problems. The research team will run 10 social clubs and 10 data science clubs mentored by women in data science recruited through the University of Miami’s Institute for Data Science and Computing. Participants will be 250 middle school-aged girls recruited in Miami, FL, and Yolo County, CA, through local and national girls’ organizations. Youth will participate in a data science club and will learn key data science concepts and skills, including data structures, storage, exploration, analysis, and visualization. These concepts will be learned from working with their own data collected in personally meaningful ways in addition to working with data collected by others in the same social game eco-system. The project will also develop facilitator materials to allow adult volunteers to create game-based informal data science learning experiences for youth in their areas. The project is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments and is co-funded by the Innovative Technology Experiences for Students and Teachers (ITEST), which seeks to engage underrepresented students in technology-rich learning environments, including skills in data literacy, and increase students’ knowledge and interest in information and communication technology (ICT) careers.
Researchers will focus on two primary research questions: 1) Across gameplay and club experiences, in what ways do participants engage with data to pursue personal or social goals? 2) How do gameplay and club experiences shape girls’ perceptions of data, data science, and their fit with data and data science? The project will use design-based research methods to iteratively design the game and social club experiences. To ensure that uses of data feel personally and socially meaningful to young girls, the virtual world’s goals, narratives, and activities will be co-designed with girls from groups underrepresented in data science. The project will research engagement with game data in two informal, game-based learning scenarios: organic, self-directed, social play club, and structured, adult-facilitated data science clubs. The research will use a combination of quantitative and qualitative methods including surveys, focus groups, interviews, and gameplay and club observations. Project evaluation will determine how gameplay and club experiences impact participants' attitudes toward and interest in data-rich futures. The project holds the potential for broadening participation and promoting interest in data science by blending game-based learning with the rich social and adult mentoring through club participation. The results will be disseminated through conference presentations, scholarly publications, and social media. The game and facilitator materials will be designed for dissemination and made freely available to the public.
DATE:
-
TEAM MEMBERS:
Lisa HardyGary GoldbergerJennifer Kahn
Through the T523: Formative Evaluation for Educational Product Development course, our team conducted a semester-long formative evaluation for the Museum of Science, Boston (MoS) Gaia Exhibit. The Gaia Exhibit (Gaia) is a new, temporary art installation located in the MoS’s Blue Wing exhibition hall. Gaia that strives to inspire appreciation for the earth and climate change awareness. The exhibit displays imagery of the Earth’s surface on a twenty feet diameter, three-dimensional globe. Additional exhibit elements include projected questions on the floor to prompt reflection and exhibit-
This award is funded in whole or in part under the American Rescue Plan Act of 2021 (Public Law 117-2).
The Accessible Oceans study will design auditory displays that support learning and understanding of ocean data in informal learning environments like museums, science centers, and aquariums. Most data presentations in these spaces use visual display techniques such as graphs, charts, and computer-generated visualizations, resulting in inequitable access for learners with vision impairment or other print-related disabilities. While music, sound effects, and environmental sounds are sometimes used, these audio methods are inadequate for conveying quantitative information. The project will use sonification (turning data into sound) to convey meaningful aspects of ocean science data to increase access to ocean data and ocean literacy. The project will advance knowledge on the design of auditory displays for all learners, with and without disabilities, as well as advance the use of technology for STEM formal and informal education. The study will include 425 participants but will reach tens of thousands through the development of education materials, public reporting, and social media. The study will partner with the Smithsonian National Museum of Natural History, Woods Hole Oceanographic Institution Ocean Discovery Center, the Georgia Aquarium, the Eugene Science Center, the Atlanta Center for the Visually Impaired, and Perkins School for the Blind.
The project will leverage existing educational ocean datasets from the NSF-funded Ocean Observatories Initiative to produce and evaluate the feasibility of using integrated auditory displays to communicate tiered learning objectives of oceanographic principles. Integrated auditory displays will each be comprised of a data sonification and a context-setting audio introduction that will help to make sure all users start with the same basic information about the phenomenon. The displays will be developed through a user-centered design process that will engage ocean science experts, visually impaired students and adults (and their teachers), and design-oriented undergraduate and graduate students. The project will support advocacy skills for inclusive design and will provide valuable training opportunities for graduate and undergraduate students in human-centered design and accessibility. The project will have foundational utility in auditory display, STEM education, human-computer interaction, and other disciplines, contributing new strategies for representing quantitative information that can be applied across STEM disciplines that use similar visual data displays. The project will generate publicly accessible resources to advance studies of inclusive approaches on motivating learners with and without disabilities to learn more about and consider careers in STEM.
This Pilots and Feasibility Studies project is supported by the Advancing Informal STEM Learning program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants.
This NOVA multiplatform media initiative consisted of a 2-hour nationally broadcast PBS documentary, Polar Extremes; a 10-part original digital series, Antarctic Extremes; an interactive game, Polar Lab; accompanying polar-themed digital shorts, radio stories, text reporting, and social media content; a collection of educational resources on PBS LearningMedia; and community screening events and virtual field trips for science classrooms. Across multiple media platforms the project’s video content had nearly 13 million views.
The research explored the potential for informal STEM learning
There is a dearth of prominent STEM role models for underrepresented populations. For example, according to a 2017 survey, only 3.1% of physicists in the United States are Black, only 2.1% are Hispanic, and only 0.5% are Native American. The project will help bridge these gaps by developing exhibits that include simulations of historical scientific experiments enacted by little-known scientists of color, virtual reality encounters that immerse participants in the scientists' discovery process, and other content that allows visitors to interact with the exhibits and explore the exhibits' themes. The project will develop transportable, interactive exhibits focusing on light: how we perceive light, sources of light from light bulbs to stars, uses of real and artificial light in human endeavors, and past and current STEM innovators whose work helps us understand, create, and harness light now. The exhibits will be developed in three stages, each exploring a characteristic of light (Color, Energy, or Time). Each theme will be explored via multiple deliveries: short documentary and animated films, virtual reality experiences, interactive "photobooths," and technology-based inquiry activities. The exhibit components will be copied at seven additional sites, which will host the exhibits for their audiences, and the project's digital assets will enable other STEM learning organizations to duplicate the exhibits. The exhibits will be designed to address common gaps in understanding, among adults as well as younger learners, about light. What light really is and does, in scientific terms, is one type of hidden story these exhibits will convey to general audiences. Two other types of science stories the exhibits will tell: how contemporary research related to light, particularly in astrophysics, is unveiling the hidden stories of our universe; and hidden stories of STEM innovators, past and present, women and men, from diverse backgrounds. These stories will provide needed role models for the adolescent learners, helping them learn complex STEM content while showing them how scientific research is conducted and the diverse community of people who can contribute to STEM innovations and discoveries.
The project deliverables will be designed to present complex physics content through coherent, immersive, and embodied learning experiences that have been demonstrated to promote engagement and deeper learning. The project will research whether participants, through interacting with these exhibits, can begin to integrate discrete ideas and make connections with complex scientific content that would be difficult without technology support. For example, students and other novices often lack the expertise necessary to make distinctions between what is needed and what is extra within scientific problems. The proposed study follows a Design-Based Research (DBR) approach characterized by iterative cycles of data collection, analysis, and reflection to inform the design of educational innovations and advance educational theory. Project research includes conceiving, building, and testing iterative phases, which will enable the project to capture the complexity of learning and engagement in informal learning settings. Research participants will complete a range of research activities, including focus group interviews, observation, and pre-post assessment of science content knowledge and dispositions.
By showcasing such role models and informing about related STEM content, this project will widen perspectives of audiences in informal learning settings, particularly adolescents from groups underrepresented in STEM fields. Research findings and methodologies will be shared widely in the informal STEM learning community, building the field's knowledge of effective ways to broaden participation in informal science learning, and thus increase broaden participation in and preparation for the STEM-based workforce.
This project is funded by the National Science Foundation's (NSF's) Advancing Informal STEM Learning (AISL) program, which supports innovative research, approaches, and resources for use in a variety of learning settings.
DATE:
-
TEAM MEMBERS:
Todd BoyetteJill HammJanice AndersonCrystal Harden
Digital Observation Technology Skills (DOTS) is a framework for integrating modern, mobile technology into outdoor, experiential science education. DOTS addresses longstanding tensions between modern technology and classical outdoor education by carefully selecting appropriate digital technology for educational purposes and by situating these tools in classical experiential pedagogy.
DATE:
TEAM MEMBERS:
R. Justin HoughamMarc NutterCaitlin Graham
Charles Darwin is largely unknown and poorly understood as a historical figure. Similarly, the fundamental principles of evolution are often miss-stated, misunderstood, or entirely rejected by large numbers of Americans. Simply trying to communicate more facts about Darwin, or facts supporting the principles of evolution is inadequate; neither students nor members of the public will care or retain the information. On the contrary, building facts into a one-on-one conversational narrative creates an memorable opportunity to learn. Here, we create a digital-media, self-guided question and answer
DATE:
TEAM MEMBERS:
David J. LampeBrinley KantorskiJohn Pollock
In March of 2016, the Exploratorium transmitted a live webcast of a total solar eclipse from Woleai, a remote island in the southwestern Pacific. The webcast reached over 1 million viewers. Evaluation reveals effective use of digital media to engage learners in solar science and related STEM content.
Edu, Inc. conducted an external evaluation study that shows clear and consistent evidence of broad distribution of STEM content through multiple online channels, social media, pre-produced videos, and an app for mobile devices. IBM Watson did a deep analysis of tweets on eclipse topics that
A quasi-experimental separate-sample pretest/posttest study was implemented in Boston with adult viewers of the IMAX film "Dolphins." A majority of adults rated the film as 'very interesting,' reporting that they liked the film's cinematography, educational value and experiential quality. Viewing the film significantly increased adult viewers' knowledge about topics associated with dolphins. A second study focused on seventh grade students who viewed only the film or viewed the film and subsequently completed film-related activities in their school classroom setting. A quasi-experimental
A summative evaluation in Jersey City, New Jersey at the Liberty Science Center’s IMAX® Dome Theater was carried out with adult (18+ years) viewers. A quasi-experimental separate-sample pretest/posttest design was used. "Island of the Sharks" made a positive impact on sample adult audience member’s knowledge of topics presented in the film and was reportedly moderately or very interesting to 81% of the adult sample. A second summative evaluation involving three classes from each of two middle schools was carried out with seventh grade students. A quasi-experimental pretest/posttest