Engagement is the cornerstone of learning in informal science education. During free-choice learning in museums and science centers, visitor engagement shapes how learners interact with exhibits, navigate through exhibit spaces, and form attitudes, interests, and understanding of science. Recent advances in multimodal learning analytics are creating novel opportunities for expanding the range and richness of measures of visitor engagement in free-choice settings. In particular, multimodal learning analytics offer significant potential for integrating multiple data sources to devise a composite picture of visitors' cognitive, affective, and behavioral engagement. The project will center on providing a rich empirical account of meaningful visitor engagement with interactive tabletop science exhibits among individual visitors and small groups, as well as uncovering broader tidal patterns in visitor engagement that unfold across exhibit spaces. A key objective of the project is creating models and practitioner-focused learning analytic tools that will inform the best practices of exhibit designers and museum educators. This project is funded by the Advancing Informal STEM Learning (AISL) program. As part of its overall strategy to enhance learning in informal environments, AISL funds research and innovative approaches and resources for use in a variety of settings. The research team will conduct data-rich investigations of visitors' learning experiences with multimodal learning analytics that fuse the rich multichannel data streams produced by fully-instrumented exhibit spaces with the data-driven modeling functionalities afforded by recent advances in machine learning and educational data mining. The research team will conduct a series of visitor studies of naturalistic engagement in solo, dyad, and group interactions as visitors explore interactive tabletop science exhibits. The studies will utilize eye trackers to capture visitors' moment-to-moment attention, facial expression analysis and quantitative field observations to track visitors' emotional states, trace logs generated by exhibit software, as well as motion-tracking sensors and coded video recordings to capture visitors' behavioral interactions. The studies will also use conversation recordings and pre-post assessment measures to capture visitors' science understanding and inquiry processes. With these multimodal data streams as training data, the research team will use probabilistic and neural machine learning techniques to devise learning analytic models of visitor engagement. The project will be conducted by a partnership between North Carolina State University and the North Carolina Museum of Natural Sciences. The research team will 1) design a data-rich multimodal visitor study methodology, 2) create the Visitor Informatics Platform, a suite of open source software tools for multimodal visitor analytics, and 3) launch the Multimodal Visitor Data Warehouse, a curated visitor experience data archive. Together, the multimodal visitor study methodology, the Visitor Informatics Platform, and the Multimodal Visitor Data Warehouse will enable researchers and practitioners in the informal science education community to utilize multimodal learning analytics in their own informal learning environments. It is anticipated that the project will advance the field of informal STEM learning by extending and enriching measures of meaningful visitor engagement, expanding the evidence base for visitor experience design principles, and providing learning analytic tools to support museum educators. By enhancing understanding of the cognitive, affective, and behavioral dynamics underlying visitor experiences in science museums, informal science educators will be well-positioned to design learning experiences that are more effective and engaging. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
DATE:
-
TEAM MEMBERS:
James LesterJames MinogueJonathan RoweNorth Carolina Museum of Natural Sciences
One way to encourage youth to pursue training in the STEM fields and enter the STEM workforce is to foster interest and engagement in STEM during adolescence. Informal STEM Learning Sites (ISLS) provide opportunities for building interest and engagement in the STEM fields through a multitude of avenues, including the programming that they provide for youth, particularly teens. Frequently, ISLS provide opportunities to participate in volunteer programs, internships or work, which allow teens both to learn relevant STEM knowledge as well as to share that knowledge with others through opportunities to serve as youth educators. While youth educator programs provide rich contexts for teens to engage as both learners and teachers in these informal STEM environments, research to date has not yet identified the relationship between serving as youth educators and STEM engagement. Thus, the goal of this project is to document the impact of youth educators on visitor learning in ISLS and to identify best practices for implementing youth educator programs. The project studies STEM interests and engagement in the youth participants and the visitors that they interact with at six different ISLS in the US and UK. This project is funded through Science Learning+, which is an international partnership between the National Science Foundation (NSF) and the Wellcome Trust with the UK Economic and Social Research Council. The goal of this joint funding effort is to make transformational steps toward improving the knowledge base and practices of informal STEM experiences. Within NSF, Science Learning+ is part of the Advancing Informal STEM Learning (AISL) program that seeks to enhance learning in informal environments and to broaden access to and engagement in STEM learning experiences.
This project examines youth educator experiences related to STEM identity, educational aspirations, and motivation. The project also identifies outcomes that the youth educators have on visitors to ISLS in terms of knowledge, interest, and engagement in STEM. The specific aims are: 1) Outcomes for Teens - To measure the longitudinal impact of participation in an extended youth educator experience in an ISLS; 2) Outcomes for Visitors - To compare visitor engagement with and learning from exhibits in ISLS when they interact with a youth educator, relative to outcomes of interacting with an adult educator or no educator; and 3) Outcomes Across Demographics and STEM Sites - To examine differences in visitor engagement based on participant characteristics such as socio-economic status (SES), age, gender, and ethnicity and to compare outcomes of youth educator experiences across different types of ISLS. This research, which draws on expectancy value theory and social cognitive theory, will follow youth participants longitudinally over the course of 5 years and use latent variable analyses to understand the impact on the youth educators as well as the visitors with whom they interact. Importantly, the results of this research will be used to develop best practices for implementing youth educator programs in ISLS and the results will be disseminated to both academic and practice-based communities.
This project has clear and measurable broader impacts in a variety of ways. First, the project provides guidance to improve programming for youth in ISLS, including both the sites involved directly in the research and to the larger community of ISLS through evaluation, development, and dissemination of best practices. Additionally, this project provides rigorous, research-based evidence to identify and describe the outcomes of youth educator programs. This study directly benefits the participants of the research, both the visiting public and the youth educators, through opportunities to engage with science. The findings speak to issues of access and inclusivity in ISLS, providing insight into how to design environments that are welcoming and accessible for diverse groups of learners. Finally, this project provides evidence for best practices for ISLS in developing programs for youth that will lead to interest in and pursuit of STEM careers by members of underrepresented groups.
As the world is increasingly dependent upon computing and computational processes associated with data analysis, it is essential to gain a better understanding of the visualization technologies that are used to make meaning of massive scientific data. It is also essential that the infrastructure, the very means by which technologies are developed for improving the public's engagement in science itself, be better understood. Thus, this AISL Innovations in Development project will address the critical need for the public to learn how to interpret and understand highly complex and visualized scientific data. The project will design, develop and study a new technology platform, xMacroscope, as a learning tool that will allow visitors at the Science Museum of Minnesota and the Center of Science and Industry, to create, view, understand, and interact with different data sets using diverse visualization types. The xMacroscope will support rapid research prototyping of public experiences at selected exhibits, such as collecting data on a runner's speed and height and the visualized representation of such data. The xMacroscope will provide research opportunities for exhibit designers, education researchers, and learning scientists to study diverse audiences at science centers in order to understand how learning about data through the xMacroscope tool may inform definitions of data literacy. The research will advance the state of the art in visualization technology, which will have broad implications for teaching and learning of scientific data in both informal and formal learning environments. The project will lead to better understanding by science centers on how to present data to the public more effectively through visualizations that are based upon massive amounts of data. Technology results and research findings will be disseminated broadly through professional publications and presentations at science, education, and technology conferences. The project is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants. The project is driven by the assumption that in the digital information age, being able to create and interpret data visualizations is an important literacy for the public. The research will seek to define, measure, and advance data visualization literacy. The project will engage the public in using the xMacrocope at the Science Museum of Minnesota and at the Center of Science and Industry's (COSI) science museum and research center in Columbus, Ohio. In both museum settings the public will interact with different datasets and diverse types of visualizations. Using the xMacroscope platform, personal attributes and capabilities will be measured and personalized data visualizations will be constructed. Existing theories of learning (constructivist and constructionist) will be extended to capture the learning and use of data visualization literacy. In addition, the project team will conduct a meta-review related to different types of literacy and will produce a definition with performance measures to assess data visualization literacy - currently broadly defined in the project as the ability to read, understand, and create data visualizations. The research has potential for significant impact in the field of science and technology education and education research on visual learning. It will further our understanding of the nature of data visualization literacy learning and define opportunities for visualizing data in ways that are both personally and culturally meaningful. The project expects to advance the understanding of the role of personalization in the learning process using iterative design-based research methodologies to advance both theory and practice in informal learning settings. An iterative design process will be applied for addressing the research questions by correlating visualizations to individual actions and contributions, exploring meaning-making studies of visualization construction, and testing the xMacroscope under various conditions of crowdedness and busyness in a museum context. The evaluation plan is based upon a logic model and the evaluation will iteratively inform the direction, process, and productivity of the project.
Co-led by the University of Washington and Science Gallery Dublin, this project aims to drive and transform the next generation of broadening participation efforts targeting teen-aged youth from communities historically underrepresented in STEM fields. This project investigates how out-of-school time (OST) programs that integrate epistemic practices of the arts, sciences, computer science, and other disciplines, in the context of consequential activities (such as creating radio segments, designing museum exhibitions, or building online games), can more broadly appeal to and engage youth who do not already identify as STEM learners. STEM-related skills and capacities (such as computational thinking, design, data visualizations, and digital storytelling) are key to productive and creative participation in many future civic and workplace activities, and are driving the 30 fastest-growing occupations in the US. But many new jobs will entail a hybrid blend of skills, such as programming and design skills that many students who have disengaged with academic STEM pathways may already have and would be eager to develop further. There is not currently a strong foundation of research-based evidence to guide the design, implementation, and evaluation transdisciplinary programs - in which STEM skills are embedded as tools for meaningful participation - or how such approaches relate to long-term outcomes. Hypothesizing that OST programs which effectively engage youth during their high-leverage teenage years can significantly impact youths' longer-term STEM learning trajectories, this project will involve: 1) Five 3-year studies documenting learning in different technology-rich contexts: Making Afterschool, Media Production, Museum Exhibition Design, Digital Arts Programs, and Pop-Up/Street Science Programs; 2) A 4-year longitudinal study, involving 100 youth from the above programs; 3) The creation of a number of practical measurement tools that can be used to monitor how programs are leveraging the intersections of the arts and sciences to support student engagement and learning; and 4) A Professional Development program conducted at informal science education conferences in the EU and US to engage the informal STEM field with emerging findings. This project is funded through Science Learning+, which is an international partnership between the National Science Foundation (NSF) and the Wellcome Trust with the UK Economic and Social Research Council. The goal of this joint funding effort is to make transformational steps toward improving the knowledge base and practices of informal STEM experiences to better understand, strengthen, and coordinate STEM engagement and learning. Within NSF, Science Learning+ is part of the Advancing Informal STEM Learning (AISL) program that seeks to enhance learning in informal environments.
Transdisciplinary, equity-oriented OST programs can provide supportive social contexts in which STEM concepts and practices are taken up as the means for meaningful participation in valued activities, building students' STEM skills in ways that can propel their future academic, career, and lifelong learning choices. This project will build the knowledge base about these emerging 21st century transdisciplinary approaches to broadening participation investigating: 1) The epistemic intersections across a range of disciplines (art, science, computation, design) that operate to broaden appeal and meaningful participation for underrepresented youth; 2) How transdisciplinary activities undertaken in the context of consequential learning (e.g., producing a radio segment, designing an exhibition for the general public) can illuminate the relevance of STEM to young people's lives, concerns, and futures; and 3) How participation in such programs can propel students' longer-term life choices and STEM learning trajectories. The project is a collaboration of the University of Washington, Science Gallery Dublin, Indiana University, Youth Radio in Oakland California, Guerilla Science in New York and London, and the London School of Economics.
One way to encourage youth to pursue training in the STEM fields and enter the STEM workforce is to foster interest and engagement in STEM during adolescence. Informal STEM Learning Sites (ISLS) provide opportunities for building interest and engagement in the STEM fields through a multitude of avenues, including the programming that they provide for youth, particularly teens. Frequently, ISLS provide opportunities to participate in volunteer programs, internships or work, which allow teens both to learn relevant STEM knowledge as well as to share that knowledge with others through opportunities to serve as youth educators. While youth educator programs provide rich contexts for teens to engage as both learners and teachers in these informal STEM environments, research to date has not yet identified the relationship between serving as youth educators and STEM engagement. Thus, the goal of this project is to document the impact of youth educators on visitor learning in ISLS and to identify best practices for implementing youth educator programs. The project studies STEM interests and engagement in the youth participants and the visitors that they interact with at six different ISLS in the US and UK. This project is funded through Science Learning+, which is an international partnership between the National Science Foundation (NSF) and the Wellcome Trust with the UK Economic and Social Research Council. The goal of this joint funding effort is to make transformational steps toward improving the knowledge base and practices of informal STEM experiences. Within NSF, Science Learning+ is part of the Advancing Informal STEM Learning (AISL) program that seeks to enhance learning in informal environments and to broaden access to and engagement in STEM learning experiences.
This project examines youth educator experiences related to STEM identity, educational aspirations, and motivation. The project also identifies outcomes that the youth educators have on visitors to ISLS in terms of knowledge, interest, and engagement in STEM. The specific aims are: 1) Outcomes for Teens - To measure the longitudinal impact of participation in an extended youth educator experience in an ISLS; 2) Outcomes for Visitors - To compare visitor engagement with and learning from exhibits in ISLS when they interact with a youth educator, relative to outcomes of interacting with an adult educator or no educator; and 3) Outcomes Across Demographics and STEM Sites - To examine differences in visitor engagement based on participant characteristics such as socio-economic status (SES), age, gender, and ethnicity and to compare outcomes of youth educator experiences across different types of ISLS. This research, which draws on expectancy value theory and social cognitive theory, will follow youth participants longitudinally over the course of 5 years and use latent variable analyses to understand the impact on the youth educators as well as the visitors with whom they interact. Importantly, the results of this research will be used to develop best practices for implementing youth educator programs in ISLS and the results will be disseminated to both academic and practice-based communities.
This project has clear and measurable broader impacts in a variety of ways. First, the project provides guidance to improve programming for youth in ISLS, including both the sites involved directly in the research and to the larger community of ISLS through evaluation, development, and dissemination of best practices. Additionally, this project provides rigorous, research-based evidence to identify and describe the outcomes of youth educator programs. This study directly benefits the participants of the research, both the visiting public and the youth educators, through opportunities to engage with science. The findings speak to issues of access and inclusivity in ISLS, providing insight into how to design environments that are welcoming and accessible for diverse groups of learners. Finally, this project provides evidence for best practices for ISLS in developing programs for youth that will lead to interest in and pursuit of STEM careers by members of underrepresented groups.
DATE:
-
TEAM MEMBERS:
Adam Hartstone-RoseMatthew IrvinKelly Lynn MulveyElizabeth ClemensLauren ShenfeldAdam RutlandMark WinterbottomFrances BalkwillPeter McOwanKatie ChambersStephanie TylerLisa Stallard
Informal learning, and by extension, museums, are inherently emotional experiences, evoking feelings of awe, excitement, and curiosity. Oftentimes, museum professionals have prioritized traditionally positive emotions such as excitement and interest as being the most desirable and useful in supporting museum learning. However, prior research into naturally occurring emotions at museums found that some visitors who experienced negative emotions, such as confusion or frustration, at exhibits also reported deeper engagement and overall feelings of satisfaction (Rappolt-Schlichtmann et al., 2017). Based on these findings and similar results from formal education (D’Mello et al., 2014), this project team has worked to develop and refine a framework of strategies for creating exhibits that invoke and support visitors through the complex emotional state called productive struggle (PS) which is defined as a three-part emotional arc characterized by: 1) disequilibrium (experienced emotionally as emotions like confusion, frustration, surprise, or unease) that arises from encountering a challenging task, phenomenon, or idea, 2) persistence through the task which is supported by exhibit design scaffolds, and 3) an emotionally productive resolution tied to the source of disequilibrium or an overall sense of effortful achievement. In deliberately attending to and supporting a range of negative and positive emotions in museums, visitors can gain access to a wider variety of complex emotional experiences, including those critical to STEM learning, and have potential to broaden participation in STEM by supporting learners' diverse emotional needs and preferences.
This multidisciplinary project team consisting of researchers and exhibit professionals utilized a design-based research (DBR) process to develop, test, and refine a definition of museum-based productive struggle, and create a framework of design strategies that support PS. Three physical exhibits and a virtual exhibit were created during this multi-year project. Additionally, a summative research study was conducted with 105 youth ages 10-17 to explore: 1) whether visitors experienced the expected emotional arc of PS; 2) how exhibit design strategies supported PS; and 3) the extent of visitors’ learning and engagement at these PS exhibits.
This project had three objectives to build knowledge with respect to advancing Informal STEM Education:
Plan, prototype, fabricate, and document a game-linked design-and-play STEM exhibit for multi-generational adult-child interaction utilizing an iterative exhibit design approach based on research and best practices in the field;
Develop and disseminate resources and models for collaborative play-based exhibits to the informal STEM learning community of practice of small and mid-size museums including an interactive, tangible tabletop design-and-play game and a related tablet-based game app for skateboarding science and technology design practice;
Conduct research on linkages between adult-child interactions and game-connected play with models in informal STEM learning environments.
Linked to these objectives were three project goals:
Develop tools to enable children ages 5-8 to collaboratively refine and test their own theories about motion by exploring fundamental science concepts in linked game and physical-object design challenge which integrates science (Newton’s Laws of Motion) with engineering (iterative design and testing), technology (computational models), and mathematics (predictions and comparisons of speed, distance, and height). [Linked to Objectives 1 & 3]
Advance the informal STEM education field’s understanding of design frameworks that integrate game environments and physical exhibit elements using tangibles and playful computational modeling and build upon the “Dimensions of Success” established STEM evaluation models. [Linked to Objectives 1 & 2]
Examine methods to strengthen collaborative learning within diverse families through opportunities to engage in STEM problem-based inquiry and examine how advance training for parents influences the extent of STEM content in conversations and the quality of interactions between caregivers and children in the museum setting. [Linked to Objectives 1 & 3]
The exhibit designed and created as a result of this grant project integrates skateboarding and STEM in an engaging context for youth ages 5 to 8 to learn about Newton’s Laws of Motion and connect traditionally underserved youth from rural and minority areas through comprehensive outreach. The exhibit design process drew upon research in the learning sciences and game design, science inquiry and exhibit design, and child development scholarship on engagement and interaction in adult-child dyads.
Overall, the project "Understanding Physics through Collaborative Design and Play: Integrating Skateboarding with STEM in a Digital and Physical Game-Based Children’s Museum Exhibit" accomplished three primary goals. First, we planned, prototyped, fabricated, and evaluated a game-linked design-and-play STEM gallery presented as a skatepark with related exhibits for adult-child interaction in a Children's Museum.
Second, we engaged in a range of community outreach and engagement activities for children traditionally underserved in Museums. We developed and disseminated resources for children to learn about the physics of the skatepark exhibit without visiting the Museum physically. For example, balance board activities were made portable, the skatepark video game was produced in app and web access formats, and ramps were created from block sets brought to off-site locations.
Third, we conducted a range of research to better understand adult-child interactions in the skatepark exhibit in the Children's Museum and to explore learning of physics concepts during physical and digital play. Our research findings collectively provide a new model for Children's Museum exhibit developers and the informal STEM education community to intentionally design, evaluate, and revise exhibit set-up, materials, and outcomes using a tool called "Dimensions of Success (DOS) for Children's Museum Exhibits." Research also produced a tool for monitoring the movement of children and families in Museum exhibit space, including time on task with exhibits, group constellation, transition time, and time in gallery. Several studies about adult-child interactions during digital STEM and traditional pretend play in the Museum produced findings about social positioning, interaction style, role, and affect during play.
DATE:
-
TEAM MEMBERS:
Deb DunkhaseKristen MissallBenjamin DeVane
Life on the Edge will be a 1,500-sq-ft traveling exhibition to engage museum guests with space, space exploration, and the search for life beyond our home planet through the lens of Earth's extremophiles. The exhibition will explore life forms in extreme, harsh environments on Earth, and how studying these creatures informs the search for extraterrestrial life and habitable environments within and beyond our solar system. This exhibition will provide open-ended challenges and hands-on activities that utilize NASA research and educational materials to inspire elementary-aged youth ages 5-11 and their families. Based in Ithaca, NY, Sciencenter will focus the tour on small, rural museums, including SpectrUM Discovery Area (Missoula, MT), Flathead Reservation (MT), Science Zone (Casper, WY), and Imagination Place Children's Museum (Gadsden, AL). Schools and other community partners of the host museums will be leveraged in presenting family science nights, field trips, and facilitated science programs. These activities will provide additional opportunities for learners to increase their knowledge of core STEM content and science-process skills related to astronomy and astrobiology. In this institutional engagement project, Sciencenter will partner with (1) Cornell University's Department of Astronomy throughout the 5-year grant period to develop the scientific content and to ensure that content remains current and relevant with up-to-date NASA research, and (2) the University of Montana's SpectrUM Discovery Area, who will serve as the STEM outreach hub for the region, including outreach with youth of the Flathead Reservation. The expected short-term outcomes for youth ages 5-11, after visiting Life on the Edge are (1) 75% of participants will have increased understanding of basic principles of astrobiology and astronomy, along with the breadth of NASA scientific research and missions, and (2) 50% of participants will have increased awareness of career possibilities in STEM, specifically astrobiology, astronomy, and related space sciences.
DATE:
-
TEAM MEMBERS:
Michelle KortenaarAlexander HayesLisa KalteneggerHolly TruittAdrienne TestaCharlie Trautmann