Identifying causal relationships is an important aspect of research and evaluation in visitor studies, such as making claims about the learning outcomes of a program or exhibit. Experimental and quasi-experimental approaches are powerful tools for addressing these causal questions. However, these designs are arguably underutilized in visitor studies. In this article, we offer examples of the use of experimental and quasi-experimental designs in science museums to aide investigators interested in expanding their methods toolkit and increasing their ability to make strong causal claims about
Although discussions of museums often revolve around exhibits, educators in these spaces have the potential to create in-depth, social learning experiences beyond what is possible at exhibits alone. There is still little empirical research, however, to inform how we understand, approach, and improve museum facilitation practices. In this study, we sought to address this gap by quantifying the impact of facilitation by trained educators working with visitors at interactive museum exhibits and comparing this to visitor engagement and learning outcomes for families without educator support. Using
As the world is increasingly dependent upon computing and computational processes associated with data analysis, it is essential to gain a better understanding of the visualization technologies that are used to make meaning of massive scientific data. It is also essential that the infrastructure, the very means by which technologies are developed for improving the public's engagement in science itself, be better understood. Thus, this AISL Innovations in Development project will address the critical need for the public to learn how to interpret and understand highly complex and visualized scientific data. The project will design, develop and study a new technology platform, xMacroscope, as a learning tool that will allow visitors at the Science Museum of Minnesota and the Center of Science and Industry, to create, view, understand, and interact with different data sets using diverse visualization types. The xMacroscope will support rapid research prototyping of public experiences at selected exhibits, such as collecting data on a runner's speed and height and the visualized representation of such data. The xMacroscope will provide research opportunities for exhibit designers, education researchers, and learning scientists to study diverse audiences at science centers in order to understand how learning about data through the xMacroscope tool may inform definitions of data literacy. The research will advance the state of the art in visualization technology, which will have broad implications for teaching and learning of scientific data in both informal and formal learning environments. The project will lead to better understanding by science centers on how to present data to the public more effectively through visualizations that are based upon massive amounts of data. Technology results and research findings will be disseminated broadly through professional publications and presentations at science, education, and technology conferences. The project is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants. The project is driven by the assumption that in the digital information age, being able to create and interpret data visualizations is an important literacy for the public. The research will seek to define, measure, and advance data visualization literacy. The project will engage the public in using the xMacrocope at the Science Museum of Minnesota and at the Center of Science and Industry's (COSI) science museum and research center in Columbus, Ohio. In both museum settings the public will interact with different datasets and diverse types of visualizations. Using the xMacroscope platform, personal attributes and capabilities will be measured and personalized data visualizations will be constructed. Existing theories of learning (constructivist and constructionist) will be extended to capture the learning and use of data visualization literacy. In addition, the project team will conduct a meta-review related to different types of literacy and will produce a definition with performance measures to assess data visualization literacy - currently broadly defined in the project as the ability to read, understand, and create data visualizations. The research has potential for significant impact in the field of science and technology education and education research on visual learning. It will further our understanding of the nature of data visualization literacy learning and define opportunities for visualizing data in ways that are both personally and culturally meaningful. The project expects to advance the understanding of the role of personalization in the learning process using iterative design-based research methodologies to advance both theory and practice in informal learning settings. An iterative design process will be applied for addressing the research questions by correlating visualizations to individual actions and contributions, exploring meaning-making studies of visualization construction, and testing the xMacroscope under various conditions of crowdedness and busyness in a museum context. The evaluation plan is based upon a logic model and the evaluation will iteratively inform the direction, process, and productivity of the project.
Researching the Value of Educator Actions for Learning (REVEAL) is an NSF-funded research project, led by the Oregon Museum of Science and Industry (OMSI) in collaboration with TERC and Oregon State University (OSU), which studied the impact of science center staff facilitation strategies at math exhibits. Building on and using exhibits from Design Zone—an interactive math exhibition which engaged visitors in exploring algebraic relationships and functions—the study was designed to add to the scant existing literature about the impact of staff facilitation on visitors. The study also aimed to
Mathematics is a notoriously disliked subject; there is so little stigma associated with being "bad at math", that educated adults openly describe themselves in this way. There are many reasons for math's unpopularity; chief among them is that school mathematics seldom offers opportunities to engage with the richness of this potentially fascinating subject. As a result, the mathematics education pipeline in the United States is more often a filter than a pump, siphoning students out rather than bringing them along. Children have libraries to help them fall in love with literature: where do they get a chance to fall in love with math? This project presents a unique opportunity to study children engaged with mathematics in an informal setting, the Minnesota State Fair, facilitated by mathematically knowledgeable volunteers. The Math On-a-Stick mathematical playground provides a place for children to engage with mathematics by exploring patters, asking quantitative questions, and investigating shape and space to mathematize their play. The project will observe and videotape this engagement to inform the design of mathematical learning environments in a variety of outside of school time settings, such as after school programs and summer camps, that are accessible to a wider range of the population. This project is co-funded by the EHR Core Research (ECR) and Advancing Informal STEM Learning (AISL) programs. The ECR program emphasizes fundamental STEM education research that generates foundational knowledge in three thematic areas: STEM learning and learning environments, broadening participation, and STEM workforce development. The AISL program seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments.
The project will investigate three research questions: (1) How does the design of various parts of the exhibit differently support rich mathematical interactions between children and mathematicians? (2) How do children engage different parts of the exhibit? How do differences in engagement relate to (a) exhibit design and (b) prior mathematical experience? (3) How do exhibit volunteers, mathematicians, and caregivers interact to support (or undermine) students' mathematical play? The project will use participant observation and videography to capture visitors' activities through the exhibit, analyzing them as qualitative case studies.
The REVEAL project is an NSF funded project to learn about how facilitation impacts family’s experiences of math exhibits. The goal of the project was to iteratively develop and refine a theoretical model of how staff facilitation deepens and extends family mathematical discourse at interactive exhibits. This model underwent rigorous testing and ultimately provided the evidence and research-based tools to support PD efforts for informal STEM educators. As part of this testing the learnings from developing at OMSI (Oregon Museum of Science and Industry, in Portland, Oregon) were applied in an
This poster was presented at the 2016 Advancing Informal STEM Learning (AISL) PI Meeting held in Bethesda, MD on February 29-March 2. REVEAL is using a two-phase research design, including a phase 1 design-based research study followed by a phase 2 experimental study, to investigate and measure the impact of staff facilitation on family mathematical discourse at exhibits and identify factors that influence the outcomes of these interactions. The project has an emphasis on culturally responsive research and education approaches.
In this article, science center and museum professionals from around the world share ways that they are engaging visitors in hands-on innovation. Work from the following organizations are discussed: Exploratorium, Discovery Center of Idaho, Lawrence Hall of Science, Iridescent, Conner Prairie Interactive History Park, Ideum, Discovery Place, Ontario Science Centre, Bootheel Youth Museum, Science Centre Singapore, Children's Museum of Phoenix, Discovery Museums (Acton, MA), Discovery Center of Springfield, Missouri, Museum of Science, Boston, Questacon--The National Science and Technology
This pathways project will study how audiences in public spaces, in this case those in a museum setting, relate to and make sense of large data displays. The project is preliminary to development of a traveling, hands-on exhibition enabling users to create and utilize representations of big data displays such as maps and charts. As the test case, the project will use science maps that provide an overview of science generally and specific areas of STEM, charting and exploring the history and future of science and technology. The data collection portion of the project will take place at the New York Hall of Science, the Marian Koshland Science Museum, COSI in Columbus, Ohio, and WonderLab Museum in Bloomington, Indiana. The project will create a foundation for the design of museum exhibits and educational programs that teach museum visitors how to explore, engage and make better sense of big data. The project is potentially transformative because big data is becoming ubiquitous and making sense out of large data displays is necessary in order to understand big data sets.
When it comes to STEM education, the nation’s K–12 public schools cannot do it all. The nature of 21st century proficiency in science, technology, engineering, and mathematics is too complex for any single institution. The good news is that schools do not have to do it alone. Museums, zoos, nature centers, aquariums, and planetariums are among the several thousand informal science institutions in the United States that regularly engage young people in observing, learning, and using STEM knowledge and skills. Providing a richness of resources unavailable in any classroom, informal science
DATE:
TEAM MEMBERS:
Community for Advancing Discovery Research in Education (CADRE)
Young people's participation in informal STEM learning activities can contribute to their academic and career achievements, but these connections are infrequently explicitly recognized or cultivated. More systemic approaches to STEM education could allow for students' experiences of formal and informal STEM learning to be aligned, coordinated, and supported across learning contexts. This Science Learning+ planning project brings together stakeholders in two digital badge systems--one in the US and one in the UK--to plan for a study to identify the specific structural features of the systems that may allow for the alignment of learning objectives across institutions. Digital badge systems may offer an inventive solution to the challenge of connecting and building on youth's STEM-related experiences in multiple learning contexts. When part of a defined system, badges could be used to represent and communicate evidence of individual learning, as well as provide youth and educators with evidence-supported indicators for other activities in the system that might be interesting or valuable. Properly designed and supported badge systems could transmit critical information within a network of informal STEM programs and schools that (1) recognize context-dependent, interest-driven learning and (2) provide opportunities to explore those interests across multiple settings. This project advances the field of informal STEM learning in two ways. First, the project documents and analyzes the processes by which two small groups of informal science education organizations and schools negotiate the meaning and value of badges, as proxies for learning objectives, and how they decide to recognize badges awarded by other institutions. This process builds capacity within the target systems while also beginning to identify the institutional, cultural, and material capacity issues that facilitate or constrain the alignment process. Second, the project conducts a pilot study with a small number of youth in the US and UK to investigate factors associated with an individual youth's likelihood of: a) identifying badges of interest; b) connecting the activities of various badge systems to each other and to non-badging institutions, such as school or industry; c) determining which badges to pursue; and d) persisting in a particular badge pathway. Findings from this pilot study will help identify institution- and individual-level factors that might be associated with advancing student interest and progression in STEM fields. Deepening and validating the understanding of those factors and their relative impact on student experiences and outcomes will be the focus of investigations in future studies.
DATE:
-
TEAM MEMBERS:
James DiamondNew York City Hive Learning NetworkMOUSEDigitalMeKatherine McMillan
Based on the number of visitors annually, zoos and aquariums are among the most popular venues for informal STEM learning in the United States and the United Kingdom. Most research into the impacts of informal STEM learning experiences at zoos and aquariums has focused on short-term changes in knowledge, attitudes and behaviors. This Science Learning+ project will identify the opportunities for and barriers to researching the long-term impacts of informal STEM learning experiences at zoos and aquariums. The project will address the following overarching research question: What are and how do we measure the long-term impacts of an informal STEM learning experience at a zoo and aquarium? While previous research has documented notable results, understanding the long-term impacts of zoo and aquarium learning experiences will provide a deeper and more nuanced understanding of the impact of these programs on STEM knowledge, skills and application. This study will use a participatory process to identify: (1) the range of potential long-term impacts of informal science learning experiences at zoos and aquariums; (2) particular activities that foster these impacts; and (3) opportunities for and barriers to measuring those impacts. First, an in-depth literature review will document previous research efforts to date within the zoo and aquarium community. Second, a series of consultative workshops (both in-person and online) will gather ideas and input from practitioners, researchers, and other stakeholders in zoo and aquarium education. The consultative workshops will focus on two questions in particular: (1) What are the different types and characteristics of informal science learning experiences that take place at zoos and aquariums? and (2) What are the long-term impacts zoos and aquariums are aiming to have on visitors in relation to knowledge, attitudes, skills and behaviors/actions? Finally, visitor surveys at zoos and aquariums in the US and UK will be conducted to gather input on what visitors believe are the long-term impacts of an informal STEM learning opportunity at a zoo or aquarium. The data gathered through all of these activities will inform the design of a five-year, mixed-methods study to investigate long-term impacts and associated indicators of an informal STEM learning experience at a zoo or aquarium. One of the aims of the five-year study will be to test instruments that could eventually be used by the global zoo and aquarium community to measure the long-term impacts of informal STEM learning programs. Designing tools to better understand the long-term impacts of informal STEM learning at zoos and aquariums will contribute to our ability to measure STEM learning outcomes. Additional benefits include improved science literacy and STEM skills amongst visitors over time and an understanding of how education programs contribute to wildlife conservation worldwide.
DATE:
-
TEAM MEMBERS:
Brian JohnsonStanford UniversityLancaster UniversitySarah ThomasNicole ArdoinMurray Saunders