This is a compilation of front-end, formative, and a partial summative evaluations, and an exploratory study using the xMacroscope, a data visualization technology developed for generating data from an exhibit using data captured from visitor actions.
As new technologies continue to dominate the world, access to and participation in science, technology, engineering, mathematics (STEM), and computing has become a critical focus of education research, practice, and policy. This issue is exceptionally relevant for American Indians, who remain underrepresented as only 0.2% of the STEM workforce, even though they make up 2% of the U.S. population. In response to this need, this Faculty Early Career Development Program (CAREER) project takes a community-driven design approach, a collaborative design process in which Indigenous partners maintain sovereignty as designers, to collaboratively create three place-based storytelling experiences, stories told in historical and cultural places through location-based media. The place-based storytelling experiences will be digital installations at three culturally, politically, and historically significant sites in the local community where the public can engage with Indigenous science. The work is being done in partnership with the Northwestern Band of the Shoshone Nation (NWBSN).
The principal investigator and the NWBSN will investigate: (a) what are effective strategies and processes to conduct community-driven design with Indigenous partners?; (b) how does designing place-based storytelling experiences develop tribal members' design, technical, and computational skills?; (c) how does designing these experiences impact tribal members' scientific, technological, and cultural identities? The goals are to establish a process of community-driven design, build infrastructure to support this process, and understand how this methodological approach can result in culturally-appropriate ways to engage with science through technology. The principal investigator will work with the tribe to complete three intergenerational design cycles (a design cycle is made up of multiple design iterations). Each design cycle will result in one place-based storytelling experience. The goal is to include roughly 15 youth (ages 6-18), 10 Elders, and 10 other community members (i.e. members ages 18-50, likely parents) in each design cycle (35 tribal members total). Some designers are likely to participate in multiple design cycles. The tribe currently has 48 youth ages 6-18 and the project aims to engage at least 30 across all three design cycles. Over four years of designing three different experiences, the NWBSN aims to recruit at least 100 tribal members (just under 20% of the tribe) to make contributions (as designers, storytellers, or to provide cultural artifacts or design feedback).
This CAREER award is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
There is a dearth of prominent STEM role models for underrepresented populations. For example, according to a 2017 survey, only 3.1% of physicists in the United States are Black, only 2.1% are Hispanic, and only 0.5% are Native American. The project will help bridge these gaps by developing exhibits that include simulations of historical scientific experiments enacted by little-known scientists of color, virtual reality encounters that immerse participants in the scientists' discovery process, and other content that allows visitors to interact with the exhibits and explore the exhibits' themes. The project will develop transportable, interactive exhibits focusing on light: how we perceive light, sources of light from light bulbs to stars, uses of real and artificial light in human endeavors, and past and current STEM innovators whose work helps us understand, create, and harness light now. The exhibits will be developed in three stages, each exploring a characteristic of light (Color, Energy, or Time). Each theme will be explored via multiple deliveries: short documentary and animated films, virtual reality experiences, interactive "photobooths," and technology-based inquiry activities. The exhibit components will be copied at seven additional sites, which will host the exhibits for their audiences, and the project's digital assets will enable other STEM learning organizations to duplicate the exhibits. The exhibits will be designed to address common gaps in understanding, among adults as well as younger learners, about light. What light really is and does, in scientific terms, is one type of hidden story these exhibits will convey to general audiences. Two other types of science stories the exhibits will tell: how contemporary research related to light, particularly in astrophysics, is unveiling the hidden stories of our universe; and hidden stories of STEM innovators, past and present, women and men, from diverse backgrounds. These stories will provide needed role models for the adolescent learners, helping them learn complex STEM content while showing them how scientific research is conducted and the diverse community of people who can contribute to STEM innovations and discoveries.
The project deliverables will be designed to present complex physics content through coherent, immersive, and embodied learning experiences that have been demonstrated to promote engagement and deeper learning. The project will research whether participants, through interacting with these exhibits, can begin to integrate discrete ideas and make connections with complex scientific content that would be difficult without technology support. For example, students and other novices often lack the expertise necessary to make distinctions between what is needed and what is extra within scientific problems. The proposed study follows a Design-Based Research (DBR) approach characterized by iterative cycles of data collection, analysis, and reflection to inform the design of educational innovations and advance educational theory. Project research includes conceiving, building, and testing iterative phases, which will enable the project to capture the complexity of learning and engagement in informal learning settings. Research participants will complete a range of research activities, including focus group interviews, observation, and pre-post assessment of science content knowledge and dispositions.
By showcasing such role models and informing about related STEM content, this project will widen perspectives of audiences in informal learning settings, particularly adolescents from groups underrepresented in STEM fields. Research findings and methodologies will be shared widely in the informal STEM learning community, building the field's knowledge of effective ways to broaden participation in informal science learning, and thus increase broaden participation in and preparation for the STEM-based workforce.
This project is funded by the National Science Foundation's (NSF's) Advancing Informal STEM Learning (AISL) program, which supports innovative research, approaches, and resources for use in a variety of learning settings.
DATE:
-
TEAM MEMBERS:
Todd BoyetteJill HammJanice AndersonCrystal Harden
Digital Observation Technology Skills (DOTS) is a framework for integrating modern, mobile technology into outdoor, experiential science education. DOTS addresses longstanding tensions between modern technology and classical outdoor education by carefully selecting appropriate digital technology for educational purposes and by situating these tools in classical experiential pedagogy.
DATE:
TEAM MEMBERS:
R. Justin HoughamMarc NutterCaitlin Graham
In March of 2016, the Exploratorium transmitted a live webcast of a total solar eclipse from Woleai, a remote island in the southwestern Pacific. The webcast reached over 1 million viewers. Evaluation reveals effective use of digital media to engage learners in solar science and related STEM content.
Edu, Inc. conducted an external evaluation study that shows clear and consistent evidence of broad distribution of STEM content through multiple online channels, social media, pre-produced videos, and an app for mobile devices. IBM Watson did a deep analysis of tweets on eclipse topics that
In this chapter, we explore making as a learning process in the context of a museum-based maker space designed for family participation. In particular, we focus on young children, and their adult learning partners, as an important demographic to consider and for which to design making environments and experiences. Importantly, we take a close look at the evolving role of museum educators in supporting young children's meaningful participation in making as an informal learning process. Through the presentation of a single case of a child's making in the museum, we identify key factors that
This is an Early-concept Grant for Exploratory Research supporting research in Smart and Connected Communities. The research supported by the award is collaborative with research at the University of Colorado. The researchers are studying the use of technologies to enable communities to connect youth and youth organizations to effectively support diverse learning pathways for all students. These communities, the youth, the youth organizations, formal and informal education organizations, and civic organizations form a learning ecology. The DePaul University researchers will design and implement a smart community infrastructure in the City of Chicago to track real-time student participation in community STEM activities and to develop mobile applications for both students and adults. The smart community infrastructure will bring together information from a variety of sources that affect students' participation in community activities. These include geographic information (e.g., where the student lives, where the activities take place, the student transportation options, the school the student attends), student related information (e.g., the education and experience background of the student, the economic status of the student, students' schedules), and activity information (e.g., location of activity, requirements for participation). The University of Colorado researchers will take the lead on analyzing these data in terms of a community learning ecologies framework and will explore computational approaches (i.e., recommender systems, visualizations of learning opportunities) to improve youth exploration and uptake of interests and programs. These smart technologies are then used to reduce the friction in the learning connection infrastructure (called L3 for informal, formal, and virtual learning) to enable the student to access opportunities for participation in STEM activities that are most feasible and most appropriate for the student. Such a flexible computational approach is needed to support the necessary diversity of potential recommendations: new interests for youth to explore; specific programs based on interests, friends' activities, or geographic accessibility; or programs needed to "level-up" (develop deeper skills) and complete skills to enhance youths' learning portfolios. Although this information was always available, it was never integrated so it could be used to serve the community of both learners and the providers and to provide measurable student learning and participation outcomes. The learning ecologies theoretical framework and supporting computational methods are a contribution to the state of the art in studying afterschool learning opportunities. While the concept of learning ecologies is not new, to date, no one has offered such a systematic and theoretically-grounded portfolio of measures for characterizing the health and resilience of STEM learning ecologies at multiple scales. The theoretical frameworks and concepts draw together multiple research and application domains: computer science, sociology of education, complexity science, and urban planning. The L3 Connects infrastructure itself represents an unprecedented opportunities for conducting "living lab" experiments to improve stakeholder experience of linking providers to a single network and linking youth to more expanded and varied opportunities. The University of Colorado team will employ three methods: mapping, modeling, and linking youth to STEM learning opportunities in school and out of school settings in a large urban city (Chicago). The recommender system will be embedded into youth and parent facing mobile apps, enabling the team to characterize the degree to which content-based, collaborative filtering, or constraint based recommendations influence youth actions. The project will result in two measurable outcomes of importance to key L3 stakeholder groups: a 10% increase in the number of providers (programs that are part of the infrastructure) in target neighborhoods and a 20% increase in the number of youth participating in programs.
This is an Early-concept Grant for Exploratory Research supporting research in Smart and Connected Communities. The research supported by the award is collaborative with research at DePaul University. The researchers are studying the use of technologies to enable communities to connect youth and youth organizations to effectively support diverse learning pathways for all students. These communities, the youth, the youth organizations, formal and informal education organizations, and civic organizations form a learning ecology. The DePaul University researchers will design and implement a smart community infrastructure in the City of Chicago to track real-time student participation in community STEM activities and to develop mobile applications for both students and adults. The smart community infrastructure will bring together information from a variety of sources that affect students' participation in community activities. These include geographic information (e.g., where the student lives, where the activities take place, the student transportation options, the school the student attends), student related information (e.g., the education and experience background of the student, the economic status of the student, students' schedules), and activity information (e.g., location of activity, requirements for participation). The University of Colorado researchers will take the lead on analyzing these data in terms of a community learning ecologies framework and will explore computational approaches (i.e., recommender systems, visualizations of learning opportunities) to improve youth exploration and uptake of interests and programs. These smart technologies are then used to reduce the friction in the learning connection infrastructure (called L3 for informal, formal, and virtual learning) to enable the student to access opportunities for participation in STEM activities that are most feasible and most appropriate for the student. Such a flexible computational approach is needed to support the necessary diversity of potential recommendations: new interests for youth to explore; specific programs based on interests, friends' activities, or geographic accessibility; or programs needed to "level-up" (develop deeper skills) and complete skills to enhance youths' learning portfolios. Although this information was always available, it was never integrated so it could be used to serve the community of both learners and the providers and to provide measurable student learning and participation outcomes. The learning ecologies theoretical framework and supporting computational methods are a contribution to the state of the art in studying afterschool learning opportunities. While the concept of learning ecologies is not new, to date, no one has offered such a systematic and theoretically-grounded portfolio of measures for characterizing the health and resilience of STEM learning ecologies at multiple scales. The theoretical frameworks and concepts draw together multiple research and application domains: computer science, sociology of education, complexity science, and urban planning. The L3 Connects infrastructure itself represents an unprecedented opportunities for conducting "living lab" experiments to improve stakeholder experience of linking providers to a single network and linking youth to more expanded and varied opportunities. The University of Colorado team will employ three methods: mapping, modeling, and linking youth to STEM learning opportunities in school and out of school settings in a large urban city (Chicago). The recommender system will be embedded into youth and parent facing mobile apps, enabling the team to characterize the degree to which content-based, collaborative filtering, or constraint based recommendations influence youth actions. The project will result in two measurable outcomes of importance to key L3 stakeholder groups: a 10% increase in the number of providers (programs that are part of the infrastructure) in target neighborhoods and a 20% increase in the number of youth participating in programs.
The National Science Festival Network project, also operating as the Science Festival Alliance, is designed to create a sustainable national network of science festivals that engages all facets of the general public in science learning. Science Festivals, clearly distinct from "science fairs", are community-wide activities engaging professional scientists and informal and K-12 educators targeting underrepresented segments of local communities historically underserved by formal or informal STEM educational activities. The initiative builds on previous work in other parts of the world (e.g. Europe, Australasia) and on recent efforts in the U.S. to create science festivals. The target audiences are families, children and youth ages 5-18, adults, professional scientists and educators in K-12 and informal science institutions, and underserved and underrepresented communities. Project partners include the MIT Museum in Cambridge, UC San Diego, UC San Francisco, and the Franklin Institute in Philadelphia. The deliverables include annual science festivals in these four cities supported by year-round related activities for K-12 and informal audiences, a partnership network, a web portal, and two national conferences. Ten science festivals will be convened in total over the 3 years of the project, each reaching 15,000 to 60,000 participants per year. STEM content includes earth and space science, oceanography, biological/biomedical science, bioinformatics, and computer, behavioral, aeronautical, nanotechnology, environmental, and nuclear science. An independent evaluator will systematically assess audience participation and perceptions, level/types of science interest stimulated in target groups, growth of partnering support at individual sites, and increasing interactions between ISE and formal K-12 education. A variety of qualitative and quantitative assessments will be designed and utilized. The project has the potential to transform public communication and understanding of science and increase the numbers of youth interested in pursuing science.
DATE:
-
TEAM MEMBERS:
Loren ThompsonJeremy BabendureBen Wiehe
This evaluation reports on the Mission: Solar System project, a 2-year project funded by NASA. The goal of the Mission: Solar System was to create a collection of resources that integrates digital media with hands-on science and engineering activities to support kids’ exploration in formal and informal education settings. Our goal in creating the resources were: For youth: (1) Provide opportunities to use science, technology, engineering, and math to solve challenges related to exploring our solar system, (2) Build and hone critical thinking, problem-solving, and design process skills, (3)
President Obama announced in April 2013 that the Corporation for National and Community Service (CNCS) would launch a STEM AmeriCorps initiative to build student interest in STEM. A RFA is currently being prepared to be released in the late fall of 2013. This project will engage in quick response research to identify an evaluation and research agenda that can begin to inform the program launch. Thus, the timeframe for informing the initial stages of STEM AmeriCorps is relatively short, and the creation of an evaluation and research agenda is very timely. The products from the RAPID proposal are: (1) a review of the evaluation and research literature on the use of volunteers and/or mentors to build students' interest in STEM; (2) to convene a workshop to identify evaluation and research priorities to guide the initiative; and (3) a summary evaluation agenda that identifies promising directions along with the strength of evidence around key issues.
The purposes of the STUDIO 3D evaluation were to collect information about the impact upon student learning as a result of participating in the STUDIO 3D Project, as well as to elicit information for program improvement. Areas of inquiry include recruiting and retention, impact on project participants, tracking student impacts, and the project as a whole.