This document is the final evaluation report for the project, which focuses both on formative evaluation of the collaborative+interdisciplinary presentation creation process and summative evaluation of audience learning outcomes.
This project will research and develop the Circuit, a mobile phone and web-based application that will empower families and the general public to discover the broad spectrum of informal Science, Technology, Engineering and Mathematics (STEM) opportunities that exist in most communities. These informal STEM resources include science and children's museums, science and computer camps, maker spaces, afterschool programs, citizen science and much more. There is currently no "one-stop" searching for these resources. Instead, participants must conduct multiple, inefficient Internet searches to find the sought for STEM resources. The Circuit will enable users to efficiently search a rich informal STEM database, identifying resources by location, geography, age levels, science discipline, type of program and other factors. The Circuit builds on SciStarter, an existing online platform that connects thousands of prospective and active citizen scientists to citizen science projects. SciStarter has made possible the collection and organization of several thousand citizen science projects that would otherwise be scattered across the web. The Circuit will build on SciStarter's technical achievements in the citizen science sector, while systematically encompassing the offerings of established national networks. By integrating existing networks of informal STEM resources, the app will afford the public with unrivaled access to informal STEM opportunities, while collecting data that reveals patterns of engagement towards understanding factors of influence between different types of STEM experiences.
The app will provide researchers with new opportunities for researching how families and adults participate in the ecosystem of informal STEM resources in their communities. The Circuit will develop web tools to aggregate and organize digital content from trusted, currently siloed, informal STEM networks of content providers. These include science festivals, science and children's museums, the American Association for the Advancement of Science (AAAS), and Discover Magazine (3 million readers), the largest general interest science publication. Each content partner will feed the app with information directly or through their membership and encourage adoption of The Circuit within their respective communities. The project will design digital tools, including APIs (application program interfaces) to acquire and share digital content, embeddable tools to record and analyze data about movement, engagement, and persistence across domains, and social media tools and related APIs to distribute, track, and analyze content, engagement and demographics. (An API is a code that allows two software programs to communicate with each other.) The project will conduct small-scale, proof-of-conduct studies, to test the viability of the platform to support future, independent full-scale research. An analytics dashboard will be designed and tested with partners, researchers, and evaluators to ensure access to data on patterns of visits, clicks, referrals, searches, "joins," bookmarks, shares, contributions, user-locations, persistence, and more, within and across domains. Because each partner will feed their analytics into the shared dashboard, this will provide unprecedented and much-needed data to advance research in informal STEM learning. The Circuit will allow the tracking of patterns of engagement across networks and programs. Anonymized analytics of behavioral data from end users of The Circuit will support new approaches to advance evidence-based understanding of connected informal STEM learning by exhibiting engagement patterns across informal STEM domains. Through volunteer participation by the public, the Circuit will explore the geographic and demographic patterns of participants in the system, and derive important design lessons for its own and future efforts to create curated systems of connected learning across STEM education in informal settings.
This project is funded by the National Science Foundation's (NSF's) Advancing Informal STEM Learning (AISL) program, which supports innovative research, approaches, and resources for use in a variety of learning settings.
Ruff Family Science is an exploratory project funded by the National Science Foundation (NSF) that aims to foster joint media engagement and hands-on science exploration among diverse, low-income parents and their 4- to 8-year-old children. Building on the success of the PBS series FETCH! with Ruff Ruffman, the project leverages FETCH’s funny and charismatic animated host, along with its proven approach to teaching science, to inspire educationally disadvantaged families to explore science together. More specifically, the project is undertaking a research and design process to create prototype
The Crowd & The Cloud, a three-year project, developed by Passport To Knowledge and funded by the National Science Foundation, uses multimedia to engage different audiences around citizen science and crowdsourcing. The project team created four episodes of a broadcast television series, which appeared on PBS stations and via PBS.org, an interactive website, and a robust social media presence in an attempt to reach three target audiences: the general public, scientists, and citizen scientists. Rockman et al (REA), an independent educational research and evaluation firm, conducted an external
Worldwide, four million people participate in geocaching--a game of discovering hidden treasures with GPS-enabled devices (including smart phones). Geocachers span all ages and tend to be interested in technology and the outdoors. To share information about the Montana Climate Assessment (MCA), an NSF-funded scientific report, Montana State University created a custom trackable geocaching coin featuring the MCA Website and logo. We then recruited volunteers to hide one coin in each of Montana’s 56 counties. Volunteer geocachers enthusiastically adopted all 56 counties, wrote blogs and social media posts about the coins, and engaged local Scout troops and schools. Other geocachers then found and circulated the coins while learning about Montana’s climate. One coin has traveled nearly 4,000 miles; several have visited other states and Canada. 95% of the volunteers said the project made them feel more connected to university research, and they told an average of seven other people about the project. Nearly all of the participants were unfamiliar with the Montana Climate Assessment prior to participating. The geocaching educational outreach project included several partnerships, including with Geocaching Headquarters in Seattle (a.k.a. “Groundspeak”); Cache Advance, Inc., an environmentally friendly outdoor gear company; and Gallatin Valley Geocachers. An advisory board of geocachers helped launch the project.
DATE:
TEAM MEMBERS:
Suzi TaylorRay CallawayM.J. NehasilCathy Whitlock
Charles Darwin is largely unknown and poorly understood as a historical figure. Similarly, the fundamental principles of evolution are often miss-stated, misunderstood, or entirely rejected by large numbers of Americans. Simply trying to communicate more facts about Darwin, or facts supporting the principles of evolution is inadequate; neither students nor members of the public will care or retain the information. On the contrary, building facts into a one-on-one conversational narrative creates an memorable opportunity to learn. Here, we create a digital-media, self-guided question and answer
DATE:
TEAM MEMBERS:
David J. LampeBrinley KantorskiJohn Pollock
In March of 2016, the Exploratorium transmitted a live webcast of a total solar eclipse from Woleai, a remote island in the southwestern Pacific. The webcast reached over 1 million viewers. Evaluation reveals effective use of digital media to engage learners in solar science and related STEM content.
Edu, Inc. conducted an external evaluation study that shows clear and consistent evidence of broad distribution of STEM content through multiple online channels, social media, pre-produced videos, and an app for mobile devices. IBM Watson did a deep analysis of tweets on eclipse topics that
This study explored how different presentations of an object in deep space affect understanding, engagement, and aesthetic appreciation. A total of n = 2,502 respondents to an online survey were randomly assigned to one of 11 versions of Cassiopeia A, comprising 6 images and 5 videos ranging from 3s to approximately 1min. Participants responded to intial items regarding what the image looked like, the aesthetic appeal of the image, perceptions of understanding, and how much the participant wanted to learn more. After the image was identified, participants indicated the extent to which the
DATE:
TEAM MEMBERS:
Lisa SmithKimberly ArcandRandall SmithJay BookbinderJeffrey Smith
As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants. The goal of this project is to make 21st century quantum science comprehensible and engaging to non-expert informal adult learners. This project has strong potential to add new knowledge about the public's perception and understanding of quantum physics. This scientific content is often difficult for informal audiences to grasp, and there are relatively few accessible learning resources for a non- professional audience. The development of this online, interactive resource with short animations, graphics, and simulations has strong potential to fill this gap. It will develop a visually driven online resource to engage non-expert audiences in understanding the basics of quantum physics. The web design will be modular, incorporating many multimedia elements and the structure will be flexible allowing for future expansion. All content would be freely available for educational use. There is potential for extensive reach and use of the resources by informal adult learners online as well as learners in museums, science centers, and schools. Project partners are the Joint Quantum Institute at the University of Maryland and the National Institute of Standards and Technology, College Park. An independent evaluation of the project will add new knowledge about informal learners' perceptions and/or knowledge about quantum science and technology. An initial needs assessment via focus groups with the general public will be designed to find out more about what they already know about quantum physics topics and terminology, as well as what they want to know and what formats they prefer (games, simulations, podcasts, etc.). In person user testing will be used with early versions of the project online resource using a structured think-aloud protocol. Later in year 1 and 2, online focus groups with the general public will be conducted to learn what they find engaging and what they learned from the content. Iterative feedback from participants during the formative stage will guide the development of the content and format of the online resources. The Summative Evaluation will gather data using a retrospective post-survey embedded with a pop-up link on the Atlas followed by interviews with a subset of online users. Google Analytics will be used to determine the breadth and depth of their online navigation, what resources they download, and what websites they visit afterward. A post-only survey of undergraduate and graduate students who participated in resource development will focus on changes in students' confidence around their science communication skills and level of quantum physics understanding.
Given the importance of learning to economic and life success, this review seeks to broaden the conception of learning beyond traditional formal education. Learning occurs every day in many ways and in a range of settings. This broad scope of learning--termed "informal learning"--is increasingly important in the rapidly changing knowledge economy. As such, in this review paper, we examine the different types of informal learning, their opportunities and challenges, and their issues of access and equity. Spanning multiple disciplines, e draw particular attention to the workplace and adult
DATE:
TEAM MEMBERS:
Michelle Van NoyHeather JamesCrystal Bedley