This document is the final evaluation report for the project, which focuses both on formative evaluation of the collaborative+interdisciplinary presentation creation process and summative evaluation of audience learning outcomes.
As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants. This project would expand the informal STEM learning field's understanding of how to use digital science media to increase STEM educational experiences and opportunities for English language learners. Across the U.S. there are significant STEM opportunity and achievement gaps for English learners with varying levels of English proficiency. This is at a time when the U.S. is facing a shortage of STEM professionals in the workforce including the life and physical science fields. This project aims to close these gaps and improve English learners' STEM learning outcomes using digital media. Within community colleges, there are multiple site-based programs to provide content to help English learners to learn English and to improve their math and literacy skills. Involving the state community college networks is a critical strategy for gathering important feedback for the pedagogical approach as well as for recruiting English learner research participants. The team will initially study an existing YouTube chemistry series produced by Complexly then produce and test new videos in Spanish using culturally relevant instructional strategies. The target audience is 18-34-year-old English learners. Project partners are Complexly, a producer of digital STEM media and EDC, a research organization with experience in studying informal STEM learning.
The project has the potential to advance knowledge about the use of culturally relevant media to improve STEM opportunities and success for English language learners. Using a Design-Based Implementation Research framework the research questions include: 1) what are the effective production and instructional strategies for creating digital media to teach science to English learners whose native language is Spanish? 2) what science content knowledge do English learners gain when the project's approach is applied to a widely available set of YouTube videos? and 3) how might the findings from the research be applied to future efforts targeting English learners? The project has the potential to significantly broaden participation in science and engineering. Phase 1 of the research will be an exploration of how to apply strategic pedagogical approaches to digital media content development. Interviews will be conducted with educators in 3 focal states with high numbers of English language learners (NY, CA, TX) to reflect on pedagogical foundations for teaching science to English learners. A survey of 30 English learners will provide feedback on the perceived strengths and weaknesses of a selection of existing YouTube chemistry videos. Phase 2 will create/test prototypes of 6 adapted chemistry videos. Forty students (ages 18-34) will be recruited and participate in cognitive interviews with researchers after viewing these videos. Based on this input additional videos will be produced with revised instructional strategies for further testing. Additional rounds of production and testing will be conducted to develop an English learners mini chemistry series. Phase 3 will be a pilot study to gauge the science learning of 75 English learners who will view an 11-episode chemistry miniseries. It will also identify gaps in expected learning to determine whether any further adjustments are necessary to the instructional approach.
This Innovations in Development award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
DATE:
-
TEAM MEMBERS:
Kelsey SavageCeridwen RileyStan MullerHeather LavigneCaroline ParkerKatrina Bledsoe
Digital Observation Technology Skills (DOTS) is a framework for integrating modern, mobile technology into outdoor, experiential science education. DOTS addresses longstanding tensions between modern technology and classical outdoor education by carefully selecting appropriate digital technology for educational purposes and by situating these tools in classical experiential pedagogy.
DATE:
TEAM MEMBERS:
R. Justin HoughamMarc NutterCaitlin Graham
Given the importance of learning to economic and life success, this review seeks to broaden the conception of learning beyond traditional formal education. Learning occurs every day in many ways and in a range of settings. This broad scope of learning--termed "informal learning"--is increasingly important in the rapidly changing knowledge economy. As such, in this review paper, we examine the different types of informal learning, their opportunities and challenges, and their issues of access and equity. Spanning multiple disciplines, e draw particular attention to the workplace and adult
DATE:
TEAM MEMBERS:
Michelle Van NoyHeather JamesCrystal Bedley
This is an Early-concept Grant for Exploratory Research supporting research in Smart and Connected Communities. The research supported by the award is collaborative with research at the University of Colorado. The researchers are studying the use of technologies to enable communities to connect youth and youth organizations to effectively support diverse learning pathways for all students. These communities, the youth, the youth organizations, formal and informal education organizations, and civic organizations form a learning ecology. The DePaul University researchers will design and implement a smart community infrastructure in the City of Chicago to track real-time student participation in community STEM activities and to develop mobile applications for both students and adults. The smart community infrastructure will bring together information from a variety of sources that affect students' participation in community activities. These include geographic information (e.g., where the student lives, where the activities take place, the student transportation options, the school the student attends), student related information (e.g., the education and experience background of the student, the economic status of the student, students' schedules), and activity information (e.g., location of activity, requirements for participation). The University of Colorado researchers will take the lead on analyzing these data in terms of a community learning ecologies framework and will explore computational approaches (i.e., recommender systems, visualizations of learning opportunities) to improve youth exploration and uptake of interests and programs. These smart technologies are then used to reduce the friction in the learning connection infrastructure (called L3 for informal, formal, and virtual learning) to enable the student to access opportunities for participation in STEM activities that are most feasible and most appropriate for the student. Such a flexible computational approach is needed to support the necessary diversity of potential recommendations: new interests for youth to explore; specific programs based on interests, friends' activities, or geographic accessibility; or programs needed to "level-up" (develop deeper skills) and complete skills to enhance youths' learning portfolios. Although this information was always available, it was never integrated so it could be used to serve the community of both learners and the providers and to provide measurable student learning and participation outcomes. The learning ecologies theoretical framework and supporting computational methods are a contribution to the state of the art in studying afterschool learning opportunities. While the concept of learning ecologies is not new, to date, no one has offered such a systematic and theoretically-grounded portfolio of measures for characterizing the health and resilience of STEM learning ecologies at multiple scales. The theoretical frameworks and concepts draw together multiple research and application domains: computer science, sociology of education, complexity science, and urban planning. The L3 Connects infrastructure itself represents an unprecedented opportunities for conducting "living lab" experiments to improve stakeholder experience of linking providers to a single network and linking youth to more expanded and varied opportunities. The University of Colorado team will employ three methods: mapping, modeling, and linking youth to STEM learning opportunities in school and out of school settings in a large urban city (Chicago). The recommender system will be embedded into youth and parent facing mobile apps, enabling the team to characterize the degree to which content-based, collaborative filtering, or constraint based recommendations influence youth actions. The project will result in two measurable outcomes of importance to key L3 stakeholder groups: a 10% increase in the number of providers (programs that are part of the infrastructure) in target neighborhoods and a 20% increase in the number of youth participating in programs.
This is an Early-concept Grant for Exploratory Research supporting research in Smart and Connected Communities. The research supported by the award is collaborative with research at DePaul University. The researchers are studying the use of technologies to enable communities to connect youth and youth organizations to effectively support diverse learning pathways for all students. These communities, the youth, the youth organizations, formal and informal education organizations, and civic organizations form a learning ecology. The DePaul University researchers will design and implement a smart community infrastructure in the City of Chicago to track real-time student participation in community STEM activities and to develop mobile applications for both students and adults. The smart community infrastructure will bring together information from a variety of sources that affect students' participation in community activities. These include geographic information (e.g., where the student lives, where the activities take place, the student transportation options, the school the student attends), student related information (e.g., the education and experience background of the student, the economic status of the student, students' schedules), and activity information (e.g., location of activity, requirements for participation). The University of Colorado researchers will take the lead on analyzing these data in terms of a community learning ecologies framework and will explore computational approaches (i.e., recommender systems, visualizations of learning opportunities) to improve youth exploration and uptake of interests and programs. These smart technologies are then used to reduce the friction in the learning connection infrastructure (called L3 for informal, formal, and virtual learning) to enable the student to access opportunities for participation in STEM activities that are most feasible and most appropriate for the student. Such a flexible computational approach is needed to support the necessary diversity of potential recommendations: new interests for youth to explore; specific programs based on interests, friends' activities, or geographic accessibility; or programs needed to "level-up" (develop deeper skills) and complete skills to enhance youths' learning portfolios. Although this information was always available, it was never integrated so it could be used to serve the community of both learners and the providers and to provide measurable student learning and participation outcomes. The learning ecologies theoretical framework and supporting computational methods are a contribution to the state of the art in studying afterschool learning opportunities. While the concept of learning ecologies is not new, to date, no one has offered such a systematic and theoretically-grounded portfolio of measures for characterizing the health and resilience of STEM learning ecologies at multiple scales. The theoretical frameworks and concepts draw together multiple research and application domains: computer science, sociology of education, complexity science, and urban planning. The L3 Connects infrastructure itself represents an unprecedented opportunities for conducting "living lab" experiments to improve stakeholder experience of linking providers to a single network and linking youth to more expanded and varied opportunities. The University of Colorado team will employ three methods: mapping, modeling, and linking youth to STEM learning opportunities in school and out of school settings in a large urban city (Chicago). The recommender system will be embedded into youth and parent facing mobile apps, enabling the team to characterize the degree to which content-based, collaborative filtering, or constraint based recommendations influence youth actions. The project will result in two measurable outcomes of importance to key L3 stakeholder groups: a 10% increase in the number of providers (programs that are part of the infrastructure) in target neighborhoods and a 20% increase in the number of youth participating in programs.
Public opinions toward emergent technologies may be highly dependent on the manner in which people are introduced to these technologies for the very first time. In this light, understanding how such first introductions are related to adolescents’ information seeking behaviors and their developing opinions may be particularly interesting because this target public can be considered to be not only future users of the technology but also future decision makers of its development. The present paper presents a case study of the introduction of ecogenomics among 246 adolescents who were asked to
DATE:
TEAM MEMBERS:
Mark BosRoy KloetCees KoolstraJaap Willems
The purposes of the STUDIO 3D evaluation were to collect information about the impact upon student learning as a result of participating in the STUDIO 3D Project, as well as to elicit information for program improvement. Areas of inquiry include recruiting and retention, impact on project participants, tracking student impacts, and the project as a whole.
The Jackprot is a didactic slot machine simulation that illustrates how mutation rate coupled with natural selection can interact to generate highly specialized proteins. Conceptualized by Guillermo Paz-y-Miño C., Avelina Espinosa, and Chunyan Y. Bai (New England Center for the Public Understanding of Science, Roger Williams University and the University of Massachusetts, Dartmouth), the Jackprot uses simplified slot-machine probability principles to demonstrate how mutation rate coupled with natural selection suffice to explain the origin and evolution of highly specialized proteins. The
The aim of the work reported here has been to give an overview of the support that the informal sector provides for learning and engagement with science. In addressing this goal, we have taken the view that engagement with science and the learning of science occur both within and without schools. What is of interest is not who provides the experience or where it is provided but the nature and diversity of opportunities for science learning and engagement that are offered in contemporary UK society. Thus in approaching the work we have taken a systems perspective and looked at informal
Goodman Research Group, Inc. (GRG) conducted a summative evaluation of the second season of NOVA scienceNOW, the PBS series that explores cutting-edge scientific and technological innovation in real time. (GRG also served as the external evaluator for NOVA scienceNOW during Season One). In addition to the television series, WGBH-TV developed a companion website, a series of high school classroom activities, and a Science Cafe outreach initiative, designed to discuss, in non-academic environments, the latest developments in science. The Season Two evaluation included: 1) a viewer study