This summary brief captures highlights from the second year of the NSF-funded WaterMarks project. The technical evaluation report for this same project period can be found on the main project page. The purpose of this document is to communicate key updates (as observed by the evaluation team) in a less technical way with the many different audiences who have an interest in keeping up with WaterMarks.
This is the evaluation report for the second year of the NSF-funded WaterMarks project. It reflects a current summary of available evidence about the intended outcomes of program activities to date, as well as commentary on how the project is using (or could use) this information moving forward.
This summary brief captures highlights from the evaluation report for the first year of the NSF-funded WaterMarks project (also available on this page). The purpose of this document is to communicate key updates from evaluation in a less technical way with the many different audiences who have an interest in keeping up with WaterMarks.
This is the evaluation report for the first year of the NSF-funded WaterMarks project. It reflects an initial summary of available evidence about the intended outcomes of program activities to date, as well as commentary on how the project is using (or could use) this information moving forward. This report contains descriptions of embedded measures (i.e. anonymized drawings and reflections captured on a thematic postcard) included in community walks and analyses of secondary data (i.e., interviews conducted by other members of hte project team), as well as reflections emerging from the
The U.S. Fish and Wildlife Service estimates that over 41 million people connect to nature through birding. Learning about birds in their natural environments offers opportunities for informal engagement in STEM by a broad range of individuals and groups. Birders often engage in scientific data gathering and analyses, geolocation and remote sensing, and phenology. They also become aware of ecological changes in bird habitats and migratory patterns due to rising temperatures and climate-related events like sea level rise, droughts, fires, and extreme weather. As such, the birding community is an ideal network to better understand and communicate the impacts of climatological changes on bird populations to the public. With this Innovations in Development project, the National Audubon Society will develop a new avian-focused, conservation and climate science community science curriculum for its Nature Centers, and test the effectiveness of the curriculum in educating the public about avian-focused conservation and climatological changes through guided nature experiences. Birding can serve as a pivotal entrée for young people into STEM fields and careers. Through its programs and partnerships, Audubon will leverage its national network to ensure that through this project a more diverse group of voices, particularly young adults and young adults of color, become involved in asking critical questions and developing solutions to address important environmental issues of the future. If successful, the broader impacts of this project on capacity building and public engagement could be far-reaching and long-lasting.
Over the three-year project duration, Audubon will bring educators from its nationwide network of thirty-four Nature Centers (including urban, suburban, and rural sites), together with over 510 young adults (ages 18-25) from its network of college campus chapters. An evidence-based curriculum and community science activities will be created and tested, relying heavily on a team of experts in ornithology, climate science research, STEM curriculum design, diversity, and informal science education. College students will advise on the design of content and activities to effectively interest and engage young adults. These students will be recruited from the new Audubon Campus Chapters Program, which includes 111 college and university campuses, among them, 19 Historically Black Colleges and Universities (HBCUs) and other Minority Serving Institutions (MSIs). The target population will be surveyed to also understand their current and likely participation in guided nature experiences and knowledge base in climate science. Current best practices in guided nature experiences will be gathered from across the Audubon network. The implementation efforts will result in a national STEM model, with train-the-trainer guides and workshops for informal science educators and public engagement opportunities focused on improving the state and condition of avian habitats and communities through climate science research. An external evaluation will be conducted and will include data collection methods such as retrospective pre and post surveys, semi-structured interviews, focus groups, and an embedded assessment to determine impact. The findings will be used to iteratively refine the evidence-based curriculum and measure STEM learning outcomes for the guided nature experience participants. The evaluation will address four areas: (1) fidelity of program implementation to promote accountability; (2) formative evaluation to understand needs and interests of young adults (ages 18-25), and subsequently inform program design; (3) outcomes for Center educators, to inform iterative improvement; and (4) outcomes for program participants, to contribute to the growing knowledge base on effective practices for STEM learning in informal settings.
This Innovations in Development project is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
DATE:
-
TEAM MEMBERS:
Loren SmithMark ScallionHeather Starck
This video presents reflections on SCIENCES: Supporting a Community’s Informal Education Needs—Confidence and Empowerment in STEM. SCIENCES brought together Eden Place Nature Center and the Chicago Zoological Society to collaboratively support environmental conservation and lifelong scientific learning in the Fuller Park neighborhood of Chicago.
The SCIENCES project began in 2013 and focused on adapting existing educational programs into a suite of environmentally focused science learning opportunities for professional, student, and public audiences in the Fuller Park neighborhood
The summative evaluation documents and articulates what SCIENCES has improved or changed, and in what ways. The final design of the summative evaluation was based on findings from the front-end and formative evaluations, including using participatory evaluation techniques to engage community members in discussing their experience with the programs and assessment of community needs and assets at the close of the project.
The goal of the summative evaluation was to address discrete program impacts in the context of the project, as well as the cross-program impact of providing a thematically
In The Nature of Community: SCIENCES, we share the lessons learned from an innovative partnership designed to leverage the strengths of two nonprofit organizations—a large cultural institution and a smaller, deeply-rooted community-based organization, both of which offer informal science education expertise.
You’ll read first-hand reflections of how staff members, community leaders and members, children, and adults experienced this partnership: the expectations, surprises, challenges, successes, and lessons learned. We hope the description of this partnership inspires other organizations to
Learn how to create opportunities for young people from low-income, ethnically diverse communities to learn about growing food, doing science, and how science can help them contribute to their community in positive ways. The authors developed a program that integrates hydroponics (a method of growing plants indoors without soil) into both in-school and out-of-school educational settings.
This is an Early-concept Grant for Exploratory Research supporting research in Smart and Connected Communities. The research supported by the award is collaborative with research at the University of Colorado. The researchers are studying the use of technologies to enable communities to connect youth and youth organizations to effectively support diverse learning pathways for all students. These communities, the youth, the youth organizations, formal and informal education organizations, and civic organizations form a learning ecology. The DePaul University researchers will design and implement a smart community infrastructure in the City of Chicago to track real-time student participation in community STEM activities and to develop mobile applications for both students and adults. The smart community infrastructure will bring together information from a variety of sources that affect students' participation in community activities. These include geographic information (e.g., where the student lives, where the activities take place, the student transportation options, the school the student attends), student related information (e.g., the education and experience background of the student, the economic status of the student, students' schedules), and activity information (e.g., location of activity, requirements for participation). The University of Colorado researchers will take the lead on analyzing these data in terms of a community learning ecologies framework and will explore computational approaches (i.e., recommender systems, visualizations of learning opportunities) to improve youth exploration and uptake of interests and programs. These smart technologies are then used to reduce the friction in the learning connection infrastructure (called L3 for informal, formal, and virtual learning) to enable the student to access opportunities for participation in STEM activities that are most feasible and most appropriate for the student. Such a flexible computational approach is needed to support the necessary diversity of potential recommendations: new interests for youth to explore; specific programs based on interests, friends' activities, or geographic accessibility; or programs needed to "level-up" (develop deeper skills) and complete skills to enhance youths' learning portfolios. Although this information was always available, it was never integrated so it could be used to serve the community of both learners and the providers and to provide measurable student learning and participation outcomes. The learning ecologies theoretical framework and supporting computational methods are a contribution to the state of the art in studying afterschool learning opportunities. While the concept of learning ecologies is not new, to date, no one has offered such a systematic and theoretically-grounded portfolio of measures for characterizing the health and resilience of STEM learning ecologies at multiple scales. The theoretical frameworks and concepts draw together multiple research and application domains: computer science, sociology of education, complexity science, and urban planning. The L3 Connects infrastructure itself represents an unprecedented opportunities for conducting "living lab" experiments to improve stakeholder experience of linking providers to a single network and linking youth to more expanded and varied opportunities. The University of Colorado team will employ three methods: mapping, modeling, and linking youth to STEM learning opportunities in school and out of school settings in a large urban city (Chicago). The recommender system will be embedded into youth and parent facing mobile apps, enabling the team to characterize the degree to which content-based, collaborative filtering, or constraint based recommendations influence youth actions. The project will result in two measurable outcomes of importance to key L3 stakeholder groups: a 10% increase in the number of providers (programs that are part of the infrastructure) in target neighborhoods and a 20% increase in the number of youth participating in programs.
This is an Early-concept Grant for Exploratory Research supporting research in Smart and Connected Communities. The research supported by the award is collaborative with research at DePaul University. The researchers are studying the use of technologies to enable communities to connect youth and youth organizations to effectively support diverse learning pathways for all students. These communities, the youth, the youth organizations, formal and informal education organizations, and civic organizations form a learning ecology. The DePaul University researchers will design and implement a smart community infrastructure in the City of Chicago to track real-time student participation in community STEM activities and to develop mobile applications for both students and adults. The smart community infrastructure will bring together information from a variety of sources that affect students' participation in community activities. These include geographic information (e.g., where the student lives, where the activities take place, the student transportation options, the school the student attends), student related information (e.g., the education and experience background of the student, the economic status of the student, students' schedules), and activity information (e.g., location of activity, requirements for participation). The University of Colorado researchers will take the lead on analyzing these data in terms of a community learning ecologies framework and will explore computational approaches (i.e., recommender systems, visualizations of learning opportunities) to improve youth exploration and uptake of interests and programs. These smart technologies are then used to reduce the friction in the learning connection infrastructure (called L3 for informal, formal, and virtual learning) to enable the student to access opportunities for participation in STEM activities that are most feasible and most appropriate for the student. Such a flexible computational approach is needed to support the necessary diversity of potential recommendations: new interests for youth to explore; specific programs based on interests, friends' activities, or geographic accessibility; or programs needed to "level-up" (develop deeper skills) and complete skills to enhance youths' learning portfolios. Although this information was always available, it was never integrated so it could be used to serve the community of both learners and the providers and to provide measurable student learning and participation outcomes. The learning ecologies theoretical framework and supporting computational methods are a contribution to the state of the art in studying afterschool learning opportunities. While the concept of learning ecologies is not new, to date, no one has offered such a systematic and theoretically-grounded portfolio of measures for characterizing the health and resilience of STEM learning ecologies at multiple scales. The theoretical frameworks and concepts draw together multiple research and application domains: computer science, sociology of education, complexity science, and urban planning. The L3 Connects infrastructure itself represents an unprecedented opportunities for conducting "living lab" experiments to improve stakeholder experience of linking providers to a single network and linking youth to more expanded and varied opportunities. The University of Colorado team will employ three methods: mapping, modeling, and linking youth to STEM learning opportunities in school and out of school settings in a large urban city (Chicago). The recommender system will be embedded into youth and parent facing mobile apps, enabling the team to characterize the degree to which content-based, collaborative filtering, or constraint based recommendations influence youth actions. The project will result in two measurable outcomes of importance to key L3 stakeholder groups: a 10% increase in the number of providers (programs that are part of the infrastructure) in target neighborhoods and a 20% increase in the number of youth participating in programs.
This poster was presented at the 2016 Advancing Informal STEM Learning (AISL) PI Meeting held in Bethesda, MD on February 29-March 2. The SCIENCES project aims to create a STEM ecosystem in Fuller Park, a chronically, severely under-resourced urban community in Chicago.