This is the evaluation report for the first year of the NSF-funded WaterMarks project. It reflects an initial summary of available evidence about the intended outcomes of program activities to date, as well as commentary on how the project is using (or could use) this information moving forward. This report contains descriptions of embedded measures (i.e. anonymized drawings and reflections captured on a thematic postcard) included in community walks and analyses of secondary data (i.e., interviews conducted by other members of hte project team), as well as reflections emerging from the
This Innovations in Development project supports racially and ethnically diverse youth in learning about climate resilience in informal settings, including community centers, afterschool programs, and museums. The project aims to: (1) build the capacity of community organizations to implement youth programming on climate resilience; (2) increase youth knowledge, skills, and self-efficacy associated with climate resilience (also referred to as environmental health literacy for climate resilience); and (3) explore how collaborating research universities and community organizations engage diverse youth in informal STEM learning. Project partners include the UNC Institute for the Environment, the University of Washington-Interdisciplinary Center for Exposures, Diseases, Genomics and Environment, the North Carolina Museum of Natural Sciences, Juntos NC, and the Duwamish River Community Coalition (DRCC). Juntos NC and DRCC actively engage Latino and Indigenous youth in their programming and seek to implement resilience-focused programming that supports youth science learning and leadership development.
Together, informal educators and participating youth will develop locally relevant solutions to climate impacts in their communities. Youth will interact with university-based climate scientists and educators to collect and analyze data and will participate in resilience-focused dialogue, planning, and actions in their communities. Youth will share what they learn with their families and peers through family events and teen summits. The project will engage dozens of educators in community organizations and at least 250 youth, who will share what they learn with their families and communities, reaching hundreds more people through communications and local action projects. Mixed-methods assessment will provide insight into the extent participating youth (a) develop environmental health literacy for climate resilience, and (b) take action to address resilience in their home communities. The team will assess how these outcomes vary by location, and the implications of any variation on potential for project replication. A participatory evaluation, led by an external evaluator, will provide insight into empowerment outcomes. Findings will be disseminated to professional audiences at local and national conferences; and curricular materials from this project will be disseminated through the project website.
This by the project external evaluation partner presents findings from the first phase of the Co-Created Public Engagement with Science project (CC-PES). The CC-PES project has sought to bring together informal science education institutions, civic partners, and community partners to create forums that address socio-scientific issues that are important to audience being served. The project is designed to lead these collaborative teams through three key steps of public engagement with science: agenda setting (identifying the topic of interest and designing a forum to address it), decision
To advance justice, equity, diversity, and inclusion in science, we must first understand and improve the dominant-culture frameworks that impede progress and, second, we must intentionally create more equitable models. The present authors call ourselves the ICBOs and Allies Workgroup (ICBOs stands for independent community-based organizations), and we represent communities historically excluded from the sciences. Together with institutional allies and advisors, we began our research because we wanted our voices to be heard, and we hoped to bring a different perspective to doing science with
DATE:
TEAM MEMBERS:
María Cecilia Alvarez RicaldeJuan Flores ValadezCatherine CrumJohn AnnoniRick BonneyMateo Luna CastelliMarilú López FrettsBrigid LuceyKaren PurcellJ. Marcelo BontaPatricia CampbellMakeda CheatomBerenice RodriguezYao Augustine FoliJosé GonzálezJosé Miguel Hernández HurtadoSister Sharon HoraceKaren KitchenPepe Marcos-IgaTanya SchuhPhyllis Edwards TurnerBobby WilsonFanny Villarreal
The pilot and feasibility study will develop instructional workshops for an adult population of quilters to introduce them to computational thinking. By leveraging pre-existing social structures, skill sets, and engagement in quilting, the researchers hope to help participants develop computer science and computational thinking knowledge and skills.
This poster was presented at the 2021 NSF AISL Awardee Meeting.
The pilot test for Changemakers: Advancing Community Science Literacy was a capacity building program integrating strategic discourse & community change theory that identified a new path for advancing community STEM literacies. The results of experiment established partnerships with locally based non-profits, and a collaborative effort to address environmental justice and social disparities in areas threatened by climate change.
The evaluation was underaken with instruments developed for multiple research projects to support cross-project comparative analysis.
This instrumemts presented
The pilot test of a capacity building program integrating strategic discourse & community change theory identified a new path for advancing community STEM literacies. The results of experiment established partnerships with locally based non-profits working to address environmental justice and social disparities in areas threatened by climate change identified five recommendations to reset the role of ISLC’s as more relevant to the communities: 1) Allocate Time to Build Relationships; 2) Develop a Shared Definition of Resilience; 3) Situate Community Aspirations as Context for STEM Learning; 4)
This project is a Smart and Connected Communities award. The community is part of Evanston, Illinois and is composed of the lead partners described below:
EvanSTEM which is a in-school/out of school time (OST) program to improve access and engagement for students in Evanston who have underperformed or been underrepresented in STEM.
McGaw YMCA which consists of 12,000 families serving 20,000 individuals and supporting technology and makerspace activities (MetaMedia) in a safe community atmosphere.
Office of Community Education Partnerships (OCEP) at Northwestern University which provides support for the university and community to collaborate on research, teaching, and service initiatives.
This partnership will develop a new approach to learning enagement through the STEAM (Science, Technology, Engineering, Arts, and Mathematics) interests of all young people in Evanston. This project is entitled Interests for All (I4All) and builds upon existing research results of the two Principal Investigators (PIs) and previous partnerships between the lead partners (EvanSTEM and MetaMedia had OCEP as a founding partner). I4All also brings together Evanston school districts, OST prividers, the city, and Evanston's Northwestern University as participants.
In particular the project builds on PI Pinkard's Cities of Learning project and co-PI Stevens' FUSE Studios project. Both of these projects have explicit goals to broaden participation in STEAM pursuits, a goal that is significantly advanced through I4All. In this project, I4All infrastructure will be evaluated using quantitative metrics that will tell the researchers whether and to what degree Evanston youth are finding and developing their STEAM interests and whether the I4All infrastructure supports a significantly more equitable distribution of opportunities to youth. The researchers will also conduct in depth qualitative case studies of youth interest development. These longitudinal studies will complement the quantitative metrics of participation and give measures that will be used in informing changes in I4All as part of the PIs Design Based Implementation Research approach. The artifacts produced in I4All include FUSE studio projects, software infrastructure to guide the students through OST and in-school activities and to provide to the students actionable information as to logistics for participation in I4All activities, and data that will be available to all stakeholders to evaluate the effectiveness of I4All. Additionally, this research has the potential to provide for scaling this model to different communities, leveraging the OST network in one community to begin to offer professional development more widely throughout the school districts and as an exemplar for other districts. These research results could also affect strategies and policies created by local school officials and community organizations regarding how to work together to create local learning environments to create an ecosystem where formal and informal learning spaces support and reinforce STEAM knowledge.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
This Pilot and Feasibility project will investigate: a) the kinds of learning outcomes that occur in an informal STEM project utilizing a Critical Environmental Justice-informed Participatory Action Research framework, co-created between researchers and homeless communities, and b) the challenges and opportunities that arise in evaluating such collaborative projects. To address these questions, a team of university and community researchers will work with two self-organized homeless communities, in Portland, OR and Baltimore, MD, to systematically examine air quality at their sites and further develop a multimedia environmental justice toolkit. The toolkit is aimed at equipping homeless communities with STEM-based information and approaches to learn about and address various environmental hazards. Homeless people are disproportionately vulnerable to the impacts of air and soil pollution, rodents, extreme weather, floods, fires, and other hazards. This collaboration will study STEM learning outcomes related to the broader historical socio-political context in which people are living. The project also anticipates finding ways to adapt evaluation processes to better suit the needs of marginalized groups participating in informal STEM projects.
Guided by this integrated framework, the team will undertake participant observation, interviews, focus groups, and document analysis to assess learning. It also anticipates gaining insights into how evaluation processes may be adapted to better suit the needs of marginalized groups participating in informal STEM learning. This project is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
DATE:
-
TEAM MEMBERS:
Chris HawnMelanie MaloneErin GoodlingDillon MahmoudiAnthony Levenda
This document is the final summative evaluation report written by EDC, the external evaluator of the STEM Guides project. The report concludes that the project was highly ambitious, with many dynamic and evolving pieces. It was deemed successful as a model of brokering connections between students aged 10-18 and STEM resources and opportunities in rural Maine communities. The STEM Guides program contributed to the increase in STEM awareness within each community, as well as connecting youth with interesting and relevant STEM experiences.
Future educational robots are emerging as social companions supporting learning. By socially interacting with such a robot, learners can potentially reason and talk about the things they are learning and receive help in seeing the relevance of STEM in their daily lives. However, little is known about how to design educational robots to work with youth at home over a long period of time. This project will develop an informal science learning program, called STEMMates, in collaboration with a local community center, for youth with little interest in science. The program will partner learners with an in-home learning companion robot, designed to read books with youth and provide science activities for them at the community center, where youth will engage in exciting and personally relevant science learning. As the learner reads books, the robot will make comments about what is happening in the book to help connect the reading to the science activities at the community center. The overarching goals of STEMMates are to: (a) positively support youth's individual interest in science and future science learning, (b) connect in-home learning experiences with out-of-school community-based learning, (c) bridge the gap between formal and informal engagement and learning in science, and (d) encourage the participation of youth who are underrepresented and who have low interest in STEM learning. This project is funded by the Advancing Informal STEM Learning program, which seeks to advance new approaches to and evidence-based understanding of the design and development of STEM learning opportunities for the public in informal environments.
Researchers will work with youth and staff at the community center, alongside experts in informal science learning, to design the program and then test how learners respond to reading with the robot and participating in the science activities and whether this program has a lasting impact on their science interest. Social interactions with a robot may help distribute cognitive load during learning activities to help youth reason about STEM and also supplement learning by improving feelings of value and belongingness in order to facilitate lasting interest development. Following a mixed-methods research approach using qualitative and quantitative data-collection techniques, the research team will investigate the following research questions: (1) What social and interest-development supports and activities can be utilized as socially situated interest scaffolds in an informal and in-home, augmented reading and science activity program to promote individual interest and learning in science for low interest learners? How can a social robot best facilitate this program? (2) How do learners perceive and interact with the robot in authentic, in-home, long-term situations, and how does this interaction change over time? (3) Does working with a robot designed with socially situated interest scaffolds increase individual interest in science when compared to a pre-intervention baseline, and do these effects impact future (long-term) interest and engagement in formal science learning? To answer these research questions, researchers will implement the science learning program during an 11-week summer deployment and utilize an AB single-case research design. Interview-based qualitative data and self-report surveys to examine the learner?s perception of the robot and their evolving interest in science and quantitative data on science learning using pre-/post-measure comparisons will be collected. Log data of time-on-task, reading rate, book selection and reading goal attainment will also be collected by the robot. The outcomes of this project will lay the groundwork for future investigations of the design of social robots for a diversity of learner populations and their use in different informal learning settings.
This project is funded by the National Science Foundation's (NSF's) Advancing Informal STEM Learning (AISL) program, which supports innovative research, approaches, and resources for use in a variety of learning settings.
This program will derive knowledge on extreme weather and its concepts to be shared with youth in the Boston and Kansas City areas. Subsequently, the youth will share this knowledge by displaying it as art work on the rapid transit systems. The art projects will culminate in broad-based exhibition at the end of each group's sessions. The project will involve 200 youth per region resulting in an impact of 1000 youth per year, 80 adult mentors and 20,000 adult transportation riders in learning about extreme weather concepts. Participant organizations are the University of Mass-Boston, University of Mass-Lowell, The Massachusetts College of Art, the University of Kansas Center for Research Inc., and the Goodman Research Group Inc.
The goals of this project are to bring the topic of extreme weather to the foreground by educating youth and in turn having them educate a selected group of adults that use the rapid transit system. Groups of youths will learn about the topic through a series of meetings with mentors who are experts on the issues around extreme weather. The youth will derive their own art-works with their interpretation. These art-works will be displayed on the rapid transit systems in New England (Merrimack Valley and Worcester regions) and the Mid-West (Topeka and Kansas City areas). Using a quasi-experimental mixed methodology (demographics, bus ridership, initial level of science awareness, and interest) the goal is to understand science learning outcomes associated with the creation and public display of youth art. Research questions of importance in this regard are 1. In what ways does blending art with the science enhance youth learning about extreme weather concepts? 2. To what extent does youth art support adult learning of science? and 3. How does regional context affect learning about extreme weather?
Broader impacts will result from the youth diversity as well as the diversity of riders of the rapid transit systems where the art of extreme weather is displayed.
This project is funded by the National Science Foundation's (NSF's) Advancing Informal STEM Learning (AISL) program, which supports innovative research, approaches, and resources for use in a variety of learning settings.
DATE:
-
TEAM MEMBERS:
Robert ChenLois HetlandJill LohmeierStephen MisholSteven SchrockClaudia Bode