This summary brief captures highlights from the evaluation report for the first year of the NSF-funded WaterMarks project (also available on this page). The purpose of this document is to communicate key updates from evaluation in a less technical way with the many different audiences who have an interest in keeping up with WaterMarks.
This is the evaluation report for the first year of the NSF-funded WaterMarks project. It reflects an initial summary of available evidence about the intended outcomes of program activities to date, as well as commentary on how the project is using (or could use) this information moving forward. This report contains descriptions of embedded measures (i.e. anonymized drawings and reflections captured on a thematic postcard) included in community walks and analyses of secondary data (i.e., interviews conducted by other members of hte project team), as well as reflections emerging from the
As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program funds innovative resources for use in a variety of settings. Informal STEM learning opportunities are often rare in rural locations where the early childhood education system is also under-resourced. Through partnerships with educational researchers, early math educators, pediatric health experts, and pediatric clinics, this project will develop and study a new opportunity for informal math learning. The project will work with pediatric clinics that serve rural immigrant families who are racially, culturally, and linguistically diverse. The project leverages the high levels of trust many caregivers have in their child’s pediatrician to improve math learning during critical early years. This project will build on a previous program where physician text messages to caregivers supported youth literacy development. In this instance the project will support caregivers’ math interactions with their 3- and 4-year-olds to cultivate children's math knowledge and skills. The text messaging program will be grounded in research in child development, mathematics learning, parenting practices, and adult behavior change. Texts will also provide caregiver supports for how to engage their children in mathematical activates in their everyday lives and provide information about the important skills children are developing. Text messages will be co-developed with caregiver input, and focus on content underlying mathematical development such as Number Sense, Classification and Patterning, Measurement, Geometry, and Reasoning. Caregivers will receive text messages from their pediatric clinics three times a week for eight months. For example, three related texts supporting Number Sense include: “FACT: Kids enjoy counting and it prepares them for K! Mealtimes are a fun time to practice counting objects;” “TIP: At a meal, say: Can you count all the cups on the table? All the plates? What else can you count? (Forks) Tell them: Great job!” and “GROWTH: You are helping kids to count & get ready for K. At the park, ask: How many bikes are there? How many birds? Count together & find out!” Throughout the planning and implementation phases of the project the team will work closely with early education math experts, key advisors, and caregivers to ensure the text messaging program is tailored to meet the cultural, linguistic, and contextual needs of rural caregivers and children.
The project will research impacts of the text messaging program on children, caregivers, and clinical staff. First, the project will investigate the impact of the texting program on children through a randomized trial, and pre-and-post measures of early childhood math skills and abilities. Second, using interviews at baseline and in a 9-month follow-up, the project will study the texting program’s impact on caregivers’ perceptions regarding the importance of math learning for young children. Third, the project will explore the impact of the text messaging program on health professionals’ understanding of math learning in early childhood by collecting qualitative data and assessing attitudes about the clinic’s role in supporting early math. Caregivers and clinic staff will also participate in focus groups to better understand impacts for each of these groups. The project will reach 1000 families, who will be randomly assigned to treatment or control groups through block-randomization, stratified by caregiver language and child’s age. This parent-informed project will build evidence toward new approaches to promoting early math in the pediatric clinic, an informal environment that can reach all families and can leverage innovative technology. Findings will be shared widely though a communication and engagement plan that includes children, caregivers, physicians and clinic staff, informal STEM educators, researchers, and policy makers.
DATE:
-
TEAM MEMBERS:
Lisa ChamberlainSusanna LoebJaime Peterson
This Innovations in Development project supports racially and ethnically diverse youth in learning about climate resilience in informal settings, including community centers, afterschool programs, and museums. The project aims to: (1) build the capacity of community organizations to implement youth programming on climate resilience; (2) increase youth knowledge, skills, and self-efficacy associated with climate resilience (also referred to as environmental health literacy for climate resilience); and (3) explore how collaborating research universities and community organizations engage diverse youth in informal STEM learning. Project partners include the UNC Institute for the Environment, the University of Washington-Interdisciplinary Center for Exposures, Diseases, Genomics and Environment, the North Carolina Museum of Natural Sciences, Juntos NC, and the Duwamish River Community Coalition (DRCC). Juntos NC and DRCC actively engage Latino and Indigenous youth in their programming and seek to implement resilience-focused programming that supports youth science learning and leadership development.
Together, informal educators and participating youth will develop locally relevant solutions to climate impacts in their communities. Youth will interact with university-based climate scientists and educators to collect and analyze data and will participate in resilience-focused dialogue, planning, and actions in their communities. Youth will share what they learn with their families and peers through family events and teen summits. The project will engage dozens of educators in community organizations and at least 250 youth, who will share what they learn with their families and communities, reaching hundreds more people through communications and local action projects. Mixed-methods assessment will provide insight into the extent participating youth (a) develop environmental health literacy for climate resilience, and (b) take action to address resilience in their home communities. The team will assess how these outcomes vary by location, and the implications of any variation on potential for project replication. A participatory evaluation, led by an external evaluator, will provide insight into empowerment outcomes. Findings will be disseminated to professional audiences at local and national conferences; and curricular materials from this project will be disseminated through the project website.
Mathematizing, Visualizing, and Power (MVP): Appalachian Youth Becoming Data Artists for Community Learning is a three-year Advancing Informal STEM Learning, Innovations and Development, project that focuses on community-centered data exploration catalyzed by youth. The project develops statistical artistry among young people in East Tennessee Appalachian communities and enables these youth to share their data visualizations with their communities to foster collective reflection and understanding. The creative work generated by the MVP project will be compelling in two ways, both as statistical art and as powerful statements giving voice to the experience of communities. Critical aspects of the MVP model include (1) youth learning sessions that position youth as owners of data and producers of knowledge and (2) Community Learning Events that support community learning as youth learning occurs. The MVP project has a primary focus on broadening the STEM participation of underrepresented communities of Appalachia. The project’s mission is to increase the learning and life outcomes of young people and communities of Appalachia by creating a meaningful foundation of data science and collective data exploration. The University of Tennessee partners with Pellissippi State Community College, Drexel University, and the Boys & Girls Club of the Tennessee Valley to bring together a convergent team of community members, practitioners, and professionals, with the expertise to carry out the project. The project will impact approximately 120 youth and 3800 of their East Tennessee community members. The research generated will inform how to engage community members in learning about community issues through the exploration of datasets relevant to participants.
The field of STEM education is in urgent need of knowledge about effective models to inspire community-based data exploration with young people as leaders in these efforts. The MVP project includes engaging youth with meaningful problems, building a discourse community with possibilities for action, re-positioning youth as knowledge producers within their own communities, leveraging linguistic and cultural resources of the youth participants and their communities, and implementing critical events that support substantial interaction between youth, community members, and the data visualizations. MVP builds on the idea that the design of data visualizations requires an understanding of both data science and artistic design. Research will inform the model of community engagement, examine data artists’ identities, and document community learning. The MVP model will be designed, developed, tested, and refined through three cycles of design-based research. The overarching research question guiding these cycles is: What affordances (and delimitations) related to identity and learning does the model provide for MVP Youth and community members? Data sources for the project include: fieldnotes, portfolios created by MVP Youth, youth pre/post interviews, observations of the learning sessions, a project documentary, surveys for youth and community members, interviews with community members, and audience feedback. The National Institute for STEM Evaluation and Research (NISER) will provide formative and summative evaluation about project activities. Formative feedback will be integrated into the ongoing research cycles. The research conducted will inform (1) the community learning model; (2) the integrated pedagogy and curriculum of the MVP Youth learning sessions that emphasize data science through design arts; and, (3) research on community learning and youth identity. Findings will be shared through conferences, academic and practitioner-focused journals, a video documentary, a Summit on Engaging Youth and Communities in Data, and a project website.
DATE:
-
TEAM MEMBERS:
Lynn HodgeElizabeth DyerJoy BertlingCarlye Clark
resourceprojectProfessional Development, Conferences, and Networks
Centering Native Traditional Knowledge within informal STEM education programs is critical for learning for Native youth. In co-created, place-based learning experiences for Native youth, interweaving cultural traditions, arts, language, and community partnerships is vital for authentic, meaningful learning. Standardized STEM curricula and Western-based pedagogies within the mainstream and formal education systems do not reflect the nature of Native STEM knowledge, nor do they make deep connections to it. The absence of this knowledge base can reinforce a deficit-based STEM identity, which can directly impact Native youths’ participation and engagement in STEM. Reframing STEM education for Native youth to prioritize the vitality of community and sustainability requires active consideration of what counts as science learning and who serves as holders and conduits of STEM knowledge. As highly regarded holders of traditional and western STEM knowledge, Native educators and cultural practitioners are critical for facilitating Native youths’ curiosity and engagement with STEM. This Innovations in Development project is Native-led and centers Native knowledge, voice, and contributions in STEM through a culturally based, dual-learning approach that emphasizes traditional and western STEM knowledge. Through this lens, a network of over a dozen tribal nations across 20 U.S. states will be established to support and facilitate the learning of Traditional and Western STEM knowledge in a culturally sustaining manner. The network will build on existing programs and develop a set of unique, interconnected, and synchronized placed-based informal STEM programs for Native youth reflecting the distinctive cultural aspects of Native American and Alaska Native Tribes. The network will also involve a Natives-In-STEM Role Models innovation, in which Native STEM professionals will provide inspiration to Native youth through conversations about their journeys in STEM within cultural contexts. In addition, the network will cultivate a professional network of STEM educators, practitioners, and tribal leaders. Network efforts and the formative evaluation will culminate in the development and dissemination of a community-based, co-created Framework for Informal STEM Education with Native Communities.
Together with Elders and other contributors of each community, local leads within the STEM for Youth in Native Communities (SYNC) Network team will identify and guide the STEM content topics, as well as co-create and implement the program within their sovereign lands with their youth. The content, practitioners, and programming in each community will be distinct, but the community-based, dual learning contextual framework will be consistent. Each community includes several partner organizations poised to contribute to the programming efforts, including tribal government departments, tribal and public K-12 schools, tribal colleges, museums and cultural centers, non-profits, local non-tribal government support agencies, colleges and universities, and various grassroots organizations. Programmatic designs will vary and may include field excursions, summer and after school STEM experiences, and workshops. In addition, the Natives-In-STEM innovation will be implemented across the programs, providing youth with access to Native STEM professionals and career pathways across the country. To understand the impacts of SYNC’s efforts, an external evaluator will explore a broad range of questions through formative and summative evaluations. The evaluation questions seek to explore: (a) the extent to which the culturally based, dual learning methods implemented in SYNC informal STEM programs affect Native youths’ self-efficacy in STEM and (b) how the components of SYNC’s overall theoretical context and network (e.g., partnerships, community contributors such as Elders, STEM practitioners and professionals) impact community attitudes and behaviors regarding youth STEM learning. Data and knowledge gained from these programs will inform the primary deliverable, a Framework for Native Informal STEM Education, which aims to support the informal STEM education community as it expands and deepens its service to Native youth and communities. Future enhanced professional development opportunities for teachers and educators to learn more about the findings and practices highlighted in the Framework are envisioned to maximize its strategic impact.
DATE:
-
TEAM MEMBERS:
Juan ChavezDaniella ScaliceWendy Todd
This by the project external evaluation partner presents findings from the first phase of the Co-Created Public Engagement with Science project (CC-PES). The CC-PES project has sought to bring together informal science education institutions, civic partners, and community partners to create forums that address socio-scientific issues that are important to audience being served. The project is designed to lead these collaborative teams through three key steps of public engagement with science: agenda setting (identifying the topic of interest and designing a forum to address it), decision
Chabot Space & Science Center will expand its work in youth development and community outreach by launching the Oakland Connected Learning Partnership program targeting underserved children and youth. Over the two-year project period, the museum will hire 12 teens each year from a Title I high school as paid interns. Museum educators will provide training and mentorship to prepare the interns to present STEM-based after-school programs for children aged 6-12 at local Girls & Boys Clubs. The teens will help to organize and present additional project activities, including Community Science Events at local libraries, public schools and places of worship. Each year of the project will culminate with a free Community Day at the museum for all participants. Community listening sessions at strategic intervals will help the project team understand resources and needs and obtain feedback on the program.
The Saginaw Chippewa Indian Tribe of Michigan's Ziibiwing Center of Anishinabe Culture and Lifeways will organize a four-day educational symposium to build a better understanding of Native American culture and history. The project will begin with a forum to foster dialogue on the 200th anniversary of the Treaty of Saginaw. The forum will discuss the treaty's impact on sovereignty and relationships between natives and non-natives and the loss of continuity of language, culture, and the practice of traditional art forms. The forum will include representatives from the 25 tribes whose children attended the Mount Pleasant Indian Industrial Boarding School. The representatives will share cultural stories and traditional methods through birch bark, black ash, elm and sweet grass basket making. The symposium will conclude on Michigan Indian Day with science, technology, engineering, art and math (STEAM) activities for area students.
Researchers and practitioners have identified numerous outcomes of place-based environmental action (PBEA) programs at both individual and community levels (e.g., promoting positive youth development, fostering science identity, building social capital, and contributing to environmental quality improvement). In many cases, the primary audience of PBEA programs are youth, with less attention given to lifelong learners or intergenerational (e.g., youth and adult) partnerships. However, there is a need for PBEA programs for lifelong learners as local conservation decisions in the United States
Making as a term has gained attention in the educational field. It signals many different meanings to many different groups, yet is not clearly defined. This project’s researchers refer to making as a term that bears social and cultural impact but with a broader more sociocultural association than definitions that center making in STEM learning. Using the theoretical lenses of critical relationality and embodiment, our research team position curriculum as a set of locally situated activities that are culturally, linguistically, socially, and politically influenced. We argue that curriculum
DATE:
TEAM MEMBERS:
Veronica OguilveWen WenEm BowenYousra AbourehabAmanda BermudezElizabeth GaxiolaJill Castek
To advance justice, equity, diversity, and inclusion in science, we must first understand and improve the dominant-culture frameworks that impede progress and, second, we must intentionally create more equitable models. The present authors call ourselves the ICBOs and Allies Workgroup (ICBOs stands for independent community-based organizations), and we represent communities historically excluded from the sciences. Together with institutional allies and advisors, we began our research because we wanted our voices to be heard, and we hoped to bring a different perspective to doing science with
DATE:
TEAM MEMBERS:
María Cecilia Alvarez RicaldeJuan Flores ValadezCatherine CrumJohn AnnoniRick BonneyMateo Luna CastelliMarilú López FrettsBrigid LuceyKaren PurcellJ. Marcelo BontaPatricia CampbellMakeda CheatomBerenice RodriguezYao Augustine FoliJosé GonzálezJosé Miguel Hernández HurtadoSister Sharon HoraceKaren KitchenPepe Marcos-IgaTanya SchuhPhyllis Edwards TurnerBobby WilsonFanny Villarreal