We present the assets that collaboration across a land grant university brought to the table, and the Winterberry Citizen Science program design elements we have developed to engage our 1080+ volunteer berry citizen scientists ages three through elder across urban and rural, Indigenous and non-Indigenous, and formal and informal learning settings.
DATE:
TEAM MEMBERS:
Katie SpellmanJasmine ShawChristine VillanoChrista MulderElena SparrowDouglas Cost
We used a youth focused wild berry monitoring program that spanned urban and rural Alaska to test this method across diverse age levels and learning settings.
DATE:
TEAM MEMBERS:
Katie SpellmanDouglas CostChristine Villano
The Arctic Harvest-Public Participation in Scientific Research (which encompasses the Winterberry Citizen Science program), a four-year citizen science project looking at the effect of climate change on berry availability to consumers has made measurable progress advancing our understanding of key performance indicators of highly effective citizen science programs.
DATE:
TEAM MEMBERS:
Angela LarsonKelly KealyMakaela Dickerson
resourceevaluationMuseum and Science Center Exhibits
The Kaulele Kapa Exhibit was created to explore the effectiveness of a Hawaiian culture-based framework and approach in increasing learner engagement and depth of knowledge in STEM among Native Hawaiian/Pacific Islander (NHPI) learners. The exhibit utilized hands-on and interactive activities, coupled with scientific and cultural information, to create relevant learning experiences for these communities. To determine the effectiveness, exhibit attendees were invited to complete a survey that asked about how the exhibit influenced their interest and understanding of STEM and Hawaiian culture
Reflections and Ideas about Collaboration with Integrity explores the work Generations of Knowledge: Traditional Ecological Knowledge and Environmental Science (National Science Foundation DRL-1010559), a six-year collaborative project between OMSI and Native partners from diverse tribes, cultures, and ecoregions that co-created traveling exhibits and programs for science and tribal museums. This legacy document reflects on and shares what we learned on this journey, including our detours and course corrections. The legacy document strongly reflects the work of the whole project both in its
DATE:
TEAM MEMBERS:
Victoria CoatsCecilia NguyenMolly SchmitzJaclyn BarberTim HecoxMarilyn JohnsonKyrie Thompson KellettTim SteevesLeah Gibson (Oglala Lakota)David Begay Diné [Navajo])Inez Bill (Tulalip/Lummi)Karen Kitchen (Osage)Katherine KrileHerb Lee, Jr. (Hawaiian)VerlieAnn Malina-Wright (Hawaiian)Nancy Maryboy (Cherokee/Diné [Navajo])Randall Melton (Seminole/CTUIR)Wenix Red Elk (CTUIR)David CozzoVicki Cruz (Cherokee)Deana Dartt (Chumash)Jill SteinShelly Valdez (Laguna Pueblo)Pamela Woodis (Jicarilla Apache)Tessa Campbell (Tulalip)
The Science Museum of Minnesota (SMM) will collaborate with four community organizations serving Black, Indigenous, and People of Color (BIPOC) audiences to research and develop a novel outdoor makerspace that engages families in STEM learning. A makerspace is a place where people work together on creative, interest driven projects. In working with BIPOC families, the project addresses three forms of historical (and present day) exclusion of community participants, including participation in the design of informal learning experiences, participation in such activities, and overall engagement in STEM. The project aims to develop activities that foster STEM learning using natural materials in an outdoor makerspace, informed through robust collaboration with local communities. This project will result in an outdoor makerspace at SMM that will include 3-4 settings (approximately 2500 square feet total) that house and support multiple making activities in an outdoor context. The proposed work will contribute to advancing knowledge through exploring how BIPOC families define learning in makerspaces and how younger children can be fully engaged in family learning. The project will share the inclusive design and community collaboration practices developed through this work with other museums, maker educators, and other community organizations that can develop or expand their own outdoor makerspaces in ways that will respect and reflect BIPOC families’ perspectives.
BIPOC families will join museum staff as contributors in the development and iteration of an outdoor makerspace and collaborators in the development of generalized design principles and dissemination of the research. Visitor-captured video of engagement in the outdoor makerspace, surveys, and memos from design meetings with community partners serve as the foundation for the process of aligning design and development of outdoor informal science education spaces with community needs and values. All research activities will be guided by a culturally responsive research framework and use strategies to ensure the multicultural validity such as video meaning-making with family research participants and member-checking instruments, data analyses, and findings with Design Partners. Project research will address three questions: (1) What are the characteristics of family learning in an outdoor nature-situated makerspace, including how BIPOC families identify and describe STEM learning and how outdoor spaces can be built to support BIPOC families’ perspectives? (2) How can the space be built to support multi-age families to engaged in making, including a focus on what design elements support preschool learner’s engagement and sustained participation by other family members? and (3) How do the design principles for making with widely available materials translate from indoor to outdoor spaces and materials? Research findings, design principles and community engagement guides will be widely disseminated to researchers, designers, program developers, informal science institutions and community organizations.
This award is funded in part under the American Rescue Plan Act of 2021 (Public Law 117-2).
This project will create the specification for a learner-controlled system to represent youth learning in Out-of-School-Time (OST) settings, to improve access to future Science, Technology, Engineering, and Mathematics (STEM) learning opportunities. For learners to pursue a STEM education, and STEM careers, they must be able to move through "gatekeeping" mechanisms that filter and sort students based on factors such as prior coursework and grades, teacher recommendations, and language proficiency assessments. Even though abundant evidence shows that such measures fail to capture all important aspects of STEM learning, they are traditionally relied upon in secondary and post-secondary STEM education contexts as indicators of preparation for future STEM learning. These systemic processes exclude certain minoritized groups, including Black, Indigenous, and other people of color (BIPOC), low income, immigrant and refugee youth, and youth learning English, from high-quality secondary and post-secondary STEM learning experiences because existing measures do not validate their prior knowledge and experiences. Yet, minoritized youth often engage in OST STEM learning opportunities, where their readiness for future learning opportunities is nurtured and valued. One challenge is to reliably document this readiness in a usable format so youth can access new STEM learning opportunities, especially in post-secondary contexts. This project builds strategically upon earlier work focusing on the democratization of STEM learning through vehicles such as digital micro-credentials or badges, and upon digital portfolios. Missing from these earlier efforts was integration of these platforms with an infrastructure that connected youth learners to OST STEM learning organizations and to future STEM learning opportunities. This Innovations in Development project brings together minoritized youth and their families, OST providers, and admissions officials from higher education institutions to explore the needed design features for OST "transcripts," and user stories that describe how software systems can support their creation and sharing. Grounded in the concept of mastery-based learning, where learning is demonstrated via action, learners will control what is included in the transcript so that they create their own narratives about their learning experiences. Recognizing that documentation is not the key focus of most STEM OST organizations, this project will provide direct support for identifying and codifying learning goals or outcomes that learners and their families find relevant and important within different STEM activities. This project is funded by the Advancing Informal STEM Learning (AISL) program which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments.
The project will take a Design-Based Implementation Research (DBIR) approach and proceed by convening representatives from three main stakeholder groups (youth and their families, OST providers, and admissions staff) to engage in a series of discovery and design activities. Project partners, including the Mastery Transcript Consortium (MA), STEAMville (IL), STUDIO (WA), and Wolverine Pathways (MI), will work together with the PIs to design templates learners can use to characterize STEM learning from each provider, aligned with different STEM learning foci (e.g., computer science, computational thinking, cross-cutting concepts, science and engineering practices, and mathematics). Data collected from these sessions will be used to address the following research questions: (1) How and why do youth and families from minoritized communities understand and choose to participate in STEM OST learning opportunities?, (2) How do youth understand and interact with STEM OST learning opportunities?, (3) How do OST providers characterize the STEM learning goals in the activities they provide?, and (4) How do college admissions personnel view the role of informal STEM learning as part of a holistic admissions process? This work has the potential to further the understanding of how OST learning can be documented and shared as a part of the larger ecosystem of STEM learning trajectories. By deeply engaging the perspectives and voices of minoritized youth and families, this project seeks to develop a valid and trustworthy instrument that recognizes and serves their STEM learning, thus broadening the participation of minoritized youth in STEM education and careers. This work will also benefit OST providers, by translating the documentation of youth STEM learning into forms that may help communicate the efficacy of their programs in ways that further their missions, including communicating evidence of effectiveness to both future participants and funders.
This Innovations in Development project is funded by the Advancing Informal STEM Learning program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants. Specifically, this project connects Native Hawaiian youth ages 12-17 and their family members to STEM by channeling their cultural relationship with ʻāina, the sustaining elements of the natural world including the land, sea, and air. This project seeks to: broaden participation of Native Hawaiian youth who have been historically underrepresented in STEM; actively uphold Native Hawaiian ways of knowing and traditional knowledge; articulate the science rooted in cultural wisdom; and bring STEM into the lives of participants as they connect to the ʻāina. In partnership with six ʻāina-based community organizations across Hawaiʻi, this project will develop, implement, and study ʻāina-centered environmental education activities that explore solutions to local environmental problems. For example, in one module youth and their families will explore of a section of a nearby stream; identify and discuss the native, non-native, and invasive species; remove invasive species from a small section of the stream and make observations leading to discussions of unintended consequences and systemic impacts; ultimately, learners will meet at additional local waterways to engage in similar explorations and discussions, transferring their knowledge to understanding the impacts of construction on local streams and coral reefs. To this effort, the community-based organizations bring their expertise in preserving Hawaiian culture and sustainable island lifestyle, including rural and urban systems such as farming and irrigation traditions and the restoration of cultural sites. University of Hawai’i faculty and staff bring expertise in Environmental Science, Biology, Hawaiian Studies and Problem-Based Learning Curriculum Development. This project further supports organizational learning and sharing among the six community-based organizations. Grounded in Hawaiian ʻAʻo, where learning and teaching are the same interaction, community-based organizations will create a Community of Practice that will co-learn Problem-Based Learning pedagogy; co-learn and engage in research and evaluation methods; and share experiential and traditional knowledge to co-develop the ʻāina-based environmental education activities.
This project is uniquely situated to study the impact of community-led culturally relevant pedagogy on Hawaiian learners’ interests and connections to environmental science, and to understand ʻāina-based learning through empirical research. Research methods draw on Community-Based Participatory Research and Indigenous Research Methods to develop a collaborative research design process incorporated into the project’s key components. Community members, researchers, and evaluators will work together to examine the following research questions: 1) How does environmental Problem-Based Learning situate within ʻāina-based informal contexts?; 2) What are the environmental education learning impacts of ʻāina-based activities on youth and family participants?; and 3) How does the ʻāina-centered Problem-Based Learning approach to informal STEM education support STEM knowledge, interest and awareness? The evaluation will employ a mixed-methods participatory design to explore program efficacy, fidelity, and implementation more broadly across community-based sites, as well as program sustainability within each community-based site. Anticipated project outcomes are a 15-week organizational learning and sharing program with six ʻāina-based community organizations and 72 staff; the design and implementation of 18 activities to reach 360 youth and at least one of their family members; and the launch of an ʻāina-based STEM Community of Practice. The project’s research and development process for ʻāina-centered environmental education activities will be shared broadly and provide a useful example for other organizations locally and nationally working in informal settings with Native or Indigenous populations.
Informal STEM education institutions seek to engage broader cross sections of their communities to address inequities in STEM participation and remain relevant in a multicultural society. In this chapter, we advance the role that evaluation can play in helping the field adopt more inclusive practices and achieve greater equity than at present through evaluation that addresses sociopolitical contexts and reflects the perspectives and values of non-dominant communities. To do this for specific projects, we argue that evaluation should privilege the voices and lived experiences of non-dominant
This guide offers insight into community engagement practices and activity development from our making and equity project, Making Connections. It includes documentation and recommendations for work that is designed to engage community partners as equal partners, and is written most of all for other practitioners.
As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understanding of deeper learning by participants. This pilot study, Akeakamai (Hawaiian, literally lover of wisdom, scientist, scholar), will explore the convergence of contemporary Western science topics with indigenous Hawaiian culture-based science experiences as a mechanism to strengthen STEM perceptions, cross-cultural science collaboration, and multi-generational community engagement with STEM. The work is grounded in the notion that STEM learning within the context of local informal indigenous community settings should be culturally responsive and culturally sustaining, and should privilege indigenous epistemologies. If successful, the results of this pilot could provide valuable insights on effective approaches to developing and implementing culturally consistent and sustainable multigenerational STEM engagement among Native Hawaiians and Pacific Islanders, and across the Pacific region.
Over a two-year duration, the study will address three research questions. (1) To what extent does inclusion of culture into curriculum designed for informal Culture-Science Explorations mitigate perceived barriers to participation in science? (2) What barriers do community members perceive to limit their participation in science? (3) What are the areas of consonance between Native Hawaiian and Western scientific approaches to knowledge and learning? Approximately 200 predominantly Native Hawaiian and Pacific Islanders, ranging in age from 8 - 85 years old, will participate in the pilot. The research team will collect participant data during all phases of the social intervention, a suite of culture-science exploration experiences held at the Halau Inana, a Native Hawaiian community collaboration space. The intervention will employ pedagogical methods that are responsive to Hawaiian cultural norms to deliver content that integrates across the interfaces of Western science and technology and indigenous knowledge, and incorporates Hawaiian language. A rigorous external evaluation will also be conducted. The results of the research and evaluation will be broadly disseminated. Ultimately, the project aims to develop a conceptual and practical cross-cultural, multi-generational framework for community-based science learning in Hawai'i that can serve as a model for future research and programs that extend into and beyond indigenous communities of the Pacific region.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
DATE:
-
TEAM MEMBERS:
Helen TurnerJonathan BakerChrystie Naeole