In fall 2019, the Bell Museum received funding via a NASA TEAM II grant to create Mars: The Ultimate Voyage, a full-dome planetarium show and accompanying hands-on activities that focus on the interdisciplinary roles that will be needed to send humans to Mars. This report from Catalyst Consulting Group presents the findings from the summative evaluation completed in March–May 2023.
This study collected data from seven planetarium email lists (one per planetarium regional organization in the United States), as well as online survey panel data from residents in each area, to describe and compare those who do and do not visit planetariums.
This project engages pre-college Latinx, Black, and Indigenous learners, educators, and collaborating undergraduates in an international, project-based learning and media-making community in areas of science, technology, engineering, and mathematics (STEM). The project addresses key challenges including broadening participation in informal STEM learning, developing capacity for leading informal STEM programs, and building stronger connections between STEM learning and personal and social identity formation during adolescence. The project’s community of participants is an asset-based learning environment that treats each participant, their background, skills, and interests as uniquely beneficial to the whole. Led by mentors at each hub (teachers, leaders from science organizations, or other out-of-school learning environments), participants collaborate with peers from the US and from other countries. The collaborations encompass a broad spectrum of STEM projects. Participants also create digital media to communicate their projects. The project activities reflect a focus on STEM content, collaboration, and communication, in a global context that includes school-age learners from the US and peers from Central and South America, the Middle East, Asia, and Sub-Sahara. The combination of the sophisticated STEM competencies skills for collaborating across international and cultural boundaries, and media-savvy communication abilities are essential to the nation’s future STEM workforce and to building a scientifically vibrant citizenry.
The project addresses two primary research questions co-developed with teachers and other informal science providers. The first research question involves understanding and optimizing conditions for broadening participation through this type of distributed or virtual collaboration across boundaries of culture, race, gender, ability, nationality, and socioeconomic status. The project features a design experiment by which the overall community of participants comprises four separate hubs, each hosted by the different project partners (primarily teachers). Educators devise, test, and revise alternative designs for organizing STEM collaborations. Publication of these teacher-led designs and their evaluation are among the primary outputs of the project. The designs modify and improve a template developed under this project’s proof-of-concept precursor (NSF1612824). The second research question addresses how growth in STEM abilities, collaboration, and communication mutually reinforce adolescent personal and social identity formation. Participating students in the US will intentionally reflect heterogeneous backgrounds. The project analysis will focus on whether cultural and national cross-boundary collaboration can strengthen the development of learners' personal identity and academic performance. The project methodology relies heavily on quantitative ethnography and epistemic network analysis. This approach enables the creation of visual models that highlight the presence or absence of connections between constructs relevant to each research question, along with changes between and within groups. The constructs include variations of autonomy, competence, and connection (pillars of self-determination theory) in tracing identity formation and STEM abilities. The quantitative ethnography approach provides statistically reliable scaffolding and insights about the hub designs and their efficacy in promoting goals of broadening participation and fostering mutually reinforcing STEM competencies and identity formation. This type of virtual collaboration, crossing boundaries of culture, nationality, ethnicity, age, gender, economic strata, or ability, can realistically be expected to play a significant role in next-generation learning environments, especially through out-of-school activities. The project is expected to reach 120 U.S. and 80 non-U.S. students annually. Research findings, design principles and curricula will be widely disseminated to researchers, designers, program developers, informal science institutions and community organizations.
DATE:
-
TEAM MEMBERS:
Eric HamiltonNastassia JonesDanielle EspinoSeung Lee
Increasing the diversity of the Science, Technology, Engineering, and Mathematics (STEM) workforce hinges on understanding the impact of the many related, pre-college experiences of the nation’s youth. While formal preparation, such as high school course-taking, has a major influence, research has shown that out-of-school-time activities have a much larger role in shaping the attitudes, identity, and career interests of students, particularly those who are members of groups historically underrepresented in STEM fields (Black, Indigenous, Latinx, and/or Pacific Islander). A wide range of both innovative adult-led (science clubs, internships, museum-going, competitions, summer camps) and personal-choice (hobbies, family talk, games, simulations, social media, online courses) options exist. This project studies the variety and availability such experiences to pre-college students. The project is particularly interested in how community cultural capital is leveraged through informal activities and experiences, drawing upon the “funds of knowledge” that culturally diverse students bring to their STEM experiences (e.g., high aspirations, multilingual facility, building of sustaining social networks, and the capacity to challenge negative stereotyping). This study has the capability to begin to reveal evidence-based measures of the absolute and relative effectiveness of promising informal educational practices, including many developed and disseminated by NSF-funded programs. Understanding the ecology of precollege influencers and the hypotheses on which they are based, along with providing initial measures of the efficacy of multiple pathways attempting to broaden participation of students from underrepresented groups in STEM majors and careers, will aid decision-making that will maximize the strategic impact of federal and local efforts.
The project first collects hypotheses from the wide variety of stakeholders (educators, researchers, and students) about the kinds of experiences that make a difference in increasing students’ STEM identity and career interest. Identifying the descriptive attributes that characterize opportunities across individual programs and validating a multi-part instrument to ascertain student experiences will be carried out through a review of relevant literature, surveying stakeholders using crowdsourced platforms, and through in-depth interviews with 50 providers. A sample of 1,000 students from 2- and 4-year college and universities, drawn from minority-serving institutions, such as Historically Black Colleges, Hispanic Serving Institutions, and Tribal Colleges and Universities will serve to establish the validity and reliability of the derived instrument and provide estimates of the availability and frequency of involvement. Psychometric methods and factor analysis will guide us in combining related variables into indices that reflect underlying constructs. Propensity score weighting will be employed for estimating effects when exposure to certain OST activities is confounded with other factors (e.g., parental education, SES). Path models and structural equation models (SEM) will be employed to build models that use causal or time related variables, for instance, students’ career interests at different times in their pre-college experience. The study goes beyond evaluation of individual experiences in addressing important questions that will help policy makers, educators, parents, and students understand which OST opportunities serve the diverse values and goals of members of underrepresented groups, boosting their likelihood of pursuing STEM careers. This project is co-funded by the Advancing Informal STEM Learning (AISL) and EHR CORE Research (ECR) programs.
The Science Museum of Minnesota (SMM) will collaborate with four community organizations serving Black, Indigenous, and People of Color (BIPOC) audiences to research and develop a novel outdoor makerspace that engages families in STEM learning. A makerspace is a place where people work together on creative, interest driven projects. In working with BIPOC families, the project addresses three forms of historical (and present day) exclusion of community participants, including participation in the design of informal learning experiences, participation in such activities, and overall engagement in STEM. The project aims to develop activities that foster STEM learning using natural materials in an outdoor makerspace, informed through robust collaboration with local communities. This project will result in an outdoor makerspace at SMM that will include 3-4 settings (approximately 2500 square feet total) that house and support multiple making activities in an outdoor context. The proposed work will contribute to advancing knowledge through exploring how BIPOC families define learning in makerspaces and how younger children can be fully engaged in family learning. The project will share the inclusive design and community collaboration practices developed through this work with other museums, maker educators, and other community organizations that can develop or expand their own outdoor makerspaces in ways that will respect and reflect BIPOC families’ perspectives.
BIPOC families will join museum staff as contributors in the development and iteration of an outdoor makerspace and collaborators in the development of generalized design principles and dissemination of the research. Visitor-captured video of engagement in the outdoor makerspace, surveys, and memos from design meetings with community partners serve as the foundation for the process of aligning design and development of outdoor informal science education spaces with community needs and values. All research activities will be guided by a culturally responsive research framework and use strategies to ensure the multicultural validity such as video meaning-making with family research participants and member-checking instruments, data analyses, and findings with Design Partners. Project research will address three questions: (1) What are the characteristics of family learning in an outdoor nature-situated makerspace, including how BIPOC families identify and describe STEM learning and how outdoor spaces can be built to support BIPOC families’ perspectives? (2) How can the space be built to support multi-age families to engaged in making, including a focus on what design elements support preschool learner’s engagement and sustained participation by other family members? and (3) How do the design principles for making with widely available materials translate from indoor to outdoor spaces and materials? Research findings, design principles and community engagement guides will be widely disseminated to researchers, designers, program developers, informal science institutions and community organizations.
Making as a term has gained attention in the educational field. It signals many different meanings to many different groups, yet is not clearly defined. This project’s researchers refer to making as a term that bears social and cultural impact but with a broader more sociocultural association than definitions that center making in STEM learning. Using the theoretical lenses of critical relationality and embodiment, our research team position curriculum as a set of locally situated activities that are culturally, linguistically, socially, and politically influenced. We argue that curriculum
DATE:
TEAM MEMBERS:
Veronica OguilveWen WenEm BowenYousra AbourehabAmanda BermudezElizabeth GaxiolaJill Castek
resourceresearchProfessional Development, Conferences, and Networks
The ChemAttitudes project recieved supplemental funding to create materials for train-the-trainer workshops in order to inoculate the chemistry outreach community with members who have the knowledge and resources to train others on strategies for stimulating interest, sense of relevance, and feelings of self-efficacy that were tested in the earlier work of the project. The project team recruited participants from minority serving professional organizations as a strategy for broadening participation. Can it work? Did it work?
This poster was presented at the 2021 NSF AISL Awardee Meeting.
To advance justice, equity, diversity, and inclusion in science, we must first understand and improve the dominant-culture frameworks that impede progress and, second, we must intentionally create more equitable models. The present authors call ourselves the ICBOs and Allies Workgroup (ICBOs stands for independent community-based organizations), and we represent communities historically excluded from the sciences. Together with institutional allies and advisors, we began our research because we wanted our voices to be heard, and we hoped to bring a different perspective to doing science with
DATE:
TEAM MEMBERS:
María Cecilia Alvarez RicaldeJuan Flores ValadezCatherine CrumJohn AnnoniRick BonneyMateo Luna CastelliMarilú López FrettsBrigid LuceyKaren PurcellJ. Marcelo BontaPatricia CampbellMakeda CheatomBerenice RodriguezYao Augustine FoliJosé GonzálezJosé Miguel Hernández HurtadoSister Sharon HoraceKaren KitchenPepe Marcos-IgaTanya SchuhPhyllis Edwards TurnerBobby WilsonFanny Villarreal
ChemAttitudes: Using Design-Based Research to Develop and Disseminate Strategies and Materials to Support Chemistry Interest, Relevance, and Self-Efficacy (ChemAttitudes, NSF DRL-1612482) is a collaborative project between the Museum of Science, Boston (MOS), the National Informal STEM Education Network (NISE Net), and the American Chemical Society (ACS) among others. As a part of this project, researchers and educators from MOS and the Science Museum of Minnesota (SMM) used design-based research to create chemistry hands-on activities meant to positively impact visitors’ attitudes towards
This project will draft a framework to guide citizen science projects in addressing issues of equity, diversity, and inclusion (EDI). Citizen science, sometimes called community science, involves volunteers who use science research procedures to collect valid scientific data for research projects and who often learn much about science in the process. These projects contribute directly to scientific research and often collect data of direct relevance to many communities. Although there are millions of citizen science volunteers, only a small proportion come from marginalized communities. The project will host a series of six, half-day virtual (online) workshops with scholars and practitioners with deep understanding of the participatory sciences and issues related to EDI. Workshop participants will discuss topics relevant to preparing a framework to provide guidance for integrating support EDI practices in citizen science. The project will disseminate the framework and workshop recommendations through publications for researchers and practitioners, a new website that will serve as a hub for relevant resources and EDI professional development, blogposts, and webinars.
This project will focus on EDI issues in institution-led, large-scale, citizen science projects. The project will organize workshops addressing issues relating to: (1) designing multipurpose projects that can be useful for empowering communities with data addressing community needs, providing researchers a large and robust data set, and providing learners with opportunities to develop a deeper understanding of research; (2) developing diverse leadership and engaging marginalized communities in framing research priorities; and (3) supporting strategies across citizen science projects to address barriers to participation, identity professional development needs, and create inclusive models that foster trust, create supportive networks, and build capacity for EDI in citizen science. The workshop will include approximately 20 participants, including researchers, project leaders and practitioners, with a majority of workshop participants belonging to groups underrepresented in science, such as Black, Hispanic, and Indigenous people.
This award is funded in part under the American Rescue Plan Act of 2021 (Public Law 117-2).
This project will create the specification for a learner-controlled system to represent youth learning in Out-of-School-Time (OST) settings, to improve access to future Science, Technology, Engineering, and Mathematics (STEM) learning opportunities. For learners to pursue a STEM education, and STEM careers, they must be able to move through "gatekeeping" mechanisms that filter and sort students based on factors such as prior coursework and grades, teacher recommendations, and language proficiency assessments. Even though abundant evidence shows that such measures fail to capture all important aspects of STEM learning, they are traditionally relied upon in secondary and post-secondary STEM education contexts as indicators of preparation for future STEM learning. These systemic processes exclude certain minoritized groups, including Black, Indigenous, and other people of color (BIPOC), low income, immigrant and refugee youth, and youth learning English, from high-quality secondary and post-secondary STEM learning experiences because existing measures do not validate their prior knowledge and experiences. Yet, minoritized youth often engage in OST STEM learning opportunities, where their readiness for future learning opportunities is nurtured and valued. One challenge is to reliably document this readiness in a usable format so youth can access new STEM learning opportunities, especially in post-secondary contexts. This project builds strategically upon earlier work focusing on the democratization of STEM learning through vehicles such as digital micro-credentials or badges, and upon digital portfolios. Missing from these earlier efforts was integration of these platforms with an infrastructure that connected youth learners to OST STEM learning organizations and to future STEM learning opportunities. This Innovations in Development project brings together minoritized youth and their families, OST providers, and admissions officials from higher education institutions to explore the needed design features for OST "transcripts," and user stories that describe how software systems can support their creation and sharing. Grounded in the concept of mastery-based learning, where learning is demonstrated via action, learners will control what is included in the transcript so that they create their own narratives about their learning experiences. Recognizing that documentation is not the key focus of most STEM OST organizations, this project will provide direct support for identifying and codifying learning goals or outcomes that learners and their families find relevant and important within different STEM activities. This project is funded by the Advancing Informal STEM Learning (AISL) program which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments.
The project will take a Design-Based Implementation Research (DBIR) approach and proceed by convening representatives from three main stakeholder groups (youth and their families, OST providers, and admissions staff) to engage in a series of discovery and design activities. Project partners, including the Mastery Transcript Consortium (MA), STEAMville (IL), STUDIO (WA), and Wolverine Pathways (MI), will work together with the PIs to design templates learners can use to characterize STEM learning from each provider, aligned with different STEM learning foci (e.g., computer science, computational thinking, cross-cutting concepts, science and engineering practices, and mathematics). Data collected from these sessions will be used to address the following research questions: (1) How and why do youth and families from minoritized communities understand and choose to participate in STEM OST learning opportunities?, (2) How do youth understand and interact with STEM OST learning opportunities?, (3) How do OST providers characterize the STEM learning goals in the activities they provide?, and (4) How do college admissions personnel view the role of informal STEM learning as part of a holistic admissions process? This work has the potential to further the understanding of how OST learning can be documented and shared as a part of the larger ecosystem of STEM learning trajectories. By deeply engaging the perspectives and voices of minoritized youth and families, this project seeks to develop a valid and trustworthy instrument that recognizes and serves their STEM learning, thus broadening the participation of minoritized youth in STEM education and careers. This work will also benefit OST providers, by translating the documentation of youth STEM learning into forms that may help communicate the efficacy of their programs in ways that further their missions, including communicating evidence of effectiveness to both future participants and funders.