In fall 2019, the Bell Museum received funding via a NASA TEAM II grant to create Mars: The Ultimate Voyage, a full-dome planetarium show and accompanying hands-on activities that focus on the interdisciplinary roles that will be needed to send humans to Mars. This report from Catalyst Consulting Group presents the findings from the summative evaluation completed in March–May 2023.
This project is designed to support collaboration between informal STEM learning (ISL) researchers, designers, and educators with sound researchers and acoustic ecologists to jointly explore the role of auditory experiences—soundscapes—on learning. In informal STEM learning spaces, where conversation advances STEM learning and is a vital part of the experience of exploring STEM phenomena with family and friends, attention to the impacts of soundscapes can have an important bearing on learning. Understanding how soundscapes may facilitate, spark, distract from, or even overwhelm thinking and conversation will provide ISL educators and designers evidence to inform their practice. The project is structured to reflect the complexity of ISL audiences and experiences; thus, partners include the North Park Village Nature Center located in in a diverse immigrant neighborhood in Chicago; Wild Indigo, a Great Lakes Audubon program primarily serving African American visitors in Midwest cities; an after-school/summer camp provider, STEAMing Ahead New Mexico, serving families in the rural southwest corner of New Mexico, and four sites in Ohio, MetroParks, Columbus Zoo and Aquarium, Franklin Park Conservatory and Botanical Gardens, and the Center of Science and Industry.
Investigators will conduct large-scale exploratory research to answer an understudied research question: How do environmental sounds impact STEM learning in informal learning spaces? Researchers and practitioners will characterize and describe the soundscapes throughout the different outdoor and indoor exhibit/learning spaces. Researchers will observe 800 visitors, tracking attraction, attention, dwell time, and shared learning. In addition to observations, researchers will join another 150 visitors for think-aloud interviews, where researchers will walk alongside visitors and capture pertinent notes while visitors describe their experience in real time. Correlational and cluster analyses using machine learning algorithms will be used to identify patterns across different sounds, soundscapes, responses, and reflections of research participants. In particular, the analyses will identify characteristics of sounds that correlate with increased attention and shared learning. Throughout the project, a team of evaluators will monitor progress and support continuous improvement, including guidance for developing culturally responsive research metrics co-defined with project partners. Evaluators will also document the extent to which the project impacts capacity building, and influences planning and design considerations for project partners. This exploratory study is the initial in a larger research agenda, laying the groundwork for future experimental study designs that test causal claims about the relationships between specific soundscapes and visitor learning. Results of this study will be disseminated widely to informal learning researchers and practitioners through workshops, presentations, journal articles, facilitated conversations, and a short film that aligns with the focus and findings of the research.
DATE:
-
TEAM MEMBERS:
Martha MersonJustin MeyerDaniel Shanahan
This poster was presented at the 2021 NSF AISL Awardee Meeting.
Collaborative robots – cobots – are designed to work with humans, not replace them. What learning affordances are created in educational games when learners program robots to assist them in a game instead of being the game? What game designs work best?
Many Black youth in both urban and rural areas lack engaging opportunities to learn mathematics in a manner that leads to full participation in STEM. The Young People’s Project (YPP), the Baltimore Algebra Project (BAP), and the Education for Liberation Network (EdLib) each have over two decades of experience working on this issue. In the city of Baltimore, where 90% of youth in poverty are Black, and only 5% of these students meet or exceed expectations in math, BAP, a youth led organization, develops and employs high school and college age youth to provide after-school tutoring in Algebra 1, and to advocate for a more just education for themselves and their peers. YPP works in urban or rural low income communities that span the country developing Math Literacy Worker programs that employ young people ages 14-22 to create spaces to help their younger peers learn math. Building on these deep and rich experiences, this Innovations in Development project studies how Black students see themselves as mathematicians in the context of paid peer-to-peer math teaching--a combined social, pedagogical, and economic strategy. Focusing primarily in Baltimore, the project studies how young people grow into new self-definitions through their work in informal, student-determined math learning spaces, structured collaboratively with adults who are experts in both mathematics and youth development. The project seeks to demonstrate the benefits of investing in young people as learners, teachers, and educational collaborators as part of a core strategy to improve math learning outcomes for all students.
The project uses a mixed methods approach to describe how mathematical identity develops over time in young people employed in a Youth-Directed Mathematics Collaboratory. 60 high school aged students with varying mathematical backgrounds (first in Baltimore and later in Boston) will learn how to develop peer- and near-peer led math activities with local young people in informal settings, after-school programs, camps, and community centers, reaching approximately 600 youth/children. The high school aged youth employed in this project will develop their own math skills and their own pedagogical skills through the already existing YPP and BAP structures, made up largely of peers and near-peers just like themselves. They will also participate in on-going conversations within the Collaboratory and with the community about the cultural significance of doing mathematics, which for YPP and BAP is a part of the ongoing Civil Rights/Human Rights movement. Mathematical identity will be studied along four dimensions: (a) students’ sequencing and interpretation of past mathematical experiences (autobiographical identity); (b) other people’s talk to them and their talk about themselves as learners, doers, and teachers of mathematics (discoursal identity); (c) the development of their own voices in descriptions and uses of mathematical knowledge and ideas (authorial identity); and (d) their acceptance or rejection of available selfhoods (socio-culturally available identity). Intended outcomes from the project include a clear description of how mathematical identity develops in paid peer-teaching contexts, and growing recognition from both local communities and policy-makers that young people have a key role to play, not only as learners, but also as teachers and as co-researchers of mathematics education.
This Innovations in Development project is funded by the Advancing Informal STEM Learning (AISL) program.
DATE:
-
TEAM MEMBERS:
Jay GillenMaisha MosesThomas NikundiweNaama LewisAlice Cook
Described by Wohlwend, Peppler, Keune and Thompson (2017) as “a range of activities that blend design and technology, including textile crafts, robotics, electronics, digital fabrication, mechanical repair or creation, tinkering with everyday appliances, digital storytelling, arts and crafts—in short, fabricating with new technologies to create almost anything” (p. 445), making can open new possibilities for applied, interdisciplinary learning in science, technology, engineering and mathematics (Martin, 2015), in ways that decenter and democratize access to ideas, and promote the construction
DATE:
TEAM MEMBERS:
Jill CastekMichelle Schira HagermanRebecca Woodland
The U.S. Fish and Wildlife Service estimates that over 41 million people connect to nature through birding. Learning about birds in their natural environments offers opportunities for informal engagement in STEM by a broad range of individuals and groups. Birders often engage in scientific data gathering and analyses, geolocation and remote sensing, and phenology. They also become aware of ecological changes in bird habitats and migratory patterns due to rising temperatures and climate-related events like sea level rise, droughts, fires, and extreme weather. As such, the birding community is an ideal network to better understand and communicate the impacts of climatological changes on bird populations to the public. With this Innovations in Development project, the National Audubon Society will develop a new avian-focused, conservation and climate science community science curriculum for its Nature Centers, and test the effectiveness of the curriculum in educating the public about avian-focused conservation and climatological changes through guided nature experiences. Birding can serve as a pivotal entrée for young people into STEM fields and careers. Through its programs and partnerships, Audubon will leverage its national network to ensure that through this project a more diverse group of voices, particularly young adults and young adults of color, become involved in asking critical questions and developing solutions to address important environmental issues of the future. If successful, the broader impacts of this project on capacity building and public engagement could be far-reaching and long-lasting.
Over the three-year project duration, Audubon will bring educators from its nationwide network of thirty-four Nature Centers (including urban, suburban, and rural sites), together with over 510 young adults (ages 18-25) from its network of college campus chapters. An evidence-based curriculum and community science activities will be created and tested, relying heavily on a team of experts in ornithology, climate science research, STEM curriculum design, diversity, and informal science education. College students will advise on the design of content and activities to effectively interest and engage young adults. These students will be recruited from the new Audubon Campus Chapters Program, which includes 111 college and university campuses, among them, 19 Historically Black Colleges and Universities (HBCUs) and other Minority Serving Institutions (MSIs). The target population will be surveyed to also understand their current and likely participation in guided nature experiences and knowledge base in climate science. Current best practices in guided nature experiences will be gathered from across the Audubon network. The implementation efforts will result in a national STEM model, with train-the-trainer guides and workshops for informal science educators and public engagement opportunities focused on improving the state and condition of avian habitats and communities through climate science research. An external evaluation will be conducted and will include data collection methods such as retrospective pre and post surveys, semi-structured interviews, focus groups, and an embedded assessment to determine impact. The findings will be used to iteratively refine the evidence-based curriculum and measure STEM learning outcomes for the guided nature experience participants. The evaluation will address four areas: (1) fidelity of program implementation to promote accountability; (2) formative evaluation to understand needs and interests of young adults (ages 18-25), and subsequently inform program design; (3) outcomes for Center educators, to inform iterative improvement; and (4) outcomes for program participants, to contribute to the growing knowledge base on effective practices for STEM learning in informal settings.
This Innovations in Development project is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
DATE:
-
TEAM MEMBERS:
Loren SmithMark ScallionHeather Starck
The independent evaluation firm Knight Williams, Inc. conducted a formative evaluation during Year 2 of the SciGirls CONNECT2 program in order to gather information about the partner educators’ use of, reflections on, and recommendations relating to the draft updated SciGirls Strategies. The evaluation aimed for two educators from each of 14 partner organizations – specifically the program leader and one educator who was familiar with the original SciGirls Seven – to provide reflections on their use of the draft SciGirls Strategies in their programs through an online survey and follow-up
This project will examine the characteristics and outcomes of a large sample of environmental education field trip programs for youth to elucidate program characteristics that most powerfully influence 21st century learning outcomes. Environmental education programs for youth, particularly day-long school trip programs, are popular and reside at the intersection of formal and informal STEM education. Such field trips provide opportunities for diverse audiences to participate in shared learning experiences, but current understanding of what leads to success in these programs is limited. This large-scale study will address this gap in knowledge by investigating the linkages between program characteristics and participant outcomes for at least 800 single-day environmental education field trip programs for youth in grades 5-8, particularly programs for diverse and underserved audiences. This study will result in the identification of evidence-based practices that will inform future program design for a wide variety of settings, including nature centers, national parks, zoos, museums, aquaria, and other locations providing informal environmental education programs.
This Research in Service to Practice study is guided by two research questions: 1) What program characteristics (context, design, and delivery) most powerfully influence learner self-determination and learner outcomes? And 2) Do the most influential program characteristics differ across diverse and underserved audiences (e.g. African American, Hispanic/Latino, economically disadvantaged) and contexts (e.g. rural versus urban)? This project will examine a wide range of program-related factors, including pedagogical approaches and contextual characteristics. A valid and reliable protocol for observing 78 program characteristics hypothesized to influence learner outcomes developed by a previous project will be used to systematically sample and observe 500 single-day environmental education field trip programs for youth in grades 5-8 distributed across at least 40 U.S. states and territories. Programs for diverse and underserved youth will be emphasized, and a diverse set of programs in terms of program type and context will be sought. Data from this sample will be combined with those of an existing sample of 334 programs provided by over 90 providers. The final combined sample of over 800 programs will provide sufficient statistical power to confidently identify which program components are most consistently linked with learning outcomes. This sample size will also enable stratification of the sample for examination of these relationships within relevant subpopulations. Principal component analyses will be used to reduce data in theoretically meaningful and statistically valid ways, and multilevel structural equation modeling will be employed to examine the influences of both participants' individual characteristics and program and context characteristics on participant outcomes. Since one research question focuses on whether program outcomes are the same across different audiences, the project will include at least 200 programs for each of three specific audiences to ensure sufficient statistical power for confidence in the results: primarily African American, primarily Hispanic/Latino, and primarily White.
This project is funded by the National Science Foundation's (NSF's) Advancing Informal STEM Learning (AISL) program, which supports innovative research, approaches, and resources for use in a variety of learning settings.
DATE:
-
TEAM MEMBERS:
Robert PowellMarc SternBrandon Frensley
This project will advance efforts of the Innovative Technology Experiences for Students and Teachers (ITEST) program to better understand and promote practices that increase students' motivations and capacities to pursue careers in fields of science, technology, engineering, or mathematics (STEM) by bringing together youth (grades 2-5), their families, librarians, and professional engineers in an informal environment centered on engaging youth with age-appropriate, technology-rich STEM learning experiences fundamental to the engineering design process. The overarching aim is to better understand how youth's learning preferences or dispositions relate to their STEM learning experiences. It also seeks to build community members' capacity to inspire and educate youth about STEM careers. The project team includes the Space Science Institute's (SSI) National Center for Interactive Learning (NCIL), the University of Virginia (UVA) and the American Society of Civil Engineers (ASCE). This team builds on the scope and reach of a prior NSF-funded project called the STAR Library Education Network (STAR_Net). As an extension of this prior work, Project BUILD will collaborate with 6 public libraries (3 urban and 3 rural) and their local ASCE Branches. Two libraries have been selected to serve as pilots: High Plains Public Library in Colorado and the African-American Research Library and Cultural Center in Florida. All partner libraries will develop a plan for recruiting participants from groups currently underrepresented in STEM professions. Project BUILD's specific aims are to 1) Engage underserved audiences, 2) Build the capacity of participating librarians and ASCE volunteers, 3) Increase interest and engagement in STEM activities for youth in grades 2-5 and their families, and 4) Conduct a comprehensive education research project. Program components include the following: 1) Community Dialogue Events, 2) a Professional Development Program for partner librarians and ASCE volunteers, and 3) Development of a Technology-rich Programming Kit and Circulating STEM Kit program. Two research questions will be addressed: 1) What common factors might identify youth who engage in project activities and what factors might differentiate between youth who continue with program engagement and those who do not? and 2) What programmatic factors (i.e. design and composition of program activities, library recruitment, librarian engagement, professional engineer engagement, etc.) might influence youth's initial and continued engagement in project activities as well as youth's reported future career interests? An external evaluation will investigate the quality of the project's process as well as its impact and effectiveness. Benefits to the participating libraries' communities, library and engineering professionals, and the education community will be achieved through 1) Community Dialogue events; 2) Library and Librarian Outreach; 3) ASCE Outreach; and 4) Publication of Research and Evaluation results.