The independent evaluation firm Knight Williams, Inc. administered an online survey to educators from 16 SciGirls CONNECT2 partner organizations to gather information about their anticipated use of, reflections on, and recommendations relating to the draft updated SciGirls Strategies. The evaluation aimed for two educators from each partner organization – specifically the program leader and one educator who was familiar with the original SciGirls Seven strategies – to complete the survey about the draft updated strategies after they were shared by TPT in March 2018 via an online webinar and a
The independent evaluation firm, Knight Williams, Inc., administered an online survey and conducted follow-up interviews with educators from 14 SciGirls CONNECT2 partner organizations to gather information about their use of, reflections on, and recommendations relating to the SciGirls Seven strategies. The evaluation aimed for two educators from each partner organization – specifically the program leader and one educator who was familiar with the SciGirls Seven – to share reflections on the strategies after they completed their Year 1 programs. In all, 24 educators from 13 partners completed
The independent evaluators at Knight Williams Inc. developed a front-end survey to gather background and baseline information about the 16 partner organizations selected to conduct outreach programs as part of SciGirls CONNECT2. The goal was for two people from each partner organization to complete the online survey about their background and prior use of the SciGirls Seven and related strategies. A total of 30 partner representatives completed the survey by the requested deadline, resulting in a response rate of 94%. The majority identified as program leaders, with smaller groups saying they
The goal for this research study was to determine the role of the SciGirls gender-equitable strategies on participating youths’ STEM identity changes in 16 participating SciGirls’ programs across the nation. The definition of STEM identity was based on Eccles (2007), Carlone and Johnson (2007) and Calabrese Barton and colleagues (2013). According to these researchers, individuals must have a positive STEM identity in order to persist in STEM careers. This positive STEM identity is affected by an individual’s expectations of success in STEM and the value they see in STEM and STEM careers
Children Investigating Science with Parents and Afterschool (CHISPA) was a collaboration between the Phillip and Patricia Frost Museum of Science, UnidosUS (formerly National Council of La Raza), and the ASPIRA Association that took place from 2014-18. CHISPA sought to address the disparity in science achievement among Latino and non-Latino children through local-level partnerships between science museums in metropolitan areas with growing Latino populations and UnidosUS and ASPIRA affiliate organizations serving the same communities through afterschool programs.
Partners included the
Mentoring is a widely accepted strategy for supporting positive socioemotional and cognitive development across a variety of sectors including education, workforce development, and the justice system. An estimated 2.5 million volunteer mentors support youth development in the United States each year. However, there is broad concern that practice has outpaced empirical testing, with significant gaps in the research literature on important modifiers of mentoring relationships and their impacts. This is especially true for mentoring youth ages 10-14 in STEM. Studying highly successful programs may be one way to better understand the role of mentoring and moderators of mentoring effectiveness. The Science Club, a community-based STEM mentoring program for middle-grade youth in the Chicago area, will provide multiple sites for a research study to examine three important issues for advancing theory and practice for STEM mentoring. These issues include (1) understanding STEM mentoring for youth in the middle grades, (2) identifying outcomes and motivations for scientist mentors to more fully participate in mentoring programs, and (3) examining a model of middle-school-focused STEM mentoring collaboration.
Through a series of three studies, the team will investigate which elements of the mentoring relationships are associated with the demonstrated STEM identity gains in youth participants. The work will also contribute much-needed data on the impact of STEM mentoring relationships on the mentors themselves. Study 1 is designed as a retrospective study of program alumni, both youth and mentors, about the nature and extent of each their STEM identity shifts during their time in Science Club. A purposeful sample of 160+ youth and 100+ mentor alumni will participate. Study 2 is a prospective study of three consecutive cohorts of active Science Club participants, built on data and findings from Study 1. In Study 2, the team will design and implement a new Identity-Focused Mentoring Observation Instrument specifically aimed at exploring the nature and quality of mentoring relationships and their role in science identity development longitudinally. Three independent cohorts of 40 youth and 20 mentors each will participate. Study 3 is retrospective, examining how participating individuals and organizations perceive and are impacted by mentoring. The three studies employ a mixed methods approach utilizing surveys, observations, individual interviews, and document review.
This proposal will fill critical gaps in the mentoring literature regarding the formative middle school years through novel, empirical research. Building on the current literature and practice, outcomes of the work will inform practice and enhance knowledge-building in the field on both mentoring relationships and the collective impact of university-school-OST partnerships.
This project is funded by the National Science Foundation's (NSF's) Advancing Informal STEM Learning (AISL) program, which supports innovative research, approaches, and resources for use in a variety of learning settings.
Increased exposure to STEM content and career pathways during out-of-school time contexts can significantly extend STEM learning and aspirational interests among middle and high school youth. Using a collective impact approach, the STEM CareerLaunch pilot project tests the feasibility of redesigning a widely used, national youth and career focused program for and by the National Boys & Girls Clubs of America to extend STEM learning and promote awareness, interest, and readiness for STEM-related occupations among youth. STEM CareerLaunch integrates extant STEM programs, such as First Robotics, Girls Who Code and Jason Learning, with newly developed STEM content and opportunities to create and test a comprehensive STEM learning and career program for youth. The results of this pilot will inform a more expansive effort to bring STEM CareerLaunch to an already networked 4,000 Boys & Girls Clubs, reaching over four million youth from predominately underrepresented groups in STEM, and youth participating in other afterschool/summer program throughout the United States.
Approximately 100 youth and informal educators in Boys & Girls Clubs in Fitchburg and Leominster, Massachusetts will participate in this pilot feasibility study. A five-pronged approach will be instituted including: (1) high quality out of school time STEM programming, (2) connected STEM career education, (3) mentorship, (4) professional development for the informal educators, and (5) incentives such as internships and field experiences for youth participants. The developmental evaluation will focus on program implementation, participant outcomes, and scale-up. Data collection methods will include quantitative and qualitative approaches such as baseline student data, project tracking logs, retrospective surveys, focus groups, staff interviews, and observations. A summative evaluation will also be conducted.
This endeavor is led by a collaborative partnership between the National Boys & Girls Clubs of America, Worcester Polytechnic Institute STEM Center, the Fitchburg and Leominster Public School Districts, and others. It is primarily funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. It is also co-funded by the Innovative Technology Experiences for Students and Teachers (ITEST) program which is committed to better understanding and promoting practices that increase students' motivations and capacities to pursue careers in fields of science, technology, engineering, and or mathematics (STEM).
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
The role of afterschool programs in the science, technology, engineering, and mathematics (STEM) learning ecosystem has grown over the past two decades, which has led to increasing efforts to support and improve program quality. These efforts include developing STEM programs and curricula, creating standards for facilitating informal STEM learning experiences, building networks of support, and developing tools for assessment and evaluation. However, such efforts may have limited impact in terms of ongoing quality improvement. STEM curricula vary in disciplinary focus, quality and may not apply to local contexts and needs. Many afterschool programs resort to using simple STEM kits or online activities rather than rigorous curricula with support for educators. The project will study how the California Department of Education's (CDE) efforts to change organizational culture to support continuous quality improvement (CQI) have affected the offerings and quality of afterschool STEM in the state's more than 4,500 publicly funded afterschool sites. The EPISTEMIC project will contribute new research findings on how CQI can increase access to higher quality STEM learning opportunities for underserved youth. Even more important, the project will provide new insights on how organizational culture affects participation in and implementation of afterschool CQI.
The team will use an organizational theory framework and a mixed methods approach to conduct three research activities: (1) Describe the organizational context through interviews, participant observations, and artifact analysis to map and describe the overall support system as a context for understanding organizational culture change; (2) Describe change over time in organizational culture, CQI processes, and STEM program offerings and quality through surveys/interviews of afterschool youth, staff, directors, and grantee representatives; and (3) Generate explanations about the relationships between organizational culture, CQI, and STEM quality in different contexts through in depth case studies. Bringing organizational culture, CQI, and STEM offerings and quality into shared focus is the most important intellectual contribution of this work. Organizational theory's sensemaking concept will guide analyses to describe, exemplify, and generate theoretical explanations for patterns in organizational culture, CQI, and STEM program changes, with attention to relevant contextual factors.
Continuous quality improvement provides tools for afterschool STEM staff to identify needs and ways to improve. The EPISTEMIC study will contribute recommendations on the systemic, organizational, and cultural aspects of improvement strategies relevant to policymakers, funders, support providers, and afterschool organizations in California, as well as other state or nongovernmental support systems around the country. The study will also produce CQI guidelines for reflecting on and incorporating changes to organizational culture as part of CQI for afterschool staff and site directors. These will be helpful for practitioners around the country. The study's focus on three organizational contexts -- school district, national afterschool, and local afterschool -- will extend the relevance of the findings and recommendations, which will be disseminated through forums, workshops, and articles in practice and policy-oriented publications. The study will also benefit the research community by providing a framework and methods for studying organizational culture and CQI. The findings on the relationships between organizational culture, CQI, and STEM offerings and outcomes will provide a foundation for further research on how these relate to STEM learning outcomes for youth. EPISTEMIC is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants.
This project is funded by the National Science Foundation's (NSF's) Advancing Informal STEM Learning (AISL) program, which supports innovative research, approaches, and resources for use in a variety of learning settings.
DATE:
-
TEAM MEMBERS:
Patrik LundhAndrea BeesleyTimothy PodkulCarrie Allen
This project is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments.This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants. The project will conduct a feasibility study of an informal youth STEM learning program. High school students from under served communities in New York City will use existing historical, cultural and environmental data to investigate selected UNESCO World Heritage sites. Participants will apply the skills and knowledge they have developed from their analysis of the UNESCO sites and apply them to their local communities. Participants will identify, map, and analyze their own community heritage sites, using relevant citizen science, environmental and cultural data. Throughout the program, the project will involve participants in maker-related activities. Participants will design devices to collect data, explore variables through model making, and communicate findings through models and artistic forms with the to spur both individual and community action for selected heritage sites.
The project will be implemented as a 9-month weekly after school program in Long Island City, New York. Most students from the school will be from low-income families and are youth of color. The research the question for the study is "How does access to STEM increase for historically underrepresented youth populations when culturally relevant curriculum connects citizen science and making practices?" During the first phase of the program, participants will engage with core STEM concepts and making/design processes through an engaging curriculum that explores damaged UNESCO World Heritage Sites. During the second phase, youth will identify, map, and plan enhancements for their own community heritage sites or environmental landmarks. A condensed version of the program will be piloted in the summer with youth from across the city. The Educational Development Corporation will conduct a process and summative evaluation of the project. Process evaluation, which will provide ongoing feedback to the project team, will include document review, observation of program implementation, and interviews with project partners. Summative evaluation will continue these methods, supplemented by pre- and post-participation participant surveys and focus-groups. Validated survey instruments, such as the Growth Mindset Scale, and the Common Instrument Suite (PEAR Institute) will be used. Resources from research and program practices will be disseminated through publications and conference presentations to the education research community, global learning and design fields, and practitioners from after school and other informal learning environments. Participants will share project results with their communities.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
This Research in Service to Practice project will bring together representatives from six long-standing youth programs, experts in the field of out-of-school-time youth programming, and education researchers to collaboratively explore the long-term (15-25 years) impact of STEM-focused, intensive (100+ hours/year), multi-year programming. The six partnering programs have maintained records with a combined total of over 3000 alums who participated between 1995 and 2005. This four-year research project uses an explanatory, sequential, mixed-method design to carry out four steps: (1) identify and describe the impact on the lives of program alums who are now ages 30 to 45; (2) identify causal pathways from program strategies to long-term outcomes; (3) develop an understanding of these pathways from the perspective of the people who experienced them; and (4) disseminate this knowledge broadly to those associated with STEM-focused programming. Research questions include: How did these programs affect youth's lives as they progressed toward and into adulthood? What program strategies and what participant attributes contributed most to the staying power of these effects? What life events and social structures supported and inhibited participant outcomes? This project describes the effects, identifies the causal pathways, and produces materials that programs can use for both strategic planning and generating support resources. Additionally, this project provides research methodology for organizations that want to conduct their own retrospective research and lays a foundation for a more comprehensive study that includes programs without historical documentation. The project aligns with NSF's Big Idea "NSF INCLUDES: Transforming education and career pathways to help broaden participation in science and engineering" by providing essential information about the long-term effect of interventions on educational and career pathways in STEM.
The project's approach involves three phases: (1) research preparation, (2) causal structural modeling of survey data from approximately 2,000 respondents, and (3) rich qualitative follow-up. Human ecological and self-determination theories inform data collection and analyses at every project phase. In the preparation phase, program staff complete program profiles from an historic perspective by identifying program strategies that may have included, for example, scientific research, robotics development, teaching science in informal settings, and working in scientific research labs. In the quantitative phase, the project will recruit alums who attended one of the 6 youth programs between 1995 and 2005 to submit a current resume and complete an online questionnaire, based on the following scaled variables: retrospective recall of basic psychological need satisfaction and frustration in relation to perceived program strategies; STEM identity (at three time periods: pre-program; post-program; and current); current well-being; career influences; and career barriers. The questionnaire also includes open-ended questions about life events related to the following categories: family and friends, school and work, and living conditions. Analysis of the questionnaire will lead to development of a causal structural model. In the qualitative phase, data will be collected from a purposefully selected sample of 30 alums based on findings from the quantitative phase. Methods include interviews, photo journals, and STEM pathways maps. Analysis of interviews, resumes, and photo journals take place within the structure of basic psychological need satisfaction and motivational quality across ecological systems over time. Qualitative analysis uses the constant comparative method, and findings are used to update and refine the final causal structural model and inform overall findings, conclusions, and recommendations of the project.
Since the 1990s, out-of-school time programs have engaged youth from underserved communities in STEM learning and in building interest in STEM careers, yet these programs often based on untested assumptions that participation has lasting effects on education, career, and life choices related to STEM. This Research in Service to Practice project has the potential to 1) guide practitioners in program improvement and improved program outcomes; 2) provide insight into achieving program goals, such as equity, increased well-being of participants, an informed citizenry, and a diversified STEM workforce; and 3) inform multi-stakeholder decision-making with respect to this type of programming. This research also builds a foundation of research data collection and analysis methods to guide and support future research on long term-impacts and youth STEM programming. Dissemination strategies include a website, webinars, video, infographics, conference presentations, and written reports to reach stakeholders including practitioners, researchers, administrators, and funders.
This project is funded by the National Science Foundation's (NSF's) Advancing Informal STEM Learning (AISL) program, which supports innovative research, approaches, and resources for use in a variety of learning settings.
The PEAR Institute: Partnerships in Education and Resilience at McLean Hospital and Harvard Medical School conducted a year-long study of the Tulsa Regional STEM Alliance (TRSA). Funded by the Overdeck Family Foundation, STEM Next Opportunity Fund, and the Charles and Lynn Schusterman Family Foundation, this study is the first of its kind among 68 national and international STEM Ecosystems.
DATE:
TEAM MEMBERS:
Kristin Lewis-WarnerPatricia AllenGil Noam
An in-depth case study of one of America’s first STEM Learning Ecosystems in Tulsa, Oklahoma, conducted by researchers at The PEAR Institute: Partnerships in Education and Resilience, finds that strong leadership, deep partnerships, and data-informed methods have led to the creation of diverse, high-quality, STEM-rich learning opportunities for Tulsa’s youth. Additionally, these efforts improved the capacity of STEM educators through high-quality professional development and supported youth pathways to STEM careers by increasing mentoring opportunities for STEM professionals.
These findings
DATE:
TEAM MEMBERS:
Kristin Lewis-WarnerPatricia AllenGil Noam