In 2017, Concord Evaluation Group (CEG) conducted a summative evaluation of Design Squad Global (DSG). DSG is produced and managed by WGBH Educational Foundation. WGBH partnered with FHI 360, a nonprofit human development organizations working in 70 countries, to implement DSG around the globe.
In the DSG program, children in afterschool and school clubs explored engineering through hands-on activities, such as designing and building an emergency shelter or a structure that could withstand an earthquake. Through DSG, children also had the chance to work alongside a partner club from another
The American Museum of Natural History (AMNH), in collaboration with New York University's Institute for Education and Social Policy and the University of Southern Maine Center for Evaluation and Policy, will develop and evaluate a new teacher education program model to prepare science teachers through a partnership between a world class science museum and high need schools in metropolitan New York City (NYC). This innovative pilot residency model was approved by the New York State (NYS) Board of Regents as part of the state’s Race To The Top award. The program will prepare a total of 50 candidates in two cohorts (2012 and 2013) to earn a Board of Regents-awarded Masters of Arts in Teaching (MAT) degree with a specialization in Earth Science for grades 7-12. The program focuses on Earth Science both because it is one of the greatest areas of science teacher shortages in urban areas and because AMNH has the ability to leverage the required scientific and educational resources in Earth Science and allied disciplines, including paleontology and astrophysics.
The proposed 15-month, 36-credit residency program is followed by two additional years of mentoring for new teachers. In addition to a full academic year of residency in high-needs public schools, teacher candidates will undertake two AMNH-based clinical summer residencies; a Museum Teaching Residency prior to entering their host schools, and a Museum Science Residency prior to entering the teaching profession. All courses will be taught by teams of doctoral-level educators and scientists.
The project’s research and evaluation components will examine the factors and outcomes of a program offered through a science museum working with the formal teacher preparation system in high need schools. Formative and summative evaluations will document all aspects of the program. In light of the NYS requirement that the pilot program be implemented in high-need, low-performing schools, this project has the potential to engage, motivate and improve the Earth Science achievement and interest in STEM careers of thousands of students from traditionally underrepresented populations including English language learners, special education students, and racial minority groups. In addition, this project will gather meaningful data on the role science museums can play in preparing well-qualified Earth Science teachers. The research component will examine the impact of this new teacher preparation model on student achievement in metropolitan NYC schools. More specifically, this project asks, "How do Earth Science students taught by first year AMNH MAT Earth Science teachers perform academically in comparison with students taught by first year Earth Science teachers not prepared in the AMNH program?.”
DATE:
-
TEAM MEMBERS:
Maritza MacdonaldMeryle WeinsteinRosamond KinzlerMordecai-Mark Mac LowEdmond MathezDavid Silvernail
Many urban universities offer precollege STEM programs aimed at broadening participation in STEM. These programs are designed to increase students' scientific content knowledge and skills, promote STEM engagement, increase self-efficacy, and prepare underserved and underrepresented minority high school students for success in undergraduate programs. However, even after demonstrating significant knowledge gains and success in these programs, students are often unable to authenticate their knowledge gains to receive favorable consideration on college applications. In fact, there is currently no systematic credentialing mechanism to assess and validate the scientific rigor of and competency gains within STEM precollege programs for college admissions purposes. This NSF INCLUDES Design and Development Launch Pilot seeks to address this gap by developing and testing credentialing and badging processes for four STEM precollege programs. Working with College Admissions Officers and project partners, the University of Pittsburgh endeavors to employ a collaborative impact approach to build and document the collaborative infrastructure needed to support STEM precollege program authentication processes. This will seed the development of a networked improvement community that supports all aspects of the work from participant support to the collaborative impact to the greater network of urban education university ecosystems involved in the pilot.
Over a two-year period, this pilot will examine a mechanism to strengthen the STEM pathway for disadvantaged and underrepresented minority students to enter postsecondary STEM programs and eventually STEM careers. Building on two social innovation theories, the technical approach will focus on four specific aims: (1) create a community engagement framework to recruit underserved and underrepresented high school students to STEM precollege programs, (2) develop a STEM Success Matrix that identifies student competencies acquired in precollege programs that prepare students for collegiate success in STEM, (3) credential precollege programs using the STEM Success Matrix, and (4) develop a student badging system for precollege program participants using the STEM Success Matrix. Four University of Pittsburgh STEM precollege programs will serve as the context for development and testing with support from a range of partners representing the broader Pittsburgh STEM ecosystem. Approximately 300 high school students are expected to participate in the pilot, across the four precollege programs. Data will be collected via participant surveys and interviews. Formative and summative evaluations will be conducted by an experienced, external evaluator. Shareable metrics, tools, and processes will be developed and disseminated using various platforms and mechanisms. If successful, this pilot could be transformative - changing admissions considerations by credentialing STEM precollege programs and increasing student interest and motivation in STEM through student badging. It would also transform the STEM ecosystem of underserved and underrepresented minority students by creating an important STEM pathway from precollege to undergraduate admissions and ultimately, STEM Careers. This pilot could serve as a baseline for a more expansive alliance with other urban education ecosystems or assisting others interested in establishing their own collaborative infrastructure and networked improvement community model to achieve similar results.
DATE:
-
TEAM MEMBERS:
Alison Slinskey LeggJennifer IritiDavid BooneAlaine AllenLori Delale-O'Connor
resourceprojectProfessional Development, Conferences, and Networks
Aligning for Impact: Computer Science Pathways Across Contexts [CS-PAC] is an NSF INCLUDES Design and Development Launch Pilot. It broadens participation of students who are underrepresented in computer science by using the convening and policy-making power of the Georgia State Department of Education to coalesce school district leaders to implement K-12 computer science education. The project provides a national model for how to work toward systemic change. With the State Department of Education's coordination, several school districts will collaboratively seek improvements in their own student participation rates. The coordination of data reporting and analysis, resources, communications, and policy promote more equitable participation in computer science education. Research emerging from this project informs other states about how to collaboratively shape computer science education policy and policy implementation.
Using a Collective Impact approach to systemic change, the project creates sustainable institutional change at the community, state, and national levels. Qualitative and quantitative data provide descriptions about how to utilize alignment strategies within Collective Impact in three different contexts: rural, suburban, and urban. Outcomes utilize a regression discontinuity analysis to justify successful implementation as well as qualitative analysis of implementation efforts that were deemed most effective by all stakeholders. The project outputs directly affect over 88,000 students across five districts and indirectly affect over 1.7 million in Georgia alone. The culminating project goal is the development of a coherent framework for aligning K-12 computer science education pathways.
DATE:
-
TEAM MEMBERS:
Caitlin DooleyBryan CoxShawn Utley
resourceprojectProfessional Development and Workshops
This is an "Inclusion across the Nation of Communities of Learners of Underrepresented Discoverers in Engineering and Science" (INCLUDES) Design and Development Launch Pilot that will implement a plan to assess the feasibility of a strategy designed to ensure high levels of improvement in K-12 grade students' mathematics achievement. The plan will focus on an often-neglected group of students--those who have been performing at the lowest quartile on state tests of mathematics, including African American, Hispanic, Native American, students with disabilities, and those segregated in urban and rural communities across the country. The project will draw on lessons learned from the nation's Civil Rights Movement and a community-organizing strategy learned during the struggle to achieve voting rights for African Americans. The Algebra Project (AP) is a national, nonprofit organization that uses mathematics as an organizing tool to ensure quality public school education for every child in America; it believes that every child has a right to a quality education to succeed in this technology-based society. AP's unique approach to school reform intentionally develops sustainable, student-centered models by building coalitions of stakeholders within the local communities, particularly the historically underserved populations. The AP works to change the deeply rooted social attitudes that encourage the disenfranchisement of a third of the nation's population. It delivers a multi-pronged approach to build demand for and support of quality public schools, including research and development, school development, and community development education reform efforts through K-12 initiatives.
The Algebra Project and the Young People's Project (YPP) will join efforts to bring together over 70 individuals and organizations, including 17 universities of which 8 are Historical Black Colleges and Universities, school districts, mathematics educators, and researchers to examine their experiences, and use collective learning to refine and hone strategies that they have piloted and tested to promote mathematics inclusion. The role of YPP in the proposed project will be to organize and facilitate the youth component, such that project activities reflect the language and culture of students, continuously leveraging and building upon their voice, creative input, and ongoing feedback. YPP will conduct workshops for students organized around math-based games that provide collective experiences in which student learning requires individual reflection, small group work, teamwork and discussion. The proposed work will comprise the design of effective learning opportunities; building and supporting a cadre of teachers who can effectively work with students learning under the proposed approach; using technologies to enhance teaching and learning; and utilizing evaluation and research to drive continuous improvement. Because bringing together an effective network with diverse expertise to collaborate towards national impact requires expert facilitation processes, the project will establish working groups around three major principles: (1) Organizing from the bottom up through students, their teachers, and others in local communities committed to their education, allied with individuals and organizations who have expertise and dedication for achieving the stated goals, can produce significant progress and the conditions for collective impact; (2) Effective learning materials and formal and informal learning opportunities in mathematics can be designed and implemented for students performing in the bottom academic quartile; and (3) Teachers and other educators can become more proficient and more confident in their capacity to produce students who are successful in learning the level of mathematics required for full participation in STEM. The working groups will also be tasked to consider two cross-cutting topics: (a) the communication structures and technologies needed to operate and expand the present network, and to create the "backbone" and other structures needed to operate and expand the network; and (b) the measurements and metrics for major needs, such as assessing students' mathematics literacy, socio-emotional development in specified areas; teachers' competencies; as well as the work of the network. The final product of this plan will be a "Theory of Collective Action and Strategic Plan". The plan will contain recommendations for collective actions needed in order for the current network to coordinate, add appropriate partners, develop the needed backbone structures, and become an NSF Alliance for national impact on the broadening participation challenge of improving the mathematics achievement. An external evaluator will conduct both formative and summative aspects of this process.
DATE:
-
TEAM MEMBERS:
Robert MosesNell CobbGregory BudzbanMaisha MosesWilliam Crombie
resourceresearchWebsites, Mobile Apps, and Online Media
This brief discusses the PLUM LANDING Explore Outdoors Toolkit, a new set of free, public media resources designed to help informal educators and parents infuse science learning into outdoor recreation. Developed by trusted media producer WGBH in partnership with researchers at Education Development Center (EDC), the Toolkit aims to get children (ages 6–9) from low-income, urban communities outside so they can explore the environment around them while debunking the myth that nature is something that only exists beyond city limits.
DATE:
TEAM MEMBERS:
Marion GoldsteinElizabeth PiersonJamie KynnLisa Famularo
This 4-year project addresses fundamental equity issues in informal Science, Technology, Engineering and Mathematics (STEM) learning. Access to, and opportunities within informal STEM learning (ISL) remain limited for youth from historically underrepresented backgrounds in both the United States and the United Kingdom. However, there is evidence that ISL experiences can expand opportunities for youth learning and development in STEM, for instance, increase positive attitudes towards educational aspirations and future careers/pursuits, improve grades and test scores in school settings, and decrease disciplinary action and dropout rates. Through research and development, this project brings together researchers and practitioners to focus on the experiences, practices and tools that will support equitable youth pathways into STEM. Working across conceptual frameworks and ISL settings (e.g. science centers, community groups, zoos) and universities in four urban contexts in two different nations, the partnership will produce a coherent knowledge base that strengthens and expands research plus practice partnerships, builds capacity towards transformative research and development, and develops new models and tools in support of equitable pathways into STEM at a global level. This project is funded through Science Learning+, which is an international partnership between the National Science Foundation (NSF) and the Wellcome Trust with the UK Economic and Social Research Council. The goal of this joint funding effort is to make transformational steps toward improving the knowledge base and practices of informal STEM experiences. Within NSF, Science Learning+ is part of the Advancing Informal STEM Learning (AISL) program that seeks to enhance learning in informal environments and to broaden access to and engagement in STEM learning experiences.
This Equity Pathways project responds to three challenges at the intersections of ISL research and practice in the United States and the United Kingdom: 1) lack of shared understanding of how youth from historically underrepresented backgrounds perceive and experience ISL opportunities across national contexts, and the practices and tools needed to support empowered movement through ISL; 2) limited shared understanding and evidence of core high-leverage practices that support such youth in progressing within and across ISL, and 3) limited understanding of how ISL might be equitable and transformative for such youth seeking to develop their own pathways into STEM. The major goal of this Partnership is for practitioners and researchers, working with youth through design-based implementation research, survey and critical ethnography, to develop new understandings of how and under what conditions they participate in ISL over time and across settings, and how they may connect these experiences towards pathways into STEM. The project will result in: 1) New understandings of ISL pathways that are equitable and transformative for youth from historically underrepresented backgrounds; 2) A set of high leverage practices and tools that support equitable and transformative informal science learning pathways (and the agency youth need to make their way through them); and 3) Strengthened and increased professional capacity to broaden participation among youth from historically underrepresented backgrounds in STEM through informal science learning. The project will be carried out by research + practice partnerships in 4 cities: London & Bristol, UK and Lansing, MI & Portland, OR, US, involving university researchers (University College London, Michigan State University, Oregon State University/Institute for Learning Innovation) practitioners in science museums (@Bristol Science Centre, Brent Lodge Park Animal Centre, Impressions 5, Oregon Museum of Science & Industry) and community-based centers (STEMettes, Knowle West Media Centre, Boys & Girls Clubs of Lansing, and Girls, Inc. of the Pacific Northwest).
As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program funds innovative research, approaches and resources for use in a variety of settings. The project will conduct research designed to deepen our fundamental knowledge about culture, experience, and ecosystems cognition and to develop innovative practices and approaches to support learning about changing ecological systems and environmental decision making. Work on cultural differences in the production of complex systems knowledge is severely lacking. This gap in knowledge may contribute to the continued reproduction of inequities in science education. More broadly findings from this project will have clear implications for theories of cognitive development, especially those pertaining to how knowledge is shaped by culture and experience. Focusing on ecosystems may represent an opportunity to not only increase engagement and achievement in science among non-dominant communities and Native youth specifically, but also advance effective learning for all communities. The primary deliverables for the project are conference presentations and research publications. However, the project will also develop additional resources freely available to researchers, educators, and the general public. These will include summer curricular materials and teaching tools, professional development workshops, practitioner briefs about research findings that can be used in professional development workshops and shared share more broadly, and evaluation reports.
A deeper understanding of cultural influences on conceptions of the natural world can serve to advance the educational needs of children, including children from diverse linguistic and cultural backgrounds. Project research will include two interrelated series of studies designed to expand knowledge about human cognition of complex ecosystems and the affordances of informal STEM learning environments in developing and supporting the critical 21st century skill of ecological systems level reasoning. The first consists of a series of experiments focused on ecological cognition and the role of humans in nature. The second consists of design-based research interventions in informal settings, summer workshops for youth and the communities, focused on ecological systems level thinking and socio-environmental decision making. The project will recruit and engage both child and adult participants from two broad cultural communities, Native Americans and European Americans living in urban and suburban communities, in part because it affords a sharp test of human-nature relations. Sampling from two different urban communities will avoid simple Native-non-Native comparative binaries and to conduct Native-to-Native comparative analysis. Based on results from this, the project will result in: 1) foundational knowledge about human learning and reasoning and ecosystems and environmental decision making, 2) culturally responsive models of learning and practice about complex ecosystems for indoors and outdoors informal learning environments, and 3) insights about research-practice-community partnerships. One important objective of the research is to broaden participation and close opportunity gaps for under-represented groups in STEM fields broadly and more specifically for Indigenous people. Members of Indigenous communities, who provide strong role models for other aspiring scholars, will be involved as postdoctoral fellows, research assistants and graduate fellows.
Lineage is a comprehensive educational media and outreach initiative that will engage individuals and families in learning about deep time and evolution, helping audiences come to newfound understandings of the connections between the past, present, and future of life on Earth. The project is a partnership between Twin Cities PBS (TPT) and the Smithsonian Institution's National Museum of Natural History and is linked to the opening of that museum's new Deep Time Fossil Hall in June 2019. The project includes a two-hour film for national broadcast on PBS, and a 20-minute short version for exhibition in science centers. The documentaries will show how scientists, using paleontology, genetics, earth science and other disciplines, can reconstruct in detail the origins of living animals like birds and elephants, revealing their ancient past as well as evidence of ecological change that can inform our understanding of Earth today. Extensive educational outreach will include the creation of "Bone Hunter," an innovative VR (Virtual Reality) game designed for family co-play that engages multiple players in the process of paleontology as they piece together a fossil in a digital lab. Bone Hunter and other collaborative educational activities will be deployed at Family Fossil Festivals that will attract multi-generational learners. One such Festival will take place at the Smithsonian Institution in Washington, D.C., while others will be based at geographically diverse institutions that serve underserved rural as well as urban communities. Lineage is a collaboration between national media producers, noted learning institutions and researchers, including Twin Cities Public Television, the Smithsonian Institution / National Museum of Natural History, Schell Games, the Institute for Learning Innovation (ILI), and Rockman et al. One of the project's primary innovations is its exploration of new learning designs for families that use cutting-edge technologies (e.g. the Bone Hunter virtual reality game) and collaborative multi-generational learning experiences that advance science knowledge and inquiry-based learning. An external research study conducted by ILI will investigate how intergenerational co-play with physical artifacts compared to virtual artifacts influences STEM (Science Technology Engineering Mathematics) learning and engagement. The findings will lead to critical strategic impacts for the field, building knowledge about ongoing innovation in the free choice learning space. The project's external evaluation will be conducted by Rockman et al and evaluative findings, as well as the educational materials derived from the project, will be widely disseminated through partnerships with professional and educator groups. Clips from the Lineage film and related learning resources will be hosted on PBS LearningMedia, so educators can incorporate these resources into their classrooms, and students and lifelong learners can explore and discover on their own. The project outcomes will have broad impact on public audiences, deepening and advancing knowledge and understanding about important scientific concepts, and promoting continued, family-based collaborative learning experiences to expand and deepen STEM knowledge. This project is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning.
Public Participation in Scientific Research (PPSR), often referred to as crowdsourcing or citizen science, engages participants in authentic research, which both advances science discovery as well as increases the potential for participants' understanding and use of science in their lives and careers. This four year research project examines youth participation in PPSR projects that are facilitated by Natural History Museums (NHMs). NHMs, like PPSR, have a dual focus on scientific research and science, technology, engineering, and mathematics (STEM) education. The NHMs in this project have established in-person and online PPSR programs and have close ties with local urban community-based organizations. Together, these traits make NHMs appropriate informal learning settings to study how young people participate in PPSR and what they learn. This study focuses on three types of PPSR experiences: short-term outdoor events like bioblitzes, long-term outdoor environmental monitoring projects, and online PPSR projects such as crowdsourcing the ID of field observations. The findings of this study will be shared through PPSR networks as well as throughout the field in informal STEM learning in order to strength youth programming in STEM, such that youth are empowered to engage in STEM research and activities in their communities. This project is funded through Science Learning+, which is an international partnership between the National Science Foundation (NSF) and the Wellcome Trust with the UK Economic and Social Research Council. The goal of this joint funding effort is to make transformational steps toward improving the knowledge base and practices of informal STEM experiences. Within NSF, Science Learning+ is part of the Advancing Informal STEM Learning (AISL) program that seeks to enhance learning in informal environments and to broaden access to and engagement in STEM learning experiences.
The study employs observations, surveys, interviews, and learning analytics to explore three overarching questions about youth learning: 1) What is the nature of the learning environments and what activities do youth engage in when participating in NHM-led PPSR? 2) To what extent do youth develop three science learning outcomes, through participation in NHM-led citizen science programs? The three are: a) An understanding of the science content, b) identification of roles for themselves in the practice of science, and c) a sense of agency for taking actions using science? 3) What program features and settings in NHM-led PPSR foster these three science learning outcomes among youth? Based on studies occurring at multiple NHMs in the US and the UK, the broader impact of this study includes providing research-based recommendations for NHM practitioners that will help make PPSR projects and learning science more accessible and productive for youth. This project is collaboration between education researchers at University of California, Davis and Open University (UK), and Oxford University (UK) and citizen science practitioners, educators, and environmental scientists at three NHMs in the US and UK: NHM London, California Academy of Sciences, and NHM Los Angeles.
As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program funds innovative research, approaches, and resources for use in a variety of settings. This project will develop a national infrastructure of state and regional partnerships to scale up The Franklin Institute's proven model of Leap into Science, an outreach program that builds the capacity of children (ages 3-10) and families from underserved communities to participate in science where they live. Leap into Science combines children's science-themed books with hands-on science activities to promote life-long interest and knowledge of science, and does so through partnerships with informal educators at libraries, museums, and other out-of-school time providers. Already field-tested and implemented in 12 cities, Leap into Science will be expanded to 90 new rural and urban communities in 15 states, and it is estimated that this expansion will reach more than 500,000 children and adults as well as 2,700 informal educators over four years. The inclusion of marginalized rural communities will provide new opportunities to evaluate and adapt the program to the unique assets and needs of rural families and communities.
The project will include evaluation and learning research activities. Evaluation will focus on: 1) the formative issues that may arise and modifications that may enhance implementation; and 2) the overall effectiveness and impact of the Leap into Science program as it is scaled across more sites and partners. Learning research will be used to investigate questions organized around how family science interest emerges and develops among 36 participating families across six sites (3 rural, 3 urban). Qualitative methods, including data synthesis and cross-case analysis using constant comparison, will be used to develop multiple case studies that provide insights into the processes and outcomes of interest development as families engage with Leap into Science and a conceptual framework that guides future research. This project involves a partnership between The Franklin Institute (Philadelphia, PA), the National Girls Collaborative Project (Seattle, WA), Education Development Center (Waltham, MA), and the Institute for Learning Innovation (Corvallis, OR).
In this case study, we highlight the work of the Bay Area STEM Ecosystem, which aims to increase equity and access to STEM learning opportunities in underserved communities. First, we lay out the problems they are trying to solve and give a high level overview of the Bay Area STEM Ecosystem’s approach to addressing them. Then, based on field observations and interviews, we highlight both the successes and some missed opportunities from the first collaborative program of this Ecosystem. Both the successes of The Bay Area STEM Ecosystem--as well as the partners’ willingness to share and examine