Program evaluators from the Education Development Center (EDC) used a mixed-methods, quasi-experimental design to evaluate the impact on girls’ awareness and interest in science, technology, engineering, and mathematics (STEM). After the final year of the project, EDC delivered a summative report to Techbridge Girls (TBG), which was based on data collected during the five-year grant period, with a particular focus on the final year that grant funds supported programming (2017-18). Data included pre- and post-surveys with TBG participants and comparison students, participant focus groups, and
The American Museum of Natural History (AMNH), in collaboration with New York University's Institute for Education and Social Policy and the University of Southern Maine Center for Evaluation and Policy, will develop and evaluate a new teacher education program model to prepare science teachers through a partnership between a world class science museum and high need schools in metropolitan New York City (NYC). This innovative pilot residency model was approved by the New York State (NYS) Board of Regents as part of the state’s Race To The Top award. The program will prepare a total of 50 candidates in two cohorts (2012 and 2013) to earn a Board of Regents-awarded Masters of Arts in Teaching (MAT) degree with a specialization in Earth Science for grades 7-12. The program focuses on Earth Science both because it is one of the greatest areas of science teacher shortages in urban areas and because AMNH has the ability to leverage the required scientific and educational resources in Earth Science and allied disciplines, including paleontology and astrophysics.
The proposed 15-month, 36-credit residency program is followed by two additional years of mentoring for new teachers. In addition to a full academic year of residency in high-needs public schools, teacher candidates will undertake two AMNH-based clinical summer residencies; a Museum Teaching Residency prior to entering their host schools, and a Museum Science Residency prior to entering the teaching profession. All courses will be taught by teams of doctoral-level educators and scientists.
The project’s research and evaluation components will examine the factors and outcomes of a program offered through a science museum working with the formal teacher preparation system in high need schools. Formative and summative evaluations will document all aspects of the program. In light of the NYS requirement that the pilot program be implemented in high-need, low-performing schools, this project has the potential to engage, motivate and improve the Earth Science achievement and interest in STEM careers of thousands of students from traditionally underrepresented populations including English language learners, special education students, and racial minority groups. In addition, this project will gather meaningful data on the role science museums can play in preparing well-qualified Earth Science teachers. The research component will examine the impact of this new teacher preparation model on student achievement in metropolitan NYC schools. More specifically, this project asks, "How do Earth Science students taught by first year AMNH MAT Earth Science teachers perform academically in comparison with students taught by first year Earth Science teachers not prepared in the AMNH program?.”
DATE:
-
TEAM MEMBERS:
Maritza MacdonaldMeryle WeinsteinRosamond KinzlerMordecai-Mark Mac LowEdmond MathezDavid Silvernail
resourceprojectProfessional Development, Conferences, and Networks
The National Writing Project (NWP) is collaborating with the Association of Science-Technology Centers (ASTC) on a four-year, full-scale development project that is designed to integrate science and literacy. Partnerships will be formed between NWP sites and ASTC member science centers and museums to develop, test, and refine innovative programs for educators and youth, resulting in the creation of a unique learning network. The project highlights the critical need for the integration of science and literacy and builds on recommendations in the Common Core State Standards and the National Research Council's publication, "A Framework for K-12 Science Education: Practices, Crosscutting Concepts, and Core Ideas." The content focus includes current topics in science and technology such as environmental science, sustainability, synthetic biology, geoengineering, and other subjects which align with science center research and exhibits. The project design is supported by a framework that incorporates a constructivist/inquiry-based approach that capitalizes on the synergy between rigorous science learning and robust literacy practices. Project deliverables include a set of 10 local partnership sites, professional development for network members, a project website, and an evaluation report highlighting lessons learned. Partnership sites will be selected based on interest, proximity, history, and expertise. Two geographically and demographically diverse cohorts, consisting of five partnerships each will be identified in Years 2 and 3. Each set of partners will be charged with creating a comprehensive two-year plan for science literacy activities and products to be implemented at local sites. It is anticipated that the pilot programs may result in the creation of new programs that merge science and writing, integrate writing into existing museum science programs, or integrate science activities into existing NWP programs. Interest-driven youth projects such as citizen science and science journalism activities are examples of programmatic approaches that may be adopted. The partners will convene periodically for planning and professional development focused on the integration of science and literacy for public and professional audiences, provided in part by national practitioners and research experts. A network Design Team that includes leadership representatives from NWP, ASTC, and the project evaluator, Inverness Research, Inc., will oversee project efforts in conjunction with a national advisory board, while a Partnership Coordinator will provide support for the local sites. Inverness Research will conduct a multi-level evaluation to address the following questions: -What is the nature and quality of the local partner arrangements, and the larger network as a whole? -What is the nature and quality of the local science literacy programs that local partners initiate, and how do they engage local participants, and develop their sense of inquiry and communication skills? First, a Designed-Based Implementation Research approach will be used for the developmental evaluation to assess the implementation process. Next, the documentation and portrayal phase will assess the benefits to youth, educators, institutions, and the field using surveys, interviews, observations of educators, and reviews of science communication efforts created by youth. Finally, the summative evaluation includes a comprehensive portfolio of evidence to document the audience impacts and an independent assessment of the project model by an Evaluation Review Board. This project will result in the creation of a robust learning community while contributing knowledge and lessons learned to the field about networks and innovative partnerships. It is anticipated that formal and informal educators will gain increased knowledge about science and literacy programs and develop skills to provide effective programs, while youth will demonstrate increased understanding of key science concepts and the ability to communicate science. Programs created by the local partnerships will serve approximately 650 educators (450 informal educators and 200 K-12 teachers) and 500 youth ages 9-18. Plans for dissemination, expansion, and sustainability will be undertaken by the sub-networks of the collaborating national organizations drawing on the 350 ASTC member institutions and nearly 200 NWP sites at colleges and universities.
In case studies of two first-year elementary classroom teachers, we explored the influence of informal science education (ISE) they experienced in their teacher education program. Our theoretical lens was identity development, delimited to classroom science teaching. We used complementary data collection methods and analysis, including interviews, electronic communications, and drawing prompts. We found that our two participants referenced as important the ISE experiences in their development of classroom science identities that included resilience, excitement and engagement in science
MobiLLab was designed by faculty at the University of Teacher Education in St.Gallen (German: Pädagogische Hochschule St.Gallen (PHSG)) to spark interest in science in secondary school pupils in the German-speaking part of Switzerland. Now in its fourth year of operation, the mobiLLab team would like to evaluate the program’s effectiveness to inform further development. Specifically, we want to better understand how mobiLLab affects pupils’ science and technology interest, attitudes and knowledge development, and how positive changes can be sustained. The background investigation brings
DATE:
TEAM MEMBERS:
University of Teacher Education, St. Gallen, SwithzerlandRebecca Cors
The Community STEM Outreach Project at the Saint Louis Science Center (SLSC) received funding from the United States Office of Naval Research (ONR) from October 2010 through September 2013. Klein Consulting, with support from Tisdal Consulting, conducted the evaluation of the three-year project. The original proposal from the SLSC to ONR laid the foundation for the Community STEM Outreach Project by describing the institution and its youth program, the Youth Exploring Science (YES) Program. Plans were underway to reach out to existing and new national partners to document and disseminate a
The Chicago Zoological Society (CZS) in collaboration with Eden Place Nature Center, the Fuller Park Community Corporation, and the University of Illinois at Chicago (UIC) will implement the SCIENCES Program, Supporting a Community's Informal Education Needs: Confidence and Empowerment in STEM. The primary goals of this Full Scale Development project are to broaden access to and participation in environmental science, strengthen partnerships between CZS, Eden Place, and UIC, and gain insights into the 'ecosystemic' learning model which promotes scientific literacy and agency in the community. The project targets a low-resource community with a minority audience while the secondary audience is informal science learning organizations and researchers who will advance research in informal learning. The theoretical framework for the project design draws on conservation psychology, informal science learning, civic ecology education, and urban science education to create an ecosystematic, geographically centered approach. The deliverables include research, curriculum, and engaging hands-on programs for youth, families, adults, and teachers, reaching both in-school and out-of-school audiences, in addition to the SCIENCES Implementation Network. Three potential curriculum themes to be explored are water conservation and protection, pollinators for healthy ecosystems, and community resilience to climate change. The SCIENCES project offers a comprehensive suite of engaging programs for community audiences. For example, the year-long Zoo Adventure Passport (ZAP) program for families includes hands-on experiments and field trips, while project-based learning experiences enable teens to create wetlands, design interpretive signage, and develop associated public programming. School-based programs include professional development for teachers on the Great Lakes ecosystem and invasive species. Existing programs that have been previously evaluated and demonstrated to show learning impacts will be adapted and modified to meet the goals of the ecosystemic learning model by providing multiple learning opportunities. New learning resources will also be created to support the content themes and provide continuity. The result will be a comprehensive approach that ensures deep community engagement by individuals, families, and organizations, with cohesiveness provided by the overarching content themes which broaden access to STEM learning resources and leverages partnerships. The project includes both a research and evaluation plan. The primary research question to be addressed is: How does a large informal science learning institution work with a community-based organization to support environmental scientific literacy and agency at all levels of the community? A sociocultural framework will be used for this mixed-methods case study research. Study participants include community leaders, youth, parents, teachers, and staff from Eden Place. The case study sample will include 20 focal individuals drawn from the participant groups and approximately 300 survey participants. Case study data will be triangulated with evaluation data and analyzed using a grounded theory approach. By examining changes from the baseline following the implementation of the community programs, the findings may provide insight on agency and science literacy among community members. The comprehensive, mixed-methods evaluation plan employs a quasi-experimental design and incorporates front-end, formative, and summative evaluation components. The evaluation questions address the quality of the processes and products, access to environmental science learning opportunities, environmental science literacy, sustainability, and barriers to implementation. An extensive dissemination plan is proposed with a dual emphasis on meeting stakeholders' needs at multiple levels. The evaluation and research teams will emphasize publication in peer reviewed journals and presentations at conferences for informal science education professionals. Findings will be shared with the Fuller Park community stakeholders using creative methods such as one-page research briefs written in layperson's language, videos, and recorded interviews with participants. The local project Advisory Board will also be actively involved in the dissemination of findings to community constituents. The SCIENCES National Amplification Network will be created and work collaboratively with the American Association of Zoos and Aquariums and the Metropolitan Green Spaces Alliance to disseminate the model. Collectively, the activities and deliverables outlined in this proposal will advance the discovery of sustainable models of community-based learning while the research will advance the understanding of informal learning support for science literacy and agency.
Techbridge has proposed a broad implementation project that will scale up a tested multi-faceted model that increases girls' interest in STEM careers. The objectives of this project are to increase girls' engineering, technology, and science skills and career interests; build STEM capacity and sustainability across communities; enhance STEM and career exploration for underrepresented girls and their families; and advance research on the scale-up, sustainability, and impact of the model with career exploration. The Techbridge approach is grounded in Eccles' expectancy value model, and helps bridge critical junctures as girls transition from elementary to middle school and middle school to high school, immersing participants in a network of peers and supportive adults. Techbridge targets girls in grades 5-12 with a model that includes five components: a previously tested and evaluated curriculum, career exploration, professional development for staff and teachers, family engagement, and dissemination. The inquiry-based curriculum introduces electrical engineering and computer science through engaging, hands-on units on Cars and Engines, Green Design, and Electrical Engineering. The Techbridge model will be enhanced to include a central repository for curriculum and support materials, electronic girl-driven career exploration resources, an online learning community and video tools for staff, and customized family guides. Project deliverables include the dissemination of the enhanced model to three cities, 24 school sites and teachers, 2,000 girls, and over 600 role models. A supplementary research component will study the broad implementation of the Techbridge model by examining the fidelity of implementation and the program's impact on girls' STEM engagement and learning. The research questions are as follows: (1) To what extent and how do new program sites demonstrate adherence to the Techbridge program model? (2) Do new sites experience similar or increased participant responsiveness to Techbridge programming with regard to scientific learning outcomes, career awareness, attitude and interest in engineering? (3)How are changes experienced by girls sustained over time, if at all? (4) To what extent and how do new sites balance instilling the Techbridge essentials, those critical components Techbridge identifies as essential for success, with the need for local adaptation and ownership of the program? and (5) Given the potential for customization in local communities, do new sites maintain programmatic quality of delivery experienced at the original site? If so, what are elements essential to success regarding quality delivery? The mixed-methods study will include document analysis, embedded assessments, participant survey scales, and observations. Qualitative data methods include interviews with teachers, role models, staff and focus groups with girls. A project evaluation will also be conducted which investigates project outcomes for participants (girls, teachers, role models, and families) and fidelity of the implementation and enhancements at expansion sites, using a quasi-experimental approach. Career and learning outcomes for girls will be determined using embedded assessments, portfolios, surveys, school data, and previously validated instruments such as the Career Interest Questionnaire and the Modified Attitudes towards Science Inventory. The Managing Complex Change model is used as a framework for the project evaluation for the purpose of examining factors related to the effectiveness of scaling. The dissemination of research and evaluation findings will be achieved through the use of publications, blogs, social media, and conferences. It is anticipated that this project will broaden the participation of Hispanic, African-American, and English language learner girls, build capacity for STEM programming and sustainability at the dissemination sites, and disseminate findings to over 1 million educators, researchers, and community members. Broader impacts include contributing to the field's understanding of how virtual role models and field trips can engage young women, increase corporate advocacy, and engage participants in research and dissemination efforts.
Using STEM America (USA) is a two-year Pathways project designed to examine the feasibility of using informal STEM learning opportunities to improve science literacy among English Language Learner (ELL) students in Imperial County, California. Project partners include the Rueben H. Fleet Science Center and the University of California, San Diego (UCSD). The project's goals are to support teachers in the development of informal science education opportunities for English learners, partner with students in grades 7-12 to create activities and exhibits, deliver student-produced products to community members, and sustain and disseminate the activities through the development of web-based teacher tools. The teachers will work with informal science education experts, STEM professionals, and undergraduate students to develop and implement the program lessons with their 7-12 grade students. The activities and exhibits designed for community audiences will be used in the Imperial Valley Discovery Zone, slated for completion in fall 2013. Special emphasis will be placed on understanding English scientific word frames and science content specific vocabulary to help ELL students express complex scientific concepts in English. The project deliverables in this pilot project include a comprehensive teacher professional development strategy, student-developed informal science activities and exhibits, a project website, and multiple teacher resources (lesson plans, how-to guides, training materials, and social networking tools). Teachers will receive 45 hours of professional development during the summer with an additional 20 hours of support provided during the school year. UCSD's Jacob's School of Engineering will provide training on solar energy micro-grids using a micro-grid observatory to be located in Imperial Valley. English language development training will be provided by the University of California's Professional Development Institute (UCPDI) and address the role of language objectives in scientific conceptual knowledge and language development; using science and language to improve classroom questioning/discussion; and teaching academic language to English learners. The informal science education component of the training provided by the Fleet Science Center will address topics such as questioning strategies, scientific reasoning frameworks, communicating science to public audiences, and learning "high level" science content using hands-on approaches. The project design builds on research which supports an active learning approach that mirrors scientific practice and is one of the strengths of informal science learning environments. The question to be addressed by the USA Project is: "Can informal STEM activities with embedded English Language development strategies assist English learner students to increase their English language competency and their interest in STEM subjects?" The PI seeks to identify the impact that teachers have on guiding students in inquiry-based informal STEM education, evaluate the academic outcomes for students, and measure changes in community interest, understanding, and attitudes towards STEM and STEM occupations. The USA Project is designed to reach approximately 200 underserved students and will promote the participation of at least 400 additional students, parents, and other rural community members. It is anticipated that this project will result in the development of a model for teacher-led informal STEM education, increased STEM learning opportunities for the community, and the development of a network of educational institutions that helps to bridge formal and informal STEM learning and learning environments.
This report is the result of a project to investigate through a sociocultural lens whether girls-only, informal STEM experiences have potential long-term influences on young women's lives, both in terms of STEM but also more generally. The authors documented young women's perceptions of their program experiences and the ways in which they influenced their future choices in education, careers, leisure pursuits, and ways of thinking about what science is and who does it. This report includes the questionnaire used in the study.
This report combines the views of education researchers, technology developers, educators, and researchers in emerging fields such as educational data mining and technology-supported evidence-centered design to present an expanded view of approaches to evidence. It presents the case for why the transition to digital learning warrants a re-examination of how we think about educational evidence. The report describes approaches to evidence-gathering that capitalize on digital learning data and draws implications for policy, education practice, and R&D funding.
DATE:
TEAM MEMBERS:
U.S. Department of EducationOffice of Educational Technology
The Learning and Youth Research and Evaluation Center (LYREC) is a collaboration of the Exploratorium, Harvard University, Kings College London, SRI International and UC Santa Cruz. LYREC provides technical assistance to NSF AYS projects, collects and synthesizes their impact data, and oversees dissemination of progress and results. This center builds on the Center for Informal Learning in Schools (CILS) that has developed a theoretical approach that takes into account the particular strengths and affordances of both Out of School Teaching (OST) and school environments. This foundation will permit strengthening the potential of the NSF AYS projects to develop strong local models that can generate valid and reliable data that can guide future investment, design and research aimed at creating coherence across OST and school settings. The overarching questions for the work are: 1. How can OST programs support K-8 engagement and learning in science, and in particular how can they contribute to student engagement with K-8 school science and beyond? 2. What is the range of science learning outcomes OST programs can promote, particularly when in collaboration with schools, IHE's, businesses, and other community partners? 3. How can classroom teachers and schools build on children's OST experiences to strengthen children's participation and achievement in K-12 school science Additionally, the data analysis will reveal: 1. How OST programs may be positioned to support, in particular, high-poverty, female and/or minority children traditionally excluded from STEM academic and career paths; and 2. The structural/organizational challenges and constraints that exist to complicate or confound efforts to provide OST experiences that support school science engagement, and conversely, the new possibilities which are created by collaboration across organizational fields. Data will be gathered from surveys, interviews, focus groups, evaluation reports, and classroom and school data.