EvaluATE is a national resource center dedicated to supporting and improving the evaluation practices of approximately 250 ATE grantees across the country. EvaluATE conducts webinars and workshops, publishes a quarterly newsletter, maintains a website with a digital resource library, develops materials to guide evaluation work, and conducts an annual survey of ATE grantees. EvaluATE's mission is to promote the goals of the ATE program by partnering with projects and centers to strengthen the program's evaluation knowledge base, expand the use of exemplary evaluation practices, and support the continuous improvement of technician education throughout the nation. EvaluATE's goals associated with this proposal are to: (1) Ensure that all ATE Principal Investigators and evaluators know the essential elements of a credible and useful evaluation; (2) Maintain a comprehensive collection of online resources for ATE evaluation; (3) Strengthen and expand the network of ATE evaluation stakeholders; and (4) Gather, synthesize, and disseminate data about the ATE program activities to advance knowledge about ATE/technician education. The Center plans to produce a comprehensive set of evaluation resources to complement other services, engaging several community college-based Principal Investigators and evaluators in that process.
EvaluATE's products are informed by current research on evaluation, the National Science Foundation's priorities for the evaluation of ATE grants, and the needs of ATE PIs and evaluators for sound guidance that is immediately relevant and usable in their contexts. The fundamental nature of EvaluATE's work is geared toward supporting ATE grantees to use evaluation regularly to improve their work and demonstrate their impacts. All of EvaluATE's products are available to the public. EvaluATE's findings from the annual survey of ATE grantees aid in advancing understanding of the status of technician education and illuminate areas for additional research. The new survey investigates ATE grantees' work to serve underrepresented and special populations, including women, people of color, and veterans. Survey data are available upon request for research and evaluation purposes.
DATE:
-
TEAM MEMBERS:
Lori WingateArlen GullicksonEmma PerkKelly RobertsonLyssa Becho
As a part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program funds research and innovative resources for use in a variety of settings. This media and technology project will scale up Youth Radio's proven model of STEM education through youth-driven multimedia journalism and related app development using the MIT App Inventor. A new Youth News Network (YNN) will implement a nationwide feeder system of youth reporters and educators using the previously developed and proven STEM curriculum. Previous research and evaluation has demonstrated that this model can engage underserved youth and put them in leadership positions in technological innovation. Key deliverables include the YNN STEM Desk that will produce 15-20 STEM-related stories each year; bootcamps (1-3 day workshops) training youth around the country focusing on app development and media links; and new toolkits providing resources to help with app development, data analysis and other STEM-specific skills. Project partners include MIT Media Lab, National Public Radio, Best Buy's Teen Tech Centers, National Writing Project, Computer Clubhouses, and PBS Learning Media among others.
Over the previous eight years, research and evaluation findings had been used to refine the project. These data served as the foundation for this scale-up project. The research conducted by the investigator and the Scholar-in-Residence in this scale-up uses an embedded ethnographic approach that combines field notes, recorded meetings and discussions, media artifacts, etc.--data that is transcribed and coded for indicators of STEM learning and critical computational literacy. The external summative evaluation will build on prior evidence regarding how this unique model engages youth and impacts their skills in STEM related media and technology.
DATE:
-
TEAM MEMBERS:
Elisabeth SoepEllin O'LearyHarold Abelson
resourceevaluationProfessional Development, Conferences, and Networks
The NISE Network Professional Impacts Summative Evaluation is a longitudinal examination of individual professionals over the final three years of NISE Net funding. This investigation is based on the NISE Network goals for professionals and explores how involvement with NISE Net impacts an individual professional’s sense of community, learning about nano, and use of nano educational products and practices. This evaluation primarily included professional partners who were: (1) Informal Science Educators (ISE): Professionals from science museums and children’s museums implementing informal
The National Writing Project (NWP) is collaborating with the Association of Science-Technology Centers (ASTC) on a four-year, full-scale development project that is designed to integrate science and literacy. Partnerships will be formed between NWP sites and ASTC member science centers and museums to develop, test, and refine innovative programs for educators and youth, resulting in the creation of a unique learning network. The project highlights the critical need for the integration of science and literacy and builds on recommendations in the Common Core State Standards and the National Research Council's publication, "A Framework for K-12 Science Education: Practices, Crosscutting Concepts, and Core Ideas." The content focus includes current topics in science and technology such as environmental science, sustainability, synthetic biology, geoengineering, and other subjects which align with science center research and exhibits. The project design is supported by a framework that incorporates a constructivist/inquiry-based approach that capitalizes on the synergy between rigorous science learning and robust literacy practices. Project deliverables include a set of 10 local partnership sites, professional development for network members, a project website, and an evaluation report highlighting lessons learned. Partnership sites will be selected based on interest, proximity, history, and expertise. Two geographically and demographically diverse cohorts, consisting of five partnerships each will be identified in Years 2 and 3. Each set of partners will be charged with creating a comprehensive two-year plan for science literacy activities and products to be implemented at local sites. It is anticipated that the pilot programs may result in the creation of new programs that merge science and writing, integrate writing into existing museum science programs, or integrate science activities into existing NWP programs. Interest-driven youth projects such as citizen science and science journalism activities are examples of programmatic approaches that may be adopted. The partners will convene periodically for planning and professional development focused on the integration of science and literacy for public and professional audiences, provided in part by national practitioners and research experts. A network Design Team that includes leadership representatives from NWP, ASTC, and the project evaluator, Inverness Research, Inc., will oversee project efforts in conjunction with a national advisory board, while a Partnership Coordinator will provide support for the local sites. Inverness Research will conduct a multi-level evaluation to address the following questions: -What is the nature and quality of the local partner arrangements, and the larger network as a whole? -What is the nature and quality of the local science literacy programs that local partners initiate, and how do they engage local participants, and develop their sense of inquiry and communication skills? First, a Designed-Based Implementation Research approach will be used for the developmental evaluation to assess the implementation process. Next, the documentation and portrayal phase will assess the benefits to youth, educators, institutions, and the field using surveys, interviews, observations of educators, and reviews of science communication efforts created by youth. Finally, the summative evaluation includes a comprehensive portfolio of evidence to document the audience impacts and an independent assessment of the project model by an Evaluation Review Board. This project will result in the creation of a robust learning community while contributing knowledge and lessons learned to the field about networks and innovative partnerships. It is anticipated that formal and informal educators will gain increased knowledge about science and literacy programs and develop skills to provide effective programs, while youth will demonstrate increased understanding of key science concepts and the ability to communicate science. Programs created by the local partnerships will serve approximately 650 educators (450 informal educators and 200 K-12 teachers) and 500 youth ages 9-18. Plans for dissemination, expansion, and sustainability will be undertaken by the sub-networks of the collaborating national organizations drawing on the 350 ASTC member institutions and nearly 200 NWP sites at colleges and universities.
The Balboa Park Cultural Partnership, in collaboration with several informal science education and other cultural and business organizations in San Diego, Chicago, and Worcester, MA are implementing a research and development project that investigates a range of possible approaches for stimulating the development of 21st Century creativity skills and innovative processes at the interface between informal STEM learning and methods for creative thinking. The goal of the research is to advance understanding of the potential impacts of creative thinking methods on the public's understanding of and engagement with STEM, with a focus on 21st Century workforce skills of teens and adults. The goal of the project's development activities is to experiment with a variety of "innovation incubator" models in cities around the country. Modeled on business "incubators" or "accelerators" that are designed to foster and accelerate innovation and creativity, these STEM incubators generate collaborations of different professionals and the public around STEM education and other STEM-related topics of local interest that can be explored with the help of creative learning methodologies such as innovative methods to generate creative ideas, ideas for transforming one STEM idea to others, drawing on visual and graphical ideas, improvisation, narrative writing, and the process of using innovative visual displays of information for creating visual roadmaps. Hosting the project's incubators are the Balboa Park Cultural Partnership (San Diego), the Museum of Science and Industry (Chicago) and the EcoTarium (Worcester, MA). National partners are the Association of Science-Technology Centers, the American Association for the Advancement of Science, and the Americans for the Arts. Activities will include: the formation and collaborative processes of three incubator sites, a research study, the development of a creative thinking curriculum infused into science education, professional development based on the curriculum, public engagement events and exhibits, a project website and tools for social networking, and project evaluation. A national advisory council includes professionals in education, science, creativity, and business.
Portal to the Public: Expanding the National Network (PoP: ENN) is implementing around the county the successful NSF-funded Portal to the Public model in which researchers are trained to communicate and interact with the general public at informal science education (ISE) institutions about the research that they are conducting. The project, which follows on a thorough evaluation of the model at eight sites and current implementation at an additional fifteen sites, will incorporate twenty new ISE sites into the growing network, provide training and mentorship to ISE professionals on the use and adaptation of the PoP implementation manual and toolkits, and develop an enhanced network website that will serve as a communication and innovation hub. The work is responsive to the needs and activities of ISE organizations which continue to expand their missions beyond presenting to the public established science, technology, engineering and math (STEM) and are working to become places where visitors can also experience the process and promise of current research via face-to-face interactions with researchers. The project is expanding both the kind and number of institutions involved around the country and is facilitating their capacity to develop a knowledge base, share experiences and best practices.
The aim of this project is to create conversations in science museums among scientists, engineers, and public audiences about an emerging research field, synthetic biology. Synthetic biology applies science and engineering to create new biological systems, and re-design existing biological systems, for useful purposes. This is an important new area of research and development that raises societal questions about potential benefits, costs, and risks. Conversations between researchers and public audiences will focus not only on what synthetic biology is and how research in the field is carried out, but also on the potential products, outcomes, and implications for society of this work. Researchers and publics will explore personal and societal values and priorities as well as desired research outcomes so that both groups can learn from each other. Public participants will benefit from knowing about this field of research, and researchers will benefit from hearing public perspectives directly from the public participants. This project will be led by the Museum of Science with partners at the American Association for the Advancement of Science, the Synthetic Biology Engineering Research Center, the Science Museum of Minnesota, the Ithaca Sciencenter, and several other universities and science museums. It is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants. This project is aimed at pushing beyond traditional modes of communicating with public audiences rooted in "public understanding of science" modalities into the mechanisms and perspectives associated with "public engagement with science" (PES). The project will support informal educational institutions as facilitators of such PES activities through which mutual learning takes place among research experts and various publics. Formative evaluation will support the development of evaluation tools that practitioners can use themselves to measure impacts of public engagement activities on both scientist and public participants. Summative evaluation will measure the impacts of the project on informal science education practitioners and researchers participating in the development of the project. In the first year of the project, two kinds of engagement activities will be tested at eight pilot sites across the U.S. The first kind will be the focus of "showcase" events, in which researchers demonstrate and talk with museum visitors about the basics of synthetic biology and their research work. The second kind will be the focus of "forum" events in which the multi-directional conversations focus on societal implications and participants' priorities for maximizing the benefits of this new field while minimizing the risks. The work of the first year will inform development of a kit of public engagement materials that will support widespread public engagement with synthetic biology in the second year at up to 200 sites across the U.S. Successful practices and infrastructure developed by the Nanoscale Informal Science Education Network to support NanoDays events will be use for this broad dissemination of public engagement in synthetic biology in year 2. When the project is complete a set of tools and guides will be provided online for developing, implementing, and evaluating engagement events that bring scientists and publics together, specifically about synthetic biology, but adaptable to other emerging research topics. The informal science education field will have a better understanding of how to get scientists, engineers, and publics to engage together in discussions about the societal implications of emerging technologies, and how to evaluate the quality of that engagement for both the researchers and the publics involved. The project will also provide a sense of informed public views on societal issues related to synthetic biology that emerge through a variety of public engagement activities that take place in science museums.
The Cyberlearning Resource Center (CRC) has responsibility for promoting integrative collaboration among cyberlearning grantees (across NSF programs); synthesis and national dissemination of cyberlearning findings, technologies, models, materials, and best practices; creating a national presence for Cyberlearning; helping the disparate Cyberlearning research and development communities coordinate efforts to build capacity; and providing infrastructure (technological and social) for supporting these efforts. Monitored through the Cyberlearning: Transforming Education program, the CRC serves as a resource for all NSF grantees and programs with cyberlearning components, helping to promote synergy and integrate projects across NSF's cyberlearning investments. Among society's central challenges are amplifying, expanding, and transforming opportunities people have for learning and more effectively drawing in, motivating, and engaging young learners. Engaging actively as a citizen and productively in the workforce requires understanding a broad variety of concepts and possessing the ability to collaborate, learn, solve problems, and make decisions. Whether learning is facilitated in school or out of school, and whether learners are youngsters or adults, to develop such knowledge and capabilities, learners must be motivated to learn, actively engage over the long term in learning activities, and put forth sustained cognitive and social effort. Consistent with NSF's mission and strategic plan, a variety of programs at NSF invest in research aimed towards achieving these goals. In support of this important thematic thrust, the Cyberlearning Resource Center works with researchers and NSF program officers to identify and disseminate findings from across programs and projects; develop ways to broker productive partnerships and collaborations; convene meetings for purposes of envisioning the future, integrating findings, and building capacity,; and monitor the cyberlearning portfolio and its influences and impacts.
The Global Soundscapes! Big Data, Big Screens, Open Ears Project uses the new science of soundscape ecology to design a variety of informal science learning experiences that engage participants through acoustic discovery Soundscape ecology is an interdisciplinary science that studies how humans relate to place through sound and how humans influence the environment through the alteration of natural sound composition. The project includes: (1) an interface to the NSF-funded Global Sustainable Soundscapes Network, which includes 12 universities around the world; (2) sound-based learning experiences targeting middle-school students (grades 5-8), visually impaired and urban students, and the general public; and (3) professional development for informal science educators. Project educational components include: the first interactive, sound-based digital theater experience; hands-on Your Ecosystem Listening Labs (YELLS), a 1-2 day program for school classes and out-of school groups; a soundscape database that will assist researchers in developing a soundscape Big Database; and iListen, a virtual online portal for learning and discovery about soundscape. The project team includes Purdue-based researchers involved in soundscape and other ecological research; Foxfire Interactive, an award-winning educational media company; science museum partners with digital theaters; the National Audubon Society and its national network of field stations; the Perkins School for the Blind; and Multimedia Research (as the external evaluator).
The National Girls Collaborative Project (NGCP) seeks to maximize access to shared resources within projects and with public and private sector organizations and institutions interested in expanding girls’ participation in science, technology, engineering, and mathematics (STEM). Funded primarily by the National Science Foundation, the NGCP is a robust national network of more than 3,000 girl-serving STEM organizations. Currently, 31 Collaboratives, serving 40 states, facilitate collaboration between more than 12,800 organizations who serve more than 7.7 million girls and 4.4 million boys. The NGCP occupies a unique role in the STEM community because it facilitates collaboration with all stakeholders who benefit from increasing diversity and engagement of women in STEM. These stakeholders form Regional Collaboratives, who are connected to local girl-serving STEM programs. Regional Collaboratives are led by leadership teams and advisory boards with representatives from K-12 education, higher education, community-based organizations, professional organizations, and industry. NGCP strengthens the capacity of girl-serving STEM projects by facilitating collaboration among programs and organizations and by sharing promising practice research, program models, and products through webinars, collaboration training, and institutes. This is accomplished through a tested comprehensive program of change that uses collaboration to expand and strengthen STEM-related opportunities for girls and women. In each replication state, the NGCP model creates a network of professionals, researchers, and practitioners, facilitating collaboration within this network, and delivering high-quality research-based professional development. Participating programs can also receive mini-grant funding to develop collaborative STEM-focused projects. To date, over 27,000 participants have been served in 241 mini-grant projects, and over 17,000 practitioners have been served through in-person events and webinars. The NGCP’s collaborative model changes the way practitioners and educators work to advance girls’ participation in STEM. It facilitates the development of practitioners in their knowledge of good gender equitable educational practices, awareness of the role of K-12 education in STEM workforce development, and mutual support of peers locally and across the United States.