EvaluATE is a national resource center dedicated to supporting and improving the evaluation practices of approximately 250 ATE grantees across the country. EvaluATE conducts webinars and workshops, publishes a quarterly newsletter, maintains a website with a digital resource library, develops materials to guide evaluation work, and conducts an annual survey of ATE grantees. EvaluATE's mission is to promote the goals of the ATE program by partnering with projects and centers to strengthen the program's evaluation knowledge base, expand the use of exemplary evaluation practices, and support the continuous improvement of technician education throughout the nation. EvaluATE's goals associated with this proposal are to: (1) Ensure that all ATE Principal Investigators and evaluators know the essential elements of a credible and useful evaluation; (2) Maintain a comprehensive collection of online resources for ATE evaluation; (3) Strengthen and expand the network of ATE evaluation stakeholders; and (4) Gather, synthesize, and disseminate data about the ATE program activities to advance knowledge about ATE/technician education. The Center plans to produce a comprehensive set of evaluation resources to complement other services, engaging several community college-based Principal Investigators and evaluators in that process.
EvaluATE's products are informed by current research on evaluation, the National Science Foundation's priorities for the evaluation of ATE grants, and the needs of ATE PIs and evaluators for sound guidance that is immediately relevant and usable in their contexts. The fundamental nature of EvaluATE's work is geared toward supporting ATE grantees to use evaluation regularly to improve their work and demonstrate their impacts. All of EvaluATE's products are available to the public. EvaluATE's findings from the annual survey of ATE grantees aid in advancing understanding of the status of technician education and illuminate areas for additional research. The new survey investigates ATE grantees' work to serve underrepresented and special populations, including women, people of color, and veterans. Survey data are available upon request for research and evaluation purposes.
DATE:
-
TEAM MEMBERS:
Lori WingateArlen GullicksonEmma PerkKelly RobertsonLyssa Becho
resourceevaluationProfessional Development, Conferences, and Networks
The Summative Study of the Nano Mini-exhibition took place during the spring and summer of 2012. After being observed during their Mini-exhibition experience, 455 visitors across six different partner institutions participated in surveys and interviews with NISE Net evaluation team members. This report begins by describing the key findings of the study in detail, with additional information about study methods, instruments, and two exploratory sub-studies found in the Appendices.
The Complex Adaptive Systems as a Model for Network Evaluations (CASNET) study was a four-year research project investigating evaluation capacity building (ECB) within a network using a complexity theory lens. The study used a case study approach to examine and understand evaluation capacity building within the Nanoscale Informal Science Education Network (NISE Net). NISE Net is a national community of researchers and informal science educators dedicated to fostering public awareness, engagement, and understanding of nanoscale science, engineering, and technology. Instituted in 2005 through NSF funding (DRL-0532536 and 0940143), NISE Net has continuously expanded and is currently comprised of close to 600 science museum and university partners. The intent of the CASNET project was to provide insights on (1) the implications of complexity theory for promoting widespread and systemic use of evaluation within a network, and (2) complex system conditions that foster or impede ECB within a network, i.e., in this case, within the NISE Net.
The Center for Advancement of Informal Science Education (CAISE), a cooperative agreement with the National Science Foundation Advancing Informal STEM Learning (AISL) program, is a partnership of the Association of Science-Technology Centers with faculty and professionals from the University of Pittsburgh Center for Learning in Out-of-School Environments (UPCLOSE), Oregon State University (OSU), the Great Lakes Science Center, KQED Public Media, advisors and other collaborators. CAISE works to support and resource ongoing improvement of, and NSF investments in, the national infrastructure for informal Science Technology Engineering and Mathematics (STEM) education. CAISE's roles are to build capacity and support continued professionalization for the field by fostering a community that bridges the many varied forms in which informal STEM learning experiences are developed and delivered for learners of all ages. To that end, CAISE activities also include: creating field-driven evidence databases about the impacts of informal STEM education; facilitating federated searches of those databases; furthering dialogue and knowledge transfer between learning research and practice; working to enhance the quality and diversity of evaluation knowledge and processes; and helping STEM researchers improve their efforts in informal STEM education, outreach and communication. For Principal Investigators (PIs) and potential PIs, CAISE provides resources that can assist in the development of evidence-based proposals. It also facilitates and strengthens networks through PI meetings, communications, and other methods that encourage sharing of deliverables, practices, outcomes and findings across projects. For the AISL Program at NSF, CAISE is assisting program officers in understanding the portfolio of awards, identifying the portfolio's impacts in key areas, and integrating the program's investments in education infrastructure.
The National Writing Project (NWP) is collaborating with the Association of Science-Technology Centers (ASTC) on a four-year, full-scale development project that is designed to integrate science and literacy. Partnerships will be formed between NWP sites and ASTC member science centers and museums to develop, test, and refine innovative programs for educators and youth, resulting in the creation of a unique learning network. The project highlights the critical need for the integration of science and literacy and builds on recommendations in the Common Core State Standards and the National Research Council's publication, "A Framework for K-12 Science Education: Practices, Crosscutting Concepts, and Core Ideas." The content focus includes current topics in science and technology such as environmental science, sustainability, synthetic biology, geoengineering, and other subjects which align with science center research and exhibits. The project design is supported by a framework that incorporates a constructivist/inquiry-based approach that capitalizes on the synergy between rigorous science learning and robust literacy practices. Project deliverables include a set of 10 local partnership sites, professional development for network members, a project website, and an evaluation report highlighting lessons learned. Partnership sites will be selected based on interest, proximity, history, and expertise. Two geographically and demographically diverse cohorts, consisting of five partnerships each will be identified in Years 2 and 3. Each set of partners will be charged with creating a comprehensive two-year plan for science literacy activities and products to be implemented at local sites. It is anticipated that the pilot programs may result in the creation of new programs that merge science and writing, integrate writing into existing museum science programs, or integrate science activities into existing NWP programs. Interest-driven youth projects such as citizen science and science journalism activities are examples of programmatic approaches that may be adopted. The partners will convene periodically for planning and professional development focused on the integration of science and literacy for public and professional audiences, provided in part by national practitioners and research experts. A network Design Team that includes leadership representatives from NWP, ASTC, and the project evaluator, Inverness Research, Inc., will oversee project efforts in conjunction with a national advisory board, while a Partnership Coordinator will provide support for the local sites. Inverness Research will conduct a multi-level evaluation to address the following questions: -What is the nature and quality of the local partner arrangements, and the larger network as a whole? -What is the nature and quality of the local science literacy programs that local partners initiate, and how do they engage local participants, and develop their sense of inquiry and communication skills? First, a Designed-Based Implementation Research approach will be used for the developmental evaluation to assess the implementation process. Next, the documentation and portrayal phase will assess the benefits to youth, educators, institutions, and the field using surveys, interviews, observations of educators, and reviews of science communication efforts created by youth. Finally, the summative evaluation includes a comprehensive portfolio of evidence to document the audience impacts and an independent assessment of the project model by an Evaluation Review Board. This project will result in the creation of a robust learning community while contributing knowledge and lessons learned to the field about networks and innovative partnerships. It is anticipated that formal and informal educators will gain increased knowledge about science and literacy programs and develop skills to provide effective programs, while youth will demonstrate increased understanding of key science concepts and the ability to communicate science. Programs created by the local partnerships will serve approximately 650 educators (450 informal educators and 200 K-12 teachers) and 500 youth ages 9-18. Plans for dissemination, expansion, and sustainability will be undertaken by the sub-networks of the collaborating national organizations drawing on the 350 ASTC member institutions and nearly 200 NWP sites at colleges and universities.
The Balboa Park Cultural Partnership, in collaboration with several informal science education and other cultural and business organizations in San Diego, Chicago, and Worcester, MA are implementing a research and development project that investigates a range of possible approaches for stimulating the development of 21st Century creativity skills and innovative processes at the interface between informal STEM learning and methods for creative thinking. The goal of the research is to advance understanding of the potential impacts of creative thinking methods on the public's understanding of and engagement with STEM, with a focus on 21st Century workforce skills of teens and adults. The goal of the project's development activities is to experiment with a variety of "innovation incubator" models in cities around the country. Modeled on business "incubators" or "accelerators" that are designed to foster and accelerate innovation and creativity, these STEM incubators generate collaborations of different professionals and the public around STEM education and other STEM-related topics of local interest that can be explored with the help of creative learning methodologies such as innovative methods to generate creative ideas, ideas for transforming one STEM idea to others, drawing on visual and graphical ideas, improvisation, narrative writing, and the process of using innovative visual displays of information for creating visual roadmaps. Hosting the project's incubators are the Balboa Park Cultural Partnership (San Diego), the Museum of Science and Industry (Chicago) and the EcoTarium (Worcester, MA). National partners are the Association of Science-Technology Centers, the American Association for the Advancement of Science, and the Americans for the Arts. Activities will include: the formation and collaborative processes of three incubator sites, a research study, the development of a creative thinking curriculum infused into science education, professional development based on the curriculum, public engagement events and exhibits, a project website and tools for social networking, and project evaluation. A national advisory council includes professionals in education, science, creativity, and business.
Expanding on the encouraging outcomes of an NSF-funded conference, this three-year project led by the National Center for Science and Civic Engagement at Harrisburg University of Science and Technology, in collaboration with the Koshland Science Museum of the National Academy of Sciences, will explore and evaluate ways to support new collaborations between professionals in institutions of higher education and informal STEM education around areas of common interest. The primary goal is to develop the educational infrastructure to grow and efficiently sustain multiple cross-organizational partnership activities at the intersection of learning about science, society and civic engagement around such possible topics as energy, environment, genetics, earth resources, computers and ethics, nanotechnology, etc. The initiative is: 1) creating a joint organizing "secretariat" to provide communications and support through low-cost shared services for at least six partnerships around the country; 2) providing partnership support and technical assistance to seed the six national partnerships, and 3) sharing evaluation and analysis services across all the partnerships. The outcomes of the work pertain to improvements in professional knowledge and practice in higher education and informal science education, as well as the improvement of learning by undergraduates and by the general public.
Portal to the Public: Expanding the National Network (PoP: ENN) is implementing around the county the successful NSF-funded Portal to the Public model in which researchers are trained to communicate and interact with the general public at informal science education (ISE) institutions about the research that they are conducting. The project, which follows on a thorough evaluation of the model at eight sites and current implementation at an additional fifteen sites, will incorporate twenty new ISE sites into the growing network, provide training and mentorship to ISE professionals on the use and adaptation of the PoP implementation manual and toolkits, and develop an enhanced network website that will serve as a communication and innovation hub. The work is responsive to the needs and activities of ISE organizations which continue to expand their missions beyond presenting to the public established science, technology, engineering and math (STEM) and are working to become places where visitors can also experience the process and promise of current research via face-to-face interactions with researchers. The project is expanding both the kind and number of institutions involved around the country and is facilitating their capacity to develop a knowledge base, share experiences and best practices.
The STAR Library Education Network: a hands-on learning program for libraries and their communities, (STAR_Net for short) is led by the National Center for Interactive Learning (NCIL) at the Space Science Institute (PI: Paul Dusenbery). STAR stands for Science-Technology, Activities and Resources. Team members include NCIL staff, the American Library Association (ALA), Lunar and Planetary Institute (LPI), and the National Girls Collaborative Project (NGCP). STAR_Net is developing two comprehensive, informal education programs: Discover Earth and Discover Tech. The project also includes a comprehensive evaluation plan and a research component that explores how public libraries can serve as a STEM learning center in rural, underserved communities. STAR_Net is supported through a grant from the National Science Foundation. The STAR_Net project includes two traveling library exhibits: Discover Earth: A Century of Change and Discover Tech: Engineers Make a World of Difference. The Discover Earth exhibition features interactive, multimedia displays that allow exhibit visitors to interact with digital information in a dynamic way, encouraging new perspectives on our planet. Discover Tech introduces the many extraordinary ways that engineers solve problems to help people and societies around the world. Similar to a science center experience, visitors and families will be able to explore and tinker with their own engineering solutions. A number of STEM activities and resources will be developed by project staff and by other organizations to help librarians and community partners offer a wide variety of programs for their patrons. Besides the traveling exhibits and programs, STAR_Net also includes library staff training (online and in-person) and a Community of Practice (CoP) for librarians (including non-host librarians) to interact and partner with STEM professionals and organizations. NCI's Kate Haley Goldman and staff from Evaluation and Research Associates are conducting the project's evaluation.
The Coalition for Science After School (CSAS) was established in 2004 in response to the growing need for more STEM (science, technology, engineering, and mathematics) learning opportunities in out-of-school time. CSAS sought to build this field by uniting STEM education goals with out-of-school time opportunities and a focus on youth development. Over a decade of work, CSAS Steering Committee members, staff and partners advocated for STEM in out-of-school-time settings, convened leaders, and created resources to support this work. CSAS leadership decided to conclude CSAS operations in 2014, as the STEM in out-of-school time movement had experienced tremendous growth of programming and attention to science-related out-of-school time opportunities on a national level. In its ten-year strategic plan, CSAS took as its vision the full integration of the STEM education and out-of-school time communities to ensure that quality out-of-school time STEM opportunities became prevalent and available to learners nationwide. Key CSAS activities included: (1) Setting and advancing a collective agenda by working with members to identify gaps in the field, organizing others to create solutions that meet the needs, identifying policy needs in the field and supporting advocates to advance them; (2) Developing and linking committed communities by providing opportunities for focused networking and learning through conferences, webinars, and other outreach activities; and (3) Identifying, collecting, capturing, and sharing information and available research and resources in the field. The leadership of the Coalition for Science After School is deeply grateful to the funders, partners, supporters, and constituents that worked together to advance STEM in out-of-school time during the last decade, and that make up today's rich and varied STEM in out-of-school time landscape. We have much to be proud of, but as a movement there is much more work to be done. As this work continues to expand and deepen, it is appropriate for the Coalition for Science After School to step down as the many other organizations that have emerged over the last decade take on leadership for the critical work that remains to be done. A timeline and summary of CSAS activities, products, and accomplishments is available for download on this page. All resources noted in the narrative are also available for download below.
NASA‘s Johnson Space Center (JSC) and Space Center Houston (SCH) will build upon the successes of NASA‘s Summer of Innovation initiative and SCH‘s highly successful Exploration Academy by developing, planning and implementing a newly defined STEM experiential program – Spaceflight Explorers. This program will not only be sustainable for the future, but will also be able to be replicated throughout the nation‘s network of museums and science centers. The Spaceflight Explorers program will: - Provide customizable STEM education content (up to 40 contact hours) to support the implementation of STEM learning programming with an emphasis on Human Spaceflight (content to include experiential STEM experiences) - Engage underrepresented and underserved middle school students in STEM learning by utilizing NASA material - Provide professional development to educators patterned after our 18 years of experience with SEEC (Space Exploration Educators Conference) our highly successful Educators Conference held here for over 500 teachers from around the country. - Create a network of visitor centers, museums, and science centers that have an interest in promoting and sharing the excitement of human spaceflight with students, educators, and the general public. Upon receiving these resources, each museum or science center will be expected to offer summer STEM learning experiences, as outlined in the Spaceflight Explorers curriculum, at their facilities targeting underserved/under-represented populations of middle school students. By adopting and utilizing The Exploration Academy concept at their facility museums and science centers are making a commitment to support current and future NASA Human Space Flight endeavors and increase student and visitor awareness of Human Spaceflight.
The National Girls Collaborative Project (NGCP) seeks to maximize access to shared resources within projects and with public and private sector organizations and institutions interested in expanding girls’ participation in science, technology, engineering, and mathematics (STEM). Funded primarily by the National Science Foundation, the NGCP is a robust national network of more than 3,000 girl-serving STEM organizations. Currently, 31 Collaboratives, serving 40 states, facilitate collaboration between more than 12,800 organizations who serve more than 7.7 million girls and 4.4 million boys. The NGCP occupies a unique role in the STEM community because it facilitates collaboration with all stakeholders who benefit from increasing diversity and engagement of women in STEM. These stakeholders form Regional Collaboratives, who are connected to local girl-serving STEM programs. Regional Collaboratives are led by leadership teams and advisory boards with representatives from K-12 education, higher education, community-based organizations, professional organizations, and industry. NGCP strengthens the capacity of girl-serving STEM projects by facilitating collaboration among programs and organizations and by sharing promising practice research, program models, and products through webinars, collaboration training, and institutes. This is accomplished through a tested comprehensive program of change that uses collaboration to expand and strengthen STEM-related opportunities for girls and women. In each replication state, the NGCP model creates a network of professionals, researchers, and practitioners, facilitating collaboration within this network, and delivering high-quality research-based professional development. Participating programs can also receive mini-grant funding to develop collaborative STEM-focused projects. To date, over 27,000 participants have been served in 241 mini-grant projects, and over 17,000 practitioners have been served through in-person events and webinars. The NGCP’s collaborative model changes the way practitioners and educators work to advance girls’ participation in STEM. It facilitates the development of practitioners in their knowledge of good gender equitable educational practices, awareness of the role of K-12 education in STEM workforce development, and mutual support of peers locally and across the United States.