EvaluATE is a national resource center dedicated to supporting and improving the evaluation practices of approximately 250 ATE grantees across the country. EvaluATE conducts webinars and workshops, publishes a quarterly newsletter, maintains a website with a digital resource library, develops materials to guide evaluation work, and conducts an annual survey of ATE grantees. EvaluATE's mission is to promote the goals of the ATE program by partnering with projects and centers to strengthen the program's evaluation knowledge base, expand the use of exemplary evaluation practices, and support the continuous improvement of technician education throughout the nation. EvaluATE's goals associated with this proposal are to: (1) Ensure that all ATE Principal Investigators and evaluators know the essential elements of a credible and useful evaluation; (2) Maintain a comprehensive collection of online resources for ATE evaluation; (3) Strengthen and expand the network of ATE evaluation stakeholders; and (4) Gather, synthesize, and disseminate data about the ATE program activities to advance knowledge about ATE/technician education. The Center plans to produce a comprehensive set of evaluation resources to complement other services, engaging several community college-based Principal Investigators and evaluators in that process.
EvaluATE's products are informed by current research on evaluation, the National Science Foundation's priorities for the evaluation of ATE grants, and the needs of ATE PIs and evaluators for sound guidance that is immediately relevant and usable in their contexts. The fundamental nature of EvaluATE's work is geared toward supporting ATE grantees to use evaluation regularly to improve their work and demonstrate their impacts. All of EvaluATE's products are available to the public. EvaluATE's findings from the annual survey of ATE grantees aid in advancing understanding of the status of technician education and illuminate areas for additional research. The new survey investigates ATE grantees' work to serve underrepresented and special populations, including women, people of color, and veterans. Survey data are available upon request for research and evaluation purposes.
DATE:
-
TEAM MEMBERS:
Lori WingateArlen GullicksonEmma PerkKelly RobertsonLyssa Becho
resourceprojectProfessional Development, Conferences, and Networks
The National Writing Project (NWP) is collaborating with the Association of Science-Technology Centers (ASTC) on a four-year, full-scale development project that is designed to integrate science and literacy. Partnerships will be formed between NWP sites and ASTC member science centers and museums to develop, test, and refine innovative programs for educators and youth, resulting in the creation of a unique learning network. The project highlights the critical need for the integration of science and literacy and builds on recommendations in the Common Core State Standards and the National Research Council's publication, "A Framework for K-12 Science Education: Practices, Crosscutting Concepts, and Core Ideas." The content focus includes current topics in science and technology such as environmental science, sustainability, synthetic biology, geoengineering, and other subjects which align with science center research and exhibits. The project design is supported by a framework that incorporates a constructivist/inquiry-based approach that capitalizes on the synergy between rigorous science learning and robust literacy practices. Project deliverables include a set of 10 local partnership sites, professional development for network members, a project website, and an evaluation report highlighting lessons learned. Partnership sites will be selected based on interest, proximity, history, and expertise. Two geographically and demographically diverse cohorts, consisting of five partnerships each will be identified in Years 2 and 3. Each set of partners will be charged with creating a comprehensive two-year plan for science literacy activities and products to be implemented at local sites. It is anticipated that the pilot programs may result in the creation of new programs that merge science and writing, integrate writing into existing museum science programs, or integrate science activities into existing NWP programs. Interest-driven youth projects such as citizen science and science journalism activities are examples of programmatic approaches that may be adopted. The partners will convene periodically for planning and professional development focused on the integration of science and literacy for public and professional audiences, provided in part by national practitioners and research experts. A network Design Team that includes leadership representatives from NWP, ASTC, and the project evaluator, Inverness Research, Inc., will oversee project efforts in conjunction with a national advisory board, while a Partnership Coordinator will provide support for the local sites. Inverness Research will conduct a multi-level evaluation to address the following questions: -What is the nature and quality of the local partner arrangements, and the larger network as a whole? -What is the nature and quality of the local science literacy programs that local partners initiate, and how do they engage local participants, and develop their sense of inquiry and communication skills? First, a Designed-Based Implementation Research approach will be used for the developmental evaluation to assess the implementation process. Next, the documentation and portrayal phase will assess the benefits to youth, educators, institutions, and the field using surveys, interviews, observations of educators, and reviews of science communication efforts created by youth. Finally, the summative evaluation includes a comprehensive portfolio of evidence to document the audience impacts and an independent assessment of the project model by an Evaluation Review Board. This project will result in the creation of a robust learning community while contributing knowledge and lessons learned to the field about networks and innovative partnerships. It is anticipated that formal and informal educators will gain increased knowledge about science and literacy programs and develop skills to provide effective programs, while youth will demonstrate increased understanding of key science concepts and the ability to communicate science. Programs created by the local partnerships will serve approximately 650 educators (450 informal educators and 200 K-12 teachers) and 500 youth ages 9-18. Plans for dissemination, expansion, and sustainability will be undertaken by the sub-networks of the collaborating national organizations drawing on the 350 ASTC member institutions and nearly 200 NWP sites at colleges and universities.
Currently, many museums present histories of science and technology, but very few are integrating scientific activity--observation, measurement, experimentation-with the time- and place-specific narratives that characterize history-learning experiences. For the Prairie Science project, Conner Prairie is combining proven science center-style activities, developed by the Science Museum of Minnesota, with family-engagement strategies developed through extensive research and testing with audiences in historical settings. The goal of this integration is to create guest experiences that are rich in both STEM and historical content and encourage family learning. One key deliverable of this project is the Create.Connect gallery, which is currently installed at Conner Prairie. Create.Connect allows the project team to evaluate and research hands-on activities, facilitation strategies and historic settings to understand how these elements combine to encourage family conversations and learning around historical narratives and STEM content. For example, in one exhibit area families can experiment with creating their own efficient wind turbine designs while learning about the innovations of the Flint & Walling windmill manufacturing company from Indiana. The activity is facilitated by a historic interpreter portraying a windmill salesman from 1900. The interpreter not only guides the family though the process of scientific inquiry, but shares his historic perspective on wind power as well. Two other exhibit areas invite hands-on exploration of electrical circuits and forces in motion as they connect to stories from Indiana history. Evaluation and research findings from the Create.Connect exhibit will be used to develop a model that can guide other history institutions that want to incorporate STEM content and thinking into their exhibits and interpretation. By partnering with the Science Museum of Minnesota, we will combine the experience of science center professionals and history museum professionals to find the best practices for incorporating science activities into historic settings. To ensure that this dissemination model is informed from many perspectives, Conner Prairie has invited the participation of four history museums: The Museum of America and the Sea, Mystic, Connecticut; the California State Railroad Museum, Sacramento, California; the Wabash County Historical Society, Wabash, Indiana; and the Oliver H. Kelley Farm, Elk River, Minnesota. Each of the four participants will install history-STEM exhibit components which will be connected to location-specific historic narratives. Drawing on the staff experience and talents of participant museums, this project will develop realistic solutions to an array of anticipated barriers. These issues and the resulting approaches will become part of a stronger, more adaptable dissemination model that will support history museums in creating STEM-based guest experiences.
Through the Scientists for Tomorrow pathways project, The Science Institute at Columbia College in Chicago will test a model for preparing non-science major, pre-service elementary school teachers to deliver three ten-week informal science education modules to youth in after school programs. The initiative will bring engineering concepts, environmental science, and technology to approximately 240 urban Chicago youth (ages 10-14 years old) and their families. The Science Institute will partner with eight minority serving community based organizations and the Museum of Science and Industry, the Field Museum, and the Garfield Park Conservatory Alliance to develop and implement all aspects of the program. The goals of the program are two-fold. First, the project will develop and implement a high-quality STEM based afterschool program for under-represented youth in STEM. Second, the professional development and experience implementing the curriculum with youth in the local communities and within informal science education (ISE) institutions will extend and enrich the pre-service teachers\' STEM content and pedagogical knowledge base and better prepare them to teach science in formal and informal settings. Thirty teachers will receive specialized professional development through a seminar, course, and other support mechanisms in order to best support the implementation of the modules, while building their STEM content expertise, confidence, and pedagogical knowledge. Each module has a different STEM content focus: alternative energy (fall), the physics and mathematics of sound and music (winter), and environmental science (spring). At the end of each module, a culminating youth-led presentation will be held at one of the partnering Chicago museums. Youth will be encouraged to participate in all three modules. The formative evaluation will be conducted by the Co-Principal Investigators. Pre and post assessments, artifact reviews, and interviews will be used for the summative evaluation, which will be conducted by an external evaluator at the Illinois Institute of Technology. The project deliverables include: (a) a teacher training program, (b) an after school curriculum, and (c) media tools - DVDs, website. Over the grant period, the project intends to reach 120 youth each year, over 100 family and community members, and 30 teachers. The larger impact of this project will be the development of a scalable model for bringing relevant STEM content and experiences to youth, their families, and non-science major pre-service teachers. As a result of this project, a cadre of pre-service teachers will have: (a) increased their STEM content knowledge, (b) gained experience presenting STEM content in informal settings, (c) learned effective approaches to deliver hands-on STEM content, and (d) learned to use museum and other ISE resources in their teaching. In fact, after the grant period nearly half of the teachers will continue to work at the centers as part-time instructors, fully supported by the partnering community centers.
The Balboa Park Cultural Partnership, in collaboration with several informal science education and other cultural and business organizations in San Diego, Chicago, and Worcester, MA are implementing a research and development project that investigates a range of possible approaches for stimulating the development of 21st Century creativity skills and innovative processes at the interface between informal STEM learning and methods for creative thinking. The goal of the research is to advance understanding of the potential impacts of creative thinking methods on the public's understanding of and engagement with STEM, with a focus on 21st Century workforce skills of teens and adults. The goal of the project's development activities is to experiment with a variety of "innovation incubator" models in cities around the country. Modeled on business "incubators" or "accelerators" that are designed to foster and accelerate innovation and creativity, these STEM incubators generate collaborations of different professionals and the public around STEM education and other STEM-related topics of local interest that can be explored with the help of creative learning methodologies such as innovative methods to generate creative ideas, ideas for transforming one STEM idea to others, drawing on visual and graphical ideas, improvisation, narrative writing, and the process of using innovative visual displays of information for creating visual roadmaps. Hosting the project's incubators are the Balboa Park Cultural Partnership (San Diego), the Museum of Science and Industry (Chicago) and the EcoTarium (Worcester, MA). National partners are the Association of Science-Technology Centers, the American Association for the Advancement of Science, and the Americans for the Arts. Activities will include: the formation and collaborative processes of three incubator sites, a research study, the development of a creative thinking curriculum infused into science education, professional development based on the curriculum, public engagement events and exhibits, a project website and tools for social networking, and project evaluation. A national advisory council includes professionals in education, science, creativity, and business.
'Be a Scientist!' is a full-scale development project that examines the impact of a scalable, STEM afterschool program which trains engineers to develop and teach inquiry-based Family Science Workshops (FSWs) in underserved communities. This project builds on three years of FSWs which demonstrate improvements in participants' science interest, knowledge, and self-efficacy and tests the model for scale, breadth, and depth. The project partners include the Viterbi School of Engineering at the University of Southern California, the Albert Nerken Engineering Department at the Cooper Union, the Los Angeles Museum of Natural History, and the New York Hall of Science. The content emphasis is physics and engineering and includes topics such as aerodynamics, animal locomotion, automotive engineering, biomechanics, computer architecture, optics, sensors, and transformers. The project targets underserved youth in grades 1-5 in Los Angeles and New York, their parents, and engineering professionals. The design is grounded in motivation theory and is intended to foster participants' intrinsic motivation and self-direction while the comprehensive design takes into account the cultural, social, and intellectual needs of diverse families. The science activities are provided in a series of Family Science Workshops which take place in afterschool programs in eight partner schools in Los Angeles and at the New York Hall of Science in New York City. The FSWs are taught by undergraduate and graduate engineering students with support from practicing engineers who serve as mentors. The primary project deliverable is a five-year longitudinal evaluation designed to assess (1) the impact of intensive training for engineering professionals who deliver family science activities in community settings and (2) families' interest in and understanding of science. Additional project deliverables include a 16-week training program for engineering professionals, 20 physics-based workshops and lesson plans, Family Science Workshops (40 in LA and 5 in NY), a Parent Leadership Program and social networking site, and 5 science training videos. This project will reach nearly one thousand students, parents, and student engineers. The multi-method evaluation will be conducted by the Center for Children and Technology at the Education Development Center. The evaluation questions are as follows: Are activities such as recruitment, training, and FSWs aligned with the project's goals? What is the impact on families' interest in and understanding of science? What is the impact on engineers' communication skills and perspectives about their work? Is the project scalable and able to produce effective technology tools and develop long-term partnerships with schools? Stage 1 begins with the creation of a logic model by stakeholders and the collection of baseline data on families' STEM experiences and knowledge. Stage 2 includes the collection of formative evaluation data over four years on recruitment, training, co-teaching by informal educators, curriculum development, FSWs, and Parent Leadership Program implementation. Finally, a summative evaluation addresses how well the project met the goals associated with improving families' understanding of science, family involvement, social networking, longitudinal impact, and scalability. A comprehensive dissemination plan extends the project's broader impacts in the museum, engineering, evaluation, and education professional communities through publications, conference presentations, as well as web 2.0 tools such as blogs, YouTube, an online social networking forum for parents, and websites. 'Be a Scientist!' advances the field through the development and evaluation of a model for sustained STEM learning experiences that helps informal science education organizations broaden participation, foster collaborations between universities and informal science education organizations, increase STEM-based social capital in underserved communities, identify factors that develop sustained interest in STEM, and empower parents to co-invest and sustain a STEM program in their communities.
Portal to the Public: Expanding the National Network (PoP: ENN) is implementing around the county the successful NSF-funded Portal to the Public model in which researchers are trained to communicate and interact with the general public at informal science education (ISE) institutions about the research that they are conducting. The project, which follows on a thorough evaluation of the model at eight sites and current implementation at an additional fifteen sites, will incorporate twenty new ISE sites into the growing network, provide training and mentorship to ISE professionals on the use and adaptation of the PoP implementation manual and toolkits, and develop an enhanced network website that will serve as a communication and innovation hub. The work is responsive to the needs and activities of ISE organizations which continue to expand their missions beyond presenting to the public established science, technology, engineering and math (STEM) and are working to become places where visitors can also experience the process and promise of current research via face-to-face interactions with researchers. The project is expanding both the kind and number of institutions involved around the country and is facilitating their capacity to develop a knowledge base, share experiences and best practices.
The STAR Library Education Network: a hands-on learning program for libraries and their communities, (STAR_Net for short) is led by the National Center for Interactive Learning (NCIL) at the Space Science Institute (PI: Paul Dusenbery). STAR stands for Science-Technology, Activities and Resources. Team members include NCIL staff, the American Library Association (ALA), Lunar and Planetary Institute (LPI), and the National Girls Collaborative Project (NGCP). STAR_Net is developing two comprehensive, informal education programs: Discover Earth and Discover Tech. The project also includes a comprehensive evaluation plan and a research component that explores how public libraries can serve as a STEM learning center in rural, underserved communities. STAR_Net is supported through a grant from the National Science Foundation. The STAR_Net project includes two traveling library exhibits: Discover Earth: A Century of Change and Discover Tech: Engineers Make a World of Difference. The Discover Earth exhibition features interactive, multimedia displays that allow exhibit visitors to interact with digital information in a dynamic way, encouraging new perspectives on our planet. Discover Tech introduces the many extraordinary ways that engineers solve problems to help people and societies around the world. Similar to a science center experience, visitors and families will be able to explore and tinker with their own engineering solutions. A number of STEM activities and resources will be developed by project staff and by other organizations to help librarians and community partners offer a wide variety of programs for their patrons. Besides the traveling exhibits and programs, STAR_Net also includes library staff training (online and in-person) and a Community of Practice (CoP) for librarians (including non-host librarians) to interact and partner with STEM professionals and organizations. NCI's Kate Haley Goldman and staff from Evaluation and Research Associates are conducting the project's evaluation.
EDC’s Center for Children and Technology (CCT), a nonprofit research and development organization (cct.edc.org), conducted the formative evaluation of the BAS project for the last three years. Iridescent has assisted CCT researchers in the successful implementation of the evaluation (e.g., organizing site visits and meetings with partners, administering surveys, collecting consent forms). As discussed in more details below, Iridescent has always taken seriously the evaluation findings and recommendations, and has acted upon them to make program improvements. This research partnership has led
Making Stuff Season Two is designed to build on the success of the first season of Making Stuff by expanding the series content to include a broader range of STEM topics, creating a larger outreach coalition model and a “community of practice,” and developing new outreach activities and digital resources. Specifically, this project created a national television 4-part miniseries, an educational outreach campaign, expanded digital content, promotion activities, station relations, and project evaluation. These project components help to achieve the following goals: 1. To increase public understanding that basic research leads to technological innovation; 2. To increase and sustain public awareness and excitement about innovation and its impact on society; and 3. To establish a community of practice that enhances the frequency and quality of collaboration among STEM researchers and informal educators. These goals were selected in order to address a wider societal issue, and an important element of the overall mission of NOVA: to inspire new generations of scientists, learners, and innovators. By creating novel and engaging STEM content, reaching out to new partners, and developing new outreach tools, the second season of Making Stuff is designed to reach new target audiences including underserved teens and college students crucial to building a more robust and diversified STEM workforce pipeline. Series Description: In this four-part special, technology columnist and best-selling author David Pogue takes a wild ride through the cutting-edge science that is powering a next wave of technological innovation. Pogue meets the scientists and engineers who are plunging to the bottom of the temperature scale, finding design inspiration in nature, and breaking every speed limit to make tomorrow's "stuff" "Colder," "Faster," "Safer," and "Wilder." Making Stuff Faster Ever since humans stood on two feet we have had the basic urge to go faster. But are there physical limits to how fast we can go? David Pogue wants to find out, and in "Making Stuff Faster," he’ll investigate everything from electric muscle cars and the America’s cup sailboat to bicycles that smash speed records. Along the way, he finds that speed is more than just getting us from point A to B, it's also about getting things done in less time. From boarding a 737 to pushing the speed light travels, Pogue's quest for ultimate speed limits takes him to unexpected places where he’ll come face-to-face with the final frontiers of speed. Making Stuff Wilder What happens when scientists open up nature's toolbox? In "Making Stuff Wilder," David Pogue explores bold new innovations inspired by the Earth's greatest inventor, life itself. From robotic "mules" and "cheetahs" for the military, to fabrics born out of fish slime, host David Pogue travels the globe to find the world’s wildest new inventions and technologies. It is a journey that sees today's microbes turned into tomorrow’s metallurgists, viruses building batteries, and ideas that change not just the stuff we make, but the way we make our stuff. As we develop our own new technologies, what can we learn from billions of years of nature’s research? Making Stuff Colder Cold is the new hot in this brave new world. For centuries we've fought it, shunned it, and huddled against it. Cold has always been the enemy of life, but now it may hold the key to a new generation of science and technology that will improve our lives. In "Making Stuff Colder," David Pogue explores the frontiers of cold science from saving the lives of severe trauma patients to ultracold physics, where bizarre new properties of matter are the norm and the basis of new technologies like levitating trains and quantum computers. Making Stuff Safer The world has always been a dangerous place, so how do we increase our odds of survival? In "Making Stuff Safer," David Pogue explores the cutting-edge research of scientists and engineers who want to keep us out of harm’s way. Some are countering the threat of natural disasters with new firefighting materials and safer buildings. Others are at work on technologies to thwart terrorist attacks. A next-generation vaccine will save millions from deadly disease. And innovations like smarter cars and better sports gear will reduce the risk of everyday activities. We’ll never eliminate danger—but science and technology are making stuff safer.
DATE:
-
TEAM MEMBERS:
WGBH Educational FoundationPaula Apsell
Journey into Space (JIS) is designed to improve student, educator, and general public understanding of earth/space science and its relationship to NASA goals and objectives through the use of a traveling GeoDome (inflatable planetarium) and engaging supporting programming at The Journey Museum. The Museum collaborates with area colleges, school districts, K-12 educators, youth serving organizations, astronomical affiliations, and others. The overall goal of JIS is to improve student, educator, and general public understanding of STEM and its relationship to NASA goals and objectives. JIS objectives are: 1) To increase student and public interest and awareness in STEM areas; 2) To increase student interest in pursuing STEM careers; 3) To improve teacher knowledge of NASA related science; 4) To increase teacher comfort level and confidence in teaching NASA related science in their classrooms; 5) To increase collaboration between informal and formal science educators; 6) To increase student and public understanding of Plains Indians ethno astronomy; and 7) To increase museum visitors’ interest and understanding of NASA related science. The Museum produced 2 films (“Cradle of Life”, “Looney Moons”) that are offered daily, 4 recurring monthly programs (Final Frontier Friday, Amazing Science, SciGirls that became Science Explorer’s Club, and Black Hills Astronomical Society meetings), summer robotics classes and teachers’ workshops, annual Earth Science Day, in addition to the GeoDome programming that has toured the region including presentations in the three poorest counties in the United States. The ethno-astronomy is underway in partnership with Oglala Lakota College and South Dakota Space Grant Consortium.
Engineering MISSION: Engineering for Middle School Science, Inspiration and Opportunity is a project that introduces the engineering design process and the field of engineering to upper elementary and middle school students. Led by the Museum of Science, Boston in partnership with the Massachusetts Space Grant Consortium and the Goddard Space Flight Center (MD), and in conjunction with the California Afterschool Network and Audience Viewpoints (VA), this collaboration introduces engineering through the lens of our quest for knowledge about the Earth, solar system, and universe as embodied in NASA missions of exploration and discovery.
The project’s goals are to increase awareness of the field of engineering, the engineering design process, and the work of engineers; to inspire the next generation of engineers through the exciting challenges of exploring the universe; and to offer professional development opportunities for informal and formal educators in support of this effort. Our evaluation informed decisions regarding the design, development, and effectiveness of this project’s efforts. Through this undertaking, the Museum of Science, in collaboration with its partners, significantly enhanced and promoted public awareness of professional opportunities in the STEM fields.
There were four resources created through this partnership: From Dream to Discovery: Inside NASA, an engineering-themed planetarium show featuring NASA’s missions to explore Earth and space; a collection of short-duration, space-engineering-themed visual assets for use in museums and schools; aeronautical and aerospace-themed out-of-school time activities from the Museum’s Engineering Adventures program; and two workshops for informal and formal educators. The planetarium show, visual assets, and out-of-school-time activities are all currently available for distribution.