This document is the final evaluation report for the project, which focuses both on formative evaluation of the collaborative+interdisciplinary presentation creation process and summative evaluation of audience learning outcomes.
The Aldrich Contemporary Art Museum will amplify its partnership with Hart Magnet School, a Title 1 elementary school in urban Stamford, Connecticut, by increasing exposure and access to the arts for first-fifth graders, their families, and educators. A new program model, leveraging the museum's artist exhibitions, will focus on technology and an inquiry-based approach to science. Students, educators, and families will be encouraged to see and think in new ways through on-site STEAM tours at the museum, artist-led workshops at Hart, teacher professional development, and afterschool family activities. Outside evaluators will work with the project team to develop goals and associated metrics to measure how the model of museum-school partnership can enhance student achievement, engage families more deeply in their child's school experience and community, and contribute to teacher professional development. The evaluator will also train museum staff on best practices for program assessment.
The Massachusetts Audubon Society will develop, pilot, and implement an evaluation framework for nature-based STEM programming that serves K-12 students visiting its network of nature centers and museums. Working with an external consultant, the society will develop the framework comprised of a logic model and theory of change for fieldtrips, and develop a toolkit of evaluation data collection methodology suitable to various child development stages. The project team will design and conduct three professional development training seminars to help Massachusetts Audubon school educators develop a working understanding of the new evaluation framework for school programs and gain the skills necessary to support protocol implementation. This project will result in the development and adoption of a universal protocol to guide the collection, management, and reporting of education program evaluation data across the 19 nature centers and museums in the Massachusetts Audubon system.
DATE:
-
TEAM MEMBERS:
Kris Scopinich
resourceresearchProfessional Development, Conferences, and Networks
As science communication programs grow worldwide, effective evaluation and assessment metrics lag. While there is no consensus on evaluation protocols specifically for science communication training, there is agreement on elements of effective training: listening, empathy, and knowing your audience — core tenets of improvisation. We designed an evaluation protocol, tested over three years, based on validated and newly developed scales for an improvisation-based communication training at the Alan Alda Center for Communicating Science. Initial results suggest that ‘knowing your audience’ should
DATE:
TEAM MEMBERS:
Christine O’ConnellMerryn McKinnonJordan Labouff
resourceevaluationProfessional Development, Conferences, and Networks
This document describes the summative project evaluation of 5 annual cohorts of STE(A)M teachers, mostly from California, Florida, and New Mexico participating in out-of-school authentic research experiences collecting fossils and learning about geology, biology, and the natural history along the Panama Canal, and their experiences with museums and research collections. The STEM content of this project is based on the Great American Biotic Interchange (GABI) of animals and plants across the Isthmus of Panama over the past 5 million years. This report also describes the efficacy of sustained
In order to engage visitors, guests, participants or audiences in positive STEM learning experiences, informal learning institutions need professionals who understand how to design for and facilitate engaging activities. Initial professional training for informal STEM educators, and subsequent ongoing professional learning create considerable challenges. There is a need for providing informal STEM educators with pathways to professionalization that guarantee high quality educators who can support successful informal STEM education. In this symposium, we propose to share research on key aspects
This video presents reflections on SCIENCES: Supporting a Community’s Informal Education Needs—Confidence and Empowerment in STEM. SCIENCES brought together Eden Place Nature Center and the Chicago Zoological Society to collaboratively support environmental conservation and lifelong scientific learning in the Fuller Park neighborhood of Chicago.
The SCIENCES project began in 2013 and focused on adapting existing educational programs into a suite of environmentally focused science learning opportunities for professional, student, and public audiences in the Fuller Park neighborhood
The summative evaluation documents and articulates what SCIENCES has improved or changed, and in what ways. The final design of the summative evaluation was based on findings from the front-end and formative evaluations, including using participatory evaluation techniques to engage community members in discussing their experience with the programs and assessment of community needs and assets at the close of the project.
The goal of the summative evaluation was to address discrete program impacts in the context of the project, as well as the cross-program impact of providing a thematically
In The Nature of Community: SCIENCES, we share the lessons learned from an innovative partnership designed to leverage the strengths of two nonprofit organizations—a large cultural institution and a smaller, deeply-rooted community-based organization, both of which offer informal science education expertise.
You’ll read first-hand reflections of how staff members, community leaders and members, children, and adults experienced this partnership: the expectations, surprises, challenges, successes, and lessons learned. We hope the description of this partnership inspires other organizations to
Consideration of the needs of individuals with a wide range of disabilities is not always considered in the early design stages of an informal STEM learning (ISL) activity or program. The primary access approach for people with disabilities becomes the provision of accommodations once the ISL product or environment is created. In contrast, the Universal Design approach considers users with a wide range of characteristics throughout the design process and works to create products and environments that are accessible, usable, and inclusive. This project, called AccessISL, led by the University of Washington's DO-IT (Disabilities, Opportunities, Internetworking and Technology) Center and Museology Program, includes an academic museology program and local ISL sites, representing museums, zoos, aquariums, makerspaces, science centers, and other sites of informal STEM learning. Insights will be gained through the engagement of people with disabilities, museology graduate students and faculty, and ISL practitioners. The AccessISL project model, composed of a set of approaches and interventions, builds on existing research and theory in the fields of education science, change management, effective ISL practices, and inclusive design processes. The project will collect evidence of policies and practices (or lack thereof) that improve the inclusiveness of ISL with respect to a wide range of disabilities and considers approaches for the design and development of new strategies; explores what stakeholders need to make change happen; uncovers challenges to the adoption of inclusive practices in public ISL settings and explores ways to overcome them; and proposes relevant content that might be included in museology curriculum. This project is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants.
This project addresses the following two objectives:
For ISL personnel and museology faculty: to increase knowledge, skills, and actions to make ISL programs, facilities, courses, and resources more welcoming and accessible to participants with disabilities and embed relevant practices within their work.
For postsecondary STEM students with disabilities and museology students: to increase knowledge and skills in advocating for ISL offerings that are welcoming and accessible to everyone, including those with a wide variety of disabilities, and to encourage individuals with disabilities to pursue careers in ISL.
The project employs a student-centered approach and a set of practices that embrace the social model of disability, social justice education, disability as a diversity issue, intersectionality, and Universal Design. A leadership team of interns--each member a STEM student with a disability or a museology graduate student--along with project staff will engage with the University of Washington's Museology Program to identify and implement strategies for making ISL activities and courses more welcoming and accessible to individuals with disabilities. An online community of practice will be developed from project partners and others nationwide. A one-day capacity building institute will be held to include presentations, student/personnel panels for sharing project and related experiences, and group discussions to explore issues and further identify systemic changes to make ISL programs more welcoming and accessible to individuals with disabilities. As prototypes of the AccessISL Model are developed, evaluation activities will primarily be formative (looking for strengths and weaknesses) and remedial (identifying/implementing changes that could be made to improve the model). The model will continue to be fine-tuned through formative evaluation. Evaluation of the model components will focus on the experience of a range of stakeholders in the project. Specifically, quantitative data collected will include levels and quality of engagement, accessibility recommendations and products developed, and delivery of ISL services. Qualitative data will be collected through observations, surveys, focus groups, interviews, and case studies.
AccessISL project products will include proceedings of an end-of-project capacity building institute, promising practices, case studies, a video, and other online resources to help ISL practitioners and museology faculty that will result in making future ISL opportunities more inclusive of people with disabilities. AccessISL will advance knowledge and ensure long-term impact using multiple strategies:
broadening the STEM participation of people with disabilities as well as women, racial/ethnic minorities, and other underrepresented groups through the application of universal design
strengthening associations and creating synergy and durable relationships among stakeholders,
encouraging teaching about disability, accessibility, and universal design in museology courses,
empowering students with disabilities and current and future ISL practitioners to advocate for accessible ISL and develops an infrastructure to promote accessible ISL programs nationwide, and
contributing to the body of promising practices with products that will (a) enhance understanding of issues related to the inclusion of people with disabilities in ISL programs and (b) promote inclusive practices.
Outcomes will benefit society by making STEM opportunities available to more people and enhancing STEM fields with the talents and perspectives of people with disabilities.
This project is funded by the National Science Foundation's (NSF's) Advancing Informal STEM Learning (AISL) program, which supports innovative research, approaches, and resources for use in a variety of learning settings.
This workshop series, convened by the Kavli, Rita Allen, Packard and Moore Foundations, was intended to view the entire system of people who support scientists’ engagement and communication efforts in order to explore how this system can be most effective and sustainable. The discussions examined where this system is thriving, the limits people within the system face and what can be done to ensure their efforts are commensurate with the demand for quality communication and engagement support.
Conducted over four closely scheduled workshops in late 2017 and early 2018, the convenings brought
Digital Observation Technology Skills (DOTS) is a framework for integrating modern, mobile technology into outdoor, experiential science education. DOTS addresses longstanding tensions between modern technology and classical outdoor education by carefully selecting appropriate digital technology for educational purposes and by situating these tools in classical experiential pedagogy.
DATE:
TEAM MEMBERS:
R. Justin HoughamMarc NutterCaitlin Graham