This document is the final evaluation report for the project, which focuses both on formative evaluation of the collaborative+interdisciplinary presentation creation process and summative evaluation of audience learning outcomes.
The Springfield Science Museum will increase participation in informal science learning by making its educational programs and learning spaces more accessible and inclusive. Museum staff will undergo Disability Inclusion and Universal Design training to redesign and enhance a core multi-use learning space and principle STEM program that can remove physical, cognitive, and social barriers to learning. External evaluators will measure access needs and learning outcomes before and after project upgrades in order to track progress and develop a scalable model of inclusive practice for all the museum’s science programming. The result will be an improved educational experience for visitors of all ages, abilities, and backgrounds.
DATE:
-
TEAM MEMBERS:
Katie Merrill
resourceprojectProfessional Development, Conferences, and Networks
Each year, millions of Americans visit science centers and museums, children’s museums, zoos, aquariums, nature centers, planetariums, and similar institutions. Recognized as trusted and popular places for educational and leisure experiences, these institutions are uniquely capable of engaging people across a spectrum of beliefs, experiences, and identities on topics related to science, technology, engineering, and mathematics (STEM), as well as addressing pressing societal problems related to science, technology, and innovation. However, the potential impacts of these institutions are largely dependent on the skills, knowledge, and abilities of the professionals working at them. Following the onset of the COVID-19 pandemic and the need to shift the nature of work, the informal STEM learning workforce was dramatically impacted. The recent period of disruption is a time for innovation. The field is well-positioned to promote new models of professional learning and development that are grounded in the values and practices of informal learning. This project will benefit local communities across the United States and society at large by advancing the capacity of science-engagement professionals to respond to societal needs, concerns, and interests more effectively through their institutions’ exhibitions, education and learning programs, and various forms of public engagement (e.g., community outreach events, supports for teachers/educators and schools). Led by the Association of Science and Technology Centers in collaboration with the Center of Science and Industry’s Center for Research and Evaluation and Oregon State University’s STEM Research Center, this work will build on the Informal STEM Learning (ISL) Professional Competency Framework developed and validated with prior National Science Foundation funding. Competency frameworks are increasingly used across many sectors to identify the suite of skills, knowledge, and capabilities necessary to be successful in a particular area of work. Given dramatic changes to the ISL workplace in the last two years, there is an even stronger potential for the Framework, particularly if newly developed supports can link the Framework to the current, emerging, and continuing needs of the workforce.
Guided by a systematic process for designing training and instructional materials, this project will first capture changes in the ways diverse informal STEM learning professionals describe, pursue, and achieve competencies to produce a revised Framework. Next, the project will collaboratively develop companion resources with diverse professionals through a series of participatory design workshops, using a sequential and iterative approach. The resources are expected to include indicators of professional competencies, self-assessments, training tools, and other types of resources that have the potential to significantly advance the professional learning (as undertaken by individuals and institutions) and the professional development capacity (as provided by institutions, associations, and other organizations) of the informal STEM learning field. Next, the project will share final products–including a refreshed website that hosts the updated Framework and newly-developed suite of tools–with professional association and network partners who can disseminate directly to the institutions and professionals who are developing and practicing these competencies. The project will gather evidence from a small number of early adopters, providing data on specific use-case scenarios. Finally, the project will document the potential impact of the Framework on the field by measuring how the Framework is perceived by informal STEM learning professionals as usable, useful, and beginning to be “in-use." Over time—and with increased use—the Framework and its companion resources hold the promise of contributing to the opening of the field to professionals with identities currently underrepresented through more transparent expectation and clearer growth pathways.
The North Carolina Museum of Natural Sciences will partner with North Carolina State University to improve the evaluation skills of informal science education providers. The project team will create a community of practice for 54 science museums across North Carolina by implementing a series of regional professional development workshops. The workshops will be designed to create a shared sense of purpose for programming and evaluation, build capacity among science museum educators to evaluate their programs, and establish a set of common metrics and methodologies for the evaluation of informal science learning across the state. The project will produce a practitioner's guide that will describe the collaborative process, lessons learned, and ways other informal science organizations can use identified evaluation goals and metrics.
The Discovery Museums will develop and implement a continuous improvement process to improve the impact of its STEM programming by strengthening staff skills in using evaluation data. The project will begin with a series of training sessions for learning programs staff based on feedback from youth regarding the quality of the museum's program delivery and an assessment of staff competencies in positive youth development. Participating staff will benefit from a deeper understanding of data and the ability to build ongoing evaluation and positive youth development practices into their program presentations in a way that supports Social-Emotional Learning outcomes. The project will potentially result in a process and set of tools to quantify the impact of STEM programming that can be shared with other informal learning organizations.
The Aldrich Contemporary Art Museum will amplify its partnership with Hart Magnet School, a Title 1 elementary school in urban Stamford, Connecticut, by increasing exposure and access to the arts for first-fifth graders, their families, and educators. A new program model, leveraging the museum's artist exhibitions, will focus on technology and an inquiry-based approach to science. Students, educators, and families will be encouraged to see and think in new ways through on-site STEAM tours at the museum, artist-led workshops at Hart, teacher professional development, and afterschool family activities. Outside evaluators will work with the project team to develop goals and associated metrics to measure how the model of museum-school partnership can enhance student achievement, engage families more deeply in their child's school experience and community, and contribute to teacher professional development. The evaluator will also train museum staff on best practices for program assessment.
DATE:
-
TEAM MEMBERS:
Namulen Bayarsaihan
resourceevaluationProfessional Development, Conferences, and Networks
Fostering STEAM provided exceptional professional development that was highly rated by participants and contributed to expected participant learning outcomes. The in-person Fostering STEAM workshop reflected professional development best practices. Likewise, the online Fostering STEAM course reflected indicators of effective online continuing education and professional development.
The Fostering STEAM professional development contributed to significant self-reported growth in principles or beliefs related to the Fostering STEAM instructional approach, as well as preparedness to develop and
The Lawrence Hall of Science will build the capacity of staff from different programmatic departments to systematically evaluate their programs in strategic alignment with the museum's newly revised Theory of Action. Based on best practices in evaluation capacity building, the museum will create a strategic plan for evaluation that provides ongoing opportunities for staff to engage in evaluation practice. To build capacity and foster a culture of continuous improvement, program teams will participate in a variety of activities such as trainings to build knowledge about evaluation; meetings to discuss purpose and methods of evaluation; group inquiry about evaluation measures, outcomes, and data; and conducting evaluation with support from evaluators. Project outcomes for museum staff will include increased knowledge of evaluation design; increased skill in designing and conducting evaluation; ongoing and continuous use of evaluation data to improve decision making and programs; and enthusiasm for evaluation as part of the process of strategic learning.
To advance justice, equity, diversity, and inclusion in science, we must first understand and improve the dominant-culture frameworks that impede progress and, second, we must intentionally create more equitable models. The present authors call ourselves the ICBOs and Allies Workgroup (ICBOs stands for independent community-based organizations), and we represent communities historically excluded from the sciences. Together with institutional allies and advisors, we began our research because we wanted our voices to be heard, and we hoped to bring a different perspective to doing science with
DATE:
TEAM MEMBERS:
María Cecilia Alvarez RicaldeJuan Flores ValadezCatherine CrumJohn AnnoniRick BonneyMateo Luna CastelliMarilú López FrettsBrigid LuceyKaren PurcellJ. Marcelo BontaPatricia CampbellMakeda CheatomBerenice RodriguezYao Augustine FoliJosé GonzálezJosé Miguel Hernández HurtadoSister Sharon HoraceKaren KitchenPepe Marcos-IgaTanya SchuhPhyllis Edwards TurnerBobby WilsonFanny Villarreal
This poster was presented at the 2021 NSF AISL Awardee Meeting.
Many informal learning institutions use STEAM approaches to engage diverse learners. Our project aims to support educators in libraries, museums, and after school programs through a STEAM professional development (PD) series. Our PD approach is centered around a set of core STEAM practices that prioritize STEAM mindset and identity work. Participants engage in exemplar activities and design new experiences for their specific teaching and learning contexts. The series involves in- person sessions, online training, and team