Various measures exist to assess learning outcomes from informal science, technology, engineering, or math (STEM) education programming, especially for youth audiences. Although there has been growing interest in the field among practitioners, researchers and evaluators for the development and implementation of shared, common STEM measures and frameworks, informal STEM learning (ISL) institutions (e.g., museums, zoos, aquaria, botanic gardens, and the like) face a unique challenge when they design and evaluate educational programs. By nature of the broad-ranging topic areas these institutions emphasize, programming seeks to reach audiences that vary in age, ability, cultural background, interest, and more. Participation also varies--for example, participants choose which in a series of exhibits they attend and how much time they spend at each. Given this planned-for variance, how do educators target appropriate learning outcomes? To what extent do outcomes map to activity complexity? How does programming success link to institutional goals? One solution may be a framework of outcome progressions. Progressions are within outcome categories (e.g., interest, attitude, knowledge, skills, behavior, science capital, etc.), the ordinal series of effects that can be mapped to programming that ranges from short duration with simple content (e.g. a hallway cart demonstration can trigger situational interest) to extended duration with complex content (e.g. a youth internship can generate well-developed personal interest). STEM educators, evaluators and researchers working in or with ISL institutions will come together to consider the utility and feasibility of an outcome progressions framework. With this framework STEM learning institution educators may be better equipped to understand the impact of their programming and how to design for it. This work will be undertaken by the COSI Center for Research and Evaluation in conjunction with the Office of Education, Outreach, and Visitor Services of the National Museum of Natural History and the PEAR Institute.
The primary aim of this multi-day virtual convening is to inform future development of the framework for outcome progressions. These progressions will reflect and accommodate outcomes targeted by existing and shared evaluation measures. The framework will provide informal educators and evaluators with (1) a way of aligning outcome expectations with programming depth and dosage; (2) outcome choices achievable with various depths of programming; (3) a strategic tool for planning institution-wide targeted outcomes across a full portfolio of institution educational programming; and (4) a tool for documenting an education department's varied outcomes across the full range of possible outcomes. To create rich and informed critical discussion, the conference will involve research consultant experts representing each outcome category; practitioners representing diverse programming; and evaluation theorists familiar with the broad needs of informal learning institutions. A conference subcommittee with expertise in culturally responsive and equitable program development and evaluation will ensure the agenda contains flexibility to generate emergent equitable and inclusive conversations. Conference participants will review and contribute to the soundness and utility of these categorizations as well as the progressions within them. These insights and inputs will inform future development of the framework as a valid and reliable tool. Deliverables include a paper summarizing the findings from the convenings and a related webinar that describes the framework and recommendations for its use. These will be shared broadly via the informalscience.org website and other professional venues.
This conference is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
In this virtual conference, The PEAR Institute convened over 40 practitioners and researchers to explore the power of an integrated vision of STEM and social-emotional development (SED). This conference aimed to 1) create a collaboration of out-of-school time (OST) leaders, practitioners, and researchers to map the overlays of STEM and SED; 2) identify best practices for integrated STEM-SED programming; 3) explore common existing and potential measures; and 4) identify how data is measured, communicated, and used for skills that are important to both STEM and SED. Through this collaboration, we aimed to initiate the conversations, establish partnerships, and build the capacity for assessment of STEM and SED in informal STEM learning programs.
Rather than seeking consensus, this conference sought to unearth questions and generate new ideas to lay a foundation for future research and practice, a way to inform the field about high-quality research and practice to promote cross-disciplinary collaboration and synergy between STEM and SED.
Conference deliverables include a conference report, an informal key findings webinar, and conference website (https://www.thepearinstitute.org/stem-sed-conference-2020). The website includes links to a participant directory, conference at-a-glance overview, agendas, readings, and additional resources identified during the conference.
This report grew out of a workshop and follow-up session sponsored by the National Science Foundation (NSF), Directorate for Education and Human Resources (EHR). The two-day workshop and subsequent meeting sought to develop and validate evaluation practices to assess the value of NSF's investment in broadening participation across all directorates and programs. Invited participants included NSF grantees, professional evaluators, and the policy community (which included representatives from Congress, the Office of Management and Budget (OMB), NSF staff, and staff from other federal agencies).
DATE:
TEAM MEMBERS:
Fitzgerald BramwellPatricia CampbellBeatriz Chu ClewellDarnella DavisNorman FortenberryAntonio GarciaDonna NelsonAdam StollVeronica Thomas
resourceevaluationProfessional Development, Conferences, and Networks
The September 6 -7 COMPASS Conference held at the Exploratorium had 19 speakers and 80 attendees. The conference purpose was to take a focused look at indoor location aware mobile (ILAM) technology and how it was used in science centers and other museums.
The three primary project goals were:
1) Form an integrated vision by consolidating expertise from disparate disciplines connected to indoor location aware mobile (ILAM) tech development;
2) transform visitor mobile tools going beyond basic digital content to more innovative forms of interaction and personalization; and,
3) open
STEAM, the use of art as a context and tool for science education, is currently a hot topic in the science education field. In almost all instances of study and practice, it involves the use of science-themed or science-informed art in science education. As such, it does not take advantage of the majority of artistic output that does not have an obvious connection to science. The National Academies of Sciences, Engineering and Medicine recently called for more research to expand the "limited but promising" evidence that integrating arts and humanities with science education leads to better learning. The goal of this 2.5-day conference is to bring together representatives of both art and science groups to have a shared discussion around how non-scientific art can influence science education in theory, and how we can apply empirical results to the theory. For purposes of this conference, "non-science art" is defined as art that was not inspired by science. Conference attendees will include researchers (art and science education researchers) and practitioners (artists, art museum interpreters, and science educators). The conference will take place during the 2020 Black Creativity exhibition at the Museum of Science and Industry, Chicago. It is anticipated that by holding the conference at that time the audience for the conference and its impact will be informed by more diverse attendance.
The conference will be implemented starting with a pre-conference reading. Attendees will be sent a copy of the white paper from the Art as a Way of Knowing report for background reading and also asked to contribute to a Google Document that describes their various contexts. Each day of the conference will focus on a theme -- state of the field and possibilities and research -- and be comprised of large and small group interactions. Attendees will be invited from the ranks of practitioners, researchers and educators in the art and science education fields; several slots will be available for open (non-invited) participants. Key outcomes include: (a) a summary of all the research that has been conducted on using non-science art in science education, (b) starting points for building a theory on why non-science art can be used in science education; and (c) a list of specific research topics that would help inform, advance, and test the theory. In addition to assessing satisfaction with the conference, evaluation will also include a one-year post conference survey to investigate impact of participation in the conference.
This conference will generate products that will give guidance to both researchers and practitioners who want to use art in science education. These products include a white paper synthesizing the discussion and appendices that include raw transcripts and a bibliography of resources. Another product is a roadmap to create interventions that can be studied, which should lead to a stronger, more rigorous theory of practice about how art can be integrated into science education.
This project is funded by the National Science Foundation's (NSF's) Advancing Informal STEM Learning (AISL) program, which supports innovative research, approaches, and resources for use in a variety of learning settings.
This report summarizes the evaluation outcomes of the Collaborative Project Management (CoPM) Institute project, a two-day convening centered on bringing proven project management skills and approaches in the for-profit world to the informal science education (ISE) community to support effective collaborations and successful outcomes. The CoPM Institute was conceptualized as a pilot effort to test the effectiveness of porting of tools and frameworks for collaborative project management from the business/for-profit environment to advance the following project goals:
Build the capacity of
A large body of research highlights the benefits of storybooks for children's learning. In the context of preschool classrooms, the use of storybooks to engage young children in STEM is a frequent topic of practitioner-oriented articles. There is also an increasing number of informal STEM education (ISE) projects exploring how to leverage storybooks to engage young children and their families in different STEM content domains. While there is universal excitement for the potential of storybooks in ISE, there is an acknowledgment of a critical need for more cross-project sharing, more research, and more efforts to synthesize and share findings. This award will catalyze new research studies and partnerships to advance efforts in ISE contexts, including the role of books in the overall learning experience or program, how books are selected or designed, and how the reading is facilitated by teachers and families. Participants will be educators and researchers working with or studying family learning for preschool-age children (three to five years) using early childhood fiction books as a tool for engaging families in STEM topics and skills.
Storybook STEM will be implemented in four phases: (1) pre-convening activities to plan, synthesize existing resources, engage a broader group of educators and researchers beyond convening attendees, and prepare convening participants to maximize the value of the in-person discussions; (2) in-person convening to catalyze cross-project discussions, outline promising practices, and identify questions and ideas for the future; (3) evaluation of the impact and value of the convening, from the perspective of participants and a project steering committee; and (4) dissemination of findings and recommendations to educators and researchers within and beyond the ISE field. Outcomes include: (1) documenting current and past work in ISE and other fields; (2) summarizing key recommendations and resources from the reading, literacy, and early childhood development fields; and (3) outlining promising directions for future work.
The findings from this project will provide a critical resource to help broadening participation efforts be more effective and inclusive for audiences across the country. Research studies motivated by the convening will address the lack of empirical work on storybooks as a tool for ISE programs and advance the ISE field's knowledge of how to integrate these books effectively. Because storybooks are a highly accessible and almost universally used family learning resource, the topic of the convening will be relevant to a wide range of audiences and will help educators broaden access to ISE for traditionally underserved and under-resourced communities.
This project is funded by the National Science Foundation's (NSF's) Advancing Informal STEM Learning (AISL) program, which supports innovative research, approaches, and resources for use in a variety of learning settings.
This project will consider how research on imaginative thinking, and perspectives on the role of imagination in STEM practice and STEM education, can be systematically applied to support STEM learning in museum contexts. Common conceptions of science as non-imaginative are persistent, but scholarship across disciplines suggests critical roles for imagination, both in the practice of STEM and in shaping learners' perceptions of themselves as part of STEM. Further, evidence from the fields of neuroscience, psychology, child development and education suggests ways that imagination can be fostered and improved, and that these understandings could be applied to the design of museum experiences in order to improve STEM outcomes.
The activities of this project, led by the Museum of Science, Boston, both synthesize and generate knowledge at the intersections of imagination, STEM, and education practice in ways that are actionable for museum professionals. Activities include: a literature review, a document review, and a survey of ISE professionals; an in-person convening of STEM professionals (researchers, practitioners, educators and others); and the development and dissemination of products designed to inform future project development. The goals of the project are to: 1) prompt conversations about imaginative thinking across the Informal Science Education (ISE) field, and between ISE and other fields; 2) identify priority areas for research and development that can advance the field's understandings at the intersections of imagination, STEM, and learning; and 3) catalyze future research and development efforts that can advance the field. The intent is for the integration scholarship on imagination, STEM, and learning within museums' research and development efforts to lead to projects that describe, test, and refine theoretical frameworks and concrete strategies for supporting imaginative thinking among public audiences through exhibitions, programs, and other designed experiences.
This project is funded by the National Science Foundation's (NSF's) Advancing Informal STEM Learning (AISL) program, which supports innovative research, approaches, and resources for use in a variety of learning settings.
Vassar College is conducting a 2.5-day conference, as well as pre- and post-conference activities, that convenes a multi-disciplinary, multi-institutional (USA and international) team to conceptualize and plan various research, education and outreach activities in informal learning, focusing on the seminal concept of tensegrity and its applications in many fields of science and mathematics. Tensegrity is the characteristic property of a stable three-dimensional structure consisting of members under tension that are contiguous and members under compression that are not.
The conference will bring together researchers and practitioners in informal learning and researchers in the various disciplines that embrace tensegrity (mathematics, engineering, biology, architecture, and art) to explore the potential that tensegrity has to engage the public in informal settings, especially through direct engagement in creating such structures. This project is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants.
To date there have been no sustained informal educational projects and research around the topic of tensegrity. However, there is considerable related work on learning through "making and tinkering" upon which the participants will adapt and expand. The intended conference outcomes are to produce prototypes of activities, a research agenda, and lines of development with the potential to engage the wider public. A key priority of the gathering is the development of new partnerships between researchers and creators of tensegrity systems and the informal learning professionals. The long-term project hypothesis is that children and adults can engage with tensegrity through tinkering with materials and becoming familiar with a growing set of basic structures and their applications. The activities will include evaluation of the conference and a social network analysis of the collaborations that result.
This project is funded by the National Science Foundation's (NSF's) Advancing Informal STEM Learning (AISL) program, which supports innovative research, approaches, and resources for use in a variety of learning settings.
DATE:
-
TEAM MEMBERS:
John McCleary
resourceresearchProfessional Development, Conferences, and Networks
On September 5–6, 2018, the Exploratorium hosted the Conference on Mobile Position Awareness Systems and Solutions (COMPASS). The 80 attendees were from throughout the United States, and from Canada, Australia, New Zealand, the United Kingdom, and Sweden. Speakers and panelists from museums, academia, industry, and research discussed indoor location projects related to informal learning, increasing accessibility, addressing privacy, and understanding visitor behavior. This publication presents the conference proceedings, with summaries of each session organized chronologically. The goals of
The STEM Effect project is a collaborative effort to engage cultural organizations around the U.S. in developing a Collaborative Action Agenda for better understanding the mid- and long-term impacts of informal STEM programs for girls.
The COMPASS conference will bring together 80 participants for two days in September 2018 at the Exploratorium in San Francisco, CA. The first dissemination will take place in a presentation at the ASTC conference the following month in October 2018. A webinar sharing insights from COMPASS and inviting others to engage will be held in March 2019 hosted by ASTC and accessible by ASTC members and non-members alike. A companion COMPASS e-publication will be released for free download, also in March 2019, with summaries of conference proceedings, key issues identified, case histories of ILAM in