Various measures exist to assess learning outcomes from informal science, technology, engineering, or math (STEM) education programming, especially for youth audiences. Although there has been growing interest in the field among practitioners, researchers and evaluators for the development and implementation of shared, common STEM measures and frameworks, informal STEM learning (ISL) institutions (e.g., museums, zoos, aquaria, botanic gardens, and the like) face a unique challenge when they design and evaluate educational programs. By nature of the broad-ranging topic areas these institutions emphasize, programming seeks to reach audiences that vary in age, ability, cultural background, interest, and more. Participation also varies--for example, participants choose which in a series of exhibits they attend and how much time they spend at each. Given this planned-for variance, how do educators target appropriate learning outcomes? To what extent do outcomes map to activity complexity? How does programming success link to institutional goals? One solution may be a framework of outcome progressions. Progressions are within outcome categories (e.g., interest, attitude, knowledge, skills, behavior, science capital, etc.), the ordinal series of effects that can be mapped to programming that ranges from short duration with simple content (e.g. a hallway cart demonstration can trigger situational interest) to extended duration with complex content (e.g. a youth internship can generate well-developed personal interest). STEM educators, evaluators and researchers working in or with ISL institutions will come together to consider the utility and feasibility of an outcome progressions framework. With this framework STEM learning institution educators may be better equipped to understand the impact of their programming and how to design for it. This work will be undertaken by the COSI Center for Research and Evaluation in conjunction with the Office of Education, Outreach, and Visitor Services of the National Museum of Natural History and the PEAR Institute.
The primary aim of this multi-day virtual convening is to inform future development of the framework for outcome progressions. These progressions will reflect and accommodate outcomes targeted by existing and shared evaluation measures. The framework will provide informal educators and evaluators with (1) a way of aligning outcome expectations with programming depth and dosage; (2) outcome choices achievable with various depths of programming; (3) a strategic tool for planning institution-wide targeted outcomes across a full portfolio of institution educational programming; and (4) a tool for documenting an education department's varied outcomes across the full range of possible outcomes. To create rich and informed critical discussion, the conference will involve research consultant experts representing each outcome category; practitioners representing diverse programming; and evaluation theorists familiar with the broad needs of informal learning institutions. A conference subcommittee with expertise in culturally responsive and equitable program development and evaluation will ensure the agenda contains flexibility to generate emergent equitable and inclusive conversations. Conference participants will review and contribute to the soundness and utility of these categorizations as well as the progressions within them. These insights and inputs will inform future development of the framework as a valid and reliable tool. Deliverables include a paper summarizing the findings from the convenings and a related webinar that describes the framework and recommendations for its use. These will be shared broadly via the informalscience.org website and other professional venues.
This conference is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
In 2017, the Education Development Center (EDC) received a grant from the National Science Foundation (NSF) to bring together PIs of STEM Program Resource Centers (PRC) funded under NSF’s Education and Human Resources (EHR), evaluators of NSF STEM projects and programs, and evaluation advisors to address concerns about the quality and consistency of STEM evaluations. According to the grant proposal, the goal was “to increase the capacity of evaluators to produce high quality, conceptually sound, methodologically appropriate evaluations of NSF programs and projects, specifically in the area of
In this virtual conference, The PEAR Institute convened over 40 practitioners and researchers to explore the power of an integrated vision of STEM and social-emotional development (SED). This conference aimed to 1) create a collaboration of out-of-school time (OST) leaders, practitioners, and researchers to map the overlays of STEM and SED; 2) identify best practices for integrated STEM-SED programming; 3) explore common existing and potential measures; and 4) identify how data is measured, communicated, and used for skills that are important to both STEM and SED. Through this collaboration, we aimed to initiate the conversations, establish partnerships, and build the capacity for assessment of STEM and SED in informal STEM learning programs.
Rather than seeking consensus, this conference sought to unearth questions and generate new ideas to lay a foundation for future research and practice, a way to inform the field about high-quality research and practice to promote cross-disciplinary collaboration and synergy between STEM and SED.
Conference deliverables include a conference report, an informal key findings webinar, and conference website (https://www.thepearinstitute.org/stem-sed-conference-2020). The website includes links to a participant directory, conference at-a-glance overview, agendas, readings, and additional resources identified during the conference.
The 15th international conference of the Public Communication of Science and Technology network took place from April 4–6, 2018. Given its location in Dunedin, New Zealand/Ōtepoti, Aotearoa, it was a natural venue for two sessions on communicating science across cultures.
The first Japan Scicom Forum in Tokyo on April 20, 2018 gathered nearly 120 attendees to discuss the growing need and demand for English-language science communication in Japan and Asia. Keynotes and workshops addressed both the philosophy and motivations for scicomm in Japan and also the best practices for international outreach. Global science communication has reached a critical mass in Japan but securing sustainable funding, integrating the community and retaining momentum present ongoing challenges. As an online community and (hopefully) a recurring event, Japan Scicom Forum will foster a
Modern science communication has emerged as a field of study, a body of practice and a profession. In the last 60 years, we have seen the birth of interactive science centres, university courses, the first research into science communication, and a growth in employment by research institutions, universities, museums, science centres and industry. Now Ireland has told its story.
DATE:
TEAM MEMBERS:
Toss Gascoigne
resourceresearchProfessional Development, Conferences, and Networks
This report grew out of a workshop and follow-up session sponsored by the National Science Foundation (NSF), Directorate for Education and Human Resources (EHR). The two-day workshop and subsequent meeting sought to develop and validate evaluation practices to assess the value of NSF's investment in broadening participation across all directorates and programs. Invited participants included NSF grantees, professional evaluators, and the policy community (which included representatives from Congress, the Office of Management and Budget (OMB), NSF staff, and staff from other federal agencies).
DATE:
TEAM MEMBERS:
Fitzgerald BramwellPatricia CampbellBeatriz Chu ClewellDarnella DavisNorman FortenberryAntonio GarciaDonna NelsonAdam StollVeronica Thomas
resourceevaluationProfessional Development, Conferences, and Networks
The September 6 -7 COMPASS Conference held at the Exploratorium had 19 speakers and 80 attendees. The conference purpose was to take a focused look at indoor location aware mobile (ILAM) technology and how it was used in science centers and other museums.
The three primary project goals were:
1) Form an integrated vision by consolidating expertise from disparate disciplines connected to indoor location aware mobile (ILAM) tech development;
2) transform visitor mobile tools going beyond basic digital content to more innovative forms of interaction and personalization; and,
3) open
The 10th World Conference of Science Journalists (San Francisco, U.S.A., 26–30 October 2017) was the most successful to date in terms of participants and probably the one with the largest presence of journalists from the developing world among its attendees and speakers. In agreement with the times, its themes were marked by ethical dilemmas in the communication of science, fake news and climate change, among others.
DATE:
TEAM MEMBERS:
Paula Leighton
resourceresearchProfessional Development, Conferences, and Networks
The challenge to the science communication field put forward by Bruce Lewenstein, of the sector becoming a ‘ghetto’ of women's over-representation (see the commentary by Lewenstein in this issue), is a very timely wake-up call. This Commentary however, elaborates and frames the pivotal and constructivist premises on which this phenomenon should be interrogated and understood on many levels. It is critical that we undertake a deeper introspection, beyond just simplistic head counts of the number of women and men in the field, if we are to make sense of the seeming paradoxes that pervade the
DATE:
TEAM MEMBERS:
Elizabeth Rasekoala
resourceprojectProfessional Development, Conferences, and Networks
This project will advance evidence-based efforts to broaden informal STEM engagement via the 2019 Inclusive Science Communication (Inclusive SciComm) Symposium, to be held September 27-29, 2019, at the University of Rhode Island in Kingston, Rhode Island. Science communication, defined as any information exchange designed to engage targeted audiences in conversations or activities related to STEM topics, is a rapidly expanding area of research and practice with the potential to significantly increase public participation and sense of belonging in STEM fields. That said, there are few opportunities for its practitioners and scholars to convene around how to make their work both inclusive and equitable, which collectively acknowledge identity, cultural differences, and epistemologies as part of broadening participation. The 2019 symposium will address this gap through panels, workshops, and posters focused on three themes that represent critical and difficult aspects of inclusive science communication: (1) New Languages, Practices, Knowledge, and Research; (2) Changing Systems and Structures through Science Communication; and (3) Social Responsibility and Ethics. Within these themes, sessions will be organized to address major barriers of absence identified by participants in the 2018 Inclusive SciComm Symposium: skills, lessons learned, and knowledge gaps, especially with regard to facilitating difficult conversations across difference (critical dialogue). The symposium also will emphasize the need to integrate research and practice to advance inclusive, equitable, and intersectional approaches to science communication.
There is an urgent need to question assumptions and examine evidence regarding how science communicators and scholars approach efforts to broaden participation, but insufficient data exist on the inputs and outputs of inclusive and equitable practice. Critical dialogue about potentially uncomfortable topics such as privilege, power, or marginalization is an essential tool for inclusive practice and pedagogy. Finding from the 2018 Inclusive SciComm Symposium indicated that many educators and practitioners lack the language, skills, or confidence to initiate this type of dialogue. This project supports the knowledge-building component of the 2019 Inclusive SciComm Symposium to inform future science communication training, practice, and scholarship, by building on preliminary data collected during the 2018 symposium and responding to the need for more robust evaluation of science communication activities. Applying the Theory of Planned Behavior, the project will employ pre/post symposium surveys to investigate how 2019 symposium activities affected knowledge, attitudes, subjective norms, and efficacy (the variables of the Theory of Planned Behavior) of attendees with regard to critical dialogue. Focus groups at the symposium will be used to identify priority research areas related to inclusion, generally, and critical dialogue, specifically, that could advance inclusive science communication practice and beneficial outcomes. The project also will evaluate symposium impacts with regard to 1) attendees' opinions on utility of symposium components for advancing inclusive science communication and 2) how attendees' experience and response orientations inform their approaches to difficult science communication conversations. Qualitative data from the surveys and focus groups will be thematically coded using constant comparison.
This project will have strategic impact for inclusive science communication practice and, therefore, for informal learning and public engagement with STEM topics. Increasing awareness and effective implementation of critical dialogue by science communicators and trainers should enhance both for ethical engagement of traditionally under-represented and marginalized groups and should foster diverse types of public participation in societal debates about scientific issues. The outcomes of this research will benefit and link the complementary, but often siloed, fields of informal science learning and science communication. A final report will summarize research findings and offer specific next steps to advance inclusive science communication practice and research, especially with regard to fostering critical dialogue. The report will be posted on inclusivescicomm.org and distributed via a national network of partners working in informal science education, science communication, and public engagement.
This project is funded by the National Science Foundation's (NSF's) Advancing Informal STEM Learning (AISL) program, which supports innovative research, approaches, and resources for use in a variety of learning settings.
STEAM, the use of art as a context and tool for science education, is currently a hot topic in the science education field. In almost all instances of study and practice, it involves the use of science-themed or science-informed art in science education. As such, it does not take advantage of the majority of artistic output that does not have an obvious connection to science. The National Academies of Sciences, Engineering and Medicine recently called for more research to expand the "limited but promising" evidence that integrating arts and humanities with science education leads to better learning. The goal of this 2.5-day conference is to bring together representatives of both art and science groups to have a shared discussion around how non-scientific art can influence science education in theory, and how we can apply empirical results to the theory. For purposes of this conference, "non-science art" is defined as art that was not inspired by science. Conference attendees will include researchers (art and science education researchers) and practitioners (artists, art museum interpreters, and science educators). The conference will take place during the 2020 Black Creativity exhibition at the Museum of Science and Industry, Chicago. It is anticipated that by holding the conference at that time the audience for the conference and its impact will be informed by more diverse attendance.
The conference will be implemented starting with a pre-conference reading. Attendees will be sent a copy of the white paper from the Art as a Way of Knowing report for background reading and also asked to contribute to a Google Document that describes their various contexts. Each day of the conference will focus on a theme -- state of the field and possibilities and research -- and be comprised of large and small group interactions. Attendees will be invited from the ranks of practitioners, researchers and educators in the art and science education fields; several slots will be available for open (non-invited) participants. Key outcomes include: (a) a summary of all the research that has been conducted on using non-science art in science education, (b) starting points for building a theory on why non-science art can be used in science education; and (c) a list of specific research topics that would help inform, advance, and test the theory. In addition to assessing satisfaction with the conference, evaluation will also include a one-year post conference survey to investigate impact of participation in the conference.
This conference will generate products that will give guidance to both researchers and practitioners who want to use art in science education. These products include a white paper synthesizing the discussion and appendices that include raw transcripts and a bibliography of resources. Another product is a roadmap to create interventions that can be studied, which should lead to a stronger, more rigorous theory of practice about how art can be integrated into science education.
This project is funded by the National Science Foundation's (NSF's) Advancing Informal STEM Learning (AISL) program, which supports innovative research, approaches, and resources for use in a variety of learning settings.