This checklist identifies and describes the elements of an evaluation report. It is intended to serve as a flexible guide for determining an evaluation report’s content. It should not be treated as a rigid set of requirements. An evaluation client’s or sponsor’s reporting requirements should take precedence over the checklist’s recommendations. Decisions about the order of content and level of detail in a report should be made with consideration of the audience’s information needs and priorities.
This checklist is strictly focused on the content of long-form technical evaluation reports
This is an introduction to four essays in a Commentary that examine contributions of universities to science communication’s development but also challenges in
consolidating those efforts.
This editorial provides an overview of the Journal of Science Communication, volume 16, issue 5. This issue of JCOM explores the question ‘what works in science communication?’ from a variety of angles, as well as focusing on the politically sensitive topic of climate change. In addition, the issue contains a set of commentaries that explore the sometimes conflicting roles of universities in science communication.
Marked by the diversity of initiatives linking science and art and by new presentation formats, the 15th Congress of the Network for Popularisation of Science and Technology in Latin America and the Caribbean (RedPOP) saw heated debates on science, culture, politics and society. Between 21st and 25th August, it brought together in Buenos Aires (Argentina) about 400 participants from 14 countries in order to share new visions, initiatives and research work in science communication. During the event, which included a vast cultural programme, a series of challenges were raised for the future
This article aims to present a critical analysis of the book entitled “Creative Research Communication ― Theory and Practice”, written by Clare Wilkinson and Emma Weitkamp (Manchester University Press, 2016). We aim to present the structure of the book, highlighting its strengths and successes. Although some chapters focus on the UK, the book offers a wide range of examples of practical activities for the communication of research of global interest and provides very useful tips. Ethical issues and the importance of evaluation, of how to do carry out such evaluation and dissemination, are also
One way to encourage youth to pursue training in the STEM fields and enter the STEM workforce is to foster interest and engagement in STEM during adolescence. Informal STEM Learning Sites (ISLS) provide opportunities for building interest and engagement in the STEM fields through a multitude of avenues, including the programming that they provide for youth, particularly teens. Frequently, ISLS provide opportunities to participate in volunteer programs, internships or work, which allow teens both to learn relevant STEM knowledge as well as to share that knowledge with others through opportunities to serve as youth educators. While youth educator programs provide rich contexts for teens to engage as both learners and teachers in these informal STEM environments, research to date has not yet identified the relationship between serving as youth educators and STEM engagement. Thus, the goal of this project is to document the impact of youth educators on visitor learning in ISLS and to identify best practices for implementing youth educator programs. The project studies STEM interests and engagement in the youth participants and the visitors that they interact with at six different ISLS in the US and UK. This project is funded through Science Learning+, which is an international partnership between the National Science Foundation (NSF) and the Wellcome Trust with the UK Economic and Social Research Council. The goal of this joint funding effort is to make transformational steps toward improving the knowledge base and practices of informal STEM experiences. Within NSF, Science Learning+ is part of the Advancing Informal STEM Learning (AISL) program that seeks to enhance learning in informal environments and to broaden access to and engagement in STEM learning experiences.
This project examines youth educator experiences related to STEM identity, educational aspirations, and motivation. The project also identifies outcomes that the youth educators have on visitors to ISLS in terms of knowledge, interest, and engagement in STEM. The specific aims are: 1) Outcomes for Teens - To measure the longitudinal impact of participation in an extended youth educator experience in an ISLS; 2) Outcomes for Visitors - To compare visitor engagement with and learning from exhibits in ISLS when they interact with a youth educator, relative to outcomes of interacting with an adult educator or no educator; and 3) Outcomes Across Demographics and STEM Sites - To examine differences in visitor engagement based on participant characteristics such as socio-economic status (SES), age, gender, and ethnicity and to compare outcomes of youth educator experiences across different types of ISLS. This research, which draws on expectancy value theory and social cognitive theory, will follow youth participants longitudinally over the course of 5 years and use latent variable analyses to understand the impact on the youth educators as well as the visitors with whom they interact. Importantly, the results of this research will be used to develop best practices for implementing youth educator programs in ISLS and the results will be disseminated to both academic and practice-based communities.
This project has clear and measurable broader impacts in a variety of ways. First, the project provides guidance to improve programming for youth in ISLS, including both the sites involved directly in the research and to the larger community of ISLS through evaluation, development, and dissemination of best practices. Additionally, this project provides rigorous, research-based evidence to identify and describe the outcomes of youth educator programs. This study directly benefits the participants of the research, both the visiting public and the youth educators, through opportunities to engage with science. The findings speak to issues of access and inclusivity in ISLS, providing insight into how to design environments that are welcoming and accessible for diverse groups of learners. Finally, this project provides evidence for best practices for ISLS in developing programs for youth that will lead to interest in and pursuit of STEM careers by members of underrepresented groups.
Northern ecosystems are rapidly changing; so too are the learning and information needs of Arctic and sub-Arctic communities who depend on these ecosystems for wild harvested foods. Public Participation in Scientific Research (PPSR) presents a possible method to increase flow of scientific and local knowledge, enhance STEM-based problem solving skills, and co-create new knowledge about phenology at local and regional or larger scales. However, there remain some key challenges that the field of PPSR research must address to achieve this goal. The proposed research will make substantial contributions to two of these issues by: 1) advancing theory on the interactions between PPSR and resilience in social-ecological systems, and 2) advancing our understanding of strategies to increase the engagement of youth and adults historically underrepresented in STEM, including Alaska Native and indigenous youth and their families who play an essential role in the sustainability of environmental monitoring in the high latitudes and rural locations throughout the globe. In particular, our project results will assist practitioners in choosing and investing in design elements of PPSR projects to better navigate the trade-offs between large-scale scientific outcomes and local cultural relevance. The data collected across the citizen science network will also advance scientific knowledge on the effects of phenological changes on berry availability to people and other animals.
The Arctic Harvest research goals are to 1) critically examine the relationship between PPSR learning outcomes in informal science environments and attributes of social-ecological resilience and 2) assess the impact of two program design elements (level of support and interaction with mentors and scientists, and an innovative story-based delivery method) on the engagement of underserved audiences. In partnership with afterschool clubs in urban and rural Alaska, we will assess the impact of participation in Winterberry, a new PPSR project that investigates the effect of changes in the timing of the seasons on subsistence berry resources. We propose to investigate individual and community-level learning outcomes expected to influence the ability for communities to adapt to climate change impacts, including attributes of engagement, higher-order thinking skills, and their influence on the level of civic action and interest in berry resource stewardship by the youth groups. Using both quantitative and qualitative approaches, we compare these outcomes with the same citizen science program delivered through two alternate methods: 1) a highly supported delivery method with increased in-person interaction with program mentors and scientists, and 2) an innovative method that weaves in storytelling based on elder experiences, youth observations, and citizen science data at all stages of the program learning cycle. This project is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants. The project also has support from the Office of Polar Programs.
DATE:
-
TEAM MEMBERS:
Katie SpellmanElena SparrowChrista MulderDeb Jones
This 4-year project addresses fundamental equity issues in informal Science, Technology, Engineering and Mathematics (STEM) learning. Access to, and opportunities within informal STEM learning (ISL) remain limited for youth from historically underrepresented backgrounds in both the United States and the United Kingdom. However, there is evidence that ISL experiences can expand opportunities for youth learning and development in STEM, for instance, increase positive attitudes towards educational aspirations and future careers/pursuits, improve grades and test scores in school settings, and decrease disciplinary action and dropout rates. Through research and development, this project brings together researchers and practitioners to focus on the experiences, practices and tools that will support equitable youth pathways into STEM. Working across conceptual frameworks and ISL settings (e.g. science centers, community groups, zoos) and universities in four urban contexts in two different nations, the partnership will produce a coherent knowledge base that strengthens and expands research plus practice partnerships, builds capacity towards transformative research and development, and develops new models and tools in support of equitable pathways into STEM at a global level. This project is funded through Science Learning+, which is an international partnership between the National Science Foundation (NSF) and the Wellcome Trust with the UK Economic and Social Research Council. The goal of this joint funding effort is to make transformational steps toward improving the knowledge base and practices of informal STEM experiences. Within NSF, Science Learning+ is part of the Advancing Informal STEM Learning (AISL) program that seeks to enhance learning in informal environments and to broaden access to and engagement in STEM learning experiences.
This Equity Pathways project responds to three challenges at the intersections of ISL research and practice in the United States and the United Kingdom: 1) lack of shared understanding of how youth from historically underrepresented backgrounds perceive and experience ISL opportunities across national contexts, and the practices and tools needed to support empowered movement through ISL; 2) limited shared understanding and evidence of core high-leverage practices that support such youth in progressing within and across ISL, and 3) limited understanding of how ISL might be equitable and transformative for such youth seeking to develop their own pathways into STEM. The major goal of this Partnership is for practitioners and researchers, working with youth through design-based implementation research, survey and critical ethnography, to develop new understandings of how and under what conditions they participate in ISL over time and across settings, and how they may connect these experiences towards pathways into STEM. The project will result in: 1) New understandings of ISL pathways that are equitable and transformative for youth from historically underrepresented backgrounds; 2) A set of high leverage practices and tools that support equitable and transformative informal science learning pathways (and the agency youth need to make their way through them); and 3) Strengthened and increased professional capacity to broaden participation among youth from historically underrepresented backgrounds in STEM through informal science learning. The project will be carried out by research + practice partnerships in 4 cities: London & Bristol, UK and Lansing, MI & Portland, OR, US, involving university researchers (University College London, Michigan State University, Oregon State University/Institute for Learning Innovation) practitioners in science museums (@Bristol Science Centre, Brent Lodge Park Animal Centre, Impressions 5, Oregon Museum of Science & Industry) and community-based centers (STEMettes, Knowle West Media Centre, Boys & Girls Clubs of Lansing, and Girls, Inc. of the Pacific Northwest).
As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants. This project will develop and test intergenerational science media resources for parents that are participating in adult education programs and their young children. The materials will build on the research-based and successful children's television program, Fetch with Ruff Ruffman. The target audience includes parents enrolled in adult education programs who lack a high school diploma or are in English as a Second Language classes. These resources will support parents' engagement in science activities with their children both in the adult education settings as well as at home. Adult and family educators will receive professional development resources and training to support their integration of the parent/child activities. Project partners include the National Center for Families Learning, Kentucky Educational Television, and Alabama Public Television,
The goals of the Ruff Family Science project are to: (1) investigate adult education settings that feature an intergenerational learning model, in order to learn about the unique characteristics of adults and families who are enrolled in these programs; (2) examine the institutional circumstances and educator practices that support joint parent/child engagement in science; (3) iteratively develop new prototype resources meet the priorities and needs of families and educators involved in intergenerational education settings; and (4) develop the knowledge needed to create a fuller set of materials in the future that will motivate and support diverse, low-income parents to investigate science with their children. The research strategy is comprised of three main components: Phase 1: Needs Assessment: Determine key motivations and behaviors common to adult education students who are also parents; surface obstacles and assets inherent in these parents' current practices; and examine the needs and available resources for supplementing parents' current engagement in family science learning. Phase 2: Prototype Development: Iteratively develop two prototype Activity Sets, along with related educator supports and training materials, designed to promote joint parent-child engagement with English and Spanish-speaking families around physical science concepts. Phase 3: Prototype Field Test: Test how the two refined prototype Activity Sets work in different educational settings (adult education, parent education, and parent and child together time). Explore factors that support or impede effective implementation. Sources of data for the study include observations of adult and parent education classes using an expert interview protocol, focus groups, adult and family educator interviews, and parent surveys.
As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program funds innovative research, approaches and resources for use in a variety of settings. The subject of physics and all of its sub-disciplines are becoming more prevalent in the public press as the research results appear to be quite interesting and important. While the physics discipline has made a Nation-wide effort to acquaint the public with physics knowledge through informal education learning experiences for years, it has not been as successful as the community desires. Thus, this project is aimed to gather all of the informal and outreach physics education efforts that have been attempted in the hope of finding the best practices for learning physics concepts and practices. A compendium will be published to inform future opportunities on how to educate the public through informal and outreach mechanisms. This project is a collaboration between Michigan State University and the University of Colorado. The physics community has a long history of engaging audiences in informal education activities. Physics institutions that facilitate informal programs include university departments, national laboratories and centers, and professional societies and organizations. There is, however, no systemic understanding of how these programs are facilitated, nor an assessment of the collective impact that these programs have on participants. This project will address numerous research questions in the broad areas of Activity Detail, Structural Aspects, and Assessment. Further, their efforts will determine the "who, what, why, where and how" of informal physics offerings, focusing on their facilitation, impact on participants, and the academic and discipline-specific cultures from which these programs originate. The study has several definite research outcomes that will emerge from this methodology: 1) They will produce a survey of the informal efforts of university physics departments, national physics labs and national physics organizations, 2) They will develop a taxonomy of informal physics programs from which we can characterize the landscape of programs, and 3) by investigating both "successful" as well as "failed" or terminated programs, they will develop an understanding of the culture and resources needed to support outreach from these research findings. In addition, they will produce published works that can be utilized by informal practitioners and administrators in physics to examine current programs and guide the development of new programs. With regards to the research questions and framework, the overarching and driving question for this research project is: "What is the landscape of informal physics learning, specifically, of those programs in the United States facilitated by physicists and physics students at academic institutions, national labs and by national physics organizations?" This study will provide a robust understanding of the state of informal physics programs and outreach by physicists in the United States today. Findings will inform practitioners and administrators as to how best to support and design informal physics programming. The results will also have broad implications for other discipline-specific informal STEM programming. The primary data collection methods will be a nationwide survey and interviews with a large sample of informal practitioners from the physics community. Site visits will be conducted with a subset of these programs in order to observe programs in action and to glean insights from university participants, community partners, public, and K-12 audiences.
A collaboration of TERC, MIT, The Woods Hole Oceanographic Institution and community-based dance centers in Boston, this exploratory project seeks to address two main issues in informal science learning: 1) broadening participation in science by exploring how to expand science access to African-American and Latino youth and 2) augmenting science learning in informal contexts, specifically learning physics in community-based dance sites. Building on the growing field of "embodied learning," the project is an outgrowth in part of activities over the past decade at TERC and MIT that have investigated approaches to linking science, human movement and dance. Research in embodied learning investigates how the whole body, not just the brain, contributes to learning. Such research is exploring the potential impacts on learning in school settings and, in this case, in out of school environments. This project is comprised of two parts, the first being an exploration of how African-American and Latino high school students experience learning in the context of robust informal arts-based learning environments such as community dance studios. In the second phase, the collaborative team will then identify and pilot an intervention that includes principles for embodied learning of science, specifically in physics. This phase will begin with MIT undergraduate and graduate students developing the course before transitioning to the community dance studios. This project is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants.
The goal of this pilot feasibility study is to build resources for science learning environments in which African-American and Latino students can develop identities as people who practice and are engaged in scientific inquiry. Youth will work with choreographers, physicists and educators to embody carefully selected physics topics. The guiding hypothesis is that authentic inquiries into scientific topics and methods through embodied learning approaches can provide rich opportunities for African-American and Latino high school-aged youth to learn key ideas in physics and to strengthen confidence in their ability to become scientists. A design- based research approach will be used, with data being derived from surveys, interviews, observational field notes, video documentation, a case study, and physical artifacts produced by participants. The study will provide the groundwork for producing a set of potential design principles for future projects relating to informal learning contexts, art and science education with African American and Latino youth.
DATE:
-
TEAM MEMBERS:
Folashade Cromwell SolomonTracey WrightLawrence Pratt
The Exploratorium’s livestream of the August 2017 Total Solar Eclipse reached over 63 million people. Live programs in English and Spanish provided an informal learning experience outside the museum. Over 2.75 million people viewed on-demand videos on eclipse science. Sixty major media providers rebroadcast the livestream telescope feed.
Edu, Inc. conducted a summative evaluation of the NASA-funded project. The study reveals that the Exploratorium successfully disseminated eclipse science and STEM content through media channels and a mobile app, delivering a museum experience to online