Summative evaluation of the Marcellus Matters: EASE project. Marcellus Matters: Engaging Adults in Science and Energy (EASE) was a program of Penn State University’s Marcellus Center for Outreach and Research (MCOR), in collaboration with other experts across the university. The first year of program activities took place in 2012, and the project continued through September 2016. EASE was a multidisciplinary initiative that provided adults in rural Pennsylvania with opportunities to increase their knowledge of science and energy systems and engage in scientific inquiry and investigation
Mathematics is a notoriously disliked subject; there is so little stigma associated with being "bad at math", that educated adults openly describe themselves in this way. There are many reasons for math's unpopularity; chief among them is that school mathematics seldom offers opportunities to engage with the richness of this potentially fascinating subject. As a result, the mathematics education pipeline in the United States is more often a filter than a pump, siphoning students out rather than bringing them along. Children have libraries to help them fall in love with literature: where do they get a chance to fall in love with math? This project presents a unique opportunity to study children engaged with mathematics in an informal setting, the Minnesota State Fair, facilitated by mathematically knowledgeable volunteers. The Math On-a-Stick mathematical playground provides a place for children to engage with mathematics by exploring patters, asking quantitative questions, and investigating shape and space to mathematize their play. The project will observe and videotape this engagement to inform the design of mathematical learning environments in a variety of outside of school time settings, such as after school programs and summer camps, that are accessible to a wider range of the population. This project is co-funded by the EHR Core Research (ECR) and Advancing Informal STEM Learning (AISL) programs. The ECR program emphasizes fundamental STEM education research that generates foundational knowledge in three thematic areas: STEM learning and learning environments, broadening participation, and STEM workforce development. The AISL program seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments.
The project will investigate three research questions: (1) How does the design of various parts of the exhibit differently support rich mathematical interactions between children and mathematicians? (2) How do children engage different parts of the exhibit? How do differences in engagement relate to (a) exhibit design and (b) prior mathematical experience? (3) How do exhibit volunteers, mathematicians, and caregivers interact to support (or undermine) students' mathematical play? The project will use participant observation and videography to capture visitors' activities through the exhibit, analyzing them as qualitative case studies.
Rural communities across the Nation are, in general, underserved in terms of the various forms of STEM education. Clearly, they are under-represented in the realm of contemporary STEM subjects often because they are geographically isolated and cannot travel to cities where there are Science and Museum Centers for informal education opportunities. As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program funds innovative resources for use in a variety of settings. This award will, in a collaborative effort within the community, bring STEM activities to selected communities in Arizona. Among the initial activities, there will be a STEM festival highlighting aspects of the community and its assets in an effort to gather support and begin to give perspective on identity for an extended effort of longevity. Further, these communities will be networked to facilitate discussion and to enhance effectiveness.
This project will develop STEM activities and STEM learning within a selected community by giving the community and its residents identity and opportunities for youth development and career choices. The selected communities in Arizona represent a diverse group that includes Native Americans and Latinos. In collaboration with community residents, a designed plan will be established that satisfies the needs and opportunities that can be derived from the extant community assets whether it is mining, tourism, or government facilities. Evaluation efforts are set to determine what the key features and methodologies are that facilitate STEM knowledge acquisition for each rural community. This project represents seminal and foundational work in the area of rural informal STEM education. Researchers will explore the following questions: 1) understanding how rural communities currently perceive, access, and engage in informal science learning, and the extent to which they identify themselves and/or their community in relation to science; and 2) the extent to which relevant, place-based networks can increase public awareness of local STEM assets, resources, and opportunities, and foster a science-related identity at both the personal and community level. These data will be compared to data on other rural community projects in the AISL portfolio. The partners in this effort include the Arizona Science Center, community leaders from four rural regions in Arizona, Arizona State University, and the Center of Science and Industry.
Adult education beyond K-12 and postsecondary levels is very important as this citizenry group is often the policy and decision makers in local communities, as well as for state and federal issues that impact the Nation. Moreover, they are responsible for advising their progeny on a myriad of choices. This project will plan, execute, and promote four annual public lecture events, working with a professional educational evaluation expert to develop an appropriate assessment tool for adult learners in the structured informal learning environment of a science café. These planned events will be used to test and refine an assessment tool for making this work widely available to the community of informal science practitioners and researchers. Further, this project is a pilot for epitomizing the use of science cafés to address the learning needs of unique citizens of Richmond, Virginia. The project is committed to including under-represented citizens including Veterans with disabilities. The evaluation and research efforts will validate the education mechanisms so science cafés can be more effective in the future. As a part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program funds research and innovative resources for use in a variety of settings.
This project is a collaboration that includes Virginia Commonwealth University (VCU), Rockman et al (an evaluation firm), Science Pub RVA (Science Pub RVA is a long-running and award-nominated Richmond, VA science café), Carver Community Partnership, East District Family Resource Center, VCU Partnership for People with Disabilities, VCU Department of Rehabilitation Counseling, VCU Medical Center, and a variety of other VCU departments. The investigators will conduct a series of science cafés to determine motivation, interests, and best practices for educating the diverse citizens of Richmond, Virginia. The objective of the research is to rigorously analyze the characteristics of participants and cohesively determine the best practices for the effective learning for each person. Further, rigorous evaluation will determine validity of the best and most effective learning practices enabling the project to derive an adaptable model. The investigators' hypothesis is that the participant's knowledge base is derived from the traditional learning which occurred in the K-12 classroom. Thus, in this work, the investigators hope to add to the participant's knowledge base with STS (Science, Technology, and Society) content and enhance the depth and breadth of knowledge and knowledge acquisition. The research scope will embrace an assessment that is based on the three vertices of a triangle composed of cognition, observation, and interpretation, all of which converge on the nature of science, the relevance of science to everyday life, and decision-making behaviors.
This is an efficacy study through which the Denver Museum of Nature and Science, the Denver Zoo, the Denver Botanic Gardens, and three of Denver's urban school districts join efforts to determine if partnerships among formal and informal organizations demonstrate an appropriate infrastructure for improving science literacy among urban middle school science students. The Metropolitan Denver Urban Advantage (UA Denver) program is used for this purpose. This program consists of three design elements: (a) student-driven investigations, (b) STEM-related content, and (c) alignment of schools and informal science education institutions; and six major components: (a) professional development for teachers, (b) classroom materials and resources, (c) access to science-rich organizations, (d) outreach to families, (e) capacity building and sustainability, and (e) program assessment and student learning. Three research questions guide the study: (1) How does the participation in the program affect students' science knowledge, skills, and attitudes toward science relative to comparison groups of students? (2) How does the participation in the program affect teachers' science knowledge, skills, and abilities relative to comparison groups of teachers? and (3) How do families' participation in the program affect their engagement in and support for their children's science learning and aspirations relative to comparison families?
The study's guiding hypothesis is that the UA Denver program should improve science literacy in urban middle school students measured by (a) students' increased understanding of science, as reflected in their science investigations or "exit projects"; (b) teachers' increased understanding of science and their ability to support students in their exit projects, as documented by classroom observations, observations of professional development activities, and surveys; and (c) school groups' and families' increased visits to participating science-based institutions, through surveys. The study employs an experimental research design. Schools are randomly assigned to either intervention or comparison groups and classrooms will be the units of analysis. Power analysis recommended a sample of 18 intervention and 18 comparison middle schools, with approximately 72 seventh grade science teachers, over 5,000 students, and 12,000 individual parents in order to detect differences among intervention and comparison groups. To answer the three research questions, data gathering strategies include: (a) students' standardized test scores from the Colorado Student Assessment Program, (b) students' pre-post science learning assessment using the Northwest Evaluation Association's Measures for Academic Progress (science), (c) students' pre-post science aspirations and goals using the Modified Attitude Toward Science Inventory, (d) teachers' fidelity of implementation using the Teaching Science as Inquiry instrument, and (e) classroom interactions using the Science Teacher Inquiry Rubric, and the Reformed Teaching Observation protocol. To interpret the main three levels of data (students, nested in teachers, nested within schools), hierarchical linear modeling (HLM), including HLM6 application, are utilized. An advisory board, including experts in research methodologies, science, informal science education, assessment, and measurement oversees the progress of the study and provides guidance to the research team. An external evaluator assesses both formative and summative aspects of the evaluation component of the scope of work.
The key outcome of the study is a research-informed and field-tested intervention implemented under specific conditions for enhancing middle school science learning and teaching, and supported by partnerships between formal and informal organizations.
This poster was presented at the 2016 Advancing Informal STEM Learning (AISL) PI Meeting held in Bethesda, MD on February 29-March 2. Teen Science cafes are a way for teens to explore the big advances in science and technology affecting their lives. Teens and local STEM experts engage in lively conversations and activities to explore a topic.
This poster was presented at the 2016 Advancing Informal STEM Learning (AISL) PI Meeting held in Bethesda, MD on February 29-March 2. The project's goal is to demonstrate an educational model fully commensurate with the demands of the 21st Century workforce, and more specifically, with the emerging “green-tech” economy.
This poster was presented at the 2016 Advancing Informal STEM Learning (AISL) PI Meeting held in Bethesda, MD on February 29-March 2. Science fairs have been taking place for more than 60 years. Yet in spite of their wide implementation and long history, there are few empirical studies that have examined the relationship between student participation in these fairs and their learning and interest in science. Additionally, there have been no studies to understand the real cost of these programs relative to the student benefits. Our 4-year study will be the first step to understanding the
DATE:
TEAM MEMBERS:
Abigail LevyJacqueline DeLisiMarian Pasquale
This poster was presented at the 2016 Advancing Informal STEM Learning (AISL) PI Meeting held in Bethesda, MD on February 29-March 2. EvalFest is a community of practice designed to test innovative evaluation methods across 24 Science Festivals to measure their impact.
The University of Alaska Fairbanks will partner with the National Optical and Astronomy Observatory, the University of Alaska Museum of the North, and the University of Washington-Bothell to bring biomaterials, optics, photonics, and nanotechnology content, art infused experiences, and career awareness to art-interested girls. This full scale development project, Project STEAM, will explore the intersections between biology, physics, and art using advanced technologies at the nano to macro scale levels. Middle school girls from predominately underrepresented Alaskan Native, Native American (Tohono O'odham, Pascula Yaqui) and Hispanic groups, their families, teachers, and Girl Scout Troop Leaders in two site locations- Anchorage, Alaska and Tucson, Arizona will participate in the project. Centered on the theme "Colors of Nature," Project STEAM will engage girls in science activities designed to enhance STEM learning and visual-spatial skills. Using advanced technologies, approximately 240 girls enrolled in the Summer Academy over the project duration will work with women scientist mentors, teachers, and Girl Scout Troop Leaders to create artistic representations of natural objects observed at the nano and macro scale levels. Forty girls will participate in the Summer Academy in year one (20 girls per site- Alaska and Arizona). In consequent years, approximately180 girls will participate in the Academy (30 girls per site). Another 1,500 girls are expected to be reached through their Girl Scout Troop Leaders (n=15) who will be trained to deliver a modified version of the program using specialized curriculum kits. In addition, over 6,000 girls and their families are expected to attend Project STEAM Science Cafe events held at local informal science education institutions at each site during the academic year. In conjunction with the programmatic activities, a research investigation will be conducted to study the impact of the program on girls' science identity. Participant discourse, pre and post assessments, and observed engagement with the scientific and artistic ideas and tools presented will be examined and analyzed. A mixed methods approach will also be employed for the formative and summative evaluations, which will be conducted by The Goldstream Group. Ultimately, the project endeavors to increase STEM learning and interest through art, build capacity through professional development, advance the research base on girls' science identity and inspire and interest girls in STEM careers.
The Balboa Park Cultural Partnership, in collaboration with several informal science education and other cultural and business organizations in San Diego, Chicago, and Worcester, MA are implementing a research and development project that investigates a range of possible approaches for stimulating the development of 21st Century creativity skills and innovative processes at the interface between informal STEM learning and methods for creative thinking. The goal of the research is to advance understanding of the potential impacts of creative thinking methods on the public's understanding of and engagement with STEM, with a focus on 21st Century workforce skills of teens and adults. The goal of the project's development activities is to experiment with a variety of "innovation incubator" models in cities around the country. Modeled on business "incubators" or "accelerators" that are designed to foster and accelerate innovation and creativity, these STEM incubators generate collaborations of different professionals and the public around STEM education and other STEM-related topics of local interest that can be explored with the help of creative learning methodologies such as innovative methods to generate creative ideas, ideas for transforming one STEM idea to others, drawing on visual and graphical ideas, improvisation, narrative writing, and the process of using innovative visual displays of information for creating visual roadmaps. Hosting the project's incubators are the Balboa Park Cultural Partnership (San Diego), the Museum of Science and Industry (Chicago) and the EcoTarium (Worcester, MA). National partners are the Association of Science-Technology Centers, the American Association for the Advancement of Science, and the Americans for the Arts. Activities will include: the formation and collaborative processes of three incubator sites, a research study, the development of a creative thinking curriculum infused into science education, professional development based on the curriculum, public engagement events and exhibits, a project website and tools for social networking, and project evaluation. A national advisory council includes professionals in education, science, creativity, and business.
EvalFest (Evaluation Use, Value, and Learning through Festivals of Science and Technology) will test innovative evaluation methods in science festivals that are being held across the country and assess in what ways and how effectively they are used. Morehead Planetarium and Science Center (at the University of North Carolina-Chapel Hill) and the University of California, San Francisco, in collaboration with over twenty science festivals, will (1) investigate whether a multisite evaluation approach is an effective model for creating common metrics for informal STEM education, (2) develop common methods to measure the effects of Festivals, (3) create a query-able database of 50,000 Festival attendees to share with the informal STEM learning field, and (4) document whether these efforts also result in new knowledge related to informal STEM education. The project will develop the Enterprise Feedback Management (EFM) system and query-able database for the festival community. EFMs are systems, including processes and software, that enable groups (such as the festival network) to collect, organize, analyze and share data. The EFM system will be designed to integrate data across sites and to allow users to extract data of interest. The project will refine evaluation tools currently used within the Science Festival Alliance that assess self-reported festival learning, and the effects of festival attendance, motivation, and future science participation. It will collect economic impact data and longitudinal festival attendee data. The project will also develop some new evaluation tools such as secret shopper observational protocols. Data from festival attendees will be collected onsite at participating festivals.