This report presents findings from a formative evaluation of Science Education for New Civic Engagements and Responsibilities-Informal Science Education (SENCER-ISE), a National Science Foundation and Noyce Foundation funded initiative to support partnerships between informal science and higher education institutions. This evaluation looked primarily at the collaborative infrastructure of SENCER-ISE, which included the web site, SENCER Summer Institute, and communications with project staff and/or the advisory board. This evaluation is the third evaluation that Randi Korn & Associates, Inc
DATE:
TEAM MEMBERS:
RK&A, Inc.
resourceevaluationProfessional Development, Conferences, and Networks
The National Center for Science and Civic Engagement (NCSCE) contracted Randi Korn & Associates, Inc. (RK&A) to conduct a summative evaluation of its SENCER-ISE project partnerships. SENCER-ISE is an initiative that brings partners from higher education (HE) together with partners from informal science education (ISE) to create projects that engage audiences in science using the lens of civic engagement. SENCER funded 10 partnerships over three years—six through the National Science Foundation (DRL #1001795) and four through the Noyce Foundation. Previously, RK&A conducted a formative
Purpose: In the most recent National Assessment of Educational Progress only 17% of 8th grade students performed at or above the proficient level in U.S. history. One way to engage students in learning history is to create history learning resources that are designed to be relevant and appealing to young people's interests and regular activities. Surveys find that almost all teenage boys and girls play digital games, and the majority of teens play daily. This project will leverage the potential of games and technology to engage students and increase history skills and content knowledge.
Project Activities: The team, consisting of graphic artists, content specialists, computer scientists, and programmers, will initially create wireframes and a functional game prototype. Following feedback from a group of students and teachers on the user-interface, the team will produce an online tablet app. Iterative refinements will be conducted at major production milestones until the intervention is fully functional. Once development is complete, the researchers will assess the usability and feasibility, fidelity of implementation, and the promise of the product to improve outcomes in a pilot study. The study will include 200 8th grade students in eight classrooms. Four classrooms will be assigned to play to game as part of the curriculum over three to five class periods, and four classrooms will be taught the same historical content using the business as usual curriculum without the game. Each group will complete pre- and post- assessments to assess differences in history knowledge and skills.
Product: This project team will develop a tablet-based interactive role-playing game that immerses 5th through 9th grade students in the history of the Great Depression. The game will provide players an experiential understanding of the hardships that beset Americans in the 1930s and their strategies for survival, as individuals and as a nation. Features of the game will include story-based immersive narrative missions where student's decisions continually drive the action, tips and hints for students who are struggling in the game, writing tools, and interactive maps. The game will can be integrated within a course or used as a supplement. A teacher dashboard will be developed to facilitate the use of the game within classroom settings. Finally, the final product will include upgrades to existing games, including City of Immigrants and the The Hardest Times. The upgrades will publish these games to tablets and will include deeper in-game assessment opportunities.
Purpose: There is concern about a decline in mathematics achievement scores among U.S. students during the middle school years. For example, while 4th grade U.S. students rank 8th overall on an international mathematics comparison, by 10th grade U.S. student's drop significantly to 25th in the same comparison. Some researchers posit that much of this decline relates to how math is taught in the U.S. and with how students become less engaged as learners in middle school. The purpose of this project is to develop a web-based game to engage 7h grade students in a narrative-based story which will apply learning of content and skills aligned to the Common Core State Standards (CCSS) in mathematics.
Project Activities: During Phase I in 2012, the team developed a functioning prototype and conducted usability and feasibility research with fourteen 7th grade students. Researchers found that the prototype functioned as intended and that students were highly engaged while playing the game. In Phase II, the team will develop a fully-functional user interface with animated characters, interactivity across student users, narrative scripts and accompanying art assets, 36 problem sets, and student and teacher dashboards and databases. After development is complete, a pilot study will examine the usability and feasibility, fidelity of implementation, and the promise of the game to improve math learning. The study will include 120 students in 6 classrooms in three schools, with one classroom per school randomly assigned to use the game and the other half assigned to a business-as-usual control. Analyses will compare student scores on pre and post mathematics measures.
Product: Empires is a web-based game that addresses 36 pre-algebra Common Core State Standards in mathematics for 7th and 8th grades. The game follows a storyline in a recreation of an ancient empire which is at the brink of agricultural revolution and of becoming a trade economy. As students play the game, they engage in math-focused activities to drive the action, such as taxing citizens to learn ratios and proportions, allocating resources to learn percentages, and measuring the distance and time between a neighboring empire by applying the principles of the Pythagorean Theorem. As a socially networked game, students will interact with other students in the class to complete trades that lead to encounters with different math problems. The game will include two helpful, funny, advisors who will scaffold learning through mathematical discourse, arguing over the next most important thing to do. The game design architecture will work on a wide range of computers, including desktops and iPads. A teacher's guide and companion website will provide guidance to classroom activities that complement the game.
The Wild Center will partner with Adirondack Museum, Cornell’s Maple Program, and New York State/Northeastern New York Maple Producers Associations to build regional identity, revitalize a heritage industry, and connect people to nature through the art, story, history, and science of maple sugaring. The Northern New York Maple Project will create interpretative exhibits with ecological, historical, and economic information. The museum will develop an instructional maple sugaring video; a touch-screen story kiosk that lets visitors share stories through the exhibit and social media; a storytelling workshop for staff, project partners, and maple producers; community events and conferences; a school education program; community sugaring workshops; and educational materials, website, social media, and outreach to industry, food enthusiasts, and the business community. Regular planning meetings on goals and deliverables will track results and an outside consultant will evaluate the overall success of the project.
With support from the Institute of Museum and Library Services, The Wild Center (TWC) engaged Insight Evaluation Services (IES) to assess the impact of specific outreach activities of the Northern New York Maple Project between September 2013 and September 2015. Data for this two-year evaluation study were collected via in-depth telephone interviews conducted with a total of 25 participants, including 16 Tupper Tappers (Tupper Lake area residents who engaged in backyard tapping to provide sap for syrup production at the museum through the Community Maple Project), four local school teachers
The Charles H. Wright Museum of African American History will enhance its staff capacity and train current educational staff in STEM (science, technology, engineering, and mathematics) education delivery, both of which will increase its ability to deliver interpretive tours and programming around the newly opened permanent exhibition, “Inspiring Minds: African Americans in Science and Technology.” The project will include hiring a full-time STEM educator to work with education and archival staff to create and implement a family learning approach to the sciences in the rich context of African American history and culture. A training facilitator will develop a cohort of STEM interns and train current staff educators to present STEM learning experiences. At the conclusion of the project, the museum will have increased staff capacity and widespread expansion and integration of STEM opportunities for the youth, families, and schools of Detroit.
The Mütter Museum will develop a new interactive online exhibit that explores the important role of medicine in American history through its unique collections. In keeping with the museum's broader medical humanities focus, the online exhibit will be presented thematically in a life cycle sequence. Representative objects will include obstetrical forceps, an iron lung, a surgical kit, William Harvey's book on blood circulation De Motu Cordis, eyeglasses, a tooth extractor, and an embalming kit. The museum will also develop a curriculum that addresses national secondary school education standards in history, science, and health by utilizing narrative stories, specimens, models, medical tools, photographs, and texts from collections. Exploring historic events and their health and medical underpinnings through an interesting narrative lens will engage audiences in critical STEM topics by connecting personal stories to the objects actually used to understand disease and heal people.
In Defense of Food (IDOF) is a media and outreach project based on Michael Pollan's best-selling book of the same title. Through the lens of food science, IDOF is designed to engage diverse audiences in learning about: (1) how science research is conducted, (2) how research findings are used in media, marketing, and public policy, and (3) how to apply food science research in everyday life. IDOF will be created by Kikim Media, an independent production company, broadcast and distributed by PBS and supported by an extensive outreach campaign and interactive website. The project's educational materials will be developed, in part, by the Teacher's College at Columbia University's Center for Food and Nutrition, with dissemination supported by the Coalition for Science After School and by Tufts University's Healthy Kids Out of School initiative, which involves nine of the leading out of school time (OST) organizations, such as Girl Scouts USA, and the National Urban League. The project advisory committee includes highly respected researchers in food, nutrition, and health. IDOF will use an integrated strategy of learning resources, combining a television documentary with online/social media, community outreach, and youth activities. Knight Williams Research Communications will conduct formative and summative evaluation of all major components of the project. The results will advance the informal science community's understanding of how the combination of a documentary with outreach, website/social media, and afterschool activities impacts motivation and learning. The evaluation study will pay special attention to the degree to which participation in the community events, social media/website, and afterschool activities motivates deeper or extended engagement with the subject. Project evaluation results and educational resources will be widely disseminated to the informal science community. IDOF includes a two-hour documentary film that will be produced in both English and Spanish; a community-level outreach campaign focused on reaching underserved audiences who may not watch public television; a set of activities for use in afterschool programs, youth programs and schools; and an interactive and content-rich website with tightly integrated social media tools. IDOF will be nationally broadcast by PBS; the Spanish-language version of IDOF will be broadcast by Vme Television. The ambitious IDOF educational materials and outreach campaign, combined with interactive web and social media, will reach large and diverse audiences. The intended impacts on audiences include increased knowledge and understanding of the scientific process by learning what food scientists do, what techniques they use, and how scientists arrive at their conclusions; the development of critical thinking skills audiences can use when evaluating messages about food and nutrition in media and advertising and when making decisions about what food to buy and eat; and becoming active learners and consumers regarding food. Evaluation results will be widely disseminated to science media producers and the informal science community via professional publications and presentations at conferences. The ultimate value of the In Defense of Food documentary and learning initiative will be to enhance public understanding of the crucial importance of science in people's everyday lives and in shaping dozens of daily decisions.
In collaboration with the libraries of Oklahoma State University (OSU) and Mount Holyoke College, the American Library Association proposes a traveling exhibit and public programs for 40 libraries examining the history and legacy of the Dust Bowl. The project spotlights Ken Burns' film "The Dust Bowl," and brings to public view two little known Dust Bowl archives: online oral history interviews of Dust Bowl survivors at OSU, and letters and essays of Caroline Henderson, a Mount Holyoke alumna who farmed in Oklahoma throughout the Dust Bowl. Libraries will display the exhibit for 6 weeks and present at least 3 public humanities programs from a list provided. The project humanities themes include the interaction between humans and nature; the different ways human beings respond to adversity; and how people living in the Dust Bowl tried to understand their social, economic, and ecological environment.
The Robert R. McCormick Foundation contracted RK&A to conduct a formative evaluation of the Freedom Express mobile museum program to assess the degree to which the program supports students toward the achievement of stated civic engagement-related outcomes. How did we approach this study? The evaluation was designed to explore the extent to which students demonstrate desired outcomes after experiencing the Freedom Express program and to identify the extent to which teachers value and recognize the benefits of the program. To capture diverse perspectives and produce reliable data, RK&A utilized
DATE:
TEAM MEMBERS:
Randi Korn & Associates, Inc.Randi Korn