New York Hall of Science Science Career Ladder Retrospective Impact Study Final Report

For more than 20 years, the Science Career Ladder Program at the New York Hall of Science has provided unique opportunities for middle school, high school, and college students to be trained and mentored as Explainers at the museum. The program is designed to encourage and support personal and professional development of these students, increasing their academic achievement, personal growth, and participation in science and teaching careers. Previous evaluations have documented the impact on participants and opportunities for program growth. This report documents the findings and conclusions from Part I of a two-part evaluation project, conducted by the Institute for Learning Innovation (ILI), to update understanding of program impact. Part I builds upon and extends a previous retrospective impact evaluation conducted by ILI in 2002. The purpose of this retrospective study was to obtain up-to-date evidence of the Science Career Ladder Program's long-term impact in supporting and encouraging its participants' personal, professional, and academic development. A sequential mixed methods design was used, beginning with a quantitative web survey of alumni (n=164), followed by a qualitative, semi-structured telephone interview with a subsample of respondents (n=16). The web survey response rate was approximately 27%, despite efforts to reach as many program alumni as possible. While many factors contributed to this somewhat low response rate, it could indicate a bias in the sample. Thus, it is advised to consider results conservatively, as a maximum level of impact. Overall, the results of this study indicate that the Science Career Ladder Program has been continually successful at achieving its goals in impacting each of the targeted areas of participants' personal, academic, and professional growth. The study demonstrates that program alumni continue to draw upon the experience gained, the lessons learned, and the skills developed during their time working as Explainers at the New York Hall of Science. Academic Achievement Nearly all former program participants go on to attain advanced education (undergraduate degree or higher) at a far higher rate than the general population of New York City, with a particularly stark contrast among those identified as Spanish/Hispanic/Latino, where program alumni attain advanced education at a rate five times higher than those in the general population. However, many participants entered with already well-defined educational goals. Because these students were already committed to advanced degrees, the program caused relatively little change. There was evidence to indicate that a smaller sub-set of participants had less well-defined goals upon entering the program, and for some individuals, the role models of fellow Explainers did encourage further academic pursuits. The strongest impact on academic achievement was developing knowledge, skills, and confidence that they have successfully applied to academic settings, including: science content knowledge, study skills, learning habits of mind, confidence in oral presentations, and problem-solving skills. Career/Professional Development There was a greater impact of the program on influencing career directions than there was on level of academic achievement, with more alumni reporting it influenced their decisions insome way. Program alumni largely choose careers and college majors in the sciences and in education/teaching. A large number of participants entered with a strong interest in pursuing one of these paths, although for some, the program helps reveal new directions. The impact of the program seemed to be less in directing individuals to a particular career, and more as a test-ground for their choices. They could gain assuredness in their career choices through experiences that highlighted existing and new interests, passions, aptitudes, and potential. Many emerged from the program more confident in their chosen path. Skills and Abilities Development of various skills and abilities seems to be one of the strongest program impacts, and evidence shows these are skills that alumni continue to use throughout their lives and careers. The main areas of skills that are developed include: communication abilities and techniques, self-confidence overall and in specific situations, problem-solving skills, interpersonal skills, and leadership abilities. Science Literacy and Engagement The program positively impacted the lifelong learning habits of participants, fostering a desire to continue to explore science topics, as well as general curiosity about the world. Many participants acquired an increased awareness of the role of science in all facets of life and increased level of noticing and drawing connections between their lives and science topics. The program also developed lasting positive attitudes about science, most notably among those participants who disliked or were intimidated by science upon entering the program. The result of participation was a changed view of science and its relevance in their lives. Program Design Motivations for entering and for continuing in the program were somewhat different and show a change in motivation that takes place over the course of involvement. Further, a majority of participants leave the program either for a job/career elsewhere or to attend school, both of which are in line with the program's objectives. Other reasons for leaving indicate there are differences in the challenges that are faced by participants during different phases of involvement in the program. The program design builds participants' confidence in their abilities, particularly through training, peer-to-peer teaching, and relatable supervisors. Due to this, participants describe the program as supportive and encouraging, with co-workers and supervisors all forming a strong social network. The evidence also suggests the program supports the growth of individuals specifically in those areas where they personally need greatest development. The appendix of this report includes the survey and interview protocol used in the study.

If you would like to edit a resource, please email us to submit your request.