“The Roads Taken” virtual conference was part of a three-phase research project designed to explore the very long-term impact of STEM youth programs (such as the iconic YouthALIVE program). In this first phase, a virtual conference was held to engage youth program practitioners in the development and testing of a Program Profile prototype, a structured document that helps institutions to characterize their own youth programs in useful ways.
Following the webinars and the completion of the Program Profile by each organization, participants were asked to complete a brief survey (included as
The Roads Taken Conference Report provides information and results from the virtual conference held in October and November 2016.
Representatives from ten long-standing youth programs, experts in out-of-school time (OST) youth programming, and researchers participated in the Roads Taken virtual conference in October and November 2016, funded by the National Science Foundation (DRL-1644479). Participants collaboratively developed a Program Profile template with dual purposes: a tool for practitioners and a tool for researchers. As the first phase the three-part plan, Program Profiles will
As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program funds innovative resources for use in a variety of settings. This conference proposal represents the first phase of a larger three-phase participatory research project that will use communities of interest as a vehicle for solving problems of common concern about designing youth-based STEM programs. It will set the stage for research over the next 10-25 years about the long-term impact of a variety of youth programs on STEM learning and career aspirations. Through a virtual format, the Association of Science-Technology Centers will bring together two representatives from ten long-standing youth programs, experts in the field of out-of-school time youth programming, and researchers to collaboratively develop a program profile template for measuring the impact of youth programming. The program profile template will help identify specific characteristics that will capture the influence of youth programs on their participation in out-of-school STEM activities.
The program profile template will be the main outcome from the conference. It will serve as the foundation for designing long-term impact studies that support the needs of program staff interested in improving youth programming in informal environments. It will also allow program staff and researchers to document and share intellectual capital, compare goals and features across programs, and support network efforts among informal agencies worldwide. The program profile template will be shared online through informalscience.org, the Association of Science-Technology Centers' communities of practice networks, and through other out-of-school-time national organizations.
This Research in Service to Practice project will bring together representatives from six long-standing youth programs, experts in the field of out-of-school-time youth programming, and education researchers to collaboratively explore the long-term (15-25 years) impact of STEM-focused, intensive (100+ hours/year), multi-year programming. The six partnering programs have maintained records with a combined total of over 3000 alums who participated between 1995 and 2005. This four-year research project uses an explanatory, sequential, mixed-method design to carry out four steps: (1) identify and describe the impact on the lives of program alums who are now ages 30 to 45; (2) identify causal pathways from program strategies to long-term outcomes; (3) develop an understanding of these pathways from the perspective of the people who experienced them; and (4) disseminate this knowledge broadly to those associated with STEM-focused programming. Research questions include: How did these programs affect youth's lives as they progressed toward and into adulthood? What program strategies and what participant attributes contributed most to the staying power of these effects? What life events and social structures supported and inhibited participant outcomes? This project describes the effects, identifies the causal pathways, and produces materials that programs can use for both strategic planning and generating support resources. Additionally, this project provides research methodology for organizations that want to conduct their own retrospective research and lays a foundation for a more comprehensive study that includes programs without historical documentation. The project aligns with NSF's Big Idea "NSF INCLUDES: Transforming education and career pathways to help broaden participation in science and engineering" by providing essential information about the long-term effect of interventions on educational and career pathways in STEM.
The project's approach involves three phases: (1) research preparation, (2) causal structural modeling of survey data from approximately 2,000 respondents, and (3) rich qualitative follow-up. Human ecological and self-determination theories inform data collection and analyses at every project phase. In the preparation phase, program staff complete program profiles from an historic perspective by identifying program strategies that may have included, for example, scientific research, robotics development, teaching science in informal settings, and working in scientific research labs. In the quantitative phase, the project will recruit alums who attended one of the 6 youth programs between 1995 and 2005 to submit a current resume and complete an online questionnaire, based on the following scaled variables: retrospective recall of basic psychological need satisfaction and frustration in relation to perceived program strategies; STEM identity (at three time periods: pre-program; post-program; and current); current well-being; career influences; and career barriers. The questionnaire also includes open-ended questions about life events related to the following categories: family and friends, school and work, and living conditions. Analysis of the questionnaire will lead to development of a causal structural model. In the qualitative phase, data will be collected from a purposefully selected sample of 30 alums based on findings from the quantitative phase. Methods include interviews, photo journals, and STEM pathways maps. Analysis of interviews, resumes, and photo journals take place within the structure of basic psychological need satisfaction and motivational quality across ecological systems over time. Qualitative analysis uses the constant comparative method, and findings are used to update and refine the final causal structural model and inform overall findings, conclusions, and recommendations of the project.
Since the 1990s, out-of-school time programs have engaged youth from underserved communities in STEM learning and in building interest in STEM careers, yet these programs often based on untested assumptions that participation has lasting effects on education, career, and life choices related to STEM. This Research in Service to Practice project has the potential to 1) guide practitioners in program improvement and improved program outcomes; 2) provide insight into achieving program goals, such as equity, increased well-being of participants, an informed citizenry, and a diversified STEM workforce; and 3) inform multi-stakeholder decision-making with respect to this type of programming. This research also builds a foundation of research data collection and analysis methods to guide and support future research on long term-impacts and youth STEM programming. Dissemination strategies include a website, webinars, video, infographics, conference presentations, and written reports to reach stakeholders including practitioners, researchers, administrators, and funders.
This project is funded by the National Science Foundation's (NSF's) Advancing Informal STEM Learning (AISL) program, which supports innovative research, approaches, and resources for use in a variety of learning settings.
In this article, Roger Miles presents a case for the realist approach for the constructivist approach, in response to George Hein's constructivist theory.
DATE:
TEAM MEMBERS:
Roger MilesVisitor Studies Association
The New York Hall of Science proposes a two-pronged workshop project that will: (1) conduct a study of and develop a draft report on the topic of STEM badges including conceptualizations, rationale, systems, key contributors and challenges and opportunities for STEM-related badges; and (2) conduct a workshop drawn from a wide range of experts to provide critical feedback on the report. An advisory board will guide and evaluate the work. Learning increasingly takes place across a wide spectrum of institutions and contexts, through different platforms and environments, and is often incentivized by badge reward systems. There is a concomitant need to understand and make explicit the nature and criteria used, the kinds of accomplishments individuals are expected to realize, and the ways that badges are interpreted by conventional credentialing bodies, such as K-12 educational systems and institutions of higher education. The workshop creates an opportunity for a diverse group of individuals at the forefront of badges to inform each other's efforts. The report that is generated will be available to a broad audience of practitioners, developers and researchers involved in STEM education in both formal and informal sectors as well as to individuals involved in setting STEM education policy.
The Sports Legends Museum at Camden Yards approached Randi Korn & Associates, Inc. (RK&A) to conduct visitor research. This project was one of several the Museum engaged in to better understand its audience. Data were collected between May and July 2008 through 319 standardized questionnaires and 34 in-depth interviews with groups of visitors. While a "sports museum" is a focused type of museum, one major consideration was whether sports museum visitors are homogenous, prompting us to ask, what types of sports fan does the Sports Legends Museum at Camden Yards attract? To answer this, RK&A
DATE:
TEAM MEMBERS:
Randi Korn & Associates, Inc.Sports Legends Museum at Camden Yards
In this article, Harris Shettel, an evaluation consultant, discusses the politics of evaluation and how the recent use of visitor data to inform decision making at all levels qualifies as a paradigm shift in the approach taken to museum public programming.
DATE:
TEAM MEMBERS:
Harris ShettelVisitor Studies Association
Describes an instrument developed to quantitatively evaluate the participation levels of students on outdoor education activities. From photographs taken at two minute intervals, students are assigned values for their observed behaviors. The correlations of values assigned by several judges are shown.
A multifaceted evaluation study was conducted of the 2,000-square-foot exhibit "Darkened Waters: Profile of an Oil Spill" between September 14 and October 12, 1991, at the Oakland Museum, Natural Sciences Department. "Darkened Waters" is the second generation of an exhibit originally conceived and built by the Pratt Museum in Homer, Alaska, to provide information and a forum for the environmental impact of the 1989 Exxon Valdez oil spill. The exhibit will travel to the Smithsonian Institution in December 1991, and will be available to other institutions through the Association of Science
The aim of the work reported here has been to give an overview of the support that the informal sector provides for learning and engagement with science. In addressing this goal, we have taken the view that engagement with science and the learning of science occur both within and without schools. What is of interest is not who provides the experience or where it is provided but the nature and diversity of opportunities for science learning and engagement that are offered in contemporary UK society. Thus in approaching the work we have taken a systems perspective and looked at informal