Inverness Research and Oregon State University, with support and input from CAISE, conducted an evaluation of the 2021 NSF AISL Awardee Meeting which was held virtually October 19-21, 2021. The evaluation effort included observing the meeting, participating in debriefing the meeting with CAISE co-PIs, the CAISE equity audit committee, and NSF Program Officers; developing and administering a post-event survey; and analyzing data collected through both the survey and Pathable, the virtual platform.
This report summarizes the key evaluation findings. It includes the following sections:
The National Science Teachers Association will convene a conference that will bring together STEM researchers and practitioners to review the growing connected science learning movement. A connected science learning environment has been described as a robust science ecology containing a wide variety of programs, across a range of institutions and places, allowing youth different and multiple ways to engage with STEM. Such environments can include small partnerships, such as a science museum and K-12 schools, or a large, community-wide network of a variety of organizations such as K-12 schools, museums, universities, government agencies, and community organizations. The conference will bring together over forty participants, who will meet in a series of several online meetings. The conference will result in a series of papers, articles in the online Connected Science Learning journal and other publications, a series of webinars and online forums where participants can engage with themes identified in the conference, and conference presentations at the annual meetings of organizations including the National Science Teachers Association, the Association of Science-Technology Centers, and others. This award is funded by the Advanced Informal STEM Learning program which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments.
The conference will: (1) document the research foundation that supports and demonstrates the impact and value of high-quality connected science learning experiences; (2) identify areas for which future research is recommended; and (3) provide effective, practitioner-focused resources that advance connected STEM learning. The conference will include participants that represent a wide range of researchers and practitioners in informal and formal STEM education, as well as representing gender, racial/ethnic and geographical diversity. The results and products of the conference will be instrumental in developing the understanding and appreciation for connecting STEM learning and ultimately improving connected STEM learning for K-12 youth. The importance of emphasizing diversity, equity, and accessibility will be strongly represented in the key evidence identified through the conference and will be reflected in the resources that will be disseminated to a broader audience.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
In 2017, the Education Development Center (EDC) received a grant from the National Science Foundation (NSF) to bring together PIs of STEM Program Resource Centers (PRC) funded under NSF’s Education and Human Resources (EHR), evaluators of NSF STEM projects and programs, and evaluation advisors to address concerns about the quality and consistency of STEM evaluations. According to the grant proposal, the goal was “to increase the capacity of evaluators to produce high quality, conceptually sound, methodologically appropriate evaluations of NSF programs and projects, specifically in the area of
The American Association for the Advancement of Science (AAAS) is creating, implementing and evaluating a forum for the NSF INCLUDES broadening participation community of practice and for engaging the NSF INCLUDES awardees and science, technology, engineering and mathematics (STEM) researchers across the nation to expand the NSF INCLUDES broadening participation network. The NSF INCLUDES program is a comprehensive national initiative designed to enhance U.S. leadership in STEM discoveries and innovations focused on NSF's commitment to diversity, inclusion, and broadening participation in these fields.
The NSF INCLUDES Open Forum will use the AAAS Trellis networking platform and the organization's experience engaging communities of practice focused on broadening participation, STEM education and STEM research. The project builds on the success of a prior NSF INCLUDES Conference award (HRD-1650509) that was addressing goals to define networking needs of the first round of NSF INCLUDES Design and Development Launch Pilots (DDLP); to develop design specifications for NSF INCLUDES networking, curating of resources, and supporting communities of practice; and to propose tools, techniques, capacities and functionalities for an NSF INCLUDES national network.
The NSF INCLUDES Open Forum project includes advisory board members with expertise in networking platforms and others with broadening participation knowledge and experience. A yearly conference for NSF INCLUDES awardees will offer participants an opportunity to learn about how Trellis platform upgrades, functionality and technology options (e.g., a smartphone application) can be used in new ways to engage a broader community of partners interested in broadening participation in STEM research and education contexts. An external evaluator will assess the activities and outcomes of the NSF INCLUDES Open Forum both during implementation and at project end. The PIs will also communicate the outcomes of the project to broader audiences, both academic and non-academic, and encourage a dialogue within the NSF INCLUDES community about the use of technology for organization and communication within a network.
DATE:
-
TEAM MEMBERS:
Shirley MalcomJosh Freeman
resourceresearchProfessional Development, Conferences, and Networks
Each year, the National Alliance for Broader Impacts (NABI) seeks to understand the current state of broader impacts (BI) in the national context. In 2017, NABI convened two forums to identify needs and solicit recommendations. The first event was a 90-minute town hall session with 120 participants facilitated by Jamie Bell at the April 2017 BI Summit. The second event was a two-day convening facilitated by Goose Creek Consulting at NSF headquarters in May 2017 of stakeholder groups including university administrators (e.g., provosts, associate provosts, vice-presidents of research)
This report presents findings from a formative evaluation of Science Education for New Civic Engagements and Responsibilities-Informal Science Education (SENCER-ISE), a National Science Foundation and Noyce Foundation funded initiative to support partnerships between informal science and higher education institutions. This evaluation looked primarily at the collaborative infrastructure of SENCER-ISE, which included the web site, SENCER Summer Institute, and communications with project staff and/or the advisory board. This evaluation is the third evaluation that Randi Korn & Associates, Inc
DATE:
TEAM MEMBERS:
RK&A, Inc.
resourceevaluationProfessional Development, Conferences, and Networks
The National Center for Science and Civic Engagement (NCSCE) contracted Randi Korn & Associates, Inc. (RK&A) to conduct a summative evaluation of its SENCER-ISE project partnerships. SENCER-ISE is an initiative that brings partners from higher education (HE) together with partners from informal science education (ISE) to create projects that engage audiences in science using the lens of civic engagement. SENCER funded 10 partnerships over three years—six through the National Science Foundation (DRL #1001795) and four through the Noyce Foundation. Previously, RK&A conducted a formative
DATE:
TEAM MEMBERS:
RK&A, Inc.William Burns
resourceprojectProfessional Development, Conferences, and Networks
The National Science Foundation's (NSF) Inclusion across the Nation of Communities of Learners of Underrepresented Discoverers in Engineering and Science (NSF INCLUDES) intiative supports models, networks, partnerships and research to ensure the broadening participation in STEM of women, members of racial and ethnic groups that have been historically underrepresented, persons of low socio-economic status, and people with disabilities.
The University of Cincinnati, lead for a tri-state (OH, KY, IN) project, will convene a three-day conference to convene national and local experts to explore the best practices that support the development of a backbone organization in the context of using a social innovation model for broadening participation in STEM. The intent is to strengthen the network among participants and leverage learning from the Cincinnati Strive experience with collective impact across the Midwest and beyond.
Results from the NextLivesHere: Social Change Innovation Summit, will be disseminated in the tri-state region through the Greater Cincinnati STEM Collaborative (GCSC and the Ohio STEM Learning Network (OSLN). National dissemination will occur through informal and formal STEM professional organizations and publications as well as through participation in the NSF-developed national backbone organization.
DATE:
-
TEAM MEMBERS:
Kathie MaynardRoss MeyerShiloh TurnerGeoffrey ZimmermanGisela Escoe
resourceprojectProfessional Development, Conferences, and Networks
The NSF INCLUDES program supports models, networks, partnerships and research to ensure the broadening participation in STEM of women, members of racial and ethnic groups that have been historically underrepresented, persons of low socio-economic status, and people with disabilities.
The University of California-Irvine (UCI), in partnership with the University of California-San Diego and the University of California-Davis will convene a state-wide conference on inclusion in science, technology, engineering and mathematics (STEM) higher education. The California STEM INCLUDES Conference and Network will share best practices for promoting STEM inclusion and provide an infrastructure to further these practices and track the outcomes. The purpose of the conference is to form a backbone for a large regional network in support of the National Science Foundation's Inclusion across the Nation of Communities of Learners of Underrepresented Discoverers in Engineering and Science (NSF INCLUDES) initiative.
The five goals of the conference are to:
--increase California educator awareness of demographic gaps in STEM participation and the research on factors influencing such participation gaps;
--engage participants in discussion of promising practices for increasing STEM inclusion
--broaden the impact of existing successful programs for STEM inclusion, through program modification and replication, scale-up, and increased collaboration;
--create a mechanism for sustained discussion, sharing and collaboration around STEM inclusion across California institutions;
--create central repository and common standards for reporting on STEM inclusion and implementation program impact for the state.
Approximately 340 individuals serving a broad cross-section of California's K-12, higher education, public, private and non-profit constituencies will participate in 2 ½ days of intense dialogue on topics such as data and research, successful implementations and sustainable networks for collaboration and sharing. The conference will be held in Spring 2017.
DATE:
-
TEAM MEMBERS:
Michael DenninSarah Eichhorn
resourceprojectProfessional Development, Conferences, and Networks
SRI International and the Carnegie Foundation for Advancement of Teaching will jointly lead a workshop with the dual goals of supporting INCLUDES (Inclusion across the Nation of Communities of Learners that have been Underrepresented for Diversity in Engineering and Science) Launch Pilots during their first year and contributing to plans for a National Backbone. The workshop will strengthen and deepen the potential for rapid, continuous improvement cycles within and across INCLUDES networks. To do so, they will combine the expertise of SRI and Carnegie in improvement science, rapid analytics, and fostering networks of researchers and practitioners to engage participants in conducting a complete improvement cycle within and across INCLUDES Launch Pilots.
The workshop will have three phases. A first phase, conducted online, will share expertise related to each of the four parts of a complete improvement cycle: (1) problem definition, (2) data collection, (3) formative evaluation, and (4) effective communication. This first phase will combine an initial presentation with facilitated, online opportunities to interactively engage in the topics. A second phase, conducted face-to-face, will work intensively with teams from INCLUDES networks to improve their operational and long-term plans. The third phase will reflect and report on the workshop and contribute plans to build capacities for the National Backbone organization. Through the combination of these three phases, they will support the first-year work of INCLUDES teams and also refine understanding of how a national network could combine online and face-to-face elements to advance INCLUDES efforts. The workshop team will create and disseminate resources that are immediately useful to INCLUDES and related projects, and the workshop will openly coordinate with other workshops to achieve synergies. The online offerings will be open, broadly advertised, and permanently available. The lessons learned regarding plans for a national backbone will be disseminated broadly. In addition to participants from INCLUDES networks, additional stakeholders will be invited to both phases so as to shape the future plans to achieve broader impacts aligned with overall INCLUDES goals.
DATE:
-
TEAM MEMBERS:
Timothy Podkul
resourceprojectProfessional Development, Conferences, and Networks
Jobs are growing most rapidly in areas that require STEM knowledge, causing business leaders to seek skilled American workers now and in the near future. Increase in the number of students pursuing engineering degrees is taking place but the percentages of underrepresented students in the engineering pipeline remains low. To address the challenge of increasing the participation of underrepresented groups in engineering, the National Society of Black Engineers, the American Indian Science and Engineering Society, the Society of Hispanic Professional Engineers, and the Society of Women Engineers have formed the 50K Coalition, a collaborative of over 40 organizations committed to increasing the number of bachelors degrees awarded to women and minorities from 30,000 annually to 50,000 by 2025, a 66% increase. The 50K Coalition is using the Collective Impact framework to develop an evidence-based approach that drives management decision-making, improvements, sharing of information, and collective action to achieve success. The first convening of the 50K Coalition in April, 2016, brought together 83 leaders of the engineering community representing 13 professional societies with over 700,000 members, deans of engineering, minority engineering and women in engineering administrators from 11 leading colleges of engineering, and corporate partners representing six global industries. Consensus was reached on the following Common Agenda items: 1.) Undergraduate support and retention; 2.) Public awareness and marketing; 3.) K-12 support; 4.) Community College linkages; 5.) Culture and climate. The Coalition will encourage member organizations to develop new programs and scale existing programs to reach the goal.
The Coalition will use shared metrics to track progress: AP® Calculus completion and high school graduation rates; undergraduate freshmen retention rates; community college transfer rates and number of engineering degrees awarded. The 50K Coalition will develop the other elements of the Collective Impact framework: Infrastructure and effective decision-making processes that will become the backbone organization with a focus on data management, communications and dissemination; a system of continuous communication including Basecamp, website, the annual Engineering Scorecard, WebEx hosted meetings and convenings; and mutually reinforcing activities such as programs, courses, seminars, webinars, workshops, promotional campaigns, policy initiatives, and institutional capacity building efforts. The National Academy of Sciences study, Expanding Underrepresented Minority Participation: America's Science and Technology Talent at the Crossroads recommended that professional associations make recruitment and retention of underrepresented groups an organizational goal and implement programs designed to reach that goal by working with their membership, academic institutions and funding agencies on new initiatives. While these types of organizations work together now in a variety of ways, the relationships are one-on-one. The 50K Coalition brings together, for the first time professional societies, engineering schools, and industry to consider what mutually reinforcing activities can most effectively encourage students from underrepresented groups to complete calculus and graduate from 4-year engineering programs.
DATE:
-
TEAM MEMBERS:
Karl ReidBarry CorderoSarah EcohawkKaren Horting
The Coalition for Science After School was launched January 28, 2004 at the Santa Fe Institute, home to the world’s leading researchers on the study of complexity. Against the dazzling backdrop of the New Mexican mesa, 40 educational leaders from diverse but overlapping domains—science, technology, engineering and mathematics education and after-school programs—met to grapple with three emerging, important trends in youth development and science learning in this country: 1. An explosion in the number of U.S. youth attending after-school programs, and increasing links between school and after
DATE:
TEAM MEMBERS:
The Coalition for Science After SchoolLeah Reisman