This project aims to advance racial equity in outdoor and environmental science education (OESE) by co-developing, implementing, and studying a replicable model for organizational capacity building and transformation.
DATE:
-
TEAM MEMBERS:
Jedda ForemanCraig StrangValeria Romero
African American and Latinx youth are often socialized towards athletic activity and sports participation, sometimes at the expense of their exploration of the range of potential career paths including those in the science, technology, engineering, and mathematics (STEM) fields. This project will immerse middle school youth in the rapidly growing world of sports data analytics and build their knowledge of statistics concepts and the data science process. The project will focus on the STEM interests and knowledge development of African American and Latinx youth, an underrepresented and underserved group in STEM. Researchers will explore the ways youths' social identities can and should serve as bridges towards future productive academic and professional identities including those associated with STEM learning and the STEM professions. The outcomes of the project will advance knowledge in promoting elements of informal learning experiences that build adolescents' motivation and persistence for productive participation in STEM courses and careers. This project is funded by the Advancing Informal STEM Learning program (AISL), which seeks to advance new approaches to and evidence-based understanding of the design and development of STEM learning opportunities for the public in informal environments, and the Innovative Technology Experiences for Students and Teachers program (ITEST), which funds projects that leverage innovative uses of technologies to prepare diverse youth for the STEM workforce, with a focus on broadening participation among underrepresented and underserved groups in STEM fields.
Over a three-year period, 250 middle school learners in the West Baltimore, Maryland and Hyattsville, Maryland areas will engage in three main learning activities: Summer Camp (three weeks), Sports Day Saturdays, and a Spring Summit. Through a partnership between the University of Maryland and Coppin State University, the project will utilize resources in multiple departments and units across both universities, and engage with youth sports leagues such as the American Athletic Union (AAU) to support participants' engagement in the data science process including collection of raw data, exploration of data, development of models, visualization, communication, and reporting of data, and data-driven decision making. Furthermore, youth participants will attend local AAU, college, and professional sporting events, and interact with members of coaching staffs to better understand the ways performance data technologies are utilized to inform recruitment and team performance. The mixed-methods research agenda for this project is guided by three main questions: (1) What elements of the project's model are most successful at supporting congruence of adolescents' academic identity, including STEM identity and social identity including athletic identity? (2) What elements support adolescents' motivation, and persistence for productive participation in current and future STEM courses? (3) To what extent did the project appear to influence participants' perceptions of their future professions? At multiple points throughout the experience, participants will complete surveys designed to document and assess statistics and data science knowledge; interest in STEM careers; academic, social and athletic identity development; and STEM course taking patterns. Researchers will also observe project activities, interview a focal group of participants, and survey participants' parents to identify elements of learning experiences that encourage and support adolescents' interest in STEM disciplines and STEM professions. The project team will develop conceptual and pedagogical frameworks that support middle school youth' engagement and interest in science, engineering, technology, and mathematics through repurposing spaces where these youths frequent. A major outcome of the project will be workforce preparation and offers a promising approach for encouraging youth to persist along STEM pathways, which may ultimately result in broadened participation in STEM workforces.
This project is funded by the National Science Foundation's (NSF's) Advancing Informal STEM Learning (AISL) program, which supports innovative research, approaches, and resources for use in a variety of learning settings.
There is a dearth of prominent STEM role models for underrepresented populations. For example, according to a 2017 survey, only 3.1% of physicists in the United States are Black, only 2.1% are Hispanic, and only 0.5% are Native American. The project will help bridge these gaps by developing exhibits that include simulations of historical scientific experiments enacted by little-known scientists of color, virtual reality encounters that immerse participants in the scientists' discovery process, and other content that allows visitors to interact with the exhibits and explore the exhibits' themes. The project will develop transportable, interactive exhibits focusing on light: how we perceive light, sources of light from light bulbs to stars, uses of real and artificial light in human endeavors, and past and current STEM innovators whose work helps us understand, create, and harness light now. The exhibits will be developed in three stages, each exploring a characteristic of light (Color, Energy, or Time). Each theme will be explored via multiple deliveries: short documentary and animated films, virtual reality experiences, interactive "photobooths," and technology-based inquiry activities. The exhibit components will be copied at seven additional sites, which will host the exhibits for their audiences, and the project's digital assets will enable other STEM learning organizations to duplicate the exhibits. The exhibits will be designed to address common gaps in understanding, among adults as well as younger learners, about light. What light really is and does, in scientific terms, is one type of hidden story these exhibits will convey to general audiences. Two other types of science stories the exhibits will tell: how contemporary research related to light, particularly in astrophysics, is unveiling the hidden stories of our universe; and hidden stories of STEM innovators, past and present, women and men, from diverse backgrounds. These stories will provide needed role models for the adolescent learners, helping them learn complex STEM content while showing them how scientific research is conducted and the diverse community of people who can contribute to STEM innovations and discoveries.
The project deliverables will be designed to present complex physics content through coherent, immersive, and embodied learning experiences that have been demonstrated to promote engagement and deeper learning. The project will research whether participants, through interacting with these exhibits, can begin to integrate discrete ideas and make connections with complex scientific content that would be difficult without technology support. For example, students and other novices often lack the expertise necessary to make distinctions between what is needed and what is extra within scientific problems. The proposed study follows a Design-Based Research (DBR) approach characterized by iterative cycles of data collection, analysis, and reflection to inform the design of educational innovations and advance educational theory. Project research includes conceiving, building, and testing iterative phases, which will enable the project to capture the complexity of learning and engagement in informal learning settings. Research participants will complete a range of research activities, including focus group interviews, observation, and pre-post assessment of science content knowledge and dispositions.
By showcasing such role models and informing about related STEM content, this project will widen perspectives of audiences in informal learning settings, particularly adolescents from groups underrepresented in STEM fields. Research findings and methodologies will be shared widely in the informal STEM learning community, building the field's knowledge of effective ways to broaden participation in informal science learning, and thus increase broaden participation in and preparation for the STEM-based workforce.
This project is funded by the National Science Foundation's (NSF's) Advancing Informal STEM Learning (AISL) program, which supports innovative research, approaches, and resources for use in a variety of learning settings.
DATE:
-
TEAM MEMBERS:
Todd BoyetteJill HammJanice AndersonCrystal Harden
The Space and Earth Informal STEM Education (SEISE) project, led by the Arizona State University with partners Science Museum of Minnesota, Museum of Science, Boston, and the University of California Berkeley’s Lawrence Hall of Science and Space Sciences Laboratory, is raising the capacity of museums and informal science educators to engage the public in Heliophysics, Earth Science, Planetary Science, and Astrophysics, and their social dimensions through the National Informal STEM Education Network (NISE Net). SEISE will also partner on a network-to-network basis with other existing coalitions and professional associations dedicated to informal and lifelong STEM learning, including the Afterschool Alliance, National Girls Collaborative Project, NASA Museum Alliance, STAR_Net, and members of the Association of Children’s Museums and Association of Science-Technology Centers. The goals for this project include engaging multiple and diverse public audiences in STEM, improving the knowledge and skills of informal educators, and encouraging local partnerships.
In collaboration with the NASA Science Mission Directorate (SMD), SEISE is leveraging NASA subject matter experts (SMEs), SMD assets and data, and existing educational products and online portals to create compelling learning experiences that will be widely use to share the story, science, and adventure of NASA’s scientific explorations of planet Earth, our solar system, and the universe beyond. Collaborative goals include enabling STEM education, improving U.S. scientific literacy, advancing national educational goals, and leveraging science activities through partnerships. Efforts will focus on providing opportunities for learners explore and build skills in the core science and engineering content, skills, and processes related to Earth and space sciences. SEISE is creating hands-on activity toolkits (250-350 toolkits per year over four years), small footprint exhibitions (50 identical copies), and professional development opportunities (including online workshops).
Evaluation for the project will include front-end and formative data to inform the development of products and help with project decision gates, as well as summative data that will allow stakeholders to understand the project’s reach and outcomes.
As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program funds innovative research, approaches, and resources for use in a variety of settings. The proposed project broadens the utility of Public Participation in Scientific Research (PPSR) approaches, which include citizen science, to support new angles in informal learning. It also extends previous work on interactive data visualizations in museums to encompass an element of active contribution to scientific data. To achieve these goals, this project will develop and research U!Scientist (pronounced `You, Scientist!')--a novel approach to using citizen science and learning research-based technology to engage museum visitors in learning about the process of science, shaping attitudes towards science, and science identity development. Through the U!Scientist multi-touch tabletop exhibit, visitors will: (1) interact with scientific data, (2) provide interpretations of data for direct use by scientists, (3) make statements based on evidence, and (4) visualize how their data classifications contribute to globe-spanning research projects. Visitors will also get to experience the process of science, gaining efficacy and confidence through these carefully designed interactions. This project brings together Zooniverse, experts in interactive design and learning based on large data visualizations in museums, and leaders in visitor experience and learning in science museums. Over fifty thousand museum visitors are expected to interact annually with U!Scientist through this effort. This impact will be multiplied by packaging the open-source platform so that others can easily instantiate U!Scientist at their institution.
The U!Scientist exhibit development process will follow rapid iterations of design, implementation, and revision driven by evaluation of experiences with museum visitors. It will involve close collaboration between specialists in computer science, human-computer interaction and educational design, informal science learning experts, and museum practitioners. The summative evaluation will be based on shadowing observations, U!Scientist and Zooniverse.org logfiles (i.e., automated collection of user behavior metrics), and surveys. Three key questions will be addressed through this effort: Q1) Will visitors participate in PPSR activities (via the U!Scientist touch table exhibit) on the museum floor, despite all the distractions and other learning opportunities competing for their attention? If so, who engages, for how long, and in what group configurations? Q2) If visitors do participate, will they re-engage with the content after the museum visit (i.e., continue on to Zooniverse.org)? Q3) Does engaging in PPSR via the touch table exhibit--with or without continued engagement in Zooniverse.org after the museum visit--lead to learning gains, improved understanding of the nature of science, improved attitudes towards science, and/or science identity development?
As the world is increasingly dependent upon computing and computational processes associated with data analysis, it is essential to gain a better understanding of the visualization technologies that are used to make meaning of massive scientific data. It is also essential that the infrastructure, the very means by which technologies are developed for improving the public's engagement in science itself, be better understood. Thus, this AISL Innovations in Development project will address the critical need for the public to learn how to interpret and understand highly complex and visualized scientific data. The project will design, develop and study a new technology platform, xMacroscope, as a learning tool that will allow visitors at the Science Museum of Minnesota and the Center of Science and Industry, to create, view, understand, and interact with different data sets using diverse visualization types. The xMacroscope will support rapid research prototyping of public experiences at selected exhibits, such as collecting data on a runner's speed and height and the visualized representation of such data. The xMacroscope will provide research opportunities for exhibit designers, education researchers, and learning scientists to study diverse audiences at science centers in order to understand how learning about data through the xMacroscope tool may inform definitions of data literacy. The research will advance the state of the art in visualization technology, which will have broad implications for teaching and learning of scientific data in both informal and formal learning environments. The project will lead to better understanding by science centers on how to present data to the public more effectively through visualizations that are based upon massive amounts of data. Technology results and research findings will be disseminated broadly through professional publications and presentations at science, education, and technology conferences. The project is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants. The project is driven by the assumption that in the digital information age, being able to create and interpret data visualizations is an important literacy for the public. The research will seek to define, measure, and advance data visualization literacy. The project will engage the public in using the xMacrocope at the Science Museum of Minnesota and at the Center of Science and Industry's (COSI) science museum and research center in Columbus, Ohio. In both museum settings the public will interact with different datasets and diverse types of visualizations. Using the xMacroscope platform, personal attributes and capabilities will be measured and personalized data visualizations will be constructed. Existing theories of learning (constructivist and constructionist) will be extended to capture the learning and use of data visualization literacy. In addition, the project team will conduct a meta-review related to different types of literacy and will produce a definition with performance measures to assess data visualization literacy - currently broadly defined in the project as the ability to read, understand, and create data visualizations. The research has potential for significant impact in the field of science and technology education and education research on visual learning. It will further our understanding of the nature of data visualization literacy learning and define opportunities for visualizing data in ways that are both personally and culturally meaningful. The project expects to advance the understanding of the role of personalization in the learning process using iterative design-based research methodologies to advance both theory and practice in informal learning settings. An iterative design process will be applied for addressing the research questions by correlating visualizations to individual actions and contributions, exploring meaning-making studies of visualization construction, and testing the xMacroscope under various conditions of crowdedness and busyness in a museum context. The evaluation plan is based upon a logic model and the evaluation will iteratively inform the direction, process, and productivity of the project.
The Maker movement has grown considerably over the past decade, both in the USA and internationally. Several varieties of Making have been developed, but there are still many important questions to ask and research to conduct about how different programmatic structures may relate to the potential impact Maker programs can have on individuals and communities. WestEd, in collaboration with the Lucile Packard Children's Hospital, the University of Michigan C. S. Mott Hospital Children's Hospital, and the Children's Hospital of Orange County, is conducting a year-long exploratory research study that will focus on the out-of-school learning by adolescents and young adults in children's hospitals. This research study will focus on mobile and dedicated Makerspaces in hospitals to support patients' learning. The application of Makerspaces to hospital environments is a unique opportunity to research a critical need of chronically ill individuals, i.e. to explore how Making can enhance patients' agency, creative STEM learning, and physical well-being. The proposed study is building on the prior work of the principal investigator and will: (1) examine the nature and processes of learning in children's hospitals; (2) revise the current design of the mobile Makerspace and the associated implementation model in response to variations in programmatic contexts across multiple hospital settings and disparate patients' conditions; and (3) investigate and test the effectiveness of the Makerspace approach as it relates to both patients' learning and health outcomes. The study would contribute to longer-term efforts to develop a comprehensive, scalable, and sustainable strategy to determine the programmatic viability of the mobile Makerspace approach across a more varied array of hospital settings. This project has the potential to have a much broader impact by reaching out to other isolated students beyond the hospital environment, including those in residential treatment facilities for behavioral and emotional problems, as well as those attending programs designed to help youth who have been in trouble with the law get back on track. This project is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants.
This project's goals are to contribute to the understanding of how to: (1) describe and measure the education and health impact of mobile Makerspaces on chronically ill patients, and (2) design and sustain implementation models in various hospital settings. Since a children's hospital is a challenging context to support a patient's learning, it is not typically conducive to learning. Patients are constantly interrupted by the demands of the illness, by the strict protocols that need to be adhered to, and by the medical staff who manage their exhaustive treatment regimens. The mobile Makerspace is intended to adjust the environment in deliberate ways, allowing researchers to study and observe what kinds of learning intervention models enable youth and young adults to recapture a sense of their own agency and enable them to see themselves as creators, and makers of things that improve their own and others' lives. The project will have two strands: one on learning and one on adaptation of the model. In the learning strand, the study will investigate how engaging with the Makerspace can enhance patients' learning by provoking their sense of curiosity, encouraging them to set up and pursue personal goals via invention, and inspiring them to feel more agentive in taking charge of their learning process i.e., development of affinity for and fluency in the ways of knowing, doing and being (the epistemologies and ontologies) of engineers or scientists. In the adaptation strand, they will identify challenges and opportunities for implementing Makerspaces and develop an implementation plan that provides a process for introducing Makerspaces into hospital settings.
The project will advance efforts by the American Association for the Advancement of Science and the Institute for Learning Innovation to bring together young adults from communities historically underrepresented in science, technology, engineering, and mathematics (STEM) to collaboratively conduct scientifically driven challenges embedded in a mobile learning tool based upon the AAAS Active Explorer platform. The project will be conducted at the Washington National Mall, San Francisco National Golden Gate Park, and the Boston Harbor Islands National Recreation Area, and will study how a mobile technology used in these settings can facilitate learner engagement in science content; how it can affect young adults' engagement in science-learning processes; and whether interest in learning science and technology has been furthered. The project is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments, including pathways for broadening access to STEM learning experiences and advancing research STEM learning. Research questions will investigate science learning inequalities by addressing how place-based augmented reality games can connect young adults to scientific practices, including observing science phenomena, analyzing data, and communicating findings; how young adults develop science skills related to their science self-efficacy through participation in augmented reality science exploration; and how mobile technologies and gaming can serve as mediators that enable young adults to improve their science identity. In addition to engaging young adults in science activities at the National Parks and increasing their science skills, the project will provide valuable information to National Park staff and scientists to assist them in designing effective tools, resources and experiences to better engage young adults. Research teams will collect data in the form of digital ethnography, focus groups, activity reports, artifacts, and surveys. The project will document learning and engagement through mobile technology in three urban national parks that will involve 60 young adults at each location, and will create innovative measurement tools to monitor how informal settings can leverage the intersections of the arts and sciences to support student engagement and learning.
This ChangeMakers project builds on a 2016 National Academies report finding that scientific literacy can be understood at a community level as opposed to a traditional focus on the individual. This is important since scientific knowledge is often seen as abstract and distant from the daily concerns of average citizens. A community focus shifts the spotlight away from individual learning to collective learning facilitated by trusted cultural institutions serving as social assets. This work brings together scientific expertise and community organizations to advance operational science literacy--scientific ways of problem-solving--for community leaders and functional science literacy--information and skills people can use in their daily lives--among their service populations. This will be done by gathering and sharing knowledge and developing skills and abilities to contribute to the community's overall well-being.
The New England Aquarium (NeAq) and Aquarium of the Pacific (AoP) will apply a community engagement model involving active listening, documentation, alignment of concerns and goals, and co-development of shared solutions that serves the needs of all participants. As part of the Advancing Community Science Literacy (ACSL) project, multi-disciplinary teams from NeAq, AoP and their regional partners will participate in training on the model. They will apply that training to build and implement action plans to advance community-driven responses to local environmental issues. Teams will be assessed with respect to how they use tools from their shared training, along with peer support and coaching, to make progress in engaging diverse community stakeholders. Results of the evaluation will offer insights and recommendations for informal science learning centers to serve their communities more effectively as engagement facilitators and change agents to support science literacy development and action. By applying techniques developed for cultural institutions to communicate about climate science, and combining those with techniques developed for libraries and other organizations to help meet emergent community concerns, such as storm surges and coastal flooding, it is possible to redefine the role informal science learning centers can play as part of a community culture.
ACSL is funded by the Advancing Informal STEM Learning (AISL) program which supports projects that provide multiple pathways for broadening access to and engagement in STEM learning experiences, advances innovative research on and assessment of STEM learning in informal environments, and develops understandings of deeper learning by participants.
Public Participation in Scientific Research (PPSR), often referred to as crowdsourcing or citizen science, engages participants in authentic research, which both advances science discovery as well as increases the potential for participants' understanding and use of science in their lives and careers. This four year research project examines youth participation in PPSR projects that are facilitated by Natural History Museums (NHMs). NHMs, like PPSR, have a dual focus on scientific research and science, technology, engineering, and mathematics (STEM) education. The NHMs in this project have established in-person and online PPSR programs and have close ties with local urban community-based organizations. Together, these traits make NHMs appropriate informal learning settings to study how young people participate in PPSR and what they learn. This study focuses on three types of PPSR experiences: short-term outdoor events like bioblitzes, long-term outdoor environmental monitoring projects, and online PPSR projects such as crowdsourcing the ID of field observations. The findings of this study will be shared through PPSR networks as well as throughout the field in informal STEM learning in order to strength youth programming in STEM, such that youth are empowered to engage in STEM research and activities in their communities. This project is funded through Science Learning+, which is an international partnership between the National Science Foundation (NSF) and the Wellcome Trust with the UK Economic and Social Research Council. The goal of this joint funding effort is to make transformational steps toward improving the knowledge base and practices of informal STEM experiences. Within NSF, Science Learning+ is part of the Advancing Informal STEM Learning (AISL) program that seeks to enhance learning in informal environments and to broaden access to and engagement in STEM learning experiences.
The study employs observations, surveys, interviews, and learning analytics to explore three overarching questions about youth learning: 1) What is the nature of the learning environments and what activities do youth engage in when participating in NHM-led PPSR? 2) To what extent do youth develop three science learning outcomes, through participation in NHM-led citizen science programs? The three are: a) An understanding of the science content, b) identification of roles for themselves in the practice of science, and c) a sense of agency for taking actions using science? 3) What program features and settings in NHM-led PPSR foster these three science learning outcomes among youth? Based on studies occurring at multiple NHMs in the US and the UK, the broader impact of this study includes providing research-based recommendations for NHM practitioners that will help make PPSR projects and learning science more accessible and productive for youth. This project is collaboration between education researchers at University of California, Davis and Open University (UK), and Oxford University (UK) and citizen science practitioners, educators, and environmental scientists at three NHMs in the US and UK: NHM London, California Academy of Sciences, and NHM Los Angeles.
The widespread accessibility of live streaming video now makes it possible for viewers around the world to watch live events together, including unprecedented, 24/7 views of wildlife. In addition, online technologies such as live chatting and forums have opened new possibilities for people to collaborate from locations around the world. The innovation that the projects provide is bringing these opportunities together, enabling real-time research and discussion as participants observe and annotate live streaming footage; sharing questions and insights through live Q&A sessions; and explore data with interactive visualization tools. Scientists will support the community's research interests, in contrast with traditional models of citizen science in which communities support the work of scientists. This project will enable people from diverse backgrounds and perspectives to co-create scientific investigations, including participants who might not otherwise have access to nature. The evaluation research for this project will advance the understanding of practices that enable interconnected communities of people to participate in more phases of scientific discovery, and how participation affects their learning outcomes. It is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of science, technology, engineering, and mathematics (STEM) learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants. As such, this project will advance a new genre of Public Participation in STEM Research (PPSR). It will also advance scientific exploration using live wildlife cams and establish a database for long-term research to understand how bird behavior and reproductive success are affected by environmental change. This project aims to deepen public involvement in science, building on knowledge and relevance for STEM learning by creating an online learning environment that expands on traditional crowdsourcing models of PPSR in which participants collect data to answer questions driven by scientists. In this project, participants are involved in co-created research investigations, including asking questions, deciding what data are needed, generating data, looking for patterns, making interpretations, reviewing results, and sharing findings. The goals are to 1) create a system that involves the public more deeply in scientific research; 2) develop participants' science skills and interests; 3) increase participants' understanding of birds and the environment; 4) generate new scientific knowledge about wildlife; and 5) advance the understanding of effective project design for co-created PPSR projects at a national scale. Through iterative design and evaluation, the project will advance the understanding of the conditions that foster online collaboration and establish design principles for supporting science and discovery in online learning environments. Through scaling and quasi-experimental studies, the evaluation research will advance the understanding of how learning outcomes may be similar or different for participants engaging in different ways, whether they observe the cams and read about the investigation, process data as contributors, provide some input as collaborators, or join in most or all of the scientific process as co-creators. Despite the popularity of live wildlife cams, with millions of people watching hundreds of cams around the world, little research has been conducted on the use of live cams for collaborative work in formal or informal science education. The infrastructure and open-source framework created for this project will expand the capacity for online communities of people from diverse career backgrounds and perspectives to collaborative on solving personally meaningful questions and contribute to new knowledge. Using this project as a prototype, cam operators from around the world could build networks of cams, enabling future studies with broader scope for comparative biological studies and discoveries. Additionally, it will serve as a model for use in classrooms or for online communities exploring other scientific fields using live-streaming content in collaborative research. By involving scientists and participants from across society as collaborators and co-creators, this project can help increase public engagement with science, technology, and environmental stewardship while advancing the understanding of the natural world and informing public decision-making.
As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program funds innovative research, approaches and resources for use in a variety of settings. The project will collaboratively design, test and study effective and efficient ways to develop embedded assessments (EAs) of citizen science (CS) volunteer scientific inquiry skills in order to better understand the impact of these CS experiences on volunteer scientific inquiry abilities. EAs are assessment activities that are integrated into the learning experience and allow learners to demonstrate their competencies in an unobtrusive way. The acquisition of scientific inquiry skills is an essential, even defining, characteristic of citizen science experiences that has a direct influence on data quality. Methods for assessing the direct impact of CS on volunteers' scientific inquiry skills are limited. The project will result in EA measures designed for use by diverse CS projects, strategies that CS projects can use to develop EA assessment tools, and research findings that document opportunities, supports and barriers of this innovative method across a range of CS contexts. Findings and initial resources will be shared with the broad array of stakeholders in CS through conferences, workshops, peer-reviewed publication, community websites and other relevant venues. The results of this work also have the potential to generalize to other informal science learning experiences that engage the public in science The project will address two research questions: (1) What processes are useful for developing broadly applicable EA methods or measures? and (2) What can we learn about gains in volunteers' scientific inquiry skills when citizen science organizations use EA? These will be addressed through design-based research focused on two streamlining strategies. For the reframing data validation strategy, six leaders from five established citizen science projects will conduct secondary analyses of their existing databases to uncover the skill gains of CS volunteers that are currently unexplored in their data. For the common measure strategy, ten CS projects will collaborate to create and test common EA measures of select identification-based skills. Data will be gathered through meeting notes, participant interviews and action plans, and volunteer skill gains to capture process and products of each strategy. Data will be analyzed using grounded theory, multiple process techniques, multilevel models, and repeated-measures analysis of variance. The design-based-research framework will significantly expand project impacts by jump-starting evaluation of the participating CS projects and by producing initial resources for two distinct EA strategies that have the potential to dramatically alter practice and impact citizen science efforts to ultimately enable more people to learn by contributing to the science endeavor. The project will directly equip the 15 participating citizen-science projects with authentic performance tools to assess the quality of their programing, which will expand their understanding of CS volunteer skills and help them better recruit and support their varied audiences (including rural, low-income and tribal communities).