As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program funds innovative research, approaches and resources for use in a variety of settings. The subject of physics and all of its sub-disciplines are becoming more prevalent in the public press as the research results appear to be quite interesting and important. While the physics discipline has made a Nation-wide effort to acquaint the public with physics knowledge through informal education learning experiences for years, it has not been as successful as the community desires. Thus, this project is aimed to gather all of the informal and outreach physics education efforts that have been attempted in the hope of finding the best practices for learning physics concepts and practices. A compendium will be published to inform future opportunities on how to educate the public through informal and outreach mechanisms. This project is a collaboration between Michigan State University and the University of Colorado. The physics community has a long history of engaging audiences in informal education activities. Physics institutions that facilitate informal programs include university departments, national laboratories and centers, and professional societies and organizations. There is, however, no systemic understanding of how these programs are facilitated, nor an assessment of the collective impact that these programs have on participants. This project will address numerous research questions in the broad areas of Activity Detail, Structural Aspects, and Assessment. Further, their efforts will determine the "who, what, why, where and how" of informal physics offerings, focusing on their facilitation, impact on participants, and the academic and discipline-specific cultures from which these programs originate. The study has several definite research outcomes that will emerge from this methodology: 1) They will produce a survey of the informal efforts of university physics departments, national physics labs and national physics organizations, 2) They will develop a taxonomy of informal physics programs from which we can characterize the landscape of programs, and 3) by investigating both "successful" as well as "failed" or terminated programs, they will develop an understanding of the culture and resources needed to support outreach from these research findings. In addition, they will produce published works that can be utilized by informal practitioners and administrators in physics to examine current programs and guide the development of new programs. With regards to the research questions and framework, the overarching and driving question for this research project is: "What is the landscape of informal physics learning, specifically, of those programs in the United States facilitated by physicists and physics students at academic institutions, national labs and by national physics organizations?" This study will provide a robust understanding of the state of informal physics programs and outreach by physicists in the United States today. Findings will inform practitioners and administrators as to how best to support and design informal physics programming. The results will also have broad implications for other discipline-specific informal STEM programming. The primary data collection methods will be a nationwide survey and interviews with a large sample of informal practitioners from the physics community. Site visits will be conducted with a subset of these programs in order to observe programs in action and to glean insights from university participants, community partners, public, and K-12 audiences.
DATE:
-
TEAM MEMBERS:
Kathleen HinkoNoah Finkelstein
resourceprojectWebsites, Mobile Apps, and Online Media
The intent of this five-year project is to design, deliver, and study professional development for Informal Science Learning (ISL) educators in the arena of equity-focused STEAM (Science, Technology, Engineering, Art, and Mathematics) teaching and learning. While the strategy of integrating art and science to promote interest, identity, and other STEM-related learning has grown in recent years, this domain is still nascent with respect to a guiding set of best practices. Through prior work, the team has developed and implemented a set of design principles that incorporate effective practices for broadening participation of girls in science via science-art integration on the topic of the biology, chemistry and optics of "Colors in Nature." The continued initiative would impact the ISL field by providing a mechanism for ISL educators in museums, libraries and after-school programs to adopt and implement these STEAM design principles into their work. The team will lead long-term (12-18 months) professional development activities for ISL educators, including: 1) in-person workshops that leverage their four previously developed kits; 2) online, asynchronous learning activities featuring interactive instructional videos around their STEAM design principles; 3) synchronous sessions to debrief content and foster communities of practice; and 4) guided design work around the development or redesign of STEAM activities. In the first four years of the project, the team will work with four core institutional partners (Sitka Sound Science Center, Sno-Isle Libraries, the Fairbanks North Star Borough School District after-school program, and the Pima County Public Library system) across three states (Alaska, Washington, and Arizona). In the project's later stages, they will disseminate their learning tools to a broad, national audience. This project is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants.
The project has three main goals: (1) To support ISL educators in offering meaningful STEAM activities, (2) To create institutional change among the partner organizations, and (3) To advance the ISL field with respect to professional development and designing for STEAM Programming. The research questions associated with the professional development activities address the ways in which change occurs and focus on all three levels: individual, institutional, and the ISL field. The methods are qualitative and quantitative, including videotaped observations, pre and post interviews, surveys and analysis of online and offline artifacts. In addition, the project evaluation will assess the implementation of the project's professional development model for effectiveness. Methods will include observations, interviews, surveys and Website analytics and program data.
As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program funds innovative research, approaches and resources for use in a variety of settings. The project will derive a nationwide online coaching/mentoring program for out of school educators in rural settings. The program builds on a Noyce Foundation pilot project. The issue to be addressed is that educators in rural settings are challenged in a multitude of ways due to isolation. This project will try to find ways to alleviate some of the consequences of isolation through resource sharing, knowledge sharing, and unique techniques for communicating with students. Partners in this effort are the Maine Mathematics and Science Alliance, the National AfterSchool Association, Development Without Limits, and the Maine State Library.
By using widely-available technologies, this project will bring fully online instructional coaching in STEM to out-of-school educators who live too remotely to attend ongoing in-person workshops. The project team will achieve this by adapting a highly promising coaching program where groups of educators video-record their own work with youth, practice key skills, and meet regularly to discuss their work. The project will: (a) test technical challenges to achieve fully virtual implementation; (b) design and adapt a specific STEM-skill curriculum to align with different levels of need; (c) customize the model to work with rural librarians; and (d) integrate the work into existing state and national accreditation systems.
A collaboration of TERC, MIT, The Woods Hole Oceanographic Institution and community-based dance centers in Boston, this exploratory project seeks to address two main issues in informal science learning: 1) broadening participation in science by exploring how to expand science access to African-American and Latino youth and 2) augmenting science learning in informal contexts, specifically learning physics in community-based dance sites. Building on the growing field of "embodied learning," the project is an outgrowth in part of activities over the past decade at TERC and MIT that have investigated approaches to linking science, human movement and dance. Research in embodied learning investigates how the whole body, not just the brain, contributes to learning. Such research is exploring the potential impacts on learning in school settings and, in this case, in out of school environments. This project is comprised of two parts, the first being an exploration of how African-American and Latino high school students experience learning in the context of robust informal arts-based learning environments such as community dance studios. In the second phase, the collaborative team will then identify and pilot an intervention that includes principles for embodied learning of science, specifically in physics. This phase will begin with MIT undergraduate and graduate students developing the course before transitioning to the community dance studios. This project is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants.
The goal of this pilot feasibility study is to build resources for science learning environments in which African-American and Latino students can develop identities as people who practice and are engaged in scientific inquiry. Youth will work with choreographers, physicists and educators to embody carefully selected physics topics. The guiding hypothesis is that authentic inquiries into scientific topics and methods through embodied learning approaches can provide rich opportunities for African-American and Latino high school-aged youth to learn key ideas in physics and to strengthen confidence in their ability to become scientists. A design- based research approach will be used, with data being derived from surveys, interviews, observational field notes, video documentation, a case study, and physical artifacts produced by participants. The study will provide the groundwork for producing a set of potential design principles for future projects relating to informal learning contexts, art and science education with African American and Latino youth.
DATE:
-
TEAM MEMBERS:
Folashade Cromwell SolomonTracey WrightLawrence Pratt
In March of 2016, a total solar eclipse occurred in the southwestern pacific; and in August of 2017, a total solar eclipse occurred across a broad swath of the United States. The Exploratorium launched a 2.5 year public education program—Navigating the Path of Totality—that used these two total solar eclipses as platforms for sparking public engagement and learning about the Sun, heliophysics, and the STEM content related to both. These sequential eclipses provided an unprecedented opportunity to build and scaffold public engagement and education. Our strategy was to start the public engagement process with the 2016 eclipse, nurture that engagement with resources, activities and outreach during the 17 months between the eclipses, so that audiences (especially in the U.S., where totality was visible in multiple areas across the country) would be excited, actively interested, and prepared for deeper engagement during the 2017 eclipse. For the August 2017 eclipse, the Exploratorium produced live telescope and program feeds from Madras, OR and Casper, WY. The Exploratorium worked with NASA to leverage what was a once-in-a-lifetime experience for millions to bring heliophysics information and research to students, educators, and the public at large through a variety of learning experiences and platforms.
The core of this project was live broadcasts/webcasts of each eclipse. To accomplish these objectives, the Exploratorium produced and disseminate live feeds of telescope-only images (no commentary) of each eclipse originating them from remote locations; produce and disseminate from the field live hosted broadcasts/webcasts of each eclipse using these telescope images; design and launch websites, apps, videos, educator resources, and shareable online materials for each eclipse; design and deliver eclipse themed video installations for our Webcast studio and Observatory gallery in the months that lead up to each eclipse and a public program during each eclipse; and conduct a formative and summative evaluation of the project.
These broadcasts/webcasts and pre-produced videos provide the backbone upon which complementary educational resources and activities can be built and delivered. Programs and videos were produced in English and Spanish languages. As a freely available resource, the broadcasts/webcasts also provide the baseline content for hundreds if not thousands of educational efforts provided by other science-rich institutions, schools, community-based organizations, and venues. Platforms such as NASA TV and NASA website, broadcast and online media outlets such as ABC, NBC, CBS, CNN, MSNBC and PBS, as well as hundreds of science institutions and thousands of classrooms streamed the Exploratorium eclipse broadcasts as part of their own educational programming, reaching 63M people. These live broadcasts were relied upon educational infrastructure during total solar eclipses for institutions and individuals on the path and off the path alike.
As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program funds innovative research, approaches and resources for use in a variety of settings. The project will conduct research designed to deepen our fundamental knowledge about culture, experience, and ecosystems cognition and to develop innovative practices and approaches to support learning about changing ecological systems and environmental decision making. Work on cultural differences in the production of complex systems knowledge is severely lacking. This gap in knowledge may contribute to the continued reproduction of inequities in science education. More broadly findings from this project will have clear implications for theories of cognitive development, especially those pertaining to how knowledge is shaped by culture and experience. Focusing on ecosystems may represent an opportunity to not only increase engagement and achievement in science among non-dominant communities and Native youth specifically, but also advance effective learning for all communities. The primary deliverables for the project are conference presentations and research publications. However, the project will also develop additional resources freely available to researchers, educators, and the general public. These will include summer curricular materials and teaching tools, professional development workshops, practitioner briefs about research findings that can be used in professional development workshops and shared share more broadly, and evaluation reports.
A deeper understanding of cultural influences on conceptions of the natural world can serve to advance the educational needs of children, including children from diverse linguistic and cultural backgrounds. Project research will include two interrelated series of studies designed to expand knowledge about human cognition of complex ecosystems and the affordances of informal STEM learning environments in developing and supporting the critical 21st century skill of ecological systems level reasoning. The first consists of a series of experiments focused on ecological cognition and the role of humans in nature. The second consists of design-based research interventions in informal settings, summer workshops for youth and the communities, focused on ecological systems level thinking and socio-environmental decision making. The project will recruit and engage both child and adult participants from two broad cultural communities, Native Americans and European Americans living in urban and suburban communities, in part because it affords a sharp test of human-nature relations. Sampling from two different urban communities will avoid simple Native-non-Native comparative binaries and to conduct Native-to-Native comparative analysis. Based on results from this, the project will result in: 1) foundational knowledge about human learning and reasoning and ecosystems and environmental decision making, 2) culturally responsive models of learning and practice about complex ecosystems for indoors and outdoors informal learning environments, and 3) insights about research-practice-community partnerships. One important objective of the research is to broaden participation and close opportunity gaps for under-represented groups in STEM fields broadly and more specifically for Indigenous people. Members of Indigenous communities, who provide strong role models for other aspiring scholars, will be involved as postdoctoral fellows, research assistants and graduate fellows.
Women continue to be underrepresented in computer science professions. In 2015, while 57% of professional occupations in the U.S. were held by women, only 25% of computing occupations were held by women. Furthermore, the share of computer science degrees going to women is smaller than any STEM field, even though technology careers are the most promising in terms of salaries and future growth. Research suggests that issues contributing to this lack of computer science participation begin early and involve complex social and environmental factors, including girls' perception that they do not belong in computer science classes or careers. Computer science instruction often alienates girls with irrelevant curriculum; non-collaborative pedagogies; a lack of opportunities to take risks or make mistakes; and a heavy reliance on lecture instead of hands-on, project-based learning. Computer science experiences that employ research-based gender equitable best practices, particularly role modeling, can help diminish the gender gap in participation. In response to this challenge, Twin Cities PBS (TPT), the National Girls Collaborative (NGC) and Code.org will lead Code: SciGirls! Media for Engaging Girls in Computing Pathways, a three-year project designed to engage 8-13 year-old girls in coding through transmedia programming which inspires and prepares them for future computer science studies and career paths. The project includes five new PBS SciGirls episodes featuring girls and female coding professionals using coding to solve real problems; a new interactive PBSKids.org game that allows children to develop coding skills; nationwide outreach programming, including professional development for informal educators and female coding professionals to facilitate activities for girls and families in diverse STEM learning environments; a research study that will advance understanding of how the transmedia components build girls' motivation to pursue additional coding experiences; and a third-party summative evaluation.
Code: SciGirls! will foster greater awareness of and engagement in computer science studies and career paths for girls. The PBS SciGirls episodes will feature girls and female computer science professionals using coding to solve real-world challenges. The project's transmedia component will leverage the television content into the online space in which much of 21st century learning takes place. The new interactive PBSKids.org game will use a narrative framework to help children develop coding skills. Drawing on narrative transportation theory and character identification theory, TPT will commission two exploratory knowledge-building studies to investigate: To what extent and how do the narrative formats of the Code: SciGirls! online media affect girls' interest, beliefs, and behavioral intent towards coding and code-related careers? The studies aim to advance understanding of how media builds girls' motivation to pursue computer science experiences, a skill set critical to building tomorrow's workforce. The project team will also raise educators' awareness about the importance of gender equitable computer science instruction, and empower them with best practices to welcome, prepare and retain girls in coding. The Code: SciGirls! Activity Guide will provide educators with a relevant resource for engaging aspiring computer scientists. The new media and guide will also reside on PBSLearningMedia.org, reaching 1.2 million teachers, and will be shared with thousands of educators across the SciGirls CONNECT and National Girls Collaborative networks. The new episodes are anticipated to reach 92% of U.S. TV households via PBS, and the game at PBSKids.org will introduce millions of children to coding. The summative evaluation will examine the reach and impact of the episodes, game and new activities. PIs will share research findings and project resources at national conferences and will submit to relevant publications. This project is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants.
As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program funds innovative research, approaches and resources for use in a variety of settings. Blind youth are generally excluded from STEM learning and careers because materials for their education are often composed for sighted individuals. In this proposed Innovations in Development project, the PIs suggest that spatial acuity is an important element in order for blind persons to understand physical and mental structures. Thus, in this investigation, efforts will be made to educated blind youth in the discipline of engineering. A total of 200 blind students, ages 12-20 along with 30 informal STEM educators will participate in the program. This effort is shared with the National Federation of the Blind, Utah State University, the Science Museum of Minnesota, and the Lifelong Learning Group.
The National Federation of the Blind, in partnership with scholars from Utah State University and educators from the Science Museum of Minnesota will develop a five-year Innovations in Development project in order to broaden the participation of blind students in STEM fields through the development of instruction and accessible tools that assess and improve the spatial ability of blind youth. The partnership with the Science Museum will facilitate the creation of informal science content for students and professional development opportunities for informal educators. Evaluation will be conducted by Lifelong Learning Group of the Columbus Center of Science and Industry. Activities will begin in year one with a week-long, engineering design program for thirty blind high-school students at the Federation of the blind headquarters in Baltimore. Year two will feature two similarly sized programs, taking place at the Science Museum. While spatial ability is linked to performance in science, research has not been pursued as to how that ability can be assessed, developed, and improved in blind youth. Further, educators are often unaware of ways to deliver science concepts to blind students in a spatially enhanced manner, and students do not know how to advocate for these accommodations, leading blind youth to abandon science directions. Literature on the influences of a community of practice on youth with disabilities, as well as nonvisual tools for experiencing engineering, is lacking. This project will advance understanding of how blind people can participate in science, and how spatial ability can be developed and bolstered through informal engineering activities and an existing community of practice.
As part of its overall effort to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program, seeks to advance new approaches to, and evidence-based understanding of, the design and development of science, technology, engineering, and mathematics (STEM) learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences, advancing innovative research on and assessment of STEM learning in informal environments, and developing understandings of deeper learning by participants. In alignment with these aims, the STEM + Digital Literacies (STEM+L) project will investigate science fiction as an effective mechanism to attract and immerse adolescents (ages 10-13) from diverse cultural backgrounds in environmental and human health content and socio-scientific issues. This work is particularly novel, as the current knowledge base is limited, and largely addresses the high school level. Therefore, the results of the proposed effort could yield important findings regarding the feasibility of this activity as an effective platform for science learning and engagement for younger students. As such, STEM+L would not only advance knowledge in the field but would also contribute to a growing AISL portfolio on digital literacy and learning.
STEM+L is an early stage Innovations in Development project that will engage thirty middle school students in out of school time experiences. Over a twenty-four-week period, students will work collaboratively in groups in-person and online with their peers and field experts to design, develop, and produce STEM content rich, multimedia science fictions. The in-person learning experiences will take place on the University of Miami campus during the summer and academic year. Culminating activities include student presentations online and at a local Science Fiction Festival. The research component will employ an iterative, design-based approach. Four research questions will be explored: (a) How do students learn science concepts and multimodal digital literacies through participating in the STEM+L Academy? (b) How do students change their views in STEM related subject matter and in pursuing STEM related careers? (c) How do students participate in the STEM+L Academy? (d) How do we best support students' participation and learning of STEM+L in face-to-face and online environments? Data collection methods include video records, student-generated artifacts, online surveys, embedded assessments, interviews, and multimodal reflections. Comparative case analysis and a mixed methods approach will be employed. A rigorous evaluation will be conducted by a critical external review board. Inclusive and innovative dissemination strategies will ensure that the results of the research and program reach a broad range of audiences including both informal and formal STEM and literacy educators and researchers, learning scientists, local communities, and policy makers through national and international conference presentations, journal publications, Web2.0 resources, and community outreach activities.
In this project, education researchers, environmental scientists, and educators will develop a computer tool to let STEM educators and curriculum developers build local environmental science models. The system will use data about land use to automatically construct map-based simulations of any area in the United States. Users will be able to choose from a range of environmental and economic issues to include in these models. The system will create simulations that ask students to change to patterns of land use -- for example, increasing land zoned for housing, or open land, or industrial development -- to try to meet environmental and social goals. As a result, students will be able to learn about the interaction of environmental and economic issues relevant to their own city, town, neighborhood, or region. These map-based simulations will be incorporated into an existing science, technology, engineering, and mathematics (STEM) education tool, Land Science, in which learners work in a fictional planning office to study how zoning affects economic and environmental issues in a community. Research has shown that Land Science is mode effective when learners are exploring issues in an area near their home, and the current study will investigate how and why local simulations improve environmental science learning. This project is funded by the Advancing Informal STEM Learning (AISL) program which supports work to enhance learning in informal environments by funding innovative research, approaches, and resources for use in a variety of settings.
In this project, the research team will build, test, and deploy a toolkit that will allow informal STEM educators and developers of informal STEM programming to easily adapt an existing environmental science learning environment, which consists of a place-based virtual internship in urban planning and ecology, to their local contexts, learning objectives, and learner populations. Land Science is a virtual internship in which young people explore the environmental and socio-economic impacts of land-use decisions. To do so, they play the role of interns at an urban planning firm developing a new land-use proposal for the city of Lowell, Massachusetts: they read reports, virtually visit sites, determine stakeholder priorities, and use a geographic information system (GIS) model to evaluate the socio-economic and environmental impacts of land-use choices. No one plan can satisfy all stakeholders, so learners must compromise to create an effective plan and justify their decisions. Land Science has been shown to improve civic engagement, interest in eco-social issues, and understanding of scientific models, but it is most effective when the location of the virtual internship is in or near the learners' home town. To improve the accessibility and impact of this effective learning intervention, the interdisciplinary research team, which includes learning scientists, land-use experts, and informal STEM educators, will develop a Local Environmental Modeling toolkit, which will allow educators to change the location of the simulation and the stakeholder groups, zoning codes, and environmental and socio-economic indicators included in the land-use model. The system will ensure that the model produced is functional, realistic, and appropriately complex. The localized versions of Land Science produced by informal STEM educators will be used in a range of contexts and locations, allowing the research team to study the effects of an online, place-based learning intervention on environmental science learning, STEM interest and motivation, and civic engagement.
DATE:
-
TEAM MEMBERS:
David ShafferKristen ScopinichHolly GibbsJeffrey Linderoth
Often called "self-plagiarism," text recycling occurs frequently in scientific writing. Over the past decade, increasing numbers of scientific journals have begun using plagiarism detection software to screen submitted manuscripts. As a result, large numbers of cases of text recycling are being identified, yet there is no consensus on what constitutes ethically acceptable practice. Text recycling is thus an increasingly important and controversial ethical issue in scientific communication. However, little actual research has been conducted on text recycling and it is rarely addressed in the ethical training of researchers or in scientific writing textbooks or websites. To promote the ethical and appropriate use of text recycling, this project will be conducted in two phases: In Phase 1, the researchers will investigate the ethical, practical, and legal aspects of text recycling as relevant for professional researchers, students, and publishers. In Phase 2, the investigators will produce educational materials and develop model language for text recycling guidelines and author-publisher contracts that can be adapted by educational institutions, research organizations, and publishers.
This project is a multi-institutional, multidisciplinary investigation of text recycling, the reuse of material from one?s previous work in a new manuscript. In Phase 1, the researchers will investigate questions such as these: What do expert researchers, students, and others involved in scientific communication believe to be appropriate practice, and why? Where is there a clear consensus among experts and where is there substantive disagreement? How often do professional scientists actually recycle material, and in what ways? Under what circumstances does text recycling violate publisher contracts or copyright laws? One facet of this research will involve interviewing and surveying experienced STEM faculty, students, journal editors, and others regarding the ethics of text recycling. A second facet will analyze a corpus of published scientific papers to investigate how researchers recycle text in practice and how this has changed over time. The third facet involves analyzing publisher contracts to better understand the rights of publishers and authors regarding text recycling and to assess their legal validity. In Phase 2, the investigators will use findings from Phase 1 to develop, test, and disseminate two kinds of materials: The first are web and print based instructional materials for STEM students (and others new to STEM research) explaining the ethical, legal, and practical issues involved with text recycling, as well as accompanying documents for faculty, administrators, and librarians. The second are model policies and guidelines for text recycling that address appropriate practice in both academic and professional settings. The investigators will obtain feedback on drafts of these materials from potential users and revise them accordingly, after which they will be disseminated.
As a part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program funds research and innovative resources for use in a variety of settings. This Broad Implementation project would scale up the CryptoClub Project, an afterschool and online program designed to engage middle school youth in mathematics and cryptography. The project builds on previous successful work and evaluation that is ready for scale up using a train-the-trainer model implemented through a partnership with the National Girls Collaborative. The project will train 160 new CryptoClub leaders who will then train 800 new leaders at 20 hub sites reaching 9600 students. In addition, professional development modules and webinars will continue to refresh leader skills. Other project components include an online multiplayer cryptography game, weekly challenges through social media, and digital cryptology badges for students.
The research uses a think-aloud method with students as they actually attempt to solve the cryptology problems using mathematical thinking. Three think-aloud studies will be performed during the Project. The research team will code transcripts of the interviews for evidence of the mathematical thinking intended to be addressed by each activity, as well as capturing unexpected kinds of thinking. Tasks will also be rated according to the type of knowledge elicited. A written report will include statistical analyses of the think-aloud and interview responses, interpreted in light of the overall CryptoClub goals. The findings will contribute to both future research efforts and practice. The evaluation by EDC uses a quasi-experimental design, which assesses project outcomes for trainers, leaders, students, and Internet users. EDC will also investigate the fidelity to the CryptoClub model as it is scaled up. These studies have strong potential for informing numerous other projects that are at a stage where scale up is under consideration.