New-York Historical Society's Tang Academy for American Democracy Program Evaluation

Overview

In 2023 Kera Collective partnered with the New-York Historical Society (N-YHS) to evaluate the Tang Academy for American Democracy (TAAD) program. TAAD aims to address 6th grade New York City learning standards by comparing the ancient Athenian model of democracy to democracy in the United States, as well as generate excitement about learning history.

TAAD had been in operation for three years (much of it taking place online during Covid-19), and N-YHS educators felt it was time to learn if the program was having their intended impact on students now that it was back to being held in person at the museum.

 Approach 

 Kera Collective designed an evaluation with three phases that built on each other:

• Phase 1 (Planning): We  collaboratively developed a Logic Model with TAAD program staff to define measurable outcomes of the program. These outcomes served as guideposts to develop instruments for the following phases.
• Phase 2 (Pilot Evaluation): We collected data about TAAD through two exploratory methods (observations and embedded assessments). These qualitative approaches helped us refine our instruments for the next phase.
• Phase 3 (Full-Scale Evaluation): Using a quasi-experimental pre-post study design, we assessed student outcomes through standardized observations and an assessment embedded as a program activity to look for statistically significant student growth.

Client Takeaways 

TAAD students demonstrated significant growth in content knowledge about the two democratic models, skills (e.g., public speaking and collaboration), and social-emotional learning and civic engagement. Observations showed that students were most engaged when presented with opportunities to connect program content to their own lived experiences.

The evaluation also highlighted areas for increased alignment between the intended outcomes of the program and actual student outcomes, particularly those around object inquiry. In reflecting on the findings with N-YHS, we noted outcomes on the Logic Model that were less aligned with the program’s long term intended impact, and which might be replaced with ones that were more closely aligned.

N-YHS plans to make these changes to the Logic Model, as well as think about possible changes to the art making components of the program to support student creativity and interest in bringing their ideas about democracy and civic engagement back home to their communities.

If you would like to edit a resource, please email us to submit your request.