Conference :: Giusti, E., Borun, M., Allen, S., Tisdal, C., & Reich, C. (2007). Assessing Learning in Informal Settings: Recent Pressures for Accountability. 20th Annual Visitor Studies Association Conference. Columbus, OH: Visitor Studies Association.

last updated: 2008-05-09 14:51:30

Abstract

As granting agencies expect accountability and measurable outcomes, more museums are wrestling with how to comply. Do we fall into the "no child left behind" cognitive testing mode? Do we conduct randomized controlled trials in summative evaluations and research? Should we insist on the value of learning in the affective realm, and if so, how do we assess it? How can we acknowledge the differences between formal and informal learning measures, and yet meet increasing pressures to prove impact of exhibits and programs? In this session, veteran visitor studies professionals Minda Borun, Ellen Giusti, Christine Reich, Sue Allen, and Carey Tisdal will facilitate dialogue about the challenges and opportunities of a changing political landscape. This session will provide visitor studies professionals, especially those responsible for measuring and reporting program outcomes to funding agencies, with opportunities to share responses to the current push for quantitative accountability, and to propose further actions (if any) that members of our field should be taking.

Session outline
To provide context, Sue Allen begins the session with a 15-minute summary of recent events in this arena, including the American Competitiveness Initiative, and statements by NIH SEPA and NSF ISE program officers regarding methodologies for research and evaluation. Participants will then join facilitators in roundtable discussions about the issues. Two rounds of discussion will allow participants to change tables. Facilitators will capture and share key points from these discussions. Finally, during the final 10 minutes of the session each participant will be asked to identify next steps to move this discussion toward action. A summary of the issues will be disseminated in the VSA E-newsletter.

Issues to be addressed include:

• What problems do we anticipate from increasing demands for quantitative accountability? Is this push likely to dissipate or increase with time? What can we learn from other fields on this topic?

• How can we ensure that qualitative and naturalistic methods are valued by funders? What are some ways to incorporate qualitative and quantitative approaches? Is testing with randomized control trials and quasi-experimental designs the only way to demonstrate accountability?

• Should we develop a field-wide logic model, and if so, how should we go about it? Should outcomes shape formative evaluation, or only summative? Should methods or instruments be standardized to some degree to support aggregation?

• What are the implications for relationships between external and internal evaluators in museums/zoos/aquariums/gardens? Is a separate external evaluator for summative evaluation more objective than one who has conducted front-end and formative evaluation for the same project?

• What is the impact of Institutional Review Board requirements on the design and implementation of studies in informal settings? How can the field support appropriate and consistent interpretations of human subjects’ regulations, particularly when this involves children?

• How can VSA actively participate in determining accountability standards and methods? What kinds of infrastructure for further conversations and work shall we create?

American Evaluation Association (2004). Guiding principles for evaluators. Available at: http://www.eval.org/Publications/GuidingPrinciplesPrintable.asp

45 CFR Part 690: Federal Policy for the Protection of Human Subjects

Tags