Rockman et al (REA), in partnership with Marti Louw and the University of Pittsburgh Center for Learning in Out-of-School Environments (UPCLOSE) conducted a summative evaluation in Summer 2014 of an aquatic macroinvertebrate digital teaching collection (macroinvertebrates.org) containing voucher specimens from the Carnegie Museum of Natural History (CMNH) in Pittsburgh, PA. The digital teaching collection groups three orders of aquatic insects (stoneflies, caddisflies, and mayflies), and users can click on a specific insect and get information on its genus, habitat, behaviors, size, abundance, pollution tolerance, and where and when it was collected. Videos and audio from expert scientists and relevant links to other websites are also provided. Each insect is tagged with interest spots that, when clicked, open up an overlay that indicates a specific diagnostic characteristic that can be used to identify that insect's genus. The images are zoomable and can be viewed in a high level of detail.
The main purpose of the evaluation was to see to what extent the aquatic macroinvertebrate digital teaching collection facilitates science communication and learning, and to demonstrate the kinds of interactive visual supports that can impact citizen science volunteers? accuracy in and confidence around identifying insects. A secondary goal of the study was to see how volunteers used and talked about the technology while engaged in observational practices, and to find out what youth volunteers, informal educators, and scientists thought about the digital teaching collection compared to more traditional paper-based resources.
Questions guiding REA's evaluation were as follows: 1.) Are volunteers able to use the aquatic macroinvertebrate digital teaching collection to accurately identify insects? 2.) Are volunteers confident in their ability to accurately identify insects using the digital teaching collection? 3.) How many diagnostic characteristics do volunteers notice when using the digital teaching collection, compared to more traditional paper-based resources? 4.) How do volunteers use the aquatic macroinvertebrate digital teaching collection? 5.) What do volunteers, informal science educators, and scientists think about the digital teaching collection? 6.) What do volunteers, informal science educators and scientists think is challenging about observing and identifying insects?
To investigate the effectiveness of the digital teaching collection in supporting users' accuracy and confidence in insect identification, REA used a mixed-methods approach to examine the impacts of the digital teaching collection on high school youth at an environmental science summer camp called Creek Connections. During the camp experience, youth collected and sorted insects from two portions of a local stream using traditional identification tools (dichotomous keys and flashcards). Afterwards, they returned to camp headquarters and were asked to fill out a brief demographic survey. Groups of 2-3 individuals were then asked to work together to identify six insects by order, genus, and family and fill out a worksheet listing their guess, as well as the diagnostic characteristics they used to make that classification. Each group was given the digital teaching collection for three of the insects, and either a dichotomous key (Tetra Tech's Family Level Key to Stream Invertebrates with illustrations and color photographs) or Voshell?s illustrated Flashcards of Common Freshwater Invertebrates for the other three insects. Evaluators took observation notes and photographs during the identification process, and audio recorded the conversations that took place within each group. After completing their identifications, all youth participated in a focus group to share what they had liked and disliked about each tool, and what they found difficult about insect identification. Participating educators from the camp, as well as scientists who contributed content to the digital teaching collection, were also interviewed to gather information about the challenges in training people to do insect identification and the affordances of the digital teaching collection.
Key Findings: (1) The digital teaching collection addresses the difficulties that volunteers have in identifying insects and meets educators' and scientists' expressed need for effective citizen science training. (2) The digital teaching collection is more effective at supporting volunteers' accuracy in and confidence around insect identification than traditional, paper-based resources. (3) The digital teaching collection is preferred by youth volunteers, although educators prefer the dichotomous key for basic identification work. (4) The digital teaching collection is accessible and can be used in a variety of educational contexts. (5) The digital teaching collection is valued by scientists as a way for citizen scientists to make an evidence-based case for the health of their local streams, and as a tool for supporting their own identification practices.
Includes instruments.
The main purpose of the evaluation was to see to what extent the aquatic macroinvertebrate digital teaching collection facilitates science communication and learning, and to demonstrate the kinds of interactive visual supports that can impact citizen science volunteers? accuracy in and confidence around identifying insects. A secondary goal of the study was to see how volunteers used and talked about the technology while engaged in observational practices, and to find out what youth volunteers, informal educators, and scientists thought about the digital teaching collection compared to more traditional paper-based resources.
Questions guiding REA's evaluation were as follows: 1.) Are volunteers able to use the aquatic macroinvertebrate digital teaching collection to accurately identify insects? 2.) Are volunteers confident in their ability to accurately identify insects using the digital teaching collection? 3.) How many diagnostic characteristics do volunteers notice when using the digital teaching collection, compared to more traditional paper-based resources? 4.) How do volunteers use the aquatic macroinvertebrate digital teaching collection? 5.) What do volunteers, informal science educators, and scientists think about the digital teaching collection? 6.) What do volunteers, informal science educators and scientists think is challenging about observing and identifying insects?
To investigate the effectiveness of the digital teaching collection in supporting users' accuracy and confidence in insect identification, REA used a mixed-methods approach to examine the impacts of the digital teaching collection on high school youth at an environmental science summer camp called Creek Connections. During the camp experience, youth collected and sorted insects from two portions of a local stream using traditional identification tools (dichotomous keys and flashcards). Afterwards, they returned to camp headquarters and were asked to fill out a brief demographic survey. Groups of 2-3 individuals were then asked to work together to identify six insects by order, genus, and family and fill out a worksheet listing their guess, as well as the diagnostic characteristics they used to make that classification. Each group was given the digital teaching collection for three of the insects, and either a dichotomous key (Tetra Tech's Family Level Key to Stream Invertebrates with illustrations and color photographs) or Voshell?s illustrated Flashcards of Common Freshwater Invertebrates for the other three insects. Evaluators took observation notes and photographs during the identification process, and audio recorded the conversations that took place within each group. After completing their identifications, all youth participated in a focus group to share what they had liked and disliked about each tool, and what they found difficult about insect identification. Participating educators from the camp, as well as scientists who contributed content to the digital teaching collection, were also interviewed to gather information about the challenges in training people to do insect identification and the affordances of the digital teaching collection.
Key Findings: (1) The digital teaching collection addresses the difficulties that volunteers have in identifying insects and meets educators' and scientists' expressed need for effective citizen science training. (2) The digital teaching collection is more effective at supporting volunteers' accuracy in and confidence around insect identification than traditional, paper-based resources. (3) The digital teaching collection is preferred by youth volunteers, although educators prefer the dichotomous key for basic identification work. (4) The digital teaching collection is accessible and can be used in a variety of educational contexts. (5) The digital teaching collection is valued by scientists as a way for citizen scientists to make an evidence-based case for the health of their local streams, and as a tool for supporting their own identification practices.
Includes instruments.
Associated Projects
TEAM MEMBERS
University of Pittsburgh Center for Learning in Out-of-School-Environments (UPCLOSE)
Contributor
Citation
Funders
NSF
Funding Program:
ISE/AISL
If you would like to edit a resource, please email us to submit your request.