Crowdsourcing StoryLines: Harnessing the Crowd for Causal Relation Annotation
- Resource Type
- Authors
- Caselli, Tomasso; Inel, Oana; Miller, Ben; van Erp, Marieke; Vossen, Piek; Palmer, Martha; Hovy, Eduard; Mitamura, Teruko; Caswell, David; Brown, Susan W.; Bonial, Claire
- Source
- Events and Stories in the News: Proceedings of the Workshop, 44-54
STARTPAGE=44;ENDPAGE=54;TITLE=Events and Stories in the News
- Subject
- storyline
event extraction
crowdsourcing
- Language
- English
This paper describes a crowdsourcing experiment on the annotation of plot-like structures in En- glish news articles. The CrowdTruth methodology and metrics have been applied to select valid annotations from the crowd. We further run an in-depth analysis of the annotated data by compar- ing it with available expert data. Our results show a valuable use of crowdsourcing annotations for such complex semantic tasks, and promote a new annotation approach that combines crowd and experts.