TU Darmstadt / ULB / TUbiblio

A Streamlined Method for Sourcing Discourse-level Argumentation Annotations from the Crowd

Miller, Tristan ; Sukhareva, Maria ; Gurevych, Iryna (2019)
A Streamlined Method for Sourcing Discourse-level Argumentation Annotations from the Crowd.
The 2019 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL 2019). Minneapolis, USA (02.06.2019-07.10.2019)
doi: 10.18653/v1/N19-1177
Conference or Workshop Item, Bibliographie

This is the latest version of this item.

Abstract

The study of argumentation and the development of argument mining tools depends on the availability of annotated data, which is challenging to obtain in sufficient quantity and quality. We present a method that breaks down a popular but relatively complex discourse-level argument annotation scheme into a simpler, iterative procedure that can be applied even by untrained annotators. We apply this method in a crowdsourcing setup and report on the reliability of the annotations obtained. The source code for a tool implementing our annotation method, as well as the sample data we obtained (4909 gold-standard annotations across 982 documents), are freely released to the research community. These are intended to serve the needs of qualitative research into argumentation, as well as of data-driven approaches to argument mining.

Item Type: Conference or Workshop Item
Erschienen: 2019
Creators: Miller, Tristan ; Sukhareva, Maria ; Gurevych, Iryna
Type of entry: Bibliographie
Title: A Streamlined Method for Sourcing Discourse-level Argumentation Annotations from the Crowd
Language: English
Date: 27 February 2019
Place of Publication: Minneapolis, Minnesota
Publisher: Association for Computational Linguistics
Book Title: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL 2019)
Event Title: The 2019 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL 2019)
Event Location: Minneapolis, USA
Event Dates: 02.06.2019-07.10.2019
DOI: 10.18653/v1/N19-1177
URL / URN: https://aclanthology.org/N19-1177
Corresponding Links:
Abstract:

The study of argumentation and the development of argument mining tools depends on the availability of annotated data, which is challenging to obtain in sufficient quantity and quality. We present a method that breaks down a popular but relatively complex discourse-level argument annotation scheme into a simpler, iterative procedure that can be applied even by untrained annotators. We apply this method in a crowdsourcing setup and report on the reliability of the annotations obtained. The source code for a tool implementing our annotation method, as well as the sample data we obtained (4909 gold-standard annotations across 982 documents), are freely released to the research community. These are intended to serve the needs of qualitative research into argumentation, as well as of data-driven approaches to argument mining.

Divisions: 20 Department of Computer Science
20 Department of Computer Science > Ubiquitous Knowledge Processing
DFG-Graduiertenkollegs
DFG-Graduiertenkollegs > Research Training Group 1994 Adaptive Preparation of Information from Heterogeneous Sources
Date Deposited: 18 Sep 2019 12:12
Last Modified: 11 Jun 2024 07:11
PPN:
Corresponding Links:
Export:
Suche nach Titel in: TUfind oder in Google

Available Versions of this Item

Send an inquiry Send an inquiry

Options (only for editors)
Show editorial Details Show editorial Details