Meyer, Christian M. ; Mieskes, Margot ; Stab, Christian ; Gurevych, Iryna (2014)
DKPro Agreement: An Open-Source Java Library for Measuring Inter-Rater Agreement.
Dublin, Ireland
Konferenzveröffentlichung, Bibliographie
Kurzbeschreibung (Abstract)
In this paper, we introduce a novel Java implementation of multiple inter-rater agreement measures, which we make available as open-source software. Besides assessing the reliability of coding tasks using S, pi, kappa, alpha, etc., we particularly support unitizing tasks by measuring alpha_U as the agreement of the boundaries of the identified annotation units. We provide a unified interface and data model for both tasks as well as multiple diagnostic devices for analyzing the results.
| Typ des Eintrags: | Konferenzveröffentlichung |
|---|---|
| Erschienen: | 2014 |
| Autor(en): | Meyer, Christian M. ; Mieskes, Margot ; Stab, Christian ; Gurevych, Iryna |
| Art des Eintrags: | Bibliographie |
| Titel: | DKPro Agreement: An Open-Source Java Library for Measuring Inter-Rater Agreement |
| Sprache: | Englisch |
| Publikationsjahr: | August 2014 |
| Buchtitel: | Proceedings of the 25th International Conference on Computational Linguistics: System Demonstrations (COLING) |
| Veranstaltungsort: | Dublin, Ireland |
| URL / URN: | http://aclweb.org/anthology/C14-2023 |
| Kurzbeschreibung (Abstract): | In this paper, we introduce a novel Java implementation of multiple inter-rater agreement measures, which we make available as open-source software. Besides assessing the reliability of coding tasks using S, pi, kappa, alpha, etc., we particularly support unitizing tasks by measuring alpha_U as the agreement of the boundaries of the identified annotation units. We provide a unified interface and data model for both tasks as well as multiple diagnostic devices for analyzing the results. |
| Freie Schlagworte: | UKP_reviewed;UKP_p_DKPro;UKP_a_WALL |
| ID-Nummer: | TUD-CS-2014-0863 |
| Fachbereich(e)/-gebiet(e): | 20 Fachbereich Informatik 20 Fachbereich Informatik > Ubiquitäre Wissensverarbeitung |
| Hinterlegungsdatum: | 31 Dez 2016 14:29 |
| Letzte Änderung: | 24 Jan 2020 12:03 |
| PPN: | |
| Export: | |
| Suche nach Titel in: | TUfind oder in Google |
 |
Frage zum Eintrag |
Optionen (nur für Redakteure)
 |
Redaktionelle Details anzeigen |
Drucken
Impressum