TU Darmstadt / ULB / TUbiblio

Lessons Learned from a Citizen Science Project for Natural Language Processing

Klie, Jan-Christoph ; Lee, Ji-Ung ; Stowe, Kevin ; Şahin, Gözde Gül ; Moosavi, Nafise Sadat ; Bates, Luke ; Petrak, Dominic ; Castilho, Richard Eckart de (2023)
Lessons Learned from a Citizen Science Project for Natural Language Processing.
17th Conference of the European Chapter of the Association for Computational Linguistics. Dubrovnik, Croatia (02.-06.05.2023)
Conference or Workshop Item, Bibliographie

Abstract

Many Natural Language Processing (NLP) systems use annotated corpora for training and evaluation. However, labeled data is often costly to obtain and scaling annotation projects is difficult, which is why annotation tasks are often outsourced to paid crowdworkers. Citizen Science is an alternative to crowdsourcing that is relatively unexplored in the context of NLP. To investigate whether and how well Citizen Science can be applied in this setting, we conduct an exploratory study into engaging different groups of volunteers in Citizen Science for NLP by re-annotating parts of a pre-existing crowdsourced dataset. Our results show that this can yield high-quality annotations and at- tract motivated volunteers, but also requires considering factors such as scalability, participation over time, and legal and ethical issues. We summarize lessons learned in the form of guidelines and provide our code and data to aid future work on Citizen Science.

Item Type: Conference or Workshop Item
Erschienen: 2023
Creators: Klie, Jan-Christoph ; Lee, Ji-Ung ; Stowe, Kevin ; Şahin, Gözde Gül ; Moosavi, Nafise Sadat ; Bates, Luke ; Petrak, Dominic ; Castilho, Richard Eckart de
Type of entry: Bibliographie
Title: Lessons Learned from a Citizen Science Project for Natural Language Processing
Language: English
Date: 2 May 2023
Publisher: ACL
Book Title: The 17th Conference of the European Chapter of the Association for Computational Linguistics - proceedings of the conference
Event Title: 17th Conference of the European Chapter of the Association for Computational Linguistics
Event Location: Dubrovnik, Croatia
Event Dates: 02.-06.05.2023
URL / URN: https://aclanthology.org/2023.eacl-main.261/
Abstract:

Many Natural Language Processing (NLP) systems use annotated corpora for training and evaluation. However, labeled data is often costly to obtain and scaling annotation projects is difficult, which is why annotation tasks are often outsourced to paid crowdworkers. Citizen Science is an alternative to crowdsourcing that is relatively unexplored in the context of NLP. To investigate whether and how well Citizen Science can be applied in this setting, we conduct an exploratory study into engaging different groups of volunteers in Citizen Science for NLP by re-annotating parts of a pre-existing crowdsourced dataset. Our results show that this can yield high-quality annotations and at- tract motivated volunteers, but also requires considering factors such as scalability, participation over time, and legal and ethical issues. We summarize lessons learned in the form of guidelines and provide our code and data to aid future work on Citizen Science.

Uncontrolled Keywords: UKP_p_EVIDENCE, UKP_p_square,UKP_p_INCEpTION,UKP_p_PEER, UKP_p_seditrah_factcheck
Divisions: 20 Department of Computer Science
20 Department of Computer Science > Ubiquitous Knowledge Processing
Zentrale Einrichtungen
Zentrale Einrichtungen > hessian.AI - The Hessian Center for Artificial Intelligence
Date Deposited: 12 Jun 2023 12:31
Last Modified: 09 Aug 2023 12:29
PPN: 510469019
Export:
Suche nach Titel in: TUfind oder in Google
Send an inquiry Send an inquiry

Options (only for editors)
Show editorial Details Show editorial Details