Dycke, Nils ; Kuznetsov, Ilia ; Gurevych, Iryna (2024)
Overview of PragTag-2023: Low-Resource Multi-Domain Pragmatic Tagging of Peer Reviews.
The 2023 Conference on Empirical Methods in Natural Language Processing: 10th Workshop on Argument Mining. Singapore (06.12.2023 - 10.12.2023)
doi: 10.26083/tuprints-00027663
Konferenzveröffentlichung, Zweitveröffentlichung, Verlagsversion
 | Es ist eine neuere Version dieses Eintrags verfügbar. |
Kurzbeschreibung (Abstract)
Peer review is the key quality control mechanism in science. The core component of peer review are the review reports – argumentative texts where the reviewers evaluate the work and make suggestions to the authors. Reviewing is a demanding expert task prone to bias. An active line of research in NLP aims to support peer review via automatic analysis of review reports. This research meets two key challenges. First, NLP to date has focused on peer reviews from machine learning conferences. Yet, NLP models are prone to domain shift and might underperform when applied to reviews from a new research community. Second, while some venues make their reviewing processes public, peer reviewing data is generally hard to obtain and expensive to label. Approaches to low-data NLP processing for peer review remain under-investigated. Enabled by the recent release of open multi-domain corpora of peer reviews, the PragTag-2023 Shared Task explored the ways to increase domain robustness and address data scarcity in pragmatic tagging – a sentence tagging task where review statements are classified by their argumentative function. This paper describes the shared task, outlines the participating systems, and summarizes the results.
| Typ des Eintrags: | Konferenzveröffentlichung |
|---|---|
| Erschienen: | 2024 |
| Autor(en): | Dycke, Nils ; Kuznetsov, Ilia ; Gurevych, Iryna |
| Art des Eintrags: | Zweitveröffentlichung |
| Titel: | Overview of PragTag-2023: Low-Resource Multi-Domain Pragmatic Tagging of Peer Reviews |
| Sprache: | Englisch |
| Publikationsjahr: | 16 Juli 2024 |
| Ort: | Darmstadt |
| Publikationsdatum der Erstveröffentlichung: | 2023 |
| Ort der Erstveröffentlichung: | Kerrville, TX, USA |
| Verlag: | ACL |
| Buchtitel: | Proceedings of the 10th Workshop on Argument Mining |
| Veranstaltungstitel: | The 2023 Conference on Empirical Methods in Natural Language Processing: 10th Workshop on Argument Mining |
| Veranstaltungsort: | Singapore |
| Veranstaltungsdatum: | 06.12.2023 - 10.12.2023 |
| DOI: | 10.26083/tuprints-00027663 |
| URL / URN: | https://tuprints.ulb.tu-darmstadt.de/27663 |
| Zugehörige Links: | |
| Herkunft: | Zweitveröffentlichungsservice |
| Kurzbeschreibung (Abstract): | Peer review is the key quality control mechanism in science. The core component of peer review are the review reports – argumentative texts where the reviewers evaluate the work and make suggestions to the authors. Reviewing is a demanding expert task prone to bias. An active line of research in NLP aims to support peer review via automatic analysis of review reports. This research meets two key challenges. First, NLP to date has focused on peer reviews from machine learning conferences. Yet, NLP models are prone to domain shift and might underperform when applied to reviews from a new research community. Second, while some venues make their reviewing processes public, peer reviewing data is generally hard to obtain and expensive to label. Approaches to low-data NLP processing for peer review remain under-investigated. Enabled by the recent release of open multi-domain corpora of peer reviews, the PragTag-2023 Shared Task explored the ways to increase domain robustness and address data scarcity in pragmatic tagging – a sentence tagging task where review statements are classified by their argumentative function. This paper describes the shared task, outlines the participating systems, and summarizes the results. |
| ID-Nummer: | 2023.argmining-1.21 |
| Status: | Verlagsversion |
| URN: | urn:nbn:de:tuda-tuprints-276631 |
| Sachgruppe der Dewey Dezimalklassifikatin (DDC): | 000 Allgemeines, Informatik, Informationswissenschaft > 004 Informatik |
| Fachbereich(e)/-gebiet(e): | 20 Fachbereich Informatik 20 Fachbereich Informatik > Ubiquitäre Wissensverarbeitung |
| Hinterlegungsdatum: | 16 Jul 2024 12:19 |
| Letzte Änderung: | 18 Jul 2024 07:11 |
| PPN: | |
| Export: | |
| Suche nach Titel in: | TUfind oder in Google |
Verfügbare Versionen dieses Eintrags
- Overview of PragTag-2023: Low-Resource Multi-Domain Pragmatic Tagging of Peer Reviews. (deposited 16 Jul 2024 12:19) [Gegenwärtig angezeigt]
 |
Frage zum Eintrag |
Optionen (nur für Redakteure)
 |
Redaktionelle Details anzeigen |
Drucken
Impressum