Pfeiffer, Jonas ; Simpson, Edwin ; Gurevych, Iryna (2020)
Low Resource Multi-Task Sequence Tagging - Revisiting Dynamic Conditional Random Fields.
doi: 10.48550/arXiv.2005.00250
Report, Bibliographie
Kurzbeschreibung (Abstract)
We compare different models for low resource multi-task sequence tagging that leverage dependencies between label sequences for different tasks. Our analysis is aimed at datasets where each example has labels for multiple tasks. Current approaches use either a separate model for each task or standard multi-task learning to learn shared feature representations. However, these approaches ignore correlations between label sequences, which can provide important information in settings with small training datasets. To analyze which scenarios can profit from modeling dependencies between labels in different tasks, we revisit dynamic conditional random fields (CRFs) and combine them with deep neural networks. We compare single-task, multi-task and dynamic CRF setups for three diverse datasets at both sentence and document levels in English and German low resource scenarios. We show that including silver labels from pretrained part-of-speech taggers as auxiliary tasks can improve performance on downstream tasks. We find that especially in low-resource scenarios, the explicit modeling of inter-dependencies between task predictions outperforms single-task as well as standard multi-task models.
| Typ des Eintrags: | Report |
|---|---|
| Erschienen: | 2020 |
| Autor(en): | Pfeiffer, Jonas ; Simpson, Edwin ; Gurevych, Iryna |
| Art des Eintrags: | Bibliographie |
| Titel: | Low Resource Multi-Task Sequence Tagging - Revisiting Dynamic Conditional Random Fields |
| Sprache: | Englisch |
| Publikationsjahr: | 1 Mai 2020 |
| Verlag: | arXiv |
| Reihe: | Computation and Language |
| Auflage: | 1. Version |
| DOI: | 10.48550/arXiv.2005.00250 |
| URL / URN: | https://arxiv.org/abs/2005.00250 |
| Kurzbeschreibung (Abstract): | We compare different models for low resource multi-task sequence tagging that leverage dependencies between label sequences for different tasks. Our analysis is aimed at datasets where each example has labels for multiple tasks. Current approaches use either a separate model for each task or standard multi-task learning to learn shared feature representations. However, these approaches ignore correlations between label sequences, which can provide important information in settings with small training datasets. To analyze which scenarios can profit from modeling dependencies between labels in different tasks, we revisit dynamic conditional random fields (CRFs) and combine them with deep neural networks. We compare single-task, multi-task and dynamic CRF setups for three diverse datasets at both sentence and document levels in English and German low resource scenarios. We show that including silver labels from pretrained part-of-speech taggers as auxiliary tasks can improve performance on downstream tasks. We find that especially in low-resource scenarios, the explicit modeling of inter-dependencies between task predictions outperforms single-task as well as standard multi-task models. |
| Freie Schlagworte: | UKP_p_FAMULUS |
| Zusätzliche Informationen: | Preprint |
| Fachbereich(e)/-gebiet(e): | 20 Fachbereich Informatik 20 Fachbereich Informatik > Ubiquitäre Wissensverarbeitung |
| Hinterlegungsdatum: | 15 Mär 2021 12:10 |
| Letzte Änderung: | 11 Jul 2024 09:54 |
| PPN: | |
| Export: | |
| Suche nach Titel in: | TUfind oder in Google |
 |
Frage zum Eintrag |
Optionen (nur für Redakteure)
 |
Redaktionelle Details anzeigen |
Drucken
Impressum