Bigoulaeva, Irina ; Sachdeva, Rachneet ; Madabushi, Harish Tayyar ; Villavicencio, Aline ; Gurevych, Iryna (2022)
Effective Cross-Task Transfer Learning for Explainable Natural Language Inference with T5.
Konferenzveröffentlichung, Bibliographie
Kurzbeschreibung (Abstract)
We compare sequential fine-tuning with a model for multi-task learning in the context where we are interested in boosting performance on two of the tasks, one of which depends on the other. We test these models on the FigLang2022 shared task which requires participants to predict language inference labels on figurative language along with corresponding textual explanations of the inference predictions. Our results show that while sequential multi-task learning can be tuned to be good at the first of two target tasks, it performs less well on the second and additionally struggles with overfitting. Our findings show that simple sequential fine-tuning of text-to-text models is an extraordinarily powerful method of achieving cross-task knowledge transfer while simultaneously predicting multiple interdependent targets. So much so, that our best model achieved the (tied) highest score on the task.
| Typ des Eintrags: | Konferenzveröffentlichung |
|---|---|
| Erschienen: | 2022 |
| Autor(en): | Bigoulaeva, Irina ; Sachdeva, Rachneet ; Madabushi, Harish Tayyar ; Villavicencio, Aline ; Gurevych, Iryna |
| Art des Eintrags: | Bibliographie |
| Titel: | Effective Cross-Task Transfer Learning for Explainable Natural Language Inference with T5 |
| Sprache: | Englisch |
| Publikationsjahr: | Dezember 2022 |
| Ort: | Abu Dhabi, United Arab Emirates |
| Verlag: | Association for Computational Linguistics |
| Buchtitel: | Proceedings of the 3rd Workshop on Figurative Language Processing (FLP) |
| URL / URN: | https://aclanthology.org/2022.flp-1.8 |
| Kurzbeschreibung (Abstract): | We compare sequential fine-tuning with a model for multi-task learning in the context where we are interested in boosting performance on two of the tasks, one of which depends on the other. We test these models on the FigLang2022 shared task which requires participants to predict language inference labels on figurative language along with corresponding textual explanations of the inference predictions. Our results show that while sequential multi-task learning can be tuned to be good at the first of two target tasks, it performs less well on the second and additionally struggles with overfitting. Our findings show that simple sequential fine-tuning of text-to-text models is an extraordinarily powerful method of achieving cross-task knowledge transfer while simultaneously predicting multiple interdependent targets. So much so, that our best model achieved the (tied) highest score on the task. |
| Freie Schlagworte: | UKP_p_texprax, UKP_p_seditrah_factcheck |
| Fachbereich(e)/-gebiet(e): | 20 Fachbereich Informatik 20 Fachbereich Informatik > Ubiquitäre Wissensverarbeitung |
| Hinterlegungsdatum: | 01 Mär 2023 10:09 |
| Letzte Änderung: | 20 Jul 2023 10:03 |
| PPN: | 508938996 |
| Export: | |
| Suche nach Titel in: | TUfind oder in Google |
 |
Frage zum Eintrag |
Optionen (nur für Redakteure)
 |
Redaktionelle Details anzeigen |
Drucken
Impressum