Eger, Steffen ; Rücklé, Andreas ; Gurevych, Iryna (2019)
Pitfalls in the Evaluation of Sentence Embeddings.
The 4th Workshop on Representation Learning for NLP (Repl4NLP 2019). Florence, Italy (02.08.2019-02.08.2019)
doi: 10.18653/v1/W19-4308
Konferenzveröffentlichung, Bibliographie
Dies ist die neueste Version dieses Eintrags.
Kurzbeschreibung (Abstract)
Deep learning models continuously break new records across different NLP tasks. At the same time, their success exposes weaknesses of model evaluation. Here, we compile several key pitfalls of evaluation of sentence embeddings, a currently very popular NLP paradigm. These pitfalls include the comparison of embeddings of different sizes, normalization of embeddings, and the low (and diverging) correlations between transfer and probing tasks. Our motivation is to challenge the current evaluation of sentence embeddings and to provide an easy-to-access reference for future research. Based on our insights, we also recommend better practices for better future evaluations of sentence embeddings.
| Typ des Eintrags: | Konferenzveröffentlichung |
|---|---|
| Erschienen: | 2019 |
| Autor(en): | Eger, Steffen ; Rücklé, Andreas ; Gurevych, Iryna |
| Art des Eintrags: | Bibliographie |
| Titel: | Pitfalls in the Evaluation of Sentence Embeddings |
| Sprache: | Englisch |
| Publikationsjahr: | 5 Juni 2019 |
| Ort: | Florence, Italy |
| Verlag: | Association for Computational Linguistics |
| Buchtitel: | Proceedings of the 4th Workshop on Representation Learning for NLP (RepL4NLP-2019) |
| Veranstaltungstitel: | The 4th Workshop on Representation Learning for NLP (Repl4NLP 2019) |
| Veranstaltungsort: | Florence, Italy |
| Veranstaltungsdatum: | 02.08.2019-02.08.2019 |
| DOI: | 10.18653/v1/W19-4308 |
| URL / URN: | https://www.aclweb.org/anthology/W19-4308 |
| Zugehörige Links: | |
| Kurzbeschreibung (Abstract): | Deep learning models continuously break new records across different NLP tasks. At the same time, their success exposes weaknesses of model evaluation. Here, we compile several key pitfalls of evaluation of sentence embeddings, a currently very popular NLP paradigm. These pitfalls include the comparison of embeddings of different sizes, normalization of embeddings, and the low (and diverging) correlations between transfer and probing tasks. Our motivation is to challenge the current evaluation of sentence embeddings and to provide an easy-to-access reference for future research. Based on our insights, we also recommend better practices for better future evaluations of sentence embeddings. |
| Fachbereich(e)/-gebiet(e): | 20 Fachbereich Informatik 20 Fachbereich Informatik > Ubiquitäre Wissensverarbeitung DFG-Graduiertenkollegs DFG-Graduiertenkollegs > Graduiertenkolleg 1994 Adaptive Informationsaufbereitung aus heterogenen Quellen |
| Hinterlegungsdatum: | 28 Jun 2019 06:42 |
| Letzte Änderung: | 05 Jun 2024 07:37 |
| PPN: | |
| Export: | |
| Suche nach Titel in: | TUfind oder in Google |
Verfügbare Versionen dieses Eintrags
-
Pitfalls in the Evaluation of Sentence Embeddings. (deposited 06 Jun 2019 08:26)
- Pitfalls in the Evaluation of Sentence Embeddings. (deposited 28 Jun 2019 06:42) [Gegenwärtig angezeigt]
 |
Frage zum Eintrag |
Optionen (nur für Redakteure)
 |
Redaktionelle Details anzeigen |
Drucken
Impressum