TU Darmstadt / ULB / TUbiblio

Concatenated Power Mean Word Embeddings as Universal Cross-Lingual Sentence Representations

Rücklé, Andreas and Eger, Steffen and Peyrard, Maxime and Gurevych, Iryna :
Concatenated Power Mean Word Embeddings as Universal Cross-Lingual Sentence Representations.
[Online-Edition: https://arxiv.org/abs/1803.01400]
In: arXiv:1803.01400
[Article] , (2018)

Official URL: https://arxiv.org/abs/1803.01400

Abstract

Average word embeddings are a common baseline for more sophisticated sentence embedding techniques. However, they typically fall short of the performances of more complex models such as InferSent. Here, we generalize the concept of average word embeddings to power mean word embeddings. We show that the concatenation of different types of power mean word embeddings considerably closes the gap to state-of-the-art methods monolingually and substantially outperforms these more complex techniques cross-lingually. In addition, our proposed method outperforms different recently proposed baselines such as SIF and Sent2Vec by a solid margin, thus constituting a much harder-to-beat monolingual baseline.

Item Type: Article
Erschienen: 2018
Creators: Rücklé, Andreas and Eger, Steffen and Peyrard, Maxime and Gurevych, Iryna
Title: Concatenated Power Mean Word Embeddings as Universal Cross-Lingual Sentence Representations
Language: English
Abstract:

Average word embeddings are a common baseline for more sophisticated sentence embedding techniques. However, they typically fall short of the performances of more complex models such as InferSent. Here, we generalize the concept of average word embeddings to power mean word embeddings. We show that the concatenation of different types of power mean word embeddings considerably closes the gap to state-of-the-art methods monolingually and substantially outperforms these more complex techniques cross-lingually. In addition, our proposed method outperforms different recently proposed baselines such as SIF and Sent2Vec by a solid margin, thus constituting a much harder-to-beat monolingual baseline.

Journal or Publication Title: arXiv:1803.01400
Uncontrolled Keywords: UKP_p_QAEduInf;AIPHES_area_b2
Divisions: Department of Computer Science
Department of Computer Science > Ubiquitous Knowledge Processing
DFG-Graduiertenkollegs
DFG-Graduiertenkollegs > Research Training Group 1994 Adaptive Preparation of Information from Heterogeneous Sources
Date Deposited: 06 Mar 2018 08:34
Official URL: https://arxiv.org/abs/1803.01400
Identification Number: TUD-CS-2018-0050
Related URLs:
Export:

Optionen (nur für Redakteure)

View Item View Item