Transcription of Sentence-BERT: Sentence Embeddings using Siamese BERT …
{{id}} {{{paragraph}}}
Example: stock market
Sentence-BERT: Sentence Embeddings using Siamese BERT …
Sentence -BERT: Sentence Embeddings using Siamese BERT-NetworksNils Reimers and Iryna GurevychUbiquitous Knowledge Processing Lab (UKP-TUDA)Department of Computer Science, Technische Universit at (Devlin et al., 2018) and RoBERTa (Liuet al., 2019) has set a new state-of-the-artperformance on Sentence -pair regression taskslike semantic textual similarity (STS). How-ever, it requires that both sentences are fedinto the network, which causes a massive com-putational overhead: Finding the most sim-ilar pair in a collection of 10,000 sentencesrequires about 50 million inference computa-tions (~65 hours) with BERT. The constructionof BERT makes it unsuitable for semantic sim-ilarity search as well as for unsupervised taskslike this publication, we present Sentence -BERT(SBERT), a modification of the pretrainedBERT network that use Siamese and triplet net-work structures to derive semantically mean-ingful Sentence Embeddings that can be com-pared using cosine-similarity.
2017). RoBERTa (Liu et al.,2019) showed, that the performance of BERT can further improved by small adaptations to the pre-training process. We also tested XLNet (Yang et al.,2019), but it led in general to worse results than BERT. A large disadvantage of the BERT network structure is that no independent sentence embed-
Loading..
Tags:
Information
Domain:
Source:
Link to this page:
Please notify us if you found a problem with this document: