The idea in plain language.
Two noisy views of the same sentence can teach a model what should remain similar.
How it works
In the unsupervised version, the same sentence passes through a language model twice with different dropout noise. Those representations form a positive pair, while other sentences provide negatives. A supervised version uses entailment and contradiction labels to define useful pairs. The objective aims to make semantically related sentences close while improving the distribution of embeddings.
What to keep in mind
Semantic-similarity benchmarks measure a particular capability. Good sentence embeddings do not guarantee factual correctness or reliable reasoning.
Source: SimCSE: Simple Contrastive Learning of Sentence Embeddings. The original manuscript contains the methods, experiments, figures, and references. An arXiv posting date may follow an earlier conference publication. Read the linked record for version history.