Asymmetric Uncertainty in Vision-Language Models
Vision-language models like CLIP embed every image and caption as a single point on a unit hypersphere. We adapt them after training so that text becomes a distribution on the sphere while images stay points.
Read Exploiting the Asymmetric Uncertainty Structure of Pre-trained VLMs on the Unit Hypersphere ↗