Attention as a Kernel: Uncovering Latent Geometry for Zero-Shot Transfer

This thesis studies whether the attention mechanism, seen as a kernel, can reveal latent geometric structure in frozen visual features to enable zero-shot transfer. Features from a frozen ResNet backbone pass through an inter-sample attention layer, and the resulting representations build a Gram kernel that feeds a multi-class SVM. We evaluate on CelebA and AwA2, including a semantic zero-shot variant using sentence-transformer embeddings of attribute names. The kernel exceeds the trained network on both CelebA (balanced accuracy 0.750) and AwA2 (0.925), with strong transfer results (0.737 and 0.886). Ablation studies show a nuanced picture: the attention output does not consistently beat the input projection, and raw frozen features often remain the strongest baseline.