Abstract: Vision-language models such as Contrastive Language-Image Pre-training (CLIP) learn joint representations of images and text from web-scale data, enabling zero-shot image classification. However, adapting these models to new domains with limited labelled data presents three challenges: existing adaptation methods are rarely strictly few-shot, simple text prompts fail to capture the fine-grained semantics, and current approaches discard spatial information from image encoders. First, we introduce the Hold-One-Shot-Out Adapter, a validation-free approach that reserves one example per class to tune the blending of prior and adapted features. This method improves on the CLIP-Adapter baseline by four percentage points on average across eleven datasets. Second, we investigate whether Large Language Models can generate side infor- mation that improves low-shot transfer. A systematic comparison reveals that visual descriptions reflect visual similarity better than dictionary definitions or taxonomic hierarchies. Our method improves low-shot accuracy by two percentage points on average across twelve datasets. Third, we examine whether aligning spatial image features with part-level text descriptions improves fine-grained image recognition. Experiments reveal that training-free visual prompting degrades accuracy, and trainable spatial adapters overfit to limited training data. Our analysis shows that distinct part descriptions can have only small angular separation in the CLIP text space, limiting the super-vision signal for local alignment. Fourth, we present PreLabellingProbe, a method that estimates dataset-level zero-shot performance using one labelled image per class. A Large Language Model is used to generate plausible and counterfactual captions, and the margin between similarity scores is used to estimate dataset-level accuracy with a Pearson correlation of 0.96. The thesis improves CLIP adaptation when labelled data are scarce and shows that current text encoders limit fine-grained transfer.