The classification of major depressive disorders (MDDs) is a challenging task in clinical practice, especially in low-resource scenarios where generalization is essential for effective adaptation. Recent progress in meta-training large language models (LLMs) via in-context learning (ICL) offers promise for robust adaptation to unseen tasks without parameter updates. However, existing methods rely on multitask fine-tuning and do not fully exploit the optimization advantages of model-agnostic meta learning (MAML) techniques, limiting their generalization. This study proposes prompt-MAML, a novel method for meta-training LLMs that enhances multimodal ICL for classifying MDD tasks. The method integrates audio-textual features through a transformer-based cross-modal alignment module and incorporates bi-level optimization to learn generalizable model parameters that adapt well to unseen tasks. Extensive experiments demonstrate that prompt-MAML outperforms strong baseline models by an average improvement in macro-F1 of +4 on seen domains, +3 on unseen domains, and +3 in few-shot settings, demonstrating robustness and effectiveness in data-scarce and cross-domain clinical scenarios. Additionally, exploration depth is shown to play a key role in task performance, and further analysis of task complexity, modality, and optimiser configurations highlights critical design considerations for meta-training LLMs.