Editing Foundation Models (section)

== <span style="color: #FFFFFF;">Remembering</span> ==
* '''Foundation model''' — A large model trained on broad data at scale, adapted to many downstream tasks. Term coined by Stanford HAI in 2021.
* '''Pre-training''' — The initial training phase on massive, diverse datasets that gives the model its general capabilities.
* '''Adaptation''' — Tailoring a foundation model for a specific task via fine-tuning, prompting, RLHF, or retrieval augmentation.
* '''Emergence''' — Capabilities that appear only at scale; not present in smaller versions of the same model.
* '''Transfer learning (foundation model sense)''' — Using the knowledge encoded in a foundation model's weights as a starting point for new tasks.
* '''Multimodal foundation model''' — A foundation model trained on multiple modalities (text, image, audio, video).
* '''GPT-4''' — OpenAI's large multimodal language foundation model.
* '''Gemini''' — Google DeepMind's natively multimodal foundation model family.
* '''CLIP''' — OpenAI's vision-language foundation model; enables zero-shot image classification.
* '''DALL-E 3''' — OpenAI's text-to-image foundation model.
* '''Llama 3''' — Meta's open-weight language foundation model family.
* '''Mistral''' — Efficient open-weight language foundation models.
* '''ESM-2''' — A protein language foundation model trained on evolutionary sequence data.
* '''SAM (Segment Anything Model)''' — Meta's vision foundation model for image segmentation.
* '''Homogenization risk''' — The risk that widespread use of a small number of foundation models spreads shared flaws, biases, and failure modes.
</div>

<div style="background-color: #006400; color: #FFFFFF; padding: 20px; border-radius: 8px; margin-bottom: 15px;">