M15 · Level III — AI Systems Engineering
Foundation Models & Fine-Tuning
Understand how broad pretrained models are formed and how they are adapted to specialised tasks, domains, modalities and system requirements.
Topics
Technical concepts
- 01The Foundation-Model Paradigm
- 02Transfer Learning as a Statistical Assumption
- 03Choosing the Adaptation Boundary
- 04Domain Adaptation and Distribution Shift
- 05Catastrophic Forgetting
- 06Supervised Fine-Tuning
- 07Instruction Tuning and Task Conditioning
- 08Training-Data Curation as Model Engineering
- 09Parameter-Efficient Fine-Tuning
- 10Low-Rank Adaptation: Mathematical Derivation
- 11Low-Rank Updates as Constrained Optimisation
- 12Freezing Policies and Layer-Wise Adaptation
- 13Fine-Tuning Learning Rates and Update Scale
- 14Regularisation During Adaptation
- 15Instruction Dataset Design
- 16Preference Data and Relative Feedback
- 17Preference Optimisation: Conceptual Frameworks
- 18Alignment as an Objective-Design Problem
- 19Evaluating Fine-Tuned Models
- 20Benchmark Leakage and Validation Contamination
- 21Multi-Task and Mixture Fine-Tuning
- 22Continual and Sequential Adaptation Bridge
- 23Quantisation-Aware Adaptation and Low-Bit Deployment Bridge
- 24Multimodal Foundation Models and Adaptation
- 25Foundation Models as Components of Agentic Systems
- 26Security of Adapted Models
- 27Quantum Technology Bridge: Foundation Models and Hybrid Adaptation
Practical implementation
- 01Laboratory I: Baseline Transfer and Fine-Tuning Matrix
- 02Laboratory II: Scratch Low-Rank Adaptation
- 03Laboratory III: Instruction-Data Curation Engine
- 04Laboratory IV: Supervised Fine-Tuning
- 05Laboratory V: Instruction-Tuning Mixture Study
- 06Laboratory VI: Catastrophic Forgetting and Retention
- 07Laboratory VII: Preference-Data Adaptation
- 08Laboratory VIII: Low-Bit Adaptation Study
- 09Laboratory IX: Foundation-Model Evaluation Harness
- 10Laboratory X: Multimodal Adaptation
Tools and technologies
Project
Adapted model + evaluation harness + release packet
Expected learning outcomes
- 01Can define a foundation-model adaptation problem and boundary.
- 02Can compare frozen, partial and full adaptation.
- 03Can derive and implement a low-rank adaptation mechanism.
- 04Can curate and audit instruction-tuning data.
- 05Can quantify catastrophic forgetting and test a mitigation.
- 06Can evaluate adapted models beyond the training objective.
- 07Can audit benchmark contamination and reproducibility.
- 08Can measure resource cost and checkpoint state.
- 09Can defend adaptation choices and limitations in a technical viva.
Assessment
- 01Daily practice
- 02Laboratories
- 03Research reproduction
- 04Integrated project
- 05Technical viva
- 06Engineering record
Source document: TUTOR Frontier Module 15 Foundation Models Fine Tuning