M04 · Level I — Computational Intelligence Foundations
Optimization & Numerical Methods for Intelligent Systems
Understand how intelligent systems turn objectives and constraints into computable optimisation and numerical procedures.
Topics
Technical concepts
- 01First Principle: What Is an Objective?
- 02Variables, Domains, Units & Scale
- 03Derivatives from First Principles
- 04Gradient, Jacobian & Hessian
- 05Taylor Expansion as the Local Model
- 06Gradient Descent from Scratch
- 07Step Size, Stability & Line Search
- 08Convexity & Global Geometry
- 09Smoothness, Strong Convexity & Conditioning
- 10Newton's Method
- 11Quasi-Newton & Curvature Approximation
- 12Constraints: Feasible Sets Before Optimality
- 13Lagrange Multipliers
- 14KKT Conditions
- 15Penalties, Barriers & Regularisation
- 16Floating-Point Arithmetic
- 17Conditioning vs Stability
- 18Root Finding: Bisection, Newton & Secant
- 19Linear Systems & Least Squares
- 20Numerical Integration & ODE Foundations
- 21Automatic Differentiation
- 22Stochastic Gradient Methods
- 23Momentum, Adaptive Methods & Optimiser Families
- 24Optimisation in Machine Learning
- 25Optimisation in Vision & Language Systems
- 26Research Transfer & Industry Practice
Practical implementation
- 01Laboratory 01 - Optimisation Engine from First Principles
- 02Laboratory 02 - Numerical Methods Benchmark
- 03Laboratory 03 - ML Loss-Surface Investigation
Tools and technologies
Project
Laboratory 03 - ML Loss-Surface Investigation
Expected learning outcomes
- 01Translate an engineering objective into a mathematical optimisation problem.
- 02Understand derivatives as local sensitivity.
- 03Use gradient-based optimisation.
- 04Explain curvature and second-order methods.
- 05Handle constraints.
- 06Understand numerical error.
- 07Solve roots, linear systems and differential equations numerically.
- 08Connect optimisation to machine learning.
- 09Make evidence-based numerical decisions.
Assessment
- 01Weight Requirement — 20%
- 02Evidence — 30%
- 03Daily practice — 10%
- 04Laboratories — 20%
- 05Research reproduction — 15%
- 06Integrated project — 5%
Source document: TUTOR Frontier Module 04 Optimization Numerical Methods for Intelligent Systems