TUTOR

M04 · Level I — Computational Intelligence Foundations

Optimization & Numerical Methods for Intelligent Systems

Understand how intelligent systems turn objectives and constraints into computable optimisation and numerical procedures.

Topics

  • First-Principles Optimization
  • Numerical Stability
  • Constrained Decisions
  • Learning Systems
  • Simulation

Technical concepts

  • 01First Principle: What Is an Objective?
  • 02Variables, Domains, Units & Scale
  • 03Derivatives from First Principles
  • 04Gradient, Jacobian & Hessian
  • 05Taylor Expansion as the Local Model
  • 06Gradient Descent from Scratch
  • 07Step Size, Stability & Line Search
  • 08Convexity & Global Geometry
  • 09Smoothness, Strong Convexity & Conditioning
  • 10Newton's Method
  • 11Quasi-Newton & Curvature Approximation
  • 12Constraints: Feasible Sets Before Optimality
  • 13Lagrange Multipliers
  • 14KKT Conditions
  • 15Penalties, Barriers & Regularisation
  • 16Floating-Point Arithmetic
  • 17Conditioning vs Stability
  • 18Root Finding: Bisection, Newton & Secant
  • 19Linear Systems & Least Squares
  • 20Numerical Integration & ODE Foundations
  • 21Automatic Differentiation
  • 22Stochastic Gradient Methods
  • 23Momentum, Adaptive Methods & Optimiser Families
  • 24Optimisation in Machine Learning
  • 25Optimisation in Vision & Language Systems
  • 26Research Transfer & Industry Practice

Practical implementation

  • 01Laboratory 01 - Optimisation Engine from First Principles
  • 02Laboratory 02 - Numerical Methods Benchmark
  • 03Laboratory 03 - ML Loss-Surface Investigation

Tools and technologies

  • Linux VM/container
  • numerical experiments
  • simulation-first

Project

Laboratory 03 - ML Loss-Surface Investigation

Expected learning outcomes

  • 01Translate an engineering objective into a mathematical optimisation problem.
  • 02Understand derivatives as local sensitivity.
  • 03Use gradient-based optimisation.
  • 04Explain curvature and second-order methods.
  • 05Handle constraints.
  • 06Understand numerical error.
  • 07Solve roots, linear systems and differential equations numerically.
  • 08Connect optimisation to machine learning.
  • 09Make evidence-based numerical decisions.

Assessment

  • 01Weight Requirement — 20%
  • 02Evidence — 30%
  • 03Daily practice — 10%
  • 04Laboratories — 20%
  • 05Research reproduction — 15%
  • 06Integrated project — 5%

Source document: TUTOR Frontier Module 04 Optimization Numerical Methods for Intelligent Systems