TUTOR

M19 · Level III — AI Systems Engineering

ML Systems, MLOps & Inference

Engineer the operational life of intelligent models: packaging, serving, inference, observability, deployment, scaling, updates, rollback and production evidence.

Topics

  • Model
  • Artifact
  • Runtime
  • Serving
  • Observability
  • Scaling
  • Update
  • Rollback
  • Evidence

Technical concepts

  • 01The Model Artifact
  • 02Training-Serving Parity
  • 03Deterministic and Reproducible Inference
  • 04Inference APIs as Typed Contracts
  • 05Serving Architectures
  • 06Latency as a Distribution
  • 07Throughput and Saturation
  • 08Dynamic Batching
  • 09Inference Caching
  • 10Memory as a Production Constraint
  • 11Numerical Precision at Inference
  • 12Model Registry and Provenance
  • 13Reproducible Runtime Environments
  • 14ML CI/CD Pipeline
  • 15Offline/Online Parity
  • 16Deployment Strategies
  • 17Rollback Engineering
  • 18Capacity and Autoscaling
  • 19Backpressure and Load Shedding
  • 20Observability: Logs, Metrics and Traces
  • 21Service Level Objectives for ML Systems
  • 22Drift and Change Detection
  • 23Online Features and Data Dependencies
  • 24Distributed Inference
  • 25Queueing Reasoning for Inference Services
  • 26Cost and Resource Efficiency
  • 27Configuration, Secrets and Runtime State
  • 28Controlled Change with Feature Flags
  • 29Load Testing
  • 30Soak Testing and Long-Run Stability
  • 31Production Fault Injection
  • 32ML Runtime Security
  • 33Model Supply Chain and Integrity
  • 34Inference Optimisation Before M20

Practical implementation

  • 01Laboratory I: Observable Inference Service
  • 02Laboratory II: Latency/Throughput Benchmark
  • 03Laboratory III: Dynamic Batching Study
  • 04Laboratory IV: Cache Correctness and Performance
  • 05Laboratory V: Canary Deployment Simulator
  • 06Laboratory VI: Shadow Evaluation
  • 07Laboratory VII: Autoscaling and Backpressure
  • 08Laboratory VIII: Rollback and Recovery Drill
  • 09Laboratory IX: ML Production Incident Analysis

Tools and technologies

  • Linux VM/container
  • local and cloud serving
  • load generation
  • profiling
  • model registry simulation
  • canary/shadow
  • rollback drills

Project

End-to-end deployed service + evidence + rollback + monitoring

Expected learning outcomes

  • 01Can package and reproduce a model artifact.
  • 02Can define typed inference contracts.
  • 03Can measure latency, throughput, resource use and saturation.
  • 04Can reason about batching, caching and memory.
  • 05Can deploy through controlled release strategies.
  • 06Can instrument logs, metrics and traces.
  • 07Can detect offline/online drift or parity failures.
  • 08Can enforce backpressure and resource boundaries.
  • 09Can execute and verify a rollback.
  • 10Can investigate an ML production incident from evidence.

Assessment

  • 01Daily practice
  • 02Laboratories
  • 03Research reproduction
  • 04Integrated project
  • 05Technical viva
  • 06Engineering record

Source document: TUTOR Frontier Module 19 ML Systems MLOps Inference