TUTOR

M12 · Level II — Machine Intelligence

Computer Vision & Multimodal Perception

Derive visual perception from physical sensing and signal representation, then engineer deep vision and multimodal systems.

Topics

  • Scene
  • Light
  • Sensor
  • Sampled Signal
  • Tensor
  • Features
  • Perception
  • Geometry
  • Multimodal State

Technical concepts

  • 01Image Formation: Scene -> Light -> Sensor
  • 02Sampling, Quantisation and Aliasing
  • 03Images as Arrays, Tensors and Coordinate Systems
  • 04Filtering: Local Structure Before Learning
  • 05Convolution and Cross-Correlation
  • 06Receptive Fields and Hierarchical Features
  • 07Convolutional Neural Networks
  • 08Pooling, Striding and Downsampling
  • 09Vision Data Normalisation and Augmentation
  • 10Visual Classification
  • 11Object Detection as Localisation + Recognition
  • 12Matching, IoU and Non-Maximum Suppression
  • 13Semantic Segmentation
  • 14Instance and Panoptic Perception
  • 15Multi-Scale Vision and Feature Pyramids
  • 16Monocular Depth and Geometry
  • 17Stereo Geometry
  • 18Motion and Optical Flow
  • 19Point Clouds and 3D Representation
  • 20Vision Transformers
  • 21CNN vs Vision Transformer: Inductive Bias
  • 22Self-Supervised Visual Representation Learning
  • 23Multimodal Perception: Shared State Across Modalities
  • 24Image-Text Alignment
  • 25Cross-Modal Attention and Grounding
  • 26Document Vision and OCR as Perception
  • 27Video Perception
  • 28Uncertainty and Ambiguity in Visual Perception
  • 29Vision Robustness and Distribution Shift
  • 30Perturbation, Adversarial Sensitivity and Failure Surfaces
  • 31Visual Attribution and Explanatory Probes
  • 32Multimodal Failure: Alignment, Dominance and Shortcutting
  • 33Quantum Technology Bridge: Vision, Kernels and Hybrid Perception

Practical implementation

  • 01Laboratory I: Vision Operators from First Principles
  • 02Laboratory II: CNN from Primitive Tensor Operations
  • 03Laboratory III: Detection and Segmentation
  • 04Laboratory IV: Stereo and Depth
  • 05Laboratory V: Image-Text Retrieval and Grounding
  • 06Laboratory VI: Vision Robustness Matrix

Tools and technologies

  • Linux VM/container
  • synthetic and public datasets
  • first-principles operators
  • framework models
  • reproducible evaluation

Project

End-to-end perception or multimodal system

Expected learning outcomes

  • 01Can explain image formation, sampling and quantisation.
  • 02Can implement basic vision operators without high-level black boxes.
  • 03Can derive and audit CNN tensor transformations.
  • 04Can evaluate detection and segmentation with task-appropriate metrics.
  • 05Can reconstruct simple 3D geometry from stereo or depth.
  • 06Can implement a small vision transformer and compare it with a CNN.
  • 07Can build a measurable image-text alignment system.
  • 08Can diagnose robustness, shift and multimodal shortcut failures.
  • 09Can reproduce a vision/multimodal research claim with documented deviations.

Assessment

  • 01Daily practice
  • 02Laboratories
  • 03Research reproduction
  • 04Integrated project
  • 05Technical viva
  • 06Engineering record

Source document: TUTOR Frontier Module 12 Computer Vision Multimodal Perception