M09 · Level II — Machine Intelligence
Neural Networks from First Principles
Derive neural networks as compositional mathematical systems, then engineer reliable training, diagnostics and deployment.
Topics
Technical concepts
- 01First Principle: What a Neural Network Computes
- 02The Affine Map: The Core Computational Primitive
- 03Scalar Neuron, Perceptron and Decision Geometry
- 04Activation Functions as Controlled Nonlinearity
- 05Multilayer Composition and Representation
- 06Expressivity and the Universal Approximation Idea
- 07Computational Graphs: The Language of Backpropagation
- 08The Chain Rule: Local Sensitivity Becomes Global Sensitivity
- 09Reverse-Mode Automatic Differentiation and Backpropagation
- 10Loss Functions Turn Behaviour into a Learning Signal
- 11Gradient Descent as Parameter Dynamics
- 12Initialisation and Symmetry Breaking
- 13Normalisation and Internal Scale Control
- 14Regularisation: Constraining Learned Solutions
- 15Training Dynamics: What Changes During Learning?
- 16Numerical Stability and Finite Precision
- 17Minibatches, Stochasticity and Throughput
- 18Capacity, Underfitting and Overfitting
- 19Learned Representations
- 20Vision Bridge: From Pixels to Feature Maps
- 21Language and Sequence Bridge
- 22Transformer Bridge: The Neural Core Before Attention
- 23Implementation I: Neural Network in NumPy
- 24Implementation II: Framework Mapping with PyTorch
- 25Gradient Checking and Verification
- 26The Reproducible Training Harness
- 27Failure Lab I: Dead or Saturated Activations
- 28Failure Lab II: Vanishing and Exploding Gradients
- 29Failure Lab III: When the Network Is Not the Problem
- 30Controlled Neural-Network Experiments
- 31Interpretability and Diagnostic Probes
- 32Inference: The Trained Network as a Runtime System
- 33Quantum Technology Bridge: Where It May Intersect with Neural Systems
Practical implementation
- 01Laboratory Matrix
Tools and technologies
Project
Laboratory Matrix
Expected learning outcomes
- 01By completion, the learner should be capable of writing a neural network from first principles, proving the gradient for a small graph, detecting numerical pathologies, comparing training interventions under controlled experiments, and translating the same principles into vision and language settings.
- 02Every implementation must be traceable to a mathematical statement, measurable objective, gradient pathway and failure mode.
Assessment
- 01Daily practice
- 02Laboratories
- 03Research reproduction
- 04Integrated project
- 05Technical viva
- 06Engineering record
Source document: TUTOR Frontier Module 09 Neural Networks from First Principles