M03 · Level I — Computational Intelligence Foundations
Probability, Statistics & Information for Intelligent Systems
Build the mathematical language for uncertainty, inference, distributions, likelihood, information and evidence.
Topics
Technical concepts
- 01First Principles: Counting Before Probability
- 02Probability Axioms & Events
- 03Random Variables & Distributions
- 04Expected Value, Variance & Covariance
- 05Conditional Probability
- 06Bayes' Theorem: Inference by Reversal
- 07Total Probability & Total Expectation
- 08Independence and Dependence
- 09Sampling: How Data Enters the Model
- 10Likelihood: Data as Evidence About Parameters
- 11Maximum Likelihood Estimation (MLE)
- 12MAP: Combining Evidence with Prior Structure
- 13The Language Model Case: Why Can “WAST” Be Interpreted?
- 14Cross-Entropy: Measuring Distributional Mismatch
- 15Entropy: Quantifying Uncertainty
- 16KL Divergence
- 17Mutual Information & Information Flow
- 18Bayes Decision Theory
- 19From Probability to Classification
- 20Calibration & Reliability
- 21Confidence Intervals, Uncertainty & Bootstrap
- 22Hypothesis Testing: Evidence, Not Ritual
- 23Permutation Tests & Randomisation
- 24Monte Carlo Simulation
- 25Probability in Computer Vision
- 26Probability in Language Intelligence
- 27Statistical Failure Modes
Practical implementation
- 01Laboratory 01 — Bayes Inference Simulator
- 02Laboratory 02 — Language Inference from Corrupted Text
- 03Laboratory 03 — Statistical Inference on Image Data
Tools and technologies
Project
Laboratory 03 — Statistical Inference on Image Data
Expected learning outcomes
- 01Construct probability spaces and random variables from real phenomena.
- 02Use conditional probability and Bayes' theorem.
- 03Compute expectation, variance, covariance and correlation.
- 04Distinguish probability distributions and statistical models.
- 05Perform estimation using likelihood and posterior reasoning.
- 06Understand entropy, cross-entropy, KL divergence and mutual information.
- 07Reason about sampling, bias and uncertainty. Samp
- 08Interpret model predictions probabilistically.
- 09Apply probabilistic reasoning to language, vision and intelligent decisions.
Assessment
- 01Weight Requirement — 20%
- 02Evidence — 30%
- 03Daily practice — 10%
- 04Laboratories — 20%
- 05Research reproduction — 15%
- 06Integrated project — 5%
Source document: TUTOR Frontier Module 03 Probability Statistics Information for Intelligent Systems