TUTOR

M06 · Level I — Computational Intelligence Foundations

Data Engineering & Representation for Intelligent Systems

Understand how raw observations become trustworthy computational representations suitable for inference, learning, simulation and decision systems.

Topics

  • Observation
  • Representation
  • Quality
  • Provenance
  • Sampling
  • Features
  • Pipelines
  • Learning Systems

Technical concepts

  • 01First Principle: What Is Data?
  • 02Schema: Types, Keys, Units & Constraints
  • 03Data Types Are Semantic Commitments
  • 04Missing Data
  • 05Outliers, Noise & Anomalies
  • 06Duplicates, Identity & Entity Resolution
  • 07Sampling & Coverage
  • 08Train / Validation / Test Separation
  • 09Data Leakage: The Hidden Failure
  • 10Normalisation, Standardisation & Scaling
  • 11Categorical Representation
  • 12Text Representation
  • 13Image Representation
  • 14Time-Series Representation
  • 15Graph Representation
  • 16Data Transformation as a Directed Graph
  • 17SQL as a Language of Data Reasoning
  • 18Columnar & Analytical Data Systems
  • 19Data Validation & Contracts
  • 20Data Lineage & Provenance
  • 21Data Versioning & Dataset Releases
  • 22Pipeline Determinism & Idempotence
  • 23Research Transfer: Data as the Input Interface of Intelligence
  • 24Industry Practice & Data Decision Record

Practical implementation

  • 01Laboratory 01 - Data Quality Engine
  • 02Laboratory 02 - Leakage-Proof Feature Pipeline
  • 03Laboratory 03 - Multi-Modal Representation Notebook
  • 04Laboratory 04 - Data Pipeline Reproducibility

Tools and technologies

  • Linux VM/container
  • Python/SQL
  • reproducible datasets
  • data pipelines
  • simulation-first

Project

Laboratory 04 - Data Pipeline Reproducibility

Expected learning outcomes

  • 01Explain the data-generating process behind a dataset.
  • 02Design schemas and data contracts.
  • 03Detect and diagnose data-quality failures.
  • 04Reason about missingness and sampling.
  • 05Prevent data leakage.
  • 06Transform and represent data appropriately.
  • 07Work across tabular, temporal, text, image and Re graph data.
  • 08Build reproducible data pipelines.
  • 09Connect representations to intelligent-system behaviour.

Assessment

  • 01Daily practice — 20%
  • 02Laboratories — 30%
  • 03Research reproduction — 10%
  • 04Integrated project — 20%
  • 05Technical viva — 15%
  • 06Engineering record — 5%

Source document: TUTOR Frontier Module 06 Data Engineering Representation for Intelligent Systems