TUTOR

M13 · Level II — Machine Intelligence

Language & Speech Intelligence

Derive language and speech intelligence from physical signals and symbolic structure, then build robust sequence, speech and multilingual systems.

Topics

  • Sound
  • Signal
  • Features
  • Symbols
  • Language
  • Context
  • Speech
  • Multilingual
  • Multimodal Intelligence

Technical concepts

  • 01Language as Structured Symbolic Data
  • 02Tokenisation as a Computational Interface
  • 03n-Gram Language Models
  • 04Cross-Entropy, Entropy and Perplexity
  • 05Embeddings and Distributional Representation
  • 06Syntax, Composition and Long-Range Dependency
  • 07Meaning, Context and Pragmatic Information
  • 08Speech as an Acoustic Signal
  • 09Speech Sampling, Quantisation and Aliasing
  • 10Framing and Short-Time Analysis
  • 11Fourier Analysis and the Short-Time Fourier Transform
  • 12Mel-Scale Features and MFCC-Style Representation
  • 13Phonetics, Phonology and Sub-Word Speech Structure
  • 14Language Models Inside Speech Recognition
  • 15Connectionist Temporal Classification
  • 16RNN-T / Transducer-Style Sequence Alignment
  • 17Attention-Based Encoder-Decoder Speech Recognition
  • 18Speech Decoding and Beam Search
  • 19ASR Evaluation and Error Decomposition
  • 20Text-to-Speech as Inverse Perception
  • 21Prosody, Timing and Expressive Speech
  • 22Speaker and Paralinguistic Information
  • 23Multilingual Language Intelligence
  • 24Multilingual Speech and Code-Switching
  • 25Text Normalisation and Evaluation Interfaces
  • 26Language Representations as a Knowledge-System Interface
  • 27Language and Speech Robustness
  • 28Generative Language Behaviour and Hallucination as an Evaluation Problem
  • 29Quantum Technology Bridge: Language, Speech and Hybrid Computation

Practical implementation

  • 01Laboratory I: Language Model from First Principles
  • 02Laboratory II: Speech Signal Processing Pipeline
  • 03Laboratory III: CTC Alignment Engine
  • 04Laboratory IV: Tiny ASR System
  • 05Laboratory V: Compact Text-to-Speech Chain
  • 06Laboratory VI: Multilingual Representation Study
  • 07Laboratory VII: Speech Robustness Matrix
  • 08Laboratory VIII: Speech-Text Alignment

Tools and technologies

  • Linux VM/container
  • signal processing labs
  • scratch sequence models
  • transformer and speech systems
  • reproducible evaluation

Project

End-to-end language, speech or audio-text system

Expected learning outcomes

  • 01Can derive tokenisation and probability interfaces.
  • 02Can implement core speech signal-processing operations.
  • 03Can explain and implement an alignment objective such as CTC on a reduced problem.
  • 04Can build and evaluate a small language model and ASR system.
  • 05Can analyse decoding and edit-distance error structure.
  • 06Can build a constrained speech-synthesis or audio-text alignment experiment.
  • 07Can evaluate multilingual, code-switching and noise robustness.
  • 08Can reproduce a language/speech research result with documented deviations.
  • 09Can defend all major design choices in a technical viva.

Assessment

  • 01Daily practice
  • 02Laboratories
  • 03Research reproduction
  • 04Integrated project
  • 05Technical viva
  • 06Engineering record

Source document: TUTOR Frontier Module 13 Language and Speech Intelligence