TUTOR

M16 · Level III — AI Systems Engineering

Retrieval, RAG & Knowledge Systems

Engineer systems that retrieve external information and condition model computation on that information with measurable relevance, provenance and reliability.

Topics

  • Information
  • Index
  • Retrieval
  • Ranking
  • Context
  • Grounding
  • Generation
  • Provenance
  • Knowledge
  • Reliable System

Technical concepts

  • 01Information Needs Before Models
  • 02Inverted Index: The Classical Retrieval Primitive
  • 03TF-IDF and Term Discrimination
  • 04BM25: Saturation and Length Normalisation
  • 05Query Processing and Lexical Normalisation
  • 06Dense Retrieval: Embeddings as Search Representations
  • 07Exact Vector Search
  • 08Approximate Nearest Neighbour Retrieval
  • 09Hybrid Retrieval: Lexical + Dense Evidence
  • 10Metadata, Filters and Access Control
  • 11Chunking: The Retrieval Unit Is a Model Choice
  • 12Chunk Overlap and Boundary Effects
  • 13Structured Data and Tables in Knowledge Systems
  • 14Multimodal Retrieval
  • 15Query Rewriting and Expansion
  • 16Reranking: Spend More Compute on Fewer Candidates
  • 17Retrieval Score Calibration
  • 18Context Packing and Evidence Selection
  • 19Retrieval-Augmented Generation: Complete Pipeline
  • 20Grounding: Evidence Before Fluency
  • 21Provenance, Citations and Source Traceability
  • 22Freshness, Index Updates and Temporal Knowledge
  • 23Conflicting and Inconsistent Sources
  • 24Knowledge Graphs and Structured Relations
  • 25Hybrid Knowledge: Graph + Text + Vectors
  • 26Multimodal RAG
  • 27Query Routing and Retrieval Planning
  • 28Retrieval Security: Indirect Prompt Injection
  • 29Retrieval Poisoning and Corpus Integrity
  • 30RAG Evaluation: Retrieve, Ground, Answer
  • 31RAG Evaluation Dataset Design
  • 32Abstention and Insufficient Evidence
  • 33Answer Verification and Claim-Level Checking
  • 34Context Contamination and Irrelevant Evidence
  • 35Caching and Knowledge-System State
  • 36Observability and Provenance Logs
  • 37Retrieval vs Tools: Where Knowledge Systems End
  • 38Quantum Technology Bridge: Retrieval, Search and Knowledge

Practical implementation

  • 01Laboratory I: Search Engine from First Principles
  • 02Laboratory II: Exact Dense Retrieval
  • 03Laboratory III: ANN Recall-Latency Trade-off
  • 04Laboratory IV: Hybrid Retrieval and Reranking
  • 05Laboratory V: End-to-End RAG
  • 06Laboratory VI: Chunking and Context-Packing Study
  • 07Laboratory VII: Graph + Text Knowledge System
  • 08Laboratory VIII: RAG Security and Integrity

Tools and technologies

  • Linux VM/container
  • lexical and vector retrieval from first principles
  • hybrid search reranking
  • RAG
  • evaluation
  • security
  • reproducibility

Project

Auditable knowledge system + evaluation harness

Expected learning outcomes

  • 01Can implement and evaluate an inverted-index search system.
  • 02Can implement a dense retrieval baseline and an ANN trade-off study.
  • 03Can combine lexical and dense evidence and rerank candidates.
  • 04Can engineer chunking and context selection under a declared token budget.
  • 05Can build an auditable RAG pipeline with source provenance.
  • 06Can evaluate grounding, abstention and conflict handling.
  • 07Can enforce metadata and access-control boundaries before generation.
  • 08Can diagnose prompt-injection and corpus-poisoning risks.
  • 09Can reproduce a retrieval/RAG research result with documented deviations.

Assessment

  • 01Daily practice
  • 02Laboratories
  • 03Research reproduction
  • 04Integrated project
  • 05Technical viva
  • 06Engineering record

Source document: TUTOR Frontier Module 16 Retrieval RAG Knowledge Systems