Skip to main content
tech futures.
Introduction to AI
All lessons
Lesson Overview
Introduction
Under the Hood
LLMs and learning
The main steps in training
Gathering training data
Tokens and Context
Tokenisation
OpenAI's Tokeniser
Activity: Counting tokens
Training data in tokens
Vocabularies
Token IDs
Tokens in Lumen
Context
Activity: Context
Transformer Architecture
Computers and maths
Transformer Architecture
Simplified Transformer Architecture
Embedding Meanings
Human Word Map Activity
Embeddings in real life
Finding word connections
Where does the training come in?
Predicting the next token
How LLMs are Used
AI Chatbots: Software with Settings
Temperature
Activity: Changing Temperature
Reinforcement Learning from Human Feedback
Activity: Rating Lumen
Trade Secrets
Context Windows
Context Windows
Grounding
Activity: Grounding
Stateless AI
Stateless AI
Activity: Stateless AI
Reasoning Models and Understanding
Reasoning Models
LLMs and Understanding
Activity: Categorising LLMs vs Rule-Based
End of lesson quiz
Lesson 5: Under the Hood
Activity: Rating Lumen
Instructions
Download the worksheet below
Work through the activity in the worksheet, rating Lumen's responses.
Complete the questions in the worksheet
Google Docs
PDF
Word
Previous
: Reinforcement Learning from Human Feedback
Next
: Trade Secrets