# AI Alignment — Syllabus In this course we will: * Develop a benchmark to track and improve model and prompt performance over time * Use moderation models to evaluate and score harmful AI outputs * Train a model towards or away from specific values * Prompt engineer a model towards or away from specific values * Create a values-evaluation model through self-consistency **Recommended Prerequisites:** - A laptop or desktop computer with a web browser - Able to join a video call through our online campus (runs in any modern browser) - You can read and write Python - You know how large language models work (tokens, inference, prompting) - Interest in AI ethics, safety, and alignment — no prior philosophy background needed - **Accounts you'll need:** OpenAI API key (pay-as-you-go) - **Helpful but not required:** Completed [[Using Large Language Models Syllabus]] or [[Prompt Engineering Syllabus]] **New to LLMs?** Start with [[../../../Using Large Language Models/Using Large Language Models Syllabus|Using Large Language Models]] to build your foundation. In this course we will: * Develop a benchmark to track and improve model and prompt performance over time * Use moderation models to evaluate and score harmful AI outputs * Train a model towards or away from specific values * Prompt engineer a model towards or away from specific values * Create a values-evaluation model through self-consistency ## Session recording — Friday 25 September 2026 <p><a href="https://drive.google.com/file/d/1y5CRkEAB_W5ipAGefhe_ialUaHjm4vcj/view">▶︎ The full day · Fri 25 Sep 2026</a></p> ## [Discussion Group 1 - Tokens, Values, and Ethics](/3ocRTCkMSverocAsXFeMcg) - Discuss and develop your own values and express them with minimal tokens ## [Discussion Group 2 - Reward Misspecification](#) - Review and discuss the Reward Misspecification problem - Review and discuss the Reward Hacking problem ## [Discussion Group 3 - Scalable Oversight](#) - Review and discuss the Scalable Oversight Problem - Lecture on the geometric problem of oversight ## [Discussion Group 4 - Secrets, Lies, and Hallucinations](#) - Discuss methods for reducing AI Hallucination - Discuss practices to ensure LLMs can keep secrets - Detection of Sleeper Agents ## [Workshop 1 - Values and Ethics Benchmarking](#) * Develop a benchmark to track and improve model and prompt performance over time * Use moderation models to evaluate and score harmful AI outputs ## [Workshop 2 - Values Prompts and Fine-Tunes](#) * Prompt engineer a model towards or away from specific values * Train a model towards or away from specific values ## [Workshop 3 - Values-Estimation Classification](#) * Create a values-evaluation model through self-consistency prompt engineering * Create a values-aligned model through synthetic data generated with self-consistency ## More recordings for this class - [gather-recording-2025-06-25T17-09-25](https://www.loom.com/share/5e97b4a317bf4fba982d4dc8495d61a8) — Detect AI-generated text using statistical analysis and modify it to bypass automated checkers - [Understanding Chat GPT for Prompting and Reasoning](https://www.loom.com/share/e9ce7fbd1e8f40cf831810c339c4e10f) — Elicit structured, parsable text from language models using iterative prompt refinement