Courses

AI Safety, Ethics and Society curriculum

Covers a wide range of risks, including loss of control, misalignment, malicious use, accidents, and enfeeblement, while leveraging concepts and frameworks from existing research fields to analyze AI safety.

Length

9 weeks

Category

Introductory

Organizer

Center for AI Safety (CAIS)

Intro to Transformative AI

Intensive 5-day course from AI Safety Fundamentals (AISF) helping people fast-track their understanding of AI’s impact on society and start taking their first steps in making a contribution to AI safety.

Length

5 days

Category

Introductory

Organizer

BlueDot Impact

AI Safety Atlas

Textbook used by various training programs systematically mapping the landscape of AI safety knowledge. Ideas are laid out in a carefully structured narrative where concepts build naturally on previous ones.

Length

–

Category

Technical Alignment, Governance

Organizer

Centre pour la Sécurité de l'IA (CeSIA)

Agent Foundations for Superintelligence-Robust Alignment

A guide to the cluster of thought which expects solving the alignment problem in a way which scales to superintelligence to require well-specified proposals guided by strong theoretical understanding.

Length

6 weeks

Category

Technical Alignment

Organizer

Alignment Ecosystem Development (AED)

AI Alignment Fast-Track

Intensive 5-day course from AI Safety Fundamentals (AISF) designed to accelerate your understanding of key concepts in AI alignment.

Length

5 days

Category

Introductory, Technical Alignment

Organizer

BlueDot Impact

AI Governance Fast-Track

Intensive 5-day course from AI Safety Fundamentals (AISF) designed to accelerate your understanding of key concepts in AI governance and policy.

Length

5 days

Category

Introductory, Governance

Organizer

BlueDot Impact

Future of AI Course

Concise, engaging introduction to AI and its potential impacts that can be completed in just two hours. Good entry point to share with people curious about AI.

Length

2 hours

Category

Introductory

Organizer

BlueDot Impact

Introduction to ML Safety

PhD-level survey course on technical ML safety. Covers various technical topics to reduce existential risks from AI, including robustness, monitoring, control, and systemic safety.

Length

40 hours

Category

Introductory, Technical Alignment

Organizer

Center for AI Safety (CAIS)

Introduction to AI Safety

What is safe AI, and how do we make it? This course explores that question, focusing on the technical challenges of creating reliable, ethical, and aligned AI systems.

Length

10 weeks

Category

Introductory, Technical Alignment

Organizer

Stanford University

Alignment Research Engineer Accelerator (ARENA) curriculum

Facilitates upskilling in machine learning engineering for the purpose of contributing directly to AI alignment in technical roles.

Length

4 weeks

Category

Technical Alignment

Organizer

Alignment Research Engineer Accelerator (ARENA)

Economics of Transformative AI

Course from AI Safety Fundamentals (AISF) designed for economists who want to develop their understanding of transformative AI and its economic impacts.

Length

5 weeks

Category

Strategy

Organizer

BlueDot Impact

Introduction to Cooperative AI

An introduction to cooperative AI, which is an emerging research field focused on improving the cooperative intelligence of advanced AI for the benefit of all.

Length

8 weeks

Category

Introductory

Organizer

Cooperative AI Foundation (CAIF)

MATS AI Safety Strategy Curriculum

As part of their program, MATS runs a series of discussion groups focused on topics they believe are relevant to prioritizing research into AI safety. This is the curriculum from the latest cohort.

Length

5 weeks

Category

Technical Alignment, Governance

Organizer

ML Alignment & Theory Scholars (MATS)

CHAI Annotated Bibliography of Recommended Materials

Reading list discussing ideas CHAI believes warrant further attention and research for the purpose of developing human-compatible AI, as well as prerequisite materials for approaching them.

Length

40 weeks

Category

Technical Alignment

Organizer

Center for Human-Compatible AI (CHAI)

Levelling Up in AI Safety Research Engineering

A level-based guide for independently upskilling in AI safety research engineering that aims to give concrete objectives, goals, and resources to help anyone go from zero to hero.

Length

6 months

Category

Technical Alignment

Organizer

Stanford AI Alignment (SAIA)

Victoria Krakovna's AI Alignment Resources

Intended to help people get up to speed on the main ideas in AI alignment – ensuring advanced AI systems do what we want them to do. This is a selected list that is not designed to be comprehensive.

Length

–

Category

Technical Alignment

Organizer

Victoria Krakovna

Learning-Theoretic Agenda Reading List

Vanessa Kosoy's math-heavy reading list for people interested in studying the learning-theoretic agenda.

Length

–

Category

Technical Alignment

Organizer

Association for Long-Term Existence and Resilience (ALTER)

Arkose AI Safety Papers

Papers, blog posts, talks, newsletters, and guides relevant to improving the safety of advanced AI systems to reduce large-scale risks.

Length

–

Category

Technical Alignment, Governance

Organizer

Arkose

Reading What We Can

Collection of books and articles for a 20 day reading challenge. Covers AI safety basics, ML engineering, ML upskilling, and AI safety-relevant sci-fi.

Length

20 days

Category

Introductory

Organizer

Apart Research

Existential Risks Introductory Course (ERIC) Curriculum

Each section covers a different topic related to existential risks. Much of the programme follows the general structure of The Precipice by Toby Ord, which forms the majority of the core reading material.

Length

8 weeks

Category

Introductory

Organizer

Cambridge Existential Risks Initiative (CERI)

Key Phenomena in AI Risk

Focused on introducing people to AI risk factors from misdirected optimization or consequentialist cognition, aiming to remain largely agnostic of solution paradigms.

Length

6 weeks

Category

Technical Alignment

Organizer

Principles of Intelligent Behavior in Biological and Social Systems (PIBBSS)

ML for Alignment Bootcamp (MLAB) Curriculum

Teaches machine learning skills and concepts that are relevant to doing the kinds of alignment research that seem most leveraged for reducing AI existential risk. Access requires registration.

Length

3 weeks

Category

Technical Alignment

Organizer

Redwood Research

The Interpretability Toolkit

A collection of tools to get started and skill-up in interpretability, including the "Quickstart Guide for Mechanistic Interpretability" by Neel Nanda. Created in 2023.

Length

–

Category

Technical Alignment

Organizer

Apart Research