Courses

AI Safety, Ethics and Society curriculum

Covers a wide range of risks, including loss of control, misalignment, malicious use, accidents, and enfeeblement, while leveraging concepts and frameworks from existing research fields to analyze AI safety.

Length

13 weeks

Category

Introductory, Governance

Organizer

Center for AI Safety

Introduction to ML Safety

PhD-level survey course on technical ML safety. Covers various technical topics to reduce existential risks from AI, including robustness, monitoring, control, and systemic safety.

Length

40 hours

Category

Introductory, Technical Alignment

Organizer

Center for AI Safety

Introduction to AI Safety

What is safe AI, and how do we make it? This course explores this question, focusing on the technical challenges of creating reliable, ethical, and aligned AI systems.

Length

11 weeks

Category

Introductory, Technical Alignment

Organizer

Stanford University

Alignment Research Engineer Accelerator (ARENA) curriculum

Facilitates upskilling in machine learning engineering for the purpose of contributing directly to AI alignment in technical roles.

Length

4 weeks

Category

Technical Alignment

Organizer

Alignment Research Engineer Accelerator

Agent Foundations for Superintelligence-Robust Alignment

A guide to the cluster of thought which expects solving the alignment problem in a way which scales to superintelligence to require well-specified proposals guided by strong theoretical understanding.

Length

6 weeks

Category

Technical Alignment

Organizer

Alignment Ecosystem Development

MATS AI Safety Strategy Curriculum

As part of their program, MATS runs a series of discussion groups focused on topics they believe are relevant to prioritizing research into AI safety. This is the curriculum from the latest cohort.

Length

5 weeks

Category

Technical Alignment, Governance

Organizer

ML Alignment & Theory Scholars

CHAI Annotated Bibliography of Recommended Materials

Reading list discussing ideas CHAI believes warrant further attention and research for the purpose of developing human-compatible AI, as well as prerequisite materials for approaching them.

Length

40 weeks

Category

Technical Alignment

Organizer

Center for Human-Compatible AI

Levelling Up in AI Safety Research Engineering

Uses levelling frameworks to break up one possible path from zero to hero in research engineering for AI safety through objectives, concrete goals, and resources.

Length

6 months

Category

Technical Alignment

Organizer

Stanford AI Alignment

Victoria Krakovna's AI Alignment Resources

Intended to help people get up to speed on the main ideas in AI alignment – ensuring advanced AI systems do what we want them to do. This is a selected list that is not designed to be comprehensive.

Length

–

Category

Technical Alignment

Organizer

Victoria Krakovna

Learning-Theoretic Agenda Reading List

Vanessa Kosoy's math-heavy reading list for people interested in studying the learning-theoretic agenda.

Length

–

Category

Technical Alignment

Organizer

Association for Long-Term Existence and Resilience

Arkose AI Safety Papers

Papers, blog posts, talks, newsletters, and guides relevant to improving the safety of advanced AI systems to reduce potential large-scale risks.

Length

–

Category

Technical Alignment

Organizer

Arkose

Reading What We Can

Collection of books and articles for a 20 day reading challenge. Covers AI safety basics, ML engineering, ML upskilling, and AI safety-relevant sci-fi.

Length

20 days

Category

Technical Alignment

Organizer

Apart Research

Existential Risks Introductory Course (ERIC) curriculum

Each section covers a different topic related to existential risks. Much of the programme follows the general structure of The Precipice by Toby Ord, which forms the majority of the core reading material.

Length

8 weeks

Category

Technical Alignment

Organizer

Cambridge Existential Risks Initiative

Key Phenomena in AI Risk

Focused on introducing people to AI risk factors from misdirected optimization or consequentialist cognition, aiming to remain largely agnostic of solution paradigms.

Length

7 weeks

Category

Technical Alignment

Organizer

Principles of Intelligent Behavior in Biological and Social Systems

ML for Alignment Bootcamp (MLAB)

Teaches machine learning skills and concepts that are relevant to doing the kinds of alignment research that seem most leveraged for reducing AI existential risk. Access requires registration.

Length

3 weeks

Category

Technical Alignment

Organizer

Redwood Research

The Interpretability Toolkit

Many tools to get started and skill-up in interpretability, including the "Quickstart Guide for Mechanistic Interpretability" by Neel Nanda. Created in 2023.

Length

–

Category

Technical Alignment

Organizer

Apart Research