Funders

Open Philanthropy (OP)

The largest funder in existential risk reduction. Most funding is done via proactive research, but there are also frequent requests for proposals in certain areas.

Type

Fund

Accepting applications

Yes

Manifund

Regranting platform where individual regranters have funding pools to direct towards listed projects. Also offers impact certificates.

Type

Platform

Accepting applications

Yes

UK AI Security Institute (AISI): Challenge Fund

Awarding grants of up to £200,000 per project to address pressing, unresolved questions in AI safety. Researchers worldwide can access grants for innovative research in fields such as cyber-attacks and AI misuse.

Type

Grant program

Accepting applications

Yes

AI Safety Fund (AISF)

Aiming to promote the responsible development and deployment of frontier models, minimize risks, and enable independent, standardized evaluations of capabilities and safety.

Type

Fund

Accepting applications

No

Future of Life Foundation (FLF)

Incubator helping start new organizations that steer transformative technology towards benefiting life and away from extreme large-scale risks.

Type

Incubator

Accepting applications

No

AI Risk Mitigation (ARM) Fund

Aiming to reduce catastrophic risks from advanced AI through grants towards technical research, policy, and training programs for new researchers.

Type

Fund

Accepting applications

No

Thiel Fellowship

Two-year program for young people who want to build new things. Fellows skip or drop out of college to receive a US$100,000 grant and support from the Thiel Foundation’s network of founders, investors, and scientists.

Type

Grant-based fellowship

Accepting applications

Yes

AI Safety Science

This Schmidt Sciences program aims to deepen our understanding of the safety properties of systems built with LLMs and to develop well-founded, concrete, implementable technical methods for testing and evaluating LLMs.

Type

Grant program

Accepting applications

Yes

ARIA

UK government R&D funding agency built to unlock scientific and technological breakthroughs that benefit everyone.

Type

Fund

Accepting applications

Yes

Foresight Institute

Funding projects in 1) research and forecasting automation, 2) neurotech to integrate with or compete against AGI, 3) security technologies for securing AI systems, and 4) safe multi-agent scenarios.

Type

Grant program

Accepting applications

Yes

Effective Altruism Infrastructure Fund (EAIF)

Aiming to increase the impact of effective altruism projects by increasing their access to talent, capital, and knowledge.

Type

Fund

Accepting applications

Yes

GiveWiki

Crowd-sourced charity evaluator and one-stop shop for AI safety funding applications. All applications are public and charities that receive the most support from donors with the best track records are ranked highest.

Type

Platform

Accepting applications

Yes

Catalyze Impact

Facilitates faster progress in solving the AI alignment problem by supporting individuals in setting up AI safety research organizations.

Type

Incubator

Accepting applications

No

Longview Philanthropy

Devises and executes bespoke giving strategies for major donors. Grants focus on reducing the risks posed by AI, nuclear war, and engineered pandemics.

Type

Fund

Accepting applications

No

Ergo Impact

Helps major donors find, fund, and scale the most promising solutions to the world’s most pressing problems.

Type

Incubator

Accepting applications

No

Cooperative AI Foundation (CAIF)

Charity foundation backed by a large philanthropic commitment supporting research into improving cooperative intelligence of advanced AI for the benefit of all.

Type

Fund

Accepting applications

No

Future of Life Institute (FLI)

Grants for studying the risks from powerful technologies and developing strategies for reducing them. The goal is to win the race between the growing power of our technology and the wisdom with which we manage it.

Type

Grant-based fellowship

Accepting applications

No

Center on Long-Term Risk (CLR) Fund

Supports projects and individuals aiming to address worst-case suffering risks from the development and deployment of advanced AI systems.

Type

Fund

Accepting applications

Yes

Tarbell Grants

Journalism will likely play a crucial role in helping the public understand AI – and in holding companies and policymakers to account. Tarbell Grants offers awards of $1–15k to support journalism on AI and its impacts.

Type

Grant program

Accepting applications

No

Constellation Residency

Year-long salaried position for experienced researchers, engineers, entrepreneurs, and other professionals to pursue self-directed work on AI safety.

Type

Grant-based fellowship

Accepting applications

Yes

AE Studio Research

Empowering innovators and scientists to increase human agency by creating the next generation of responsible AI. Provides support, resources, and open-source software.

Type

Grant program

Accepting applications

Yes

Emergent Ventures

Launched by the Mercatus Center, this program seeks to support entrepreneurs and brilliant minds with highly scalable, “zero to one” ideas for meaningfully improving society.

Type

Grant-based fellowship

Accepting applications

Yes

Def/acc at Entrepreneur First

Incubation program focusing on "defensive" tech – the idea that the most powerful solution to technological risk is often more technology.

Type

Incubator

Accepting applications

No

Saving Humanity from Homo Sapiens (SHfHS)

Small organization with a long history of finding people doing impactful work to prevent human-created existential risks and financially supporting them.

Type

Fund

Accepting applications

No

Macroscopic Ventures

Swiss nonprofit focused on reducing suffering risks, including that posed by catastrophic AI misuse and conflict. Previously known as Polaris Ventures.

Type

Fund

Accepting applications

No

Seldon

12-week program in San Francisco, USA, for pre-seed companies composed of top-tier talent aiming to make an impact with AI safety and assurance technologies.

Type

Incubator

Accepting applications

Yes

Economics of AI Fellowship

Fellowship run by Stripe for graduate students and early-career researchers who are interested in pursuing foundational academic research around the economics of AI.

Type

Grant-based fellowship

Accepting applications

Yes

Good Ventures

Project by Dustin Moskovitz and Cari Tuna providing funding in a range of cause areas, including minimizing risks from potentially world-changing science and technology.

Type

Fund

Accepting applications

No

AI2050

Program from Schmidt Sciences supporting exceptional people working on key opportunities and hard problems that are critical to get right for society to benefit from AI.

Type

Grant-based fellowship

Accepting applications

No

Patrick J McGovern Foundation

Tech-inspired global changemakers, optimists, and visionaries advancing AI and data solutions to create a thriving, equitable, and sustainable future for all.

Type

Grant program

Accepting applications

No

Founders Pledge

Helping entrepreneurs direct their charitable giving where it will do the most good by researching the world’s biggest problems and identifying the most effective solutions.

Type

Fund

Accepting applications

No

National Science Foundation (NSF): Safe Learning-Enabled Systems

Funds research into the design and implementation of learning-enabled systems in which safety is ensured with high levels of confidence.

Type

Grant program

Accepting applications

No

Lionheart Ventures

VC firm investing in ethical founders developing transformative technologies that have the potential to impact humanity on a meaningful scale.

Type

Venture capitalist

Accepting applications

No

Juniper Ventures

VC firm investing in companies working to help make AI safe, secure, and beneficial for humanity. Also publishes the AI Assurance Tech Report.

Type

Venture capitalist

Accepting applications

No

Nonlinear Network

Platform taking grant applications and presenting them to a donor circle, who then reach out if they are interested.

Type

Platform

Accepting applications

No

Lightspeed Grants

Funds projects that have a chance of substantially changing humanity's future trajectory for the better. Disbursed 8 million USD in the 2023 round.

Type

Fund

Accepting applications

No

The Audacious Project

Funding initiative housed at TED helping entrepreneurs shape impactful ideas into viable multi-year plans and launching them to the world alongside visionary philanthropists.

Type

Incubator

Accepting applications

Yes

Interact Fellowship

For technologists with an enthusiasm for making intentional impact. Fellows attend two summer retreats and receive access to grants programs for their personal projects, and accountability and support mechanisms for founders.

Type

Grant-based fellowship

Accepting applications

No

Mythos Ventures

Aiming to empower founders building a radically better world with safe AI systems by investing in ambitious teams with defensible strategies that can scale to post-AGI.

Type

Venture capitalist

Accepting applications

No

The Navigation Fund

Offers grants to high-impact organizations and projects that are taking bold action and making significant changes.

Type

Fund

Accepting applications

No

Superlinear Prizes

Decentralized bounty platform offering prize money for delivering on specific AI safety projects.

Type

Platform

Accepting applications

No

Lightspeed Grants

Funds projects that have a chance of substantially changing humanity's future trajectory for the better. Disbursed 8 million USD in the 2023 round.

Type

Fund

Accepting applications

No