AI Safety logo
AI Safety logo

Founder toolkit

Resources for starting and growing an AI safety organization – including incubators, fiscal sponsors, VCs, and practical tools.

35 resources

  • Provides a temporary home for nonprofit projects so they can test ideas, develop operational capabilities, and launch as independent entities. Fee: 7%.

    Type: Fiscal sponsor

  • Large, generalist fiscal sponsor with over 600 charitable projects worldwide. Streamlined onboarding process. Fee: 6%.

    Type: Fiscal sponsor

  • Accelerating initiatives by providing comprehensive support services, enabling organizations to focus on their core mission and scale efficiently. Fee: 14%+.

    Type: Fiscal sponsor

  • Providing operational infrastructure and fiscal sponsorship to research groups working on existential risk, especially those based at a university. Fee: 7–10%.

    Type: Fiscal sponsor

  • New fiscal sponsor providing admin support and bureaucracy shielding for researchers safeguarding humanity from catastrophic risk. Fee: 5–10%.

    Type: Fiscal sponsor

  • Provides fiscal sponsorship, incubation, and operations support for AI safety nonprofit founders to guide them toward effective impact from day one. Fee: 6–12%.

    Type: Fiscal sponsor

  • An overview of what fiscal sponsorship is, its benefits and disadvantages, and best practices.

    Type: Article/tool

  • Free tool for creating a theory of change. Just drop relevant documents in and it will create a draft ToC which you can then adjust and export.

    Type: Article/tool

  • Discord group aimed at entrepreneurially-minded people wanting to explore what it takes to build in AI safety.

    Type: Article/tool

  • 3-month program run by Manifund and Mox for founders building software for AI safety, epistemics, and coordination. Includes $100k investment and mentorship.

    Type: Incubator

  • 5-day incubator for technically capable people working in or adjacent to AI safety (and other fields). Funding available.

    Type: Incubator

  • Three-day program for people looking to found a nonprofit or for-profit in AI safety. Fellows learn from founders and ideate with researchers. Optional $10k grant or investment.

    Type: Incubator

  • Six-month program in Berkeley, USA, helping founders launch an AI safety for-profit or non-profit.

    Type: Incubator

  • 12-week program in San Francisco, USA, for pre-seed companies aiming to make an impact with AI safety and assurance technologies.

    Type: Incubator

  • Incubator helping start new organizations that steer transformative technology towards benefiting life and away from extreme large-scale risks.

    Type: Incubator

  • 12–14 week program run by Fifty Years, helping scientists and engineers build startups working on AI safety.

    Type: Incubator

  • Identifying leaders from business, policy, and academia, and helping them take on ambitious projects in AI safety (and other areas). Helped launch Goodfire, Transluce, etc.

    Type: Incubator

  • Identifies unowned problems, maps stakeholders, drafts milestones, sources early funders, and recruits an expert leader to take ownership.

    Type: Incubator

  • Helping philanthropists find, fund, and scale the most promising people and solutions to the world’s most pressing problems – including AI safety.

    Type: Incubator

  • For experienced professionals looking to found high-impact nonprofits or for-profits in AI safety and other fields. Includes a founder cohort, tailored guidance, and a curated network.

    Type: Incubator

  • Residential incubator for effective altruist researchers, founders, and career transitioners, with a strong focus on AI safety.

    Type: Incubator

  • Funding initiative housed at TED helping entrepreneurs shape impactful ideas into viable multi-year plans and launching them to the world alongside visionary philanthropists.

    Type: Incubator

  • Accelerator by Entrepreneurs First (EF) for early-career builders. Its safety and resilience track, run with BlueDot Impact, covers alignment, monitoring, and interpretability.

    Type: Incubator

  • New VC fund by Igor Babuschkin (xAI co-founder). Backs AI safety research and startups building agentic systems.

    Type: Venture capitalist

  • Early-stage venture fund supporting startups developing tools to enhance AI safety. Provides both financial investment and mentorship.

    Type: Venture capitalist

  • VC firm investing in ethical founders developing transformative technologies that have the potential to impact humanity on a meaningful scale.

    Type: Venture capitalist

  • VC firm investing in AI safety startups. Run by exited founders and backed by Reid Hoffman, Eric Ries, and Geoff Ralston.

    Type: Venture capitalist

  • Swiss VC focused on reducing suffering risks, including that posed by catastrophic AI misuse and AI conflict. Makes both grants to nonprofits and investments in for-profit companies.

    Type: Venture capitalist

  • Evergreen VC and fund by Jaan Tallinn backing tech that supports humanity's long-term survival. Profits from the VC go towards funding AI safety nonprofits.

    Type: Venture capitalist

  • $100M fund by Menlo Ventures and Anthropic backing AI startups from seed to Series A, including those working on "trust and safety tooling".

    Type: Venture capitalist

  • Private foundation providing grants to projects working to make the future with advanced AI go well. Also invests in for-profits in order to redeploy capital to AI safety nonprofits.

    Type: Venture capitalist

  • VC investing in startups shaping the future of AI and humanity, committed to delivering financial returns while building a resilient and beneficial future.

    Type: Venture capitalist

  • VC aiming to empower founders building a radically better world with safe AI systems by investing in ambitious teams with defensible strategies that can scale to post-AGI.

    Type: Venture capitalist