What is the EleutherAI Institute?

Empowering Open-Source Artificial Intelligence Research

Hello Everyone,

What is the distinction between an A.I. lab and an A.I. startup and one that is open and one that is not? In 2023, these are blurry lines and questions even as more A.I. startup tether themselves to “Cloud partners” and BigTech funding.

I don’t pretend to understand EleutherAI vs. CarperAI and all of these spin-offs from OpenAI easily. However we can at least try to make sense of it all and the likely trajectory for open-source in a world where LLMs seem more corporate and private than ever in their gritty details of how they actually function.

OpenAI selling out to Microsoft hasn’t exactly been great for the democratization of A.I., depending on who you talk to and what you consider open-access. Everyone from TechCrunch to the Decoder give an interesting glimpse into this.

Europe pretends it’s somehow more progressive in A.I. ethics than the U.S., but it’s not clear if this is even true.

EleutherAI is one of the pioneers of open-source research in generative AI, especially in language models. It’s now becoming a not-for-profit organization with full-time researchers.

While several open source efforts have attempted to reverse-engineer proprietary, closed source systems created by commercial labs such as Alphabet’s DeepMind and OpenAI, the lack of funding and talent is a bottleneck, not to mention the expensive compute.

One community research group, EleutherAI, is forming a nonprofit foundation. The organization today announced it’ll found a not-for-profit research institute, the EleutherAI Institute, funded by donations and grants from backers, including AI startups Hugging Face and Stability AI, former GitHub CEO Nat Friedman, Lambda Labs and Canva.

EleutherAI’s motto is “empowering open-source artificial intelligence research”. Everyone from AnthropicAI to OpenAI at its conception, claimed similar things. So when does an A.I. lab become an A.I. startup and than a private for profit corporate entity?

“Formalizing as an organization allows us to build a full time staff and engage in longer and more involved projects than would be feasible as a volunteer group,” Stella Biderman, an AI researcher at Booz Allen Hamilton who will co-run the EleutherAI Institute

Where the Open-Access Movement is Going?

While there seems to be a lot of ML researchers who believe in open-source, open-access and more decentralization in collaboration with regards the future of R&D around A.I, companies like Stability.AI and AnthropicAI now seem destined to become just ordinary OpenAI competitors.

Even HuggingFace seems on such a trajectory now that it’s partnered with AWS.

EleutherAI started several years ago as a grassroots collection of developers working to open source AI research. Its founding members — Connor Leahy, Leo Gao and Sid Black — wrote the code and collected the data needed to create a machine learning model close to OpenAI’s text-generating GPT-3. But how quickly things change!

The regulation of general-purpose AI (GPAI) is currently being debated by the European Union’s legislative bodies as they work on the Artificial Intelligence Act (AIA). Meanwhile the U.S. seems like the wild-wild west when it comes to bias in foundational models, the dangers of deepfakes or the hallucinations of LLMs at scale and what it will do to society, content and the automation of tasks done by humans.

EleutherAI is credited with curating the open sourced The Pile, a collection of datasets designed to be used to train GPT-3-like models to complete text, write code and more. And it released several models under the Apache 2.0 license, including GPT-J and GPT-NeoX, language models that for a while fueled an entirely new wave of startups, according to TechCrunch and VentureBeat.

With an OpenAI API and Hugging Face partnering with AWS, the democration of AI really just means AI as a service for enterprises for profit. The EleutherAI research collective is professionalizing, says the Decoder. Over the past two and a half years, it has evolved from a group of programmers on Discord to what it calls an open science community. Now, EleutherAI is becoming a non-profit research institute, according to its blog. 20 scientists can now work full-time for EleutherAI.