What is the AI Energy Score for LLMs? ♻️
Boris, Salesforce and Understanding Energy scores of LLMs and SLMs.
Good Afternoon,
As I’ve been digging into AI Infrastructure and energy, there’s a Paris based researcher from Salesforce that really has caught my attention.
His name is Boris, and here’s why I think he’s interesting. During the Paris AI Summit he has introduced a fascinating Energy Score for LLMs.
Lack of transparency is a fundamental challenge to AI sustainability. This matters because an AI Energy Score aims to address the lack of transparency about the environmental impact of AI models.
This is going to become a rather big deal.
Introducing 𝐀𝐈 𝐄𝐧𝐞𝐫𝐠𝐲 𝐒𝐜𝐨𝐫𝐞 – a standardized way to measure and compare the energy efficiency of AI models.
Interactive Leaderboard – data from hundreds of models already scored:
Submission Portal – score both open and closed-source models
Label Generator – share results far and wide
Documentation + FAQ – learn more about the approach
The way we think about this is also set to evolve a lot in the years ahead.
See the original post by Boris here.
Follow the Discussions on this.
This article is just an announcement stub, on February 10th, 2025 about this.
Credits:
Dr. Sasha Luccioni Clem Delangue 🤗 Yacine Jernite Régis Pierrard Sara Hooker Silvio Savarese Itai Asseo Denise Pérez Michael Weimann Hannah Downey Julie Ravillon Laurent Monjole Serena Ingre, and Boris too of course!
Links:
In Brief
Salesforce, Hugging Face, Cohere, and Carnegie Mellon introduce a standardized tool to measure AI model energy efficiency.
As part of the initiative, public ratings for 166 commonly-used AI models are being released to help developers and AI users identify models that use less energy.
AI models consume vast amounts of energy, yet transparency on their environmental impact has been lacking. The AI Energy Score provides a clear, standardized metric—similar to the ENERGY STAR rating for appliances—to drive sustainable AI development.
Congrats to Salesforce, Hugging Face, Cohere and the other partners on this.
Energy Ratings
Standardized Energy Ratings: A standardized framework for measuring and comparing AI model energy efficiency.
Public Leaderboard: A comprehensive leaderboard that features scores for 10 common AI tasks — such as text generation, image generation, and summarization — performed by 166 models, including Salesforce’s SFR-Embedding, xLAM, and SF-TextBase.
Benchmarking Portal: A platform where AI developers can submit their open or proprietary AI models to be evaluated and added to the leaderboard. Open models can be automatically tested, while closed models can be evaluated through a secured testing sandbox.
Recognizable Energy Use Label: A new 1- to 5-star label that rates AI model energy use, with five stars indicating the highest efficiency. This helps developers and users easily identify and choose more sustainable models. Once rated, AI developers can generate standardized labels to share their models’ energy score, with built-in guidance on the proper label display for visibility and impact.
I’m sure a lot more will be written about this, and I’ll be following those developments and the reactions to this actively. This is a developing story.
Boris is head of AI Sustainability at Salesforce, one of my favorite people to follow on LinkedIn around Energy and LLMs.