Many companies aim to measure sustainability-related effects with AI such as weather and energy use, but fewer talk about mitigating AI’s water- and power-hungry nature in the first place. Running generative AI sustainably could reduce some of the impact of climate change and look good to investors who want to contribute positively to the Earth.
This article will examine the environmental impact of generative AI workloads and processes and how some tech giants are addressing those issues. We spoke to Dell, Google Cloud, IBM and Microsoft.
How much energy does generative AI consume, and what is the possible impact of that usage?
How much energy generative AI consumes depends on factors including physical location, the size of the model, the intensity of the training and more. Excessive energy use can contribute to drought, animal habitat loss and climate change.
A team of researchers from Microsoft, Hugging Face, the Allen Institute for AI and several universities proposed a standard in 2022. Using it, they found that training a small language transformer model on 8 NVIDIA V100 GPUs for 36 hours used 37.3 kWh. How much carbon emissions this translates to depends a lot on the region in which the training is performed, but on average, training the language model emits about as much carbon dioxide as using one gallon of gas. Training just a fraction of a theoretical large model — a 6 billion parameter language model — would emit about as much carbon dioxide as powering a home does for a year.
Another study found AI technology could grow to consume 29.3 terawatt-hours per year — the same amount of electricity used by the entire country of Ireland.
A conversation of about 10 to 50 responses with GPT-3 consumes a half-liter of fresh water, according to Shaolei Ren, an associate professor of electrical and computer engineering at UC Riverside, speaking to Yale Environment 360.
Barron’s reported SpaceX and Tesla mogul Elon Musk suggested during the Bosch ConnectedWorld conference in February 2024 that generative AI chips could lead to an electricity shortage.
Generative AI’s energy use depends on the data center
The amount of energy consumed or emissions created depends a lot on the location of the data center, the time of year and time of day.
“Training AI models can be energy-intensive, but energy and resource consumption depend on the type of AI workload, what technology is used to run those workloads, age of the data centers and other factors,” said Alyson Freeman, customer innovation lead, sustainability and ESG at Dell.
Nate Suda, senior director analyst at Gartner, pointed out in an email to TechRepublic that it’s important to differentiate between data centers’ energy sources, data centers’ power usage effectiveness and embedded emissions in large language models hardware.
A data center hosting a LLM may be relatively energy efficient compared to an organization that creates a LLM from scratch in their own data center, since hyperscalers have “material investments in low-carbon electricity, and highly efficient data centers,” said Suda.
On the other hand, massive data centers getting increasingly efficient can kick off the Jevons effect, in which decreasing the amount of resources needed for one technology increases demand and therefore resource use overall.
How are tech giants addressing AI sustainability in terms of electricity use?
Many tech giants have sustainability goals, but fewer are specific to generative AI and electricity use. For Microsoft, one goal is to power all data centers and facilities with 100% additional new renewable energy generation. Plus, Microsoft emphasizes power purchase agreements with renewable power projects. In a power purchase agreement, the customer negotiates a preset price for energy over the next five to twenty years, providing a steady revenue stream for the utility and a fixed price for the customer.
“We’re also working on solutions that enable datacenters to provide energy capacity back to the grid to contribute to local energy supply during times of high demand,” said Sean James, director of datacenter research at Microsoft, in an email to TechRepublic.
“Don’t use a sledgehammer to crack open a nut”
IBM is addressing sustainable electricity use around generative AI through “recycling” AI models; this is a technique developed with MIT in which smaller models “grow” instead of a larger model having to be trained from scratch.
“There are definitely ways for organizations to reap the benefits of AI while minimizing energy use,” said Christina Shim, global head of IBM sustainability software, in an email to TechRepublic. “Model choice is hugely important. Using foundation models vs. training new models from scratch helps ‘amortize’ that energy-intensive training across a long lifetime of use. Using a small model trained on the right data is more energy efficient and can achieve the same results or better. Don’t use a sledgehammer to crack open a nut.”
Ways to reduce energy use of generative AI in data centers
One way to reduce energy use of generative AI is to make sure the data centers running it use less; this may involve novel heating and cooling methods, or other methods, which include:
- Renewable energy, such as electricity from sustainable sources like wind, solar or geothermal.
- Switching from diesel backup generators to battery-powered generators.
- Efficient heating, cooling and software architecture to minimize data centers’ emissions or electricity use. Efficient cooling techniques include water cooling, adiabatic (air pressure) systems or novel refrigerants.
- Commitments to net zero carbon emissions or carbon neutrality, which sometimes include carbon offsets.
Benjamin Lee, professor of electrical and systems engineering and computer and information science at the University of Pennsylvania, pointed out to TechRepublic in an email interview that running AI workloads in a data center creates greenhouse gas emissions in two ways.
- Embodied carbon costs, or emissions associated with the manufacturing and fabricating of AI chips, are relatively small in data centers, Lee said.
- Operational carbon costs, or the emissions from supplying the chips with electricity while running processes, are larger and increasing.
Energy efficiency or sustainability?
“Energy efficiency does not necessarily lead to sustainability,” Lee said. “The industry is rapidly building datacenter capacity and deploying AI chips. Those chips, no matter how efficient, will increase AI’s electricity usage and carbon footprint.”
Neither sustainability efforts like energy offsets nor renewable energy installations are likely to grow fast enough to keep up with datacenter capacity, Lee found.
“If you think about running a highly efficient form of accelerated compute with our own in-house TPUs, we leverage liquid cooling for those TPUs that allows them to run faster, but also in a much more energy efficient and as a result a more cost effective way,” said Mark Lohmeyer, vice president and general manager of compute and AI/ML Infrastructure at Google Cloud, in an interview with TechRepublic at NVIDIA GTC in March.
Google Cloud approaches power sustainability from the angle of using software to manage up-time.
“What you don’t want to have is a bunch of GPUs or any type of compute deployed using power but not actively producing, you know, the outcomes that we’re looking for,” he said. “And so driving high levels of utilization of the infrastructure is also key to sustainability and energy efficiency.”
Lee agreed with this strategy: “Because Google runs so much computation on its chips, the average embodied carbon cost per AI task is small,” he told TechRepublic in an email.
Right-sizing AI workloads
Freeman noted Dell sees the importance of right-sizing AI workloads as well, plus using energy-efficient infrastructure in data centers.
“With the rapidly increasing popularity of AI and its reliance on higher processing speeds, more pressure will be put on the energy load required to run data centers,” Freeman wrote to TechRepublic. “Poor utilization of IT assets is the single biggest cause of energy waste in the data center, and with energy costs typically accounting for 40-60% of data center’s operating costs, reducing total power consumption will likely be something at the top of customers’ minds.”
She encouraged organizations to use energy-efficient hardware configurations, optimized thermals and cooling, green energy sources and responsible retirement of old or obsolete systems.
When planning around energy use, Shim said IBM considers how long data has to travel, space utilization, energy-efficient IT and datacenter infrastructure, and open source sustainability innovations.
How are tech giants addressing AI sustainability in terms of water use?
Water use has been a concern for large corporations for decades. This concern isn’t specific to generative AI, since the problems overall — habitat loss, water loss and increased global warming — are the same no matter what a data center is being used for. However, generative AI could accelerate those threats.
The need for more efficient water use intersects with increased generative AI use in data center operations and cooling. Microsoft doesn’t separate out generative AI processes in its environmental reports, but the company does show that its total water consumption jumped from 4,196,461 cubic meters in 2020 to 6,399,415 cubic meters in 2022.
“Water use is something that we have to be mindful of for all computing, not just AI,” said Shim. “Like with energy use, there are ways businesses can be more efficient. For example, a data center could have a blue roof that collects and stores rainwater. It could recirculate and reuse water. It could use more efficient cooling systems.”
Shim said IBM is working on water sustainability through some upcoming projects. Ongoing modernization of the venerable IBM research data center in Hursley, England will include an underground reservoir to help with cooling and may go off-grid for some periods of time.
Microsoft has contracted water replenishment projects: recycling water, using reclaimed water and investing in technologies such as air-to-water generation and adiabatic cooling.
“We take a holistic approach to water reduction across our business, from design to efficiency, looking for immediate opportunities through operational usage and, in the longer term, through design innovation to reduce, recycle and repurpose water,” said James.
Microsoft addresses water use in five ways, James said:
- Reducing water use intensity.
- Replenishing more water than the organization consumes.
- Increasing access to water and sanitation services for people across the globe.
- Driving innovation to scale water solutions.
- Advocating for effective water policy.
Organizations can recycle water used in data centers, or invest in clean water initiatives elsewhere, such as Google’s Bay View office’s effort to preserve wetlands.
How do tech giants disclose their environmental impact?
Organizations interested in large tech companies’ environmental impact can find many sustainability reports publicly:
Some AI-specific callouts in these reports are:
- IBM used AI to capture and analyze IBM’s energy data, creating a more thorough picture of energy consumption
- NVIDIA focuses on the social impact of AI instead of the environmental impact in their report, committing to “models that comply with privacy laws, provide transparency about the model’s design and limitations, perform safely and as intended, and with unwanted bias reduced to the extent possible.”
Potential gaps in environmental impact reports
Many large organizations include carbon offsets as part of their efforts to reach carbon neutrality. Carbon offsets can be controversial. Some people argue that claiming credits for preventing environmental damage elsewhere in the world results in inaccuracies and does little to preserve local natural places or places already in harm’s way.
Tech giants are aware of the potential impacts of resource shortages, but may also fall into the trap of “greenwashing,” or focusing on positive efforts while obscuring larger negative impacts. Greenwashing can happen accidentally if companies do not have sufficient data on their current environmental impact compared to their climate targets.
When to not use generative AI
Deciding not to use generative AI would technically reduce energy consumption by your organization, just as declining to open a new facility might, but doing so isn’t always practical in the business world.
“It is vital for organizations to measure, track, understand and reduce the carbon emissions they generate,” said Suda. “For most organizations making significant investments in genAI, this ‘carbon accounting’ is too large for one person and a spreadsheet. They need a team and technology investments, both in carbon accounting software, and in the data infrastructure to ensure that an organization’s carbon data is maximally used for proactive decision making.”
Apple, NVIDIA and OpenAI declined to comment for this article.
