Google’s AI operations recently made headlines due to their significant environmental impact, particularly regarding carbon emissions. The company’s AI activities, including training and deploying large language models (LLMs), have led to a 48% increase in greenhouse gas emissions over the past five years. Google’s annual environmental report revealed that emissions from its data centers and supply chain were the main contributors to this rise. In 2023, emissions surged by 13% from the previous year, totaling 14.3 million metric tons, underscoring the pressing need to address the environmental effects of AI’s rapid growth.
Power and Water Consumption: The Hidden Costs of LLM Functioning
The carbon footprint of LLMs includes two main components: the operational footprint, from energy used by hardware, and the embodied footprint, from emissions during model training. LLMs require significant energy and water, often from non-renewable sources, for both training and inference (generating responses to prompts). Continuous updates and user interactions further increase energy consumption, sometimes surpassing training needs. It is estimated that energy consumption of data centers will rise to 1,000 TWh by 2026.
Water usage is another critical aspect of LLM functioning. Data centers rely on vast quantities of water for cooling servers. ChatGPT uses around 500 milliliters per prompt, and by 2027, global AI demand could lead to 4.2–6.6 billion cubic meters of water use—equivalent to the annual water withdrawal of 4–6 Denmark or half of the UK. This level of consumption is particularly concerning in regions with limited water resources, where the strain on local water supplies can have severe environmental and social consequences.
Source: AI Index Report 2023
Energy and Resource Allocation: Where It All Goes
Training LLMs is a resource-intensive process involving several key stages, each contributing to the environmental footprint.
Model Size: The size of an LLM is usually determined by the number of parameters it has. These parameters are essentially the variables that the model learns from the data during the training process. The size of the model is directly proportional to its energy consumption. This means that larger models, which have more parameters, require more computational power and thus consume more energy.
For instance, GPT-3, which is a very large model with 175 billion parameters, is reported to have consumed approximately 1,287 MWh (megawatt-hours) of electricity during its training. However, smaller models like GPT-2, which has 1.5 billion parameters, require significantly less energy for training. This is because they have fewer parameters and thus require less computational power.
Model Training: Model training is a resource-intensive process critical for developing LLMs. It involves optimizing model parameters by processing vast data through complex algorithms, relying heavily on Graphics Processing Unit (GPU) chips. Training LLMs is not a one-time event; it often involves multiple iterations to improve accuracy and efficiency. Each iteration requires GPUs to run continuous computations, consuming significant amounts of energy.
The production of GPUs involves energy-intensive raw material mining and manufacturing, contributing to environmental degradation. Once manufactured, thousands of GPUs are required to train large models like ChatGPT, further increasing energy usage. For example, training a single AI model can generate over 626,000 pounds of CO2, equivalent to nearly five times the lifetime emissions of an average American car. Additionally, disposing of GPUs adds to e-waste, further increasing the environmental footprint of LLMs.
Training Hours: The energy required to train a neural network scales with the amount of time the training process runs. Training a model involves repeatedly processing vast amounts of data through the network, adjusting weights and biases based on the feedback received. Each training iteration involves extensive computations, and the longer the training period, the more computational resources are used. This extended runtime translates into increased energy consumption.
For instance, training BERT on a large dataset required around 64 TPU days, leading to substantial energy consumption. However, smaller models or those trained on less extensive datasets might only need a few days or even hours, resulting in significantly lower energy usage.
Server Cooling: Long training periods generate substantial heat in GPUs and TPUs, necessitating effective cooling systems to prevent overheating. These cooling systems, including air conditioning, refrigeration, cooling towers and water-based chillers consume significant electricity and often rely on water, which can strain local resources, particularly in water-scarce areas. The energy used for cooling often results in increased greenhouse gas emissions, and the discharge of warm water can cause thermal pollution.
Cooling systems account for about 40% of a data center’s total energy use, and as AI operations expand, their cooling demands increase accordingly. This energy consumption contributes to higher greenhouse gas emissions and can lead to thermal pollution from discharged warm water.
Mitigation Strategies: Reducing the Environmental Footprint of LLMs
Addressing the environmental impact of LLMs requires a multi-faceted approach, incorporating both technological innovation and strategic policy-making.
Efficiency Improvements: Advances in AI technology for estimating carbon footprints are making it possible to analyze and reduce the energy consumption of LLMs. While existing tools like mlco2 are limited—they only apply to CNNs, overlook key architectural parameters, and focus solely on GPUs— new tools like LLMCarbon addresses these gaps.
LLMCarbon improves upon previous methods by providing an end-to-end carbon footprint projection model that accurately predicts emissions during training, inference, experimentation, and storage phases. LLMCarbon incorporates essential parameters such as LLM parameter count, hardware type, and data center efficiency, allowing for more accurate modeling of both operational and embodied carbon footprints. Its results have been validated against Google’s published LLM carbon footprints, showing differences of only ≤ 8.2%, which is more accurate than existing tools.
Renewable Energy Integration: Integrating renewable energy into data centers is a key strategy for reducing the carbon footprint of LLMs. By powering data centers with sources like wind, solar, or hydroelectric power, the reliance on fossil fuels for electricity generation is diminished, leading to a substantial decrease in greenhouse gas emissions. This shift not only lowers the operational carbon footprint associated with training and running LLMs but also supports the broader goal of sustainable AI development.
Water Usage Optimization: Reducing water consumption in data centers is another critical area of focus. Techniques like using recycled water for cooling and adopting more efficient cooling systems can significantly reduce water consumption. By recycling water within cooling processes and employing advanced cooling technologies, data centers can lower their dependence on freshwater resources and mitigate the strain on local water supplies.
Microsoft aims to decrease its data center water usage by 95% by 2024 and ultimately eliminate it. Currently, they use adiabatic cooling, which relies on outside air and consumes less water than traditional systems. When temperatures rise above 85°F, an evaporative cooling system, similar to a “swamp cooler,” uses water to cool the air. These measures help manage water use more sustainably and reduce the overall environmental footprint.
Model Pruning and Distillation: Techniques such as model pruning and distillation are effective in reducing the size and complexity of LLMs while maintaining their performance. Pruning involves removing redundant or less critical parameters from a model, making it more efficient. Distillation transfers knowledge from a large model to a smaller, more streamlined version, preserving essential functionality while cutting down on computational demands. These approaches help lower the energy consumption during training and inference, thus reducing the overall carbon footprint of LLMs.
Hardware Advancements: The adoption of energy-efficient hardware, such as specialized AI accelerators, significantly contributes to lowering the carbon footprint of LLMs. AI accelerators, designed to optimize the performance of machine learning tasks, consume less power compared to traditional GPUs or CPUs. By utilizing these advanced hardware solutions, data centers can reduce their energy consumption during both model training and deployment, leading to a decrease in greenhouse gas emissions associated with LLM operations.
As the adoption of LLMs continues to grow, so does the need to address their environmental impact. The tech industry must take proactive steps to mitigate the carbon footprint, energy consumption, and water usage associated with these models. By investing in efficiency improvements, renewable energy, and sustainable AI practices, we can ensure that the benefits of AI are realized without compromising the health of our planet.