How to use the Impact Framework to measure the environmental impact of your website

Co-written by: Mike Heavers Rachael Robinson Rachana Kabi Irete Hamdani Omar Faris Alexander zur Bonsen

Are you interested in using the Green Software Foundation’s Impact Framework for measuring the environmental impact of your website, but don’t know where to start? We’re a group of technologists finding our way in the world of sustainability and wanted to share our advice. This is the guide we wish we’d had when we first tried the Impact Framework out for ourselves. It was written as part of the Carbon Hack 24 hackathon, along with a video tutorial for running the Impact Framework.

Before we get started — key concepts

In this guide, we’ll be referring to a number of Green Software concepts. If you’re new to this area, have a look at the Green Software Foundation’s free training, which will familiarize you with the important ideas within a few hours. In particular, we will reference:

• Carbon equivalent: a unit of measurement for any greenhouse gas that standardizes its impact against a gram of carbon dioxide
• Operational energy usage: the electrical energy used in computation
• Embodied/embedded carbon: the carbon dioxide equivalent that is created by the manufacture and disposal of hardware
• Carbon intensity: the proportion of electrical energy used that comes from greenhouse gas emitting sources
• Functional unit: the unit of measurement an organization chooses to compare their emissions against — e.g. are they interested in grams of carbon equivalent emitted per second, per hour, per page load, per database transaction, per completed user journey…
• The Software Carbon Intensity standard: a way to quantify the level of greenhouse gases emitted by a software product that takes into account operational energy, embodied carbon, carbon intensity and a team’s functional unit

If you’re reading this and are also thinking: ‘But what even is the Impact Framework?’, then we’d strongly recommend looking at the Green Software Foundation’s introduction here. In summary, the Impact Framework is a way to compute and report the environmental impacts of software applications. It can:

• Transform observations into impacts
• Engage an existing library of plugins or create new plugins for your use case
• Explore What-If Scenarios
• Allow others to understand, verify and challenge the entire process through a decentralized and transparent structure
• Empower others to run the analysis themselves and validate your findings

The Impact Framework is an early-stage, open-source project with a large number of contributors from different organizations. As it’s in this early stage, key details are still being worked out and may change. It also means that users of the project like us have the opportunity to shape its direction with feedback and by building new resources and components.

You will need access to GitHub to use the Impact Framework project. GitHub is a collaboration platform that is popular with coders, but not used exclusively by them. If you would like a guide to using GitHub, then check out this video.

So, you’ve decided to use the Impact Framework for your website, what now?

The Impact Framework is designed to allow users to tweak it for their specific needs. To personalize the Impact Framework for your website, you’ll need to write a specially formatted .yaml file called the ‘manifest file’. This takes a few steps:

1. Decide on the project scope
2. Choose (or make) evaluation ‘plugins’
3. Collect your input data
4. Write up the manifest file
5. Run the Impact Framework

Once you’ve run the Impact Framework, you can use the resulting output data to understand and make improvements to your system.

Stage 1: Decide on project scope

One of the advantages of the Impact Framework is that it is incredibly flexible, so it could theoretically be used for calculating:

• The amount of carbon dioxide emitted by somebody going through the full web journey of applying for a passport online
• The amount of water used when fine-tuning a particular image recognition model to differentiate between photos of different brands of cars
• The per-hour cost of electricity to run an on-premises server for an individual hospital IT system

This flexibility means that you will need to make some decisions about what kind of impact you want to be evaluating with the impact framework. There are a few key decisions:

1. What kind of impact are you interested in? At the moment, there are more plugins to support you with calculating carbon emissions (or their equivalent), but don’t feel you have to be limited to this. As in the examples above, you could look at water use or financial cost — or something else entirely.
2. What’s the ‘functional unit’ you’re interested in? You could calculate impact per hour, or per HTTP request. However, it’s worth considering a more outcome-focused unit — e.g. ‘a user completing a passport application journey online’.
3. What are you counting as part of your website’s impact? Where does your website create an environmental impact? The most obvious sources are the electricity usage of your users accessing the site, and of your servers hosting and processing data. There’s also the electricity used by developers when writing, testing and deploying code and all the resources used in creating the devices used to develop, host or access the website.
4. How are you going to divide up your project? When calculating the impact of your website, the Impact Framework suggests dividing up the site into different components and calculating the impact of each of these separately. For example this could involve: a) the website bundle transferred to the user; b) the storage bucket containing your site code; c) the separate storage bucket containing your images (if you use one); d) the engineer writing and updating your site
5. In the diagram below, this means: data center emissions, networks, devices and production.

The diagram (based on these articles from Fershad Irani and data from here) outlines options for calculating the impact of a website. The Impact Framework is an example of using the Consequential model for estimation, as it involves considering the impact of each component individually.

If you find yourself getting overwhelmed with the number of project components or the breadth of potential impact of your site, then choose one area to focus on first. If you’re not sure where to start, have a look at the current options for evaluation plugins — see the section below — and choose an area that’s better supported by the Impact Framework community.

Stage 2: Choose (or make) evaluation plugins

What is an Impact Framework plugin?

Unfortunately, there’s no magic device we can plug into a user’s computer or server that will tell us the carbon dioxide emitted by them using a website. Instead, we have to find a way to transform the data we can collect (e.g. kb transferred in an HTTP request) into what we want to calculate (e.g. energy consumption).

Researchers have developed a number of ‘plugins’, which are tools that perform these calculations. For example, the SCI-O plugin from the Green Software Foundation calculates the carbon emitted by an application. This is based on information the user provides about energy used by the application and the carbon intensity of the electricity grid while the application was used.

In a different example, the TEADS AWS plugin calculates both the energy (in kWh) used by an Amazon Web Services server while running a site over a particular time period and the carbon used in creating and disposing of the server. This is based on the type of server instance, the percentage CPU usage and the length of time measured.

The Impact Framework is composable — so you can take the output of one plugin and feed it into another: for example, taking the energy calculation from the TEADS-AWS plugin and using it as an input in the SCI-O plugin.

Here’s an example of a pipeline with three plugins used to calculate the carbon emissions for a website hosted on AWS. The final plugin in the chain brings together the embodied carbon from the first plugin and the operational carbon from the second, along with information about the time spent and the user’s choice of ‘functional unit’. It uses these to work out not only the total carbon emissions during the period, but also the carbon emissions per functional unit.

How do I choose plugins?

Firstly, have a look at what plugins are available. Plugins maintained by the Green Software Foundation are listed in their if-models repository. In the early days of the Impact Framework, other organizations’ plugins were put in the if-unofficial-models repository, but increasingly these are now found in organizations’ own GitHub spaces. See this list compiled by the Green Software Foundation for a summary of what’s currently available.

Choose a plugin that corresponds to the website component you want to examine. For static sites or application front ends, look at the if-webpage-plugins MeasureWebpage plugin, which calculates the transfer weight of a website in bytes, also modeling the impact of repeat visits. If you’re interested in the impact of server-side activities, the Cloud Carbon Footprint calculates the operational energy and embodied carbon impact of a cloud-hosted application.

Of course, you may not be able to find the perfect plugin for your website. You can always have a go at writing your own!

How do I turn plugins into a pipeline?

A good place to start is looking into existing examples of pipelines that you can use or adapt. You can find sample pipelines in the example manifests shared by the Green Software Foundation. Many of the plugin developers have also shared example pipelines and manifests. For example, if you are interested in estimating the impact of your webpage across all segments of the internet (from consumer device use, network use, data center use and hardware production), the if-webpage-plugins repo has a sample pipeline.

The measure-webpage and green-hosting plugin gather information that is needed to estimate the carbon impact of your webpage. These parameters are then passed to the sustainable web design model (swd) that is included in the co2js plugin. You can configure further optional parameters of the swd model as you see fit for your use case (see co2js README). This pipeline follows the same approach as for example websitecarbon.com and ecograder.com. It has the advantage that you can fully configure the parameters that go into your swd model / co2js plugin and make them transparent in the manifest file. The MeasureWebpage Plugin also makes an attempt to approximate the amount of reloaded data on a repeat visit. And if you wish you can also generate an additional lighthouse report.

The timer steps are auxiliary methods that create an accurate timestamp and duration for your measurement.

For further details on the estimation approach of the swd model, check out this link.

Alternatively, if you are investigating the impact of your website server, you could start with the pipeline from our video. This calculates the carbon dioxide equivalent that would be emitted by a theoretical cloud-hosted application.

Here’s the existing pipeline:

If you were going to adapt this pipeline, you could replace the mock-observations plugin with real observations from your own website server. Or, if you want to expand the outputs of the pipeline to account for embodied carbon and calculate a Software Carbon Intensity Framework (SCI) impact, you could use elements from this pipeline.

The other option is to create a pipeline for your plugins from scratch. Once you have chosen the plugins you need, you can find others by looking at the data inputted or outputted. Either find plugins that can provide the data your plugin needs, or ones that can transform the data your plugin generates. Do this step by step and you can build a pipeline from data you can observe to a specific impact calculation (e.g. of grams of carbon per network request).

If you’re looking into more than one website component, you may use a different pipeline for each. For example, one pipeline built around the if-webpage-plugins example for calculating the impact of your front end and another built around the pipeline in our video for the impact of your server-side processing.

Stage 3: Collect your input data

The plugins in the impact framework have various data input requirements. Some of these will be provided by the other plugins in your pipeline. But at some point you’ll have to get some information about your service. As there are so many plugins and more are being developed, we can’t account for every case, but there are some key places you can look for observational data.

If you’re collecting data about a single-page application or the front end of a traditional website, then you might want to look at bytes transferred. You could try this yourself using your browser’s Developer Tools or use a tool like Puppeteer. This is what the MeasureWebpage plugin uses under the hood, and you can use this to return the size of a website by providing a URL.

Moving to the back end, plugins calculating the energy usage of cloud-hosted servers such as Cloud Carbon Footprint might ask for information about the cloud instance where your project is hosted. For example, the name of the instance type, a figure for the percentage CPU utilization of the server and how long the CPU utilization was observed (the ‘duration’). To get these results, you’ll need access to either the web console (eg: AWS, Azure or Google Cloud Platform) or the CLI for your cloud provider. If you don’t have this access, talk to a colleague or supplier with that access to either give you permissions to get the data or retrieve it for you. Your cloud provider should have guidance to help you find the data you need (e.g. for CPU utilization: AWS, Azure and Google Cloud Platform), and for common cloud providers the answer might be one internet search away.

Stage 4: Write up the manifest file

Once you’ve made your decisions about which plugins to include and how they would link together into pipelines, it’s time to write the manifest file. This is a .yaml file that is used to tell the Impact Framework what you want to calculate, how you want to calculate it and any observational data you’re giving.

As the Impact Framework works as an automated process, it has expectations of the structure of this input file. At the highest level, it is broken down into three parts:

1. Metadata: basic information about your project
2. Initialize: this is where you specify the plugins you will be using in your pipelines
3. Tree: this is used for defining what you will be measuring

We’ll be looking at Mike’s manifest from the video as an example, but you can also find other examples to help you build your manifest.

Metadata

.yaml example

# PART 1: Metadata
# This is meant to be read by humans, so can contain
# any information you like
name: My Manifest File
description: My First Manifest File 💚

The information in the metadata section is not used directly by the Impact Framework, so you can be more flexible with what it contains. It’s designed to be a guide for people outside your project looking at the manifest file to explain what your website does.

Initialize

.yaml example

# Part 2: Initialize
# This is where you set up your plugins
initialize:
 outputs: ['yaml']
 plugins:
   group-by:
     path: 'builtin'
     method: GroupBy
   operational-carbon:
     path: '@grnsft/if-plugins'
     method: Multiply
     global-config:
       input-parameters: ['cpu/energy', 'grid/carbon-intensity']
       output-parameter: 'carbon'
   watttime:
     path: '@grnsft/if-unofficial-plugins'
     method: WattTimeGridEmissions
   teads-curve:
     path: '@grnsft/if-unofficial-plugins'
     method: TeadsCurve
     global-config:
       interpolation: spline
   cloud-metadata:
     method: CloudMetadata
     path: "@grnsft/if-plugins"
   mock-observations:
     path: '@grnsft/if-plugins'
     method: MockObservations
     global-config:
       timestamp-from: '2024-03-05T00:00:00.000Z'
       timestamp-to: '2024-03-05T00:01:00.000Z'
       duration: 10
       components:
         - name: server-1
           cloud/instance-type: Standard_E64_v3
           cloud/region: westus3
         - name: server-2
           cloud/instance-type: Standard_E64_v3
           cloud/region: westus3
       generators:
         common:
           cloud/vendor: azure
         randint:
           cpu/utilization:
             min: 1
             max: 99
   'time-sync':
     method: TimeSync
     path: "builtin"
     global-config:
       start-time: '2024-03-05T00:00:00.000Z'
       end-time: '2024-03-05T00:01:00.000Z'
       interval: 5
       allow-padding: true

Plugins must be listed here before they can be used in a pipeline. For each plugin, you should separately give the path, method and (if required) global-config.

Tree

.yaml example

# Part 3: Tree
# This is where you list what you will be measuring
# In more complex applications, you might have several nested components
# listed in the tree, each with their own pipeline, defaults, config, inputs etc.
tree:
 pipeline:
   - mock-observations #provides sample usage data from a given geographic region with a simulated cpu / memory utilization.
   - group-by #groups inputs, in this case cloud and region
   - time-sync #standardizes the time format of observations
   - cloud-metadata #determine an instance's physical processor and thermal design power (TDP) based on its instance name (cloud vendor e.g. aws and instance type e.g. m5.large)
   - watttime #determine the carbon intensity of the grid based on the region (e.g. CAISO)
   - teads-curve #estimating CPU usages across varying type of CPUs using a Teads Curve (requires CPU TDP, Utilization, and Duration)
   - operational-carbon #combines two or more elements in an array, in this case energy in kWH and grid carbon intensity)
 defaults:
 config:
   group-by:
     group:
       - cloud/region
       - name
 inputs: null

If you are examining a number of different website components (e.g. front and back end; production and development environments) each with different plugins and pipelines, you might create a nested configuration of your components in this tree. We’ve kept it as simple as possible for this demonstration, but if you’re interested in looking at a nested project, there’s an example manifest here.

Configuring Plugins

When we started writing our own manifest files, it was difficult to understand where to put plugin setup information. The GitHub documentation for individual plugins usually states where in the manifest file data and configuration should go. However, it’s worth understanding the difference between the multiple places where configuration can occur within a manifest file:

1. Global config: Used for data that is identical wherever the plugin is invoked in any pipeline anywhere in the manifest, even if it is invoked several times across multiple components (e.g. in a frontend component, a backend component and a database component). An example could be the interpolation method used to generate values in the `teads-curve` plugin — it is one choice that applies everywhere.

2. Node config: This is for data that stays constant from timestep to timestep, but varies across invocations within a manifest. Maybe you want to use the same plugin but with different configuration in your frontend, backend and database components. An example might be operational-carbon — you might want to invoke the plugin for every component, but each one is located somewhere different in the world so you provide your grid carbon intensity value at the node level.

3. Input data: The plugins all also receive data from the array of timestamped observations in the manifest file. This is data that can vary from one timestep to the next.

4. Defaults: You can also provide a set of default values in each component that get injected into each element in the inputs array IF that value does not already exist. This can be used as a fallback in case an API fails, or it can be used as a convenience tool to quickly add values to every timestep across the array.

Stage 5: Run the Impact Framework

Once you have your manifest ready with the necessary data, you are ready to run the Impact Framework tool to let the plugins do their calculations. Watch Mike take you through this process step by step in his video. He’ll show you:

• What you need to install to run the Impact Framework and how to do this
• What’s included in his manifest file and why
• How to set up a WattTime account to use their plugin
• How to run the command-line prompts (note: this is currently for a Mac/Linux device; Windows users may have to look up alternative commands)
• How to make sense of the data that’s returned.

If you’ve got this far with your Impact Framework journey, congratulations!

Take a look at your output calculation of your project or component’s impact. Your next steps are to find ways to lower that impact. Now that you’ve worked with it, if you have ideas of how to improve the Impact Framework, please share them with the wider community.

We can’t wait to see where the Framework goes next.