CATS: Climate-Aware Task Scheduler for environmentally conscious developers
Bartholomew, Sadie L.; Colling, Lincoln; Dasgupta, Abhishek; Greenberg, Anthony J.; Lannelongue, Loïc; Lestang, Thibault; Martinez-Ortiz, Carlos; Sauzé, Colin; Walker, Andrew M.
- Publisher
- Zenodo
- Language
- en
Abstract
The Climate-Aware Task Scheduler is a lightweight Python package designed to schedule tasks based on the estimated carbon intensity of the electricity grid at any given moment. This tool uses real-time carbon intensity data from the UK's National Grid ESO via their API to estimate the carbon intensity of the electricity grid, and schedules tasks at times when the estimated carbon intensity is lowest. This helps to reduce the carbon emissions associated with running computationally intensive tasks, making it an ideal solution for environmentally conscious developers. The source code can be found at https://github.com/GreenScheduler/cats.Acknowledgements We are grateful to the staff of the Software Sustainability Institute and the organisers of Collaborations Workshop 2023 (CW23), whose efforts allowed us to enjoy the process of beginning the development of CATS as part of the CW23 Hack Day, and to others who contributed to the development via bug reports, questions, and the other contributions that help open-source software evolve. The author list is in alphabetical order. This work has been supported by the Software Sustainability Institute EPSRC, BBSRC, ESRC, NERC, AHRC, STFC and MRC (EP/S021779/1) and UKRI (AH/Z000114/1) grants.
Full text
CATS: Climate-Aware Task Scheduler for environmentally conscious developers Border image credits: ‘Climate Stripes’ infographic designed by Prof. Ed Hawkins (University of Reading), see showyourstripes.info Sadie L. Bartholomew1, Lincoln Colling2, Abhishek Dasgupta3, Anthony J. Greenberg4, Loïc Lannelongue5, Thibault Lestang6, Carlos Martinez-Ortiz7, Colin Sauzé8, Adam Ward8 , Andrew M. Walker3 1 National Centre for Atmospheric Science, United Kingdom. 2 University of Sussex, United Kingdom. 3 University of Oxford, United Kingdom. 4 Bayesic Research, USA. 5 University of Cambridge, United Kingdom. 6 Météo-France, France. 7 Netherlands eScience Center, Netherlands. 8 National Oceanography Centre, United Kingdom the world us doing our computational work (High performance) computing always requires energy (electricity etc.) - how can we do it in a sustainable way to not exacerbate the climate crisis? Image credits: https://i.imgflip.com/208mpa.jpg, from IT Crowd (Channel 4) How much impact does (High Performance) Computing have? Breakdown of the four sources of Australian astronomers’ emissions considered in one study from 2019. *Source: B. Li et al., 2023, Toward Sustainable HPC: Carbon Footprint Estimation and Environmental Implications of HPC Systems ✝Source: A. Stevens et al, 2019, The imperative to reduce carbon emissions in astronomy How does CATS work? The Climate Aware Task Scheduler calculates the optimal time to run a job to minimise its carbon intensity. It uses data from the National Energy System Operator (NESO) via it’s carbonintensity.org.uk API. This provides 48 hour carbon intensity forecasts. Instead of running your job immediately, say at this time… …CATS calculates you should run it a bit later to minimise carbon intensity of the job Plot from: https://carbonintensity.org.uk/ Basic CATS usage cats -d <job duration in mins> --loc <postcode> Directly schedule jobs using CATS Use the --scheduler argument. We currently support the at and sbatch. To run a Python script work.py expected to take an hour on a computer in Warwick use: A windy & quite sunny day across UK, Grid carbon intensity varies considerably both geographically and in time. Typical values 0-400g CO2e/kWh. Windy and/or sunny days have low carbon intensity (<50g) and windless cloudy days have a high carbon intensity (>200g). Download CATS You can download CATS from: https://github.com/GreenScheduler/cats Or run: pip install climate-aware-task-scheduler CATS only works when a system isn’t running at 100% load. It is best suited for smaller clusters that aren’t always busy. Currently CATS must be run by the user submitting the job(s). We are working on a SLURM plugin to create a “green” queue that prioritises carbon intensity. Further Reading CATS Documentation - https://cats.readthedocs.io/ JOSS Paper - https://joss.theoj.org/papers/10.21105/joss.08251 What is the problem? A neither windy nor sunny day across UK Limitations cats -d 60 --loc CV4 --scheduler sbatch --command 'python work.py'
