Eurostat European Big Data Hackathon 2025

Eurostat European Big Data Hackathon 2025#

Eurostat is organizing the fifth round of the European Big Data Hackathon from 6 to 11 March 2025 (including the presentation by the winners at the NTTS) in Brussels.

The purpose of the 2025 hackathon is to foster expertise in integrating Earth Observation data with official statistics for producing innovative ideas for statistical products and tools relevant for the EU policies.

The European Big Data Hackathon takes place every two years and gathers teams from all over Europe to compete for the best solution to a statistical challenge. The teams develop innovative approaches, applications and data products combining official statistics and big data that can help to answer pressing EU policy and/or statistical questions.

Source: https://cros.ec.europa.eu/2025EuropeanBigDataHackathon

How to download, visualise and run some basic statistics on ERA5 data#

Written by William Ray for the participants of the 5th European Big Data Hackathon 2025.

In this notebook you are shown how to:

Download ERA5 data from the Climate Data Store using API.
How to visualise the downloaded data using xarray.
Run some basic statistics such as the average temperature over a year.
Visualise the average temperature difference between 1950 and 2020.

Before you get started you will need to create an account at the Climate Data Store.

# Python Standard Libraries
import os
import datetime as dt

# Data Manipulation Libraries
import numpy as np
import xarray as xr

# Visualization Libraries
import matplotlib.pyplot as plt
import cartopy
import cartopy.crs as ccrs


# Climate Data Store API for retrieving climate data
import cdsapi

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
Cell In[1], line 6
      3 import datetime as dt
      5 # Data Manipulation Libraries
----> 6 import numpy as np
      7 import xarray as xr
      9 # Visualization Libraries

ModuleNotFoundError: No module named 'numpy'

Downloading the Data#

First, we’ll load ERA5 data from the Climate Data Store (CDS) using the cdsapi, including the land-sea mask. To do this, save your CDS API key in the $HOME/.cdsapirc file. In addition you have to have accepted Terms of use in the CDS portal.

New to CDS? Consider reading the CDS tutorial for a detailed guide.

file_name = {}  # dictionary containing [data source : file name]

# Add the data sources and file names
file_name.update({"era5": "temperature_era5.nc"})

# Create the paths to the files
path_to = {
    source: os.path.join(f"data/{source}/", file) for source, file in file_name.items()
}

# Create necessary directories if they do not exist
for path in path_to.values():
    os.makedirs(
        os.path.dirname(path), exist_ok=True
    )  # create the folder if not available

path_to

{'era5': 'data/era5/temperature_era5.nc'}

%%time 

client = cdsapi.Client()

dataset = 'reanalysis-era5-single-levels-monthly-means'
request = {
  'product_type': ['monthly_averaged_reanalysis'],
  'variable': ['2m_temperature', 'land_sea_mask'],
  'year': ['1950', '2020'],
  'month': list(range(1, 13)),
  'time': '00:00',
  'data_format': 'netcdf',
  'download_format': 'unarchived' 
}
target = path_to['era5']

client.retrieve(dataset, request, target)

2025-02-13 07:33:55,201 INFO [2024-09-26T00:00:00] Watch our [Forum](https://forum.ecmwf.int/) for Announcements, news and other discussed topics.
2025-02-13 07:33:55,202 WARNING [2024-06-16T00:00:00] CDS API syntax is changed and some keys or parameter names may have also changed. To avoid requests failing, please use the "Show API request code" tool on the dataset Download Form to check you are using the correct syntax for your API request.
2025-02-13 07:33:55,407 INFO Request ID is 06878979-6513-4d60-b490-559df4414b76
2025-02-13 07:33:55,475 INFO status has been updated to accepted
2025-02-13 07:34:03,843 INFO status has been updated to running
2025-02-13 07:34:09,065 INFO status has been updated to successful

CPU times: user 151 ms, sys: 70.2 ms, total: 222 ms
Wall time: 15.5 s

'data/era5/temperature_era5.nc'

Eurostat European Big Data Hackathon 2025

Contents

Eurostat European Big Data Hackathon 2025#

How to download, visualise and run some basic statistics on ERA5 data#

Downloading the Data#

Opening the dataset#

Calculate the yearly average for 1950#

Calculate the yearly average for 2020 and then visualise the difference between the average temperature in 1950 and 2020#