Building cloud-based data services to enable earth-science workflows across HPC centres

This resource first appeared in issue #6 on 14 Feb 2020 and has tags Technical Leadership: Cloud, Technical Leadership: HPC

Building cloud-based data services to enable earth-science workflows across HPC centres - John Hanley, ECMWF

Also from both the UK and FOSDEM, this is a really nice overview of a very sophisticated solution to making archival simulation data from the outputs of the European Centre for Medium-Range Weather Forecasts available in a cloud environment for querying and reanalysis. Groups like ECMWF run with operational requirements that would keep most of us awake at nights in panic-induced sweats - it turns out that governments, companies, navies etc. really care about having their weather forecasts correct and timely - and so while they have to push 100TB daily around the globe for their users, they also want to make the most of all that compute and data. So there is cloud environment with those outputs (that you can play with! https://pypi.org/project/ecmwf-api-client/ https://apps.ecmwf.int/datasets/) for secondary analysis and pilot projects. A really ambitious and interesting project.