IEEE Geoscience and Remote Sensing Magazine

Big Data Challenges in Climate Science
Schnase JL, Lee TJ, Mattmann CA, Lynnes CS, Cinquini L, Ramirez PM, Hart AF, Williams DN, Waliser D, Rinsland P, Webster WP, Duffy DQ, McInerney MA, Tamkin GS, Potter GL and Carrier L
The knowledge we gain from research in climate science depends on the generation, dissemination, and analysis of high-quality data. This work comprises technical practice as well as social practice, both of which are distinguished by their massive scale and global reach. As a result, the amount of data involved in climate research is growing at an unprecedented rate. Climate model intercomparison (CMIP) experiments, the integration of observational data and climate reanalysis data with climate model outputs, as seen in the Obs4MIPs, Ana4MIPs, and CREATE-IP activities, and the collaborative work of the Intergovernmental Panel on Climate Change (IPCC) provide examples of the types of activities that increasingly require an improved cyberinfrastructure for dealing with large amounts of critical scientific data. This paper provides an overview of some of climate science's big data problems and the technical solutions being developed to advance data publication, climate analytics as a service, and interoperability within the Earth System Grid Federation (ESGF), the primary cyberinfrastructure currently supporting global climate research activities.