TY - GEN

T1 - Extreme-Scale Task-Based Cholesky Factorization Toward Climate and Weather Prediction Applications

AU - Cao, Qinglei

AU - Pei, Yu

AU - Akbudak, Kadir

AU - Mikhalev, Aleksandr

AU - Bosilca, George

AU - Ltaief, Hatem

AU - Keyes, David E.

AU - Dongarra, Jack

N1 - KAUST Repository Item: Exported on 2020-10-01

PY - 2020/6/18

Y1 - 2020/6/18

N2 - Climate and weather can be predicted statistically via geospatial Maximum Likelihood Estimates (MLE), as an alternative to running large ensembles of forward models. The MLE-based iterative optimization procedure requires the solving of large-scale linear systems that performs a Cholesky factorization on a symmetric positive-definite covariance matrix—a demanding dense factorization in terms of memory footprint and computation. We propose a novel solution to this problem: at the mathematical level, we reduce the computational requirement by exploiting the data sparsity structure of the matrix off-diagonal tiles by means of low-rank approximations; and, at the programming-paradigm level, we integrate PaRSEC, a dynamic, task-based runtime to reach unparalleled levels of efficiency for solving extreme-scale linear algebra matrix operations. The resulting solution leverages fine-grained computations to facilitate asynchronous execution while providing a flexible data distribution to mitigate load imbalance. Performance results are reported using 3D synthetic datasets up to 42M geospatial locations on 130, 000 cores, which represent a cornerstone toward fast and accurate predictions of environmental applications.

AB - Climate and weather can be predicted statistically via geospatial Maximum Likelihood Estimates (MLE), as an alternative to running large ensembles of forward models. The MLE-based iterative optimization procedure requires the solving of large-scale linear systems that performs a Cholesky factorization on a symmetric positive-definite covariance matrix—a demanding dense factorization in terms of memory footprint and computation. We propose a novel solution to this problem: at the mathematical level, we reduce the computational requirement by exploiting the data sparsity structure of the matrix off-diagonal tiles by means of low-rank approximations; and, at the programming-paradigm level, we integrate PaRSEC, a dynamic, task-based runtime to reach unparalleled levels of efficiency for solving extreme-scale linear algebra matrix operations. The resulting solution leverages fine-grained computations to facilitate asynchronous execution while providing a flexible data distribution to mitigate load imbalance. Performance results are reported using 3D synthetic datasets up to 42M geospatial locations on 130, 000 cores, which represent a cornerstone toward fast and accurate predictions of environmental applications.

UR - http://hdl.handle.net/10754/656453

UR - https://dl.acm.org/doi/10.1145/3394277.3401846

U2 - 10.1145/3394277.3401846

DO - 10.1145/3394277.3401846

M3 - Conference contribution

SN - 9781450379939

BT - Proceedings of the Platform for Advanced Scientific Computing Conference

PB - ACM

ER -