Automatic Search Guided Code Optimization Framework for Mixed-Precision Scientific Applications.

Jienan Yao,Wei Xue
DOI: https://doi.org/10.1145/3624062.3624108
2023-01-01
Abstract:The rapid development in machine learning (ML) has prompted demand for low-precision arithmetic hardware that can deliver faster computing speed. Weather simulation applications typically exhibit higher sensitivity towards small perturbation on the input data, but the inherent uncertainty paves the way for opportunities in mixed-precision computing (MPC) by trading accuracy for performance. To determine an acceptable precision allocation for variables without degrading the simulation quality, challenges include exploring an exponential search space of mixed-precision configurations. We propose a mixed-precision code tuning framework for automatic search of suitable precision configurations for weather modeling applications with black-box optimization algorithms. These search algorithms demonstrate adequate sample efficiency with favorable optimality in the search space of two weather modeling miniapps, and the search results achieve up to 30% performance gain that stays within the tolerance level, offering a workflow to facilitate the identification of variables sensitive to precision change.
What problem does this paper attempt to address?