Interruptible Tasks

Lu Fang,Khanh Nguyen,Guoqing Xu,Brian Demsky,Shan Lu
DOI: https://doi.org/10.1145/2815400.2815407
2015-01-01
Abstract:Real-world data-parallel programs commonly suffer from great memory pressure, especially when they are executed to process large datasets. Memory problems lead to excessive GC effort and out-of-memory errors, significantly hurting system performance and scalability. This paper proposes a systematic approach that can help data-parallel tasks survive memory pressure, improving their performance and scalability without needing any manual effort to tune system parameters. Our approach advocates interruptible task (ITask), a new type of data-parallel tasks that can be interrupted upon memory pressure---with part or all of their used memory reclaimed---and resumed when the pressure goes away.
What problem does this paper attempt to address?