Data Object Cache in Spark Computing Engine

CHENKang,WANGBin,FENGLing
DOI: https://doi.org/10.3969/j.issn.1009-6868.2016.02.006
2016-01-01
Abstract:In this paper, Spark paral el computing cluster for memory is studied. Its main work is about modeling and analysis of memory behavior in the computing engine and making the cache strategy automatic. Thus, the scheduler can recognize a valuable data object to be cached in the memory. A new cache replacement algorithm is proposed to replace least recently used (LRU) and have better performance in some applications. Thus, the performance and reliability of the Spark computing engine can be improved.
What problem does this paper attempt to address?