Mining Frequent Patterns from Very High Dimensional Data: A Top-Down Row Enumeration Approach
Hongyan Liu,Jiawei Han,Xin Luna Dong,Shu Zheng
DOI: https://doi.org/10.1137/1.9781611972764.25
2006-01-01
Abstract:Previous chapter Next chapter Full AccessProceedings Proceedings of the 2006 SIAM International Conference on Data Mining (SDM)Mining Frequent Patterns from Very High Dimensional Data: A Top-Down Row Enumeration ApproachHongyan Liu, Jiawei Han, Dong Xin, and Zheng ShaoHongyan Liu, Jiawei Han, Dong Xin, and Zheng Shaopp.282 - 293Chapter DOI:https://doi.org/10.1137/1.9781611972764.25PDFBibTexSections ToolsAdd to favoritesExport CitationTrack CitationsEmail SectionsAboutAbstract Data sets of very high dimensionality, such as microarray data, pose great challenges on efficient processing to most existing data mining algorithms. Recently, there comes a row-enumeration method that performs a bottom-up search of row combination space to find corresponding frequent patterns. Due to a limited number of rows in microarray data, this method is more efficient than column enumerationbased algorithms. However, the bottom-up search strategy cannot take an advantage of user-specified minimum support threshold to effectively prune search space, and therefore leads to long runtime and much memory overhead. In this paper we propose a new search strategy, top-down mining, integrated with a novel rowenumeration tree, which makes full use of the pruning power of the minimum support threshold to cut down search space dramatically. Using this kind of searching strategy, we design an algorithm, TD-Close, to find a complete set of frequent closed patterns from very high dimensional data. Furthermore, an effective closeness-checking method is also developed that avoids scanning the dataset multiple times. Our performance study shows that the TD-Close algorithm outperforms substantially both Carpenter, a bottom-up searching algorithm, and FPclose, a column enumeration-based frequent closed pattern mining algorithm. Previous chapter Next chapter RelatedDetails Published:2006ISBN:978-0-89871-611-5eISBN:978-1-61197-276-4 https://doi.org/10.1137/1.9781611972764Book Series Name:ProceedingsBook Code:PR124Book Pages:xii + 646