Incremental optimization mechanism for constructing a balanced very fast decision tree for big data
Yang H.2; Fong S.1
AbstractBig data is a popular topic that highly attracts the attentions of researchers from all over the world. How to mine valuable information from such huge volumes of data remains an open problem. As the most widely used technology of decision tree, imperfect data stream leads to tree size explosion and detrimental accuracy problems. Over-fitting problem and the imbalanced class distribution reduce the performance of the original decision tree algorithm for stream mining. In this chapter, we propose an Optimized Very Fast Decision Tree (OVFDT) that possesses an optimized node-splitting control mechanism using Hoeffding bound. Accuracy, tree size, and learning time are the significant factors influencing the algorithm’s performance. Naturally, a bigger tree size takes longer computation time. OVFDT is a pioneer model equipped with an incremental optimization mechanism that seeks for a balance between accuracy and tree size for data stream mining. OVFDT operates incrementally by a test-then-train approach. Two new methods of functional tree leaves are proposed to improve the accuracy with which the tree model makes a prediction for a new data stream in the testing phase. The optimized node-splitting mechanism controls the tree model growth in the training phase. The experiment shows that OVFDT obtains an optimal tree structure in numeric and nominal datasets.
URLView the original
Fulltext Access
Citation statistics
Document TypeBook
CollectionUniversity of Macau
Affiliation1.Universidade de Macau
2.China Southern Power Grid
Recommended Citation
GB/T 7714
Yang H.,Fong S.. Incremental optimization mechanism for constructing a balanced very fast decision tree for big data[M],2017.
Files in This Item:
There are no files associated with this item.
Related Services
Recommend this item
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Yang H.]'s Articles
[Fong S.]'s Articles
Baidu academic
Similar articles in Baidu academic
[Yang H.]'s Articles
[Fong S.]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Yang H.]'s Articles
[Fong S.]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.