UM  > 科技學院  > 電腦及資訊科學系
Efficient extreme learning machine via very sparse random projection
Chen, Chuangquan; Vong, Chi-Man; Wong, Chi-Man; Wang, Weiru; Wong, Pak-Kin
2018-06
Source PublicationSOFT COMPUTING
ISSN1432-7643
Volume22Issue:11Pages:3563-3574
Abstract

Extreme learning machine (ELM) is a kind of random projection-based neural networks, whose advantages are fast training speed and high generalization. However, three issues can be improved in ELM: (1) the calculation of output weights takes time (with N training samples and L hidden nodes), which is relatively slow to train a model for large N and L; (2) the manual tuning of L is tedious, exhaustive and time-consuming; (3) the redundant or irrelevant information in the hidden layer may cause overfitting and may hinder high generalization. Inspired from compressive sensing theory, we propose an efficient ELM via very sparse random projection (VSRP) called VSRP-ELM for training with large N and L. The proposed VSRP-ELM adds a novel compression layer between the hidden layer and output layer, which compresses the dimension of the hidden layer from to under projection with random sparse-Bernoulli matrix. The advantages of VSRP-ELM are (1) faster training time is obtained for large L; (2) the tuning time of L can be significantly reduced by initializing a large L, and then shrunk to k using just a few trials, while maintaining a comparable result of the original model accuracy; (3) higher generalization may be benefited from the cleaning of redundant or irrelevant information through VSRP. From the experimental results, the proposed VSRP-ELM can speed ELM up to 7 times, while the accuracy can be improved up to 6%.

KeywordExtreme Learning Machine (Elm) Sparse-bernoulli Matrix Very Sparse Random Projection Dimension Reduction Compression Layer
DOI10.1007/s00500-018-3128-7
URLView the original
Indexed BySCI
Language英语
WOS Research AreaComputer Science
WOS SubjectComputer Science, Artificial Intelligence ; Computer Science, Interdisciplinary Applications
WOS IDWOS:000431669200009
PublisherSPRINGER
The Source to ArticleWOS
Fulltext Access
Citation statistics
Cited Times [WOS]:3   [WOS Record]     [Related Records in WOS]
Document TypeJournal article
CollectionDEPARTMENT OF COMPUTER AND INFORMATION SCIENCE
DEPARTMENT OF ELECTROMECHANICAL ENGINEERING
Affiliation1.Department of Computer of Information Science,University of Macau,Taipa,China
2.Department of Electromechanical Engineering,University of Macau,Taipa,China
Recommended Citation
GB/T 7714
Chen, Chuangquan,Vong, Chi-Man,Wong, Chi-Man,et al. Efficient extreme learning machine via very sparse random projection[J]. SOFT COMPUTING,2018,22(11):3563-3574.
APA Chen, Chuangquan,Vong, Chi-Man,Wong, Chi-Man,Wang, Weiru,&Wong, Pak-Kin.(2018).Efficient extreme learning machine via very sparse random projection.SOFT COMPUTING,22(11),3563-3574.
MLA Chen, Chuangquan,et al."Efficient extreme learning machine via very sparse random projection".SOFT COMPUTING 22.11(2018):3563-3574.
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Chen, Chuangquan]'s Articles
[Vong, Chi-Man]'s Articles
[Wong, Chi-Man]'s Articles
Baidu academic
Similar articles in Baidu academic
[Chen, Chuangquan]'s Articles
[Vong, Chi-Man]'s Articles
[Wong, Chi-Man]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Chen, Chuangquan]'s Articles
[Vong, Chi-Man]'s Articles
[Wong, Chi-Man]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.