UM  > 科技學院  > 電腦及資訊科學系
Empirical kernel map-based multilayer extreme learning machines for representation learning
Chi-Man Vong1; Chen, Chuangquan1; Wong, Pak-Kin2
2018-10-08
Source PublicationNEUROCOMPUTING
ISSN0925-2312
Volume310Pages:265-276
Abstract

Recently, multilayer extreme learning machine (ML-ELM) and hierarchical extreme learning machine (HELM) were developed for representation learning whose training time can be reduced from hours to seconds compared to traditional stacked autoencoder (SAE). However, there are three practical issues in ML-ELM and H-ELM: (1) the random projection in every layer leads to unstable and suboptimal performance; (2) the manual tuning of the number of hidden nodes in every layer is time-consuming; and (3) under large hidden layer, the training time becomes relatively slow and a large storage is necessary. More recently, issues (1) and (2) have been resolved by kernel method, namely, multilayer kernel ELM (ML-KELM), which encodes the hidden layer in form of a kernel matrix (computed by using kernel function on the input data), but the storage and computation issues for kernel matrix pose a big challenge in large-scale application. In this paper, we empirically show that these issues can be alleviated by encoding the hidden layer in the form of an approximate empirical kernel map (EKM) computed from low-rank approximation of the kernel matrix. This proposed method is called ML-EKM-ELM, whose contributions are: (1) stable and better performance is achieved under no random projection mechanism; (2) the exhaustive manual tuning on the number of hidden nodes in every layer is eliminated; (3) EKM is scalable and produces a much smaller hidden layer for fast training and low memory storage, thereby suitable for large-scale problems. Experimental results on benchmark datasets demonstrated the effectiveness of the proposed ML-EKM-ELM. As an illustrative example, on the NORB dataset, ML-EKM-ELM can be respectively up to 16 times and 37 times faster than ML-KELM for training and testing with a little loss of accuracy of 0.35%, while the memory storage can be reduced up to 1/9. (C) 2018 Elsevier B.V. All rights reserved.

KeywordKernel Learning Multilayer Extreme Learning Machine (Ml-elm) Empirical Kernel Map (Ekm) Representation Learning Stacked Autoencoder (Sae)
DOI10.1016/j.neucom.2018.05.032
URLView the original
Indexed BySCI
Language英语
WOS Research AreaComputer Science
WOS SubjectComputer Science, Artificial Intelligence
WOS IDWOS:000437299800023
PublisherELSEVIER SCIENCE BV
The Source to ArticleWOS
Fulltext Access
Citation statistics
Cited Times [WOS]:10   [WOS Record]     [Related Records in WOS]
Document TypeJournal article
CollectionDEPARTMENT OF COMPUTER AND INFORMATION SCIENCE
DEPARTMENT OF ELECTROMECHANICAL ENGINEERING
Corresponding AuthorChi-Man Vong
Affiliation1.Department of Computer and Information Science, University of Macau, Macau
2.Department of Electromechanical Engineering, University of Macau, Macau
First Author AffilicationUniversity of Macau
Corresponding Author AffilicationUniversity of Macau
Recommended Citation
GB/T 7714
Chi-Man Vong,Chen, Chuangquan,Wong, Pak-Kin. Empirical kernel map-based multilayer extreme learning machines for representation learning[J]. NEUROCOMPUTING,2018,310:265-276.
APA Chi-Man Vong,Chen, Chuangquan,&Wong, Pak-Kin.(2018).Empirical kernel map-based multilayer extreme learning machines for representation learning.NEUROCOMPUTING,310,265-276.
MLA Chi-Man Vong,et al."Empirical kernel map-based multilayer extreme learning machines for representation learning".NEUROCOMPUTING 310(2018):265-276.
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Chi-Man Vong]'s Articles
[Chen, Chuangquan]'s Articles
[Wong, Pak-Kin]'s Articles
Baidu academic
Similar articles in Baidu academic
[Chi-Man Vong]'s Articles
[Chen, Chuangquan]'s Articles
[Wong, Pak-Kin]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Chi-Man Vong]'s Articles
[Chen, Chuangquan]'s Articles
[Wong, Pak-Kin]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.