UM  > Faculty of Science and Technology
Sampling sparse representations with randomized measurement langevin dynamics
Wang,Kafeng1; Xiong,Haoyi2; Bian,Jiang3; Zhu,Zhanxing4; Gao,Qian5; Guo,Zhishan3; Xu,Cheng Zhong6; Huan,Jun7; Dou,Dejing2
Source PublicationACM Transactions on Knowledge Discovery from Data
AbstractStochastic Gradient Langevin Dynamics (SGLD) have been widely used for Bayesian sampling from certain probability distributions, incorporating derivatives of the log-posterior. With the derivative evaluation of the log-posterior distribution, SGLD methods generate samples from the distribution through performing as a thermostats dynamics that traverses over gradient flows of the log-posterior with certainly controllable perturbation. Even when the density is not known, existing solutions still can first learn the kernel density models from the given datasets, then produce new samples using the SGLD over the kernel density derivatives. In this work, instead of exploring new samples from kernel spaces, a novel SGLD sampler, namely, Randomized Measurement Langevin Dynamics (RMLD) is proposed to sample the high-dimensional sparse representations from the spectral domain of a given dataset. Specifically, given a random measurement matrix for sparse coding, RMLD first derives a novel likelihood evaluator of the probability distribution from the loss function of LASSO, then samples from the high-dimensional distribution using stochastic Langevin dynamics with derivatives of the logarithm likelihood and Metropolis-Hastings sampling. In addition, new samples in low-dimensional measuring spaces can be regenerated using the sampled high-dimensional vectors and the measurement matrix. The algorithm analysis shows that RMLD indeed projects a given dataset into a high-dimensional Gaussian distribution with Laplacian prior, then draw new sparse representation from the dataset through performing SGLD over the distribution. Extensive experiments have been conducted to evaluate the proposed algorithm using real-world datasets. The performance comparisons on three real-world applications demonstrate the superior performance of RMLD beyond baseline methods.
KeywordCompressive sensing Hamiltonian Monte Carlo LASSO
URLView the original
Scopus ID2-s2.0-85103959900
Fulltext Access
Citation statistics
Cited Times [WOS]:0   [WOS Record]     [Related Records in WOS]
Document TypeJournal article
CollectionFaculty of Science and Technology
Corresponding AuthorXiong,Haoyi
Affiliation1.Shenzhen Institutes of Advanced Technology,Chinese Academy of Sciences,University of Chinese Academy of Sciences,Shenzhen, Guangdong,518055,China
2.Big Data Lab,Baidu Inc.,Haidian, Beijing,100193,China
3.Department of Electrical and Computer Engineering,University of Central Florida,Orlando,32816,United States
4.School of Mathematical Sciences,Peking University,Haidian, Beijing,100871,China
5.Business Group of Baidu Search,Baidu Inc.,Haidian, Beijing,100193,China
6.State Key Laboratory of Internet of Things for Smart City,Faculty of Science and Technology,University of Macau,Taipa,Macao
7.StylingAI Inc.,Haidian, Beijing,100193,China
Recommended Citation
GB/T 7714
Wang,Kafeng,Xiong,Haoyi,Bian,Jiang,et al. Sampling sparse representations with randomized measurement langevin dynamics[J]. ACM Transactions on Knowledge Discovery from Data,2021,15(2).
APA Wang,Kafeng,Xiong,Haoyi,Bian,Jiang,Zhu,Zhanxing,Gao,Qian,Guo,Zhishan,Xu,Cheng Zhong,Huan,Jun,&Dou,Dejing.(2021).Sampling sparse representations with randomized measurement langevin dynamics.ACM Transactions on Knowledge Discovery from Data,15(2).
MLA Wang,Kafeng,et al."Sampling sparse representations with randomized measurement langevin dynamics".ACM Transactions on Knowledge Discovery from Data 15.2(2021).
Files in This Item:
There are no files associated with this item.
Related Services
Recommend this item
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Wang,Kafeng]'s Articles
[Xiong,Haoyi]'s Articles
[Bian,Jiang]'s Articles
Baidu academic
Similar articles in Baidu academic
[Wang,Kafeng]'s Articles
[Xiong,Haoyi]'s Articles
[Bian,Jiang]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Wang,Kafeng]'s Articles
[Xiong,Haoyi]'s Articles
[Bian,Jiang]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.