Blind recognition of touched keys on mobile devices
Yue, Qinggang1; Ling, Zhen2; Fu, Xinwen1; Liu, Benyuan1; Ren, Kui3; Zhao, Wei4
Conference Name21st ACM Conference on Computer and Communications Security, CCS 2014
Source PublicationProceedings of the ACM Conference on Computer and Communications Security
Conference Date11 3, 2014 - 11 7, 2014
Conference PlaceScottsdale, AZ, United states
Author of SourceAssociation for Computing Machinery
AbstractIn this paper, we introduce a novel computer vision based attack that automatically discloses inputs on a touch-enabled device while the attacker cannot see any text or popup in a video of the victim tapping on the touch screen. We carefully analyze the shadow formation around the fingertip, apply the optical flow, deformable part-based model (DPM), k-means clustering and other computer vision techniques to automatically locate the touched points. Planar ho-mography is then applied to map the estimated touched points to a reference image of software keyboard keys. Recognition of passwords is extremely challenging given that no language model can be applied to correct estimated touched keys. Our threat model is that a webcam, smartphone or Google Glass is used for stealthy attack in scenarios such as conferences and similar gathering places. We address both cases of tapping with one finger and tapping with multiple fingers and two hands. Extensive experiments were performed to demonstrate the impact of this attack. The per-character (or per-digit) success rate is over 97% while the success rate of recognizing 4-character passcodes is more than 90%. Our work is the first to automatically and blindly recognize random passwords (or passcodes) typed on the touch screen of mobile devices with a very high success rate. Copyright © 2014 ACM.
Fulltext Access
Citation statistics
Cited Times [WOS]:30   [WOS Record]     [Related Records in WOS]
Document TypeConference paper
CollectionUniversity of Macau
Affiliation1.University of Massachusetts, Lowell, United States;
2.Southeast University, China;
3.University at Buffalo, United States;
4.University of Macau, China
Recommended Citation
GB/T 7714
Yue, Qinggang,Ling, Zhen,Fu, Xinwen,et al. Blind recognition of touched keys on mobile devices[C]//Association for Computing Machinery,2017:1403-1414.
Files in This Item:
There are no files associated with this item.
Related Services
Recommend this item
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Yue, Qinggang]'s Articles
[Ling, Zhen]'s Articles
[Fu, Xinwen]'s Articles
Baidu academic
Similar articles in Baidu academic
[Yue, Qinggang]'s Articles
[Ling, Zhen]'s Articles
[Fu, Xinwen]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Yue, Qinggang]'s Articles
[Ling, Zhen]'s Articles
[Fu, Xinwen]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.