UM  > 科技學院
TQDL: Integrated Models for Cross-Language Document Retrieval
Long-Yue WANG; Derek F. WONG; Lidia S. CHAO
Source PublicationInternational Journal of Computational Linguistics and Chinese Language Processing

This paper proposed an integrated approach for Cross-Language Information Retrieval (CLIR), which integrated with four statistical models: Translation model, Query generation model, Document retrieval model and Length Filter model. Given a certain document in the source language, it will be translated into the target language of the statistical machine translation model. The query generation model then selects the most relevant words in the translated version of the document as a query. Instead of retrieving all the target documents with the query, the length-based model can help to filter out a large amount of irrelevant candidates according to their length information. Finally, the left documents in the target language are scored by the document searching model, which mainly computes the similarities between query and document. Different from the traditional parallel corpora-based model which relies on IBM algorithm, we divided our CLIR model into four independent parts but all work together to deal with the term disambiguation, query generation and document retrieval. Besides, the TQDL method can efficiently solve the problem of translation ambiguity and query expansion for disambiguation, which are the big issues in Cross-Language Information Retrieval. Another contribution is the length filter, which are trained from a parallel corpus according to the ratio of length between two languages. This can not only improve the recall value due to filtering out lots of useless documents dynamically, but also increase the efficiency in a smaller search space. Therefore, the precision can be improved but not at the cost of recall.In order to evaluate the retrieval performance of the proposed model on cross-languages document retrieval, a number of experiments have been conducted on different settings. Firstly, the Europarl corpus which is the collection of parallel texts in 11 languages from the proceedings of the European Parliament was used for evaluation. And we tested the models extensively to the case that: the lengths of texts are uneven and some of them may have similar contents under the same topic, because it is hard to be distinguished and make full use of the resources. After comparing different strategies, the experimental results show a significant performance of the method. The precision is normally above 90% by using a larger query size. The length-based filter plays a very important role in improving the F-measure and optimizing efficiency. This fully illustrates the discrimination power of the proposed method. It is of a great significance to both cross-language searching on the Internet and the parallel corpus producing for statistical machine translation systems. In the future work, the TQDL system will be evaluated for Chinese language, which is a big changing and more meaningful to CLIR.

KeywordCross-language Document Retrieval Length-based Filter Document Translation-based Tf-idf Statistical Machine Translation
Fulltext Access
Citation statistics
Document TypeJournal article
CollectionFaculty of Science and Technology
AffiliationNatural Language Processing & Portuguese-Chinese Machine Translation Laboratory, Department of Computer and Information Science, University of Macau, Macau S. A. R., China
First Author AffilicationUniversity of Macau
Recommended Citation
GB/T 7714
Long-Yue WANG,Derek F. WONG,Lidia S. CHAO. TQDL: Integrated Models for Cross-Language Document Retrieval[J]. International Journal of Computational Linguistics and Chinese Language Processing,2012,17(4):15-32.
APA Long-Yue WANG,Derek F. WONG,&Lidia S. CHAO.(2012).TQDL: Integrated Models for Cross-Language Document Retrieval.International Journal of Computational Linguistics and Chinese Language Processing,17(4),15-32.
MLA Long-Yue WANG,et al."TQDL: Integrated Models for Cross-Language Document Retrieval".International Journal of Computational Linguistics and Chinese Language Processing 17.4(2012):15-32.
Related Services
Recommend this item
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Long-Yue WANG]'s Articles
[Derek F. WONG]'s Articles
[Lidia S. CHAO]'s Articles
Baidu academic
Similar articles in Baidu academic
[Long-Yue WANG]'s Articles
[Derek F. WONG]'s Articles
[Lidia S. CHAO]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Long-Yue WANG]'s Articles
[Derek F. WONG]'s Articles
[Lidia S. CHAO]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.