Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

A partition-based method for string similarity joins with edit-distance constraints

A partition-based method for string similarity joins with edit-distance constraints A Partition-Based Method for String Similarity Joins with Edit-Distance Constraints GUOLIANG LI, DONG DENG, and JIANHUA FENG, Tsinghua University As an essential operation in data cleaning, the similarity join has attracted considerable attention from the database community. In this article, we study string similarity joins with edit-distance constraints, which find similar string pairs from two large sets of strings whose edit distance is within a given threshold. Existing algorithms are efficient either for short strings or for long strings, and there is no algorithm that can efficiently and adaptively support both short strings and long strings. To address this problem, we propose a new filter, called the segment filter. We partition a string into a set of segments and use the segments as a filter to find similar string pairs. We first create inverted indices for the segments. Then for each string, we select some of its substrings, identify the selected substrings from the inverted indices, and take strings on the inverted lists of the found substrings as candidates of this string. Finally, we verify the candidates to generate the final answer. We devise efficient techniques to select substrings and prove that our method can minimize the number http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png ACM Transactions on Database Systems (TODS) Association for Computing Machinery

A partition-based method for string similarity joins with edit-distance constraints

Loading next page...
 
/lp/association-for-computing-machinery/a-partition-based-method-for-string-similarity-joins-with-edit-GSTEpDEYHC

References

References for this paper are not available at this time. We will be adding them shortly, thank you for your patience.

Publisher
Association for Computing Machinery
Copyright
Copyright © 2013 by ACM Inc.
ISSN
0362-5915
DOI
10.1145/2487259.2487261
Publisher site
See Article on Publisher Site

Abstract

A Partition-Based Method for String Similarity Joins with Edit-Distance Constraints GUOLIANG LI, DONG DENG, and JIANHUA FENG, Tsinghua University As an essential operation in data cleaning, the similarity join has attracted considerable attention from the database community. In this article, we study string similarity joins with edit-distance constraints, which find similar string pairs from two large sets of strings whose edit distance is within a given threshold. Existing algorithms are efficient either for short strings or for long strings, and there is no algorithm that can efficiently and adaptively support both short strings and long strings. To address this problem, we propose a new filter, called the segment filter. We partition a string into a set of segments and use the segments as a filter to find similar string pairs. We first create inverted indices for the segments. Then for each string, we select some of its substrings, identify the selected substrings from the inverted indices, and take strings on the inverted lists of the found substrings as candidates of this string. Finally, we verify the candidates to generate the final answer. We devise efficient techniques to select substrings and prove that our method can minimize the number

Journal

ACM Transactions on Database Systems (TODS)Association for Computing Machinery

Published: Jun 1, 2013

There are no references for this article.