Scholars Central

Research articlePublished Mar 2014

An Iterative Approach to Record Deduplication

Authors: M. Roshini Karunya, S. Lalitha, B.Tech., M.E.

Ask about this article

Abstract

Select any words in the title or abstract, or press this and choose a sentence.

Record deduplication is the task of identifying, in a data repository, records that refer to the same real world entity or object in spite of misspelling words, typos, different writing styles or even different schema representations or data types [1]. The existing system aims at providing Unsupervised Duplication Detection method which can be used to identify and remove the duplicate records from different data sources. UDD, which for a given query, can effectively identify duplicates from the query result records of multiple web databases. Two cooperating classifiers, a Weighted Component Similarity Summing Classifier (WCSS) and Support Vector Machine (SVM) are used to iteratively identify the duplicate records from the non duplicate record and we also present a Genetic Programming (GP) approach to identify record deduplication. Since record deduplication is a time consuming task even for small repositories, our aim is to foster a method that finds a proper combination of the best pieces of evidence, thus yielding a deduplication function that maximizes performance using a small representative portion of the corresponding data for training purposes. We propose two more algorithms namely Particle Swarm Optimization (PSO), Bat Algorithm (BA) to improve the optimization. Index Terms – Data mining, duplicate records, genetic algorithm

Ask about this article

Answers come only from this article's title, abstract and keywords, each sentence pointing to where it comes from. Your question is sent to OpenAI to write the answer. Scholars Central does not store it.

Try:

Similar by topic

More from this journal

The newest in International Journal of Innovative Research in Computer and Communication Engineering.

All its articles

Conferences in this field

Upcoming conferences in this field.

Follow

New articles from what you follow go into your email digest, once you agree to alert emails.

  • JournalInternational Journal of Innovative Research in Computer and Communication Engineering
  • SubjectComputer Science & IT
  • SubjectEngineering
  • SubjectSocial Sciences & Humanities
  • BylineArticles whose byline prints “M. Roshini Karunya”
  • BylineArticles whose byline prints “S. Lalitha”
  • BylineArticles whose byline prints “B.Tech.”

For authors and organisers

Publish, present and keep up with your field

Find the right journal for your paper and submit your manuscript.

Photo: Pexels