Scholars Central

Research articlePublished Mar 2014

Enhancing Map-Reduce Framework for Bigdata with Hierarchical Clustering

Authors: Vadive.M, Raghunath.V

Ask about this article

Abstract

Select any words in the title or abstract, or press this and choose a sentence.

MapReduce is a software framework that allows certain kinds of parallelizable or distributable problems involving large data sets to be solved using computing clusters. This paper introduces our experience of grouping internet users by mining a huge volume of web access log of up to 500 gigabytes. The application is realized using hierarchical clustering algorithms with Map-Reduce, a parallel processing framework over clusters. However, the immediate implementation of the algorithms suffers from efficiency problem for both inadequate memory and higher execution time. This paper presents an efficient hierarchical clustering method of mining large datasets with Map-Reduce. The method includes two optimization techniques: Batch Updating to reduce the computational time and communication costs among cluster nodes, and Co-occurrence based feature selection to decrease the dimension of feature vectors and eliminate noise features.

Ask about this article

Answers come only from this article's title, abstract and keywords, each sentence pointing to where it comes from. Your question is sent to OpenAI to write the answer. Scholars Central does not store it.

Try:

Similar by topic

More from this journal

The newest in International Journal of Innovative Research in Computer and Communication Engineering.

All its articles

Conferences in this field

Upcoming conferences in this field.

Follow

New articles from what you follow go into your email digest, once you agree to alert emails.

  • JournalInternational Journal of Innovative Research in Computer and Communication Engineering
  • SubjectComputer Science & IT
  • SubjectEngineering
  • SubjectSocial Sciences & Humanities
  • BylineArticles whose byline prints “Vadive.M”
  • BylineArticles whose byline prints “Raghunath.V”

For authors and organisers

Publish, present and keep up with your field

Find the right journal for your paper and submit your manuscript.

Photo: Pexels