Scholars Central

Research articlePublished Dec 2014

Extract Transform and Load Strategy for Unstructured Data into Data Warehouse Using Map Reduce Paradigm and Big Data Analytics

Authors: P.Saravana kumar, M.Athigopal, S.Vetrivel

Ask about this article

Abstract

Select any words in the title or abstract, or press this and choose a sentence.

Analytics over the huge volume of data is now possible with Big data. Data keep on accumulated on every minute from multitude data sources such as social media, mobile devices, and sensors. In order to extract insights from diverse information feeds from multiple, often unrelated sources, data need to be correlated or harmonized to a common level of granularity. Loading Unstructured Data into Data warehouse getting complex. A strategy for fetching the unstructured data into Hadoop Distributed File System is discussed. Data cleansing and profiling of extracted data is important to overcome data quality concerns. Transform phase carried with map reduce frame work. Computation ratio, Network band width and Data locality parameters are monitored with full dump and Incremental load operations. Pig Latin is used to process data from Hadoop Distributed File System and finally load the process data into HDFS file or Data warehouse. Aggregated data from Pig is minimal Subset of Data is Loaded to Data warehouse for Business Analytics and Enterprise Reporting. Based on the Performance related parameters appropriate strategy is suggested for Different type of application.

Ask about this article

Answers come only from this article's title, abstract and keywords, each sentence pointing to where it comes from. Your question is sent to OpenAI to write the answer. Scholars Central does not store it.

Try:

Similar by topic

More from this journal

The newest in International Journal of Innovative Research in Computer and Communication Engineering.

All its articles

Conferences in this field

Upcoming conferences in this field.

Follow

New articles from what you follow go into your email digest, once you agree to alert emails.

  • JournalInternational Journal of Innovative Research in Computer and Communication Engineering
  • SubjectComputer Science & IT
  • SubjectEngineering
  • SubjectSocial Sciences & Humanities
  • BylineArticles whose byline prints “P.Saravana kumar”
  • BylineArticles whose byline prints “M.Athigopal”
  • BylineArticles whose byline prints “S.Vetrivel”

For authors and organisers

Publish, present and keep up with your field

Find the right journal for your paper and submit your manuscript.

Photo: Pexels