Research article · 25 Jul 2015
Research articlePublished Dec 2014
Extract Transform and Load Strategy for Unstructured Data into Data Warehouse Using Map Reduce Paradigm and Big Data Analytics
Abstract
Select any words in the title or abstract, or press this and choose a sentence.
Analytics over the huge volume of data is now possible with Big data. Data keep on accumulated on every minute from multitude data sources such as social media, mobile devices, and sensors. In order to extract insights from diverse information feeds from multiple, often unrelated sources, data need to be correlated or harmonized to a common level of granularity. Loading Unstructured Data into Data warehouse getting complex. A strategy for fetching the unstructured data into Hadoop Distributed File System is discussed. Data cleansing and profiling of extracted data is important to overcome data quality concerns. Transform phase carried with map reduce frame work. Computation ratio, Network band width and Data locality parameters are monitored with full dump and Incremental load operations. Pig Latin is used to process data from Hadoop Distributed File System and finally load the process data into HDFS file or Data warehouse. Aggregated data from Pig is minimal Subset of Data is Loaded to Data warehouse for Business Analytics and Enterprise Reporting. Based on the Performance related parameters appropriate strategy is suggested for Different type of application.
Ask about this article
Answers come only from this article's title, abstract and keywords, each sentence pointing to where it comes from. Your question is sent to OpenAI to write the answer. Scholars Central does not store it.
Similar by topic
Research article · 3 Jul 2017
Big Data and Analytics – Technology Transforming Health Care
Research article · Oct 2014
Big Data Analytics Security Issues in Data Driven Information System
Research article · 2014
Big Data in Educational Data Mining and Learning Analytics
Research article · Nov 2014
Data Mining with Rough Set Using Map- Reduce
More from this journal
The newest in International Journal of Innovative Research in Computer and Communication Engineering.
Commentary · 2023
Computer Vision and Pattern Recognition
Commentary · 2023
Computer Network and Network Optimization
Perspective · 2023
Cloud Computing and Big Data Analysis
Conferences in this field
Upcoming conferences in this field.
Follow
New articles from what you follow go into your email digest, once you agree to alert emails.
- JournalInternational Journal of Innovative Research in Computer and Communication Engineering
- SubjectComputer Science & IT
- SubjectEngineering
- SubjectSocial Sciences & Humanities
- BylineArticles whose byline prints “P.Saravana kumar”
- BylineArticles whose byline prints “M.Athigopal”
- BylineArticles whose byline prints “S.Vetrivel”
