Conference abstractPublished 4 May 2015
Sentiment analysis of twitter data using parallel write approach of replica placement in Hadoop cluster
Abstract
Select any words in the title or abstract, or press this and choose a sentence.
In recent years social networking has turned out to be very popular. Twitter, a micro-blogging service, is appraised to have about 200 million registered users and these users create about 65 million tweets a day. Twitter users usually express their views about topics of their interest. The challenge is that each tweet is partial up to 140 characters, and is hence very short. It may contain jargon and misspelled texts. Thus, it is hard to apply traditional NLP techniques which are designed for working with formal languages, into Twitter domain. One more challenge is that the total volume of tweets is tremendously high, and it takes a long time to process. In this project, we have described a Hadoop distributed system for real-time Twitter sentiment analysis. Our system consists of three components: A lexicon builder, a sentiment classifier and Hadoop new distributed file system for replica placement. These three components are capable of running on a large-scale distributed system since they are implemented using a Hive, Flume Map Reduce framework, HBase database model and other Hadoop environment. Thus, our sentiment classifier and lexicon builder are scalable with the number of machines and the size of data. The experiments also show that our lexicon has a good quality in opinion extraction, and the accuracy of the sentiment classifier can be improved by combining the lexicon with a machine learning technique. Another Part in our project is of HDFS; we are experiencing an information explosion era. Due to which huge amount of distributed data is being achieved and put in storage? To manage this type of data, application uses distributed file system. For replica placement the HDFS is used to save time and for better reuse. Recent state of the art design & implementation of the HDFS implement the new approach for efficient replica placement in Hadoop DFS which can improve throughout and data transfer rate.
Subjects
Computer Science & ITAsk about this article
Answers come only from this article's title, abstract and keywords, each sentence pointing to where it comes from. Your question is sent to OpenAI to write the answer. Scholars Central does not store it.
Follow
New articles from what you follow go into your email digest, once you agree to alert emails.
- JournalJournal of Computer Science & Systems Biology
- SubjectComputer Science & IT
- BylineArticles whose byline prints “Divyesh Patel”
