Editorial · 2020
Conference abstractPublished 25 Nov 2021
A Real-Time Intrusion Detection System based on Machine Learning and Big Data Techniques
Abstract
Select any words in the title or abstract, or press this and choose a sentence.
Cybersecurity ventures expects cyber-attacks damage costs will rise to $11.5 billion in 2019 and that a business will fall victim to a cyber-attack every 14 seconds. Notice here that the time frame for such event is seconds. With data generated by peta bytes each day this is a challenging task for traditional intrusion detection systems (IDSs). Protecting sensitive information is a major concern for both businesses and governments. Therefore, the need for a real-time, large-scale and effective IDS is a must. In this work we present a cloud based, fault-tolerant, scalable and distributed IDS that uses Apache Spark Structured Streaming (PySpark) and its Machine Learning library (MLlib) to detect intrusions in real-time. To demonstrate the efficacy and effectivity of this system, we implement the proposed system within Microsoft Azure Cloud as it provides both processing power and storage capabilities. A decision tree algorithm is used to predict incoming data’s nature. For this task, the use of the MAWILab dataset as a data source will give better insights about the system capabilities against cyber-attacks. The experimental results showed a 99,95 % accuracy and more than 55175 events per second were processed by the proposed system on a small cluster.
Subjects
Computer Science & ITAsk about this article
Answers come only from this article's title, abstract and keywords, each sentence pointing to where it comes from. Your question is sent to OpenAI to write the answer. Scholars Central does not store it.
Similar by topic
Conference abstract · 30 Aug 2018
Machine learning for data acquisition in dynamic real-time
Article · 30 Mar 2025
Machine Learning Algorithms for Real-Time Object Detection in Robotics
Perspective · 31 Mar 2025
Hybrid Machine Learning Models for Real-Time Financial Fraud Detection
Conference abstract · 30 Dec 2021
Data Fusion Based on Big Data Techniques in Intrusion Detection Context
More from this journal
The newest in American Journal of Computer Science and Information Technology.
Brief report · 31 Mar 2025
Deep Learning Optimization for Real-Time Image Recognition in Autonomous Vehicles
Commentary · 31 Mar 2025
A Novel IoT Framework for Smart City Traffic Management
Opinion · 31 Mar 2025
Autonomous Drone Navigation Using Deep Reinforcement Learning
Conferences in this field
Upcoming conferences in this field.
Follow
New articles from what you follow go into your email digest, once you agree to alert emails.
- JournalAmerican Journal of Computer Science and Information Technology
- SubjectComputer Science & IT
- BylineArticles whose byline prints “Farah Jemili”

