Anomaly Detection in Encrypted Traffic: Machine Learning-Based Analysis for Encrypted Malicious Traffic
Project Abstract
With considerable changes in information technology, users are concerned about the privacy and security of network communication. In recent times, a significant portion of internet browsing traffic is encrypted using Security protocols such as Secure Sockets Layer (SSL). While this encryption enhances data privacy, it also presents an opportunity for attackers and malware to conceal their malicious activities and evade detection by leveraging a legitimate protocol. Organizations can employ signature-based detection methods to efficiently identify known Indicators of Compromise (IoCs) and detect malicious traffic However, rather than relying solely on signature-based detection methods, which unable to detect malicious encrypted traffic, Encrypted Traffic Analysis (ETA) employs advanced techniques such as machine learning algorithms to analyse encrypted traffic flows and identify anomalies or deviations from normal behaviour. Other toolsets, such as Suricata, utilize signature-based detection that relies on known TLS/SSL certificates parameters. While other toolsets, such as Zeek, can detect some malicious traffic by analysing patterns in the encrypted traffic. In this project, the power of signature-based, pattern analysis and machine learning will be combined. The project aims to extract fields of interest from the encrypted traffic as features and study these features against multiple datasets to build a machine learning model that can detect unknown encrypted malicious traffic.
Keywords: Encrypted Traffic AnalysisM, Malicious Traffic Detection, Network Security
Conference Details
Session: Presentation Stream 2 at Presentation Slot 8
Location: GH043 at Tuesday 7th 13:30 – 17:00
Markers: Mukesh Tiwary, Muneeb Ahmad
Course: MSc Computer Science, Masters PG
Future Plans: I’m continuing studies