Online Anomaly Detection in Big Data: The First Line of Defense Against Intruders

Document Type

Article

Publication Date

1-1-2017

Publication Title

Studies in Big Data

Volume

24

First Page

83

Keywords

Big data, Computational intelligence, Cyber threat detection, Cyber-physical-human systems, Cyber-security, Intrusion detection, Likelihood ratio test, Malware, Online anomaly detection, Quickest detection of changes

Last Page

107

Abstract

We live in a world of abundance of information, but lack the ability to fully benefit from it, as succinctly described by John Naisbitt in his 1982 book, “we are drowning in information, but starved for knowledge”. The information, collected by various sensors and humans, is corrupted by noise, ambiguity and distortions and suffers from the data deluge problem. Combining the noisy, ambiguous and distorted information that comes from a variety of sources scattered around the globe in order to synthesize accurate and actionable knowledge is a challenging problem. To make things even more complex, there are intentionally developed intrusive mechanisms that aim to disturb accurate information fusion and knowledge extraction; these mechanisms include cyber attacks, cyber espionage and cyber crime, to name a few. Intrusion detection has become a major research focus over the past two decades and several intrusion detection approaches, such as rule-based, signature-based and computer intelligence based approaches were developed. Out of these, computational intelligence based anomaly detection mechanisms show the ability to handle hitherto unknown intrusions and attacks. However, these approaches suffer from two different issues: (i) they are not designed to detect similar attacks on a large number of devices, and (ii) they are not designed for quickest detection. In this chapter, we describe an approach that helps to scale-up existing computational intelligence approaches to implement quickest anomaly detection in millions of devices at the same time.

DOI

10.1007/978-3-319-53474-9_4

ISSN

21976503

E-ISSN

21976511

Link to Full Text

Share

COinS