Online anomaly detection in big data

Document Type

Conference Proceeding

Publication Date

10-3-2014

Publication Title

FUSION 2014 - 17th International Conference on Information Fusion

Keywords

anomaly detection, anomaly detection, Big data analytics, Page test, principal component analysis (PCA), video on demand (VOD)

Abstract

In this paper, the problem of online anomaly detection in multi-attributed, asynchronous data from a large number of individual devices is considered. It has become increasingly common for many services, such as video-on-demand (VOD), to have connected customers where hundreds of millions of subscribers access a cluster of content servers for online services. It is important to monitor these transactions online, in order to ensure acceptable quality of experience to the customers as well as for detecting any abnormal or undesirable activities. Our proposed anomaly detection strategy works in two phases: First we perform intermittent anomaly detection in space, using data from the entire set of devices for a short duration in time. This phase employs principal component analysis (PCA) for data reduction and captures models of normal and abnormal features. Then, these identified models are used to monitor each subscriber's devices online in order to quickly detect any abnormalities. The proposed approach is demonstrated on Comcast's Xfinity video streaming data.

ISBN

9788490123553

This document is currently not available here.

Share

COinS