Anomaly Detection

By for April 22, 2015
Detect different anomalous patterns in your time series data using machine learning algorithms. Level changes, trend changes, spikes are supported on seasonal and non-seasonal time series.
Anomaly Detection is an API built with Azure Machine Learning that is useful for detecting different types of anomalous patterns in your time series data. The API assigns an anomaly score to each data point in the time series, which can be used for generating alerts, monitoring through dashboards or connecting with your ticketing systems. The [Anomaly Detection API](https://docs.microsoft.com/en-us/azure/machine-learning/machine-learning-apps-anomaly-detection-api) can detect the following types of anomalies on time series data: - *Spikes and Dips:* For example, when monitoring the number of login failures to a service or number of checkouts in an e-commerce site, unusual spikes or dips could indicate security attacks or service disruptions. - *Positive and negative trends:* When monitoring memory usage in computing, for instance, shrinking free memory size is indicative of a potential memory leak; when monitoring service queue length, a persistent upward trend may indicate an underlying software issue. - *Level changes and changes in dynamic range of values:* For example, level changes in latencies of a service after a service upgrade or lower levels of exceptions after upgrade can be interesting to monitor. The machine learning based API enables: - *Flexible and robust detection:* The anomaly detection models allow users to configure sensitivity settings and detect anomalies among seasonal and non-seasonal data sets. Users can adjust the anomaly detection model to make the detection API less or more sensitive according to their needs. This would mean detecting the less or more visible anomalies in data with and without seasonal patterns.  - *Scalable and timely detection:* The traditional way of monitoring with preset thresholds set by experts' domain knowledge are costly and not scalable to millions of dynamically changing data sets. The anomaly detection models in this API are learned and models are tuned automatically from both historical and real-time data. - *Proactive and actionable detection:* Slow trend and level change detection can be applied for early anomaly detection. The early abnormal signals detected can be used to direct humans to investigate and act on the problem areas.  In addition, root cause analysis models and alerting tools can be developed on top of this anomaly detection API service.  The anomaly detection API is an effective and efficient solution for a wide range of scenarios like service health & KPI monitoring, IoT, performance monitoring, and network traffic monitoring. Here are some popular scenarios where this API can be useful:   1. IT departments need tools to track events, error code, usage log, and performance (CPU, Memory and so on) in a timely manner. 2. Online commerce sites wants to track customer activities, page views, clicks, and so on. 3. Utility companies want to track consumption of water, gas, electricity and other resources. 4. Facility/Building management services want to monitor temperature, moisture, traffic and so on. 5. IoT/manufacturers want to use sensor data in time series to monitor work flow, quality and so on. 6. Service providers, such as call centers need to monitor service demand trend, incident volume, wait queue length and so on. 7. Business analytics groups want to monitor business KPIs' (such as sales volume, customer sentiments, pricing) abnormal movement in real time.