PLATFORM
Coreset Technology
Early detection is critical when an IoT cyber-attack or major operational event threatens to take down key physical services, cause significant financial damage and even loss of life. Explore how Shield-IoT Coreset data compression can detect anomalies in minutes compared to days and weeks required by alternative methods.
KEY CHALLENGES
The Analysis of Big Data Takes Time and Effort
The main challenge is to overcome the problem of scale. As IoT networks become larger (from dozens of thousands, to millions of devices), most of current off-the-shelf anomaly detection algorithms fail to deliver results within a reasonable timeframe. Performing anomaly detection on extremely large datasets can take hours and even days. These solutions are not viable when an IoT cyber-attack or significant operational events threatens to take down key physical services, cause significant financial damage and even loss of life.
CURRENT SOLUTIONS
Off-the-Shelf Solutions
Are Limited
Are Limited
High False Positive Levels
Most anomaly detection algorithms perform heuristics without provable guarantees performance in either running time and especially quality of anomaly detection.
Impractical Running Time
Many existing anomaly detection algorithms have provable guarantees that are totally impractical, usually in terms of running time.
High False Positive Levels
Most anomaly detection algorithms perform heuristics without provable guarantees performance in either running time and especially quality of anomaly detection.
Not Suitable for Big Data Computation
Most avaialable anomaly detection algorithms do not support modern big data computation models such as streaming of unbounded data with very limited memory, edge computing, distributed computation, or parallel computations that uses GPUs devices.
CORESET TECHNOLOGY
Accurate Anomaly Detection At Scale
AI-based coresets is a powerful technique which enables the use of smaller sets of data instead of larger ones without compromising the quality of the output. This approach is based on highly efficient set of algorithms that reduce the data input on one hand, and execute anomaly detection routines on the reduced dataset (coreset), on the other hand. This technology has been applied to fixed big datasets, distributed data or to streaming data, which is highly relevant feature when we aim at network traffic anomaly detection.
INNOVATION
Higher Detection Rates with Lower False Alarms
Our "Secret Sauce"
The “magic” of ShieldIOT Coreset technology is that it delivers accurate analytics at scale in a fraction of the time required by alternative off-the-shelf products. In practice, Coreset does not directly solve the anomaly detection problem, instead it tries to solve the data problem by using much smaller datasets.
Data Compression is Crtitical
Coresets are mathematical constructs that compress the data in real-time streaming mode, from 1 million to 20 data points (n to log(n)), with almost zero loss of energy. By running Coresets on reduced datasets before applying existing AI/ML anomaly detection algorithms, Coreset-based solutions can deliver optimal results in minutes compared to hours or days. These results include significant lower false alarms levels (false positives) as well as improved detection rates (true positives).
Heuristics Models Are Impractical
Existing non-Coreset based solutions, need to perform various heuristics to enable analysis of mass scale data sets. This method results in very high levels of false alarms (in some cases exceeding 95% false positives).
INTELLECTUAL PROPERTY
Research Results
Stastical research was undertaken to compare ShieldIOT AI error rate (false positive) versus RANSAC, a state of the art anomaly detection algorithm (data: security camera, 114 features per data sample). The results show that even for a very small number of samples (2500) Shield-IoT exhibited a superior accuracy of more than X5 better, and that the accuracy-level is almost identical to naively running on the entire data set (which is of course not feasible in large IoT networks / memory-sensitive edge devices). The proven ability to reduce the data into coreset, without losing important and relevant information is at the heart of the innovation of this AI-based anomaly detection solution. Since the coreset is small compared to the original big data, it is possible to execute and run more anomaly detection algorithms in a given time. Thus, the effective data reduction enables significantly better execution time, allowing faster detection and response time to any anomaly over network traffic.
ShieldI-IoT is 20x more accurate compared to other anomaly detection vendors
ShieldI-IoT reduces detection time from one week to only one minute
INTELLECTUAL PROPERTY
Based on 15 Years Academic Research
Shield-IoT technology is based on over 15 years of academic research in MIT and Haifa University, and backed by over 70 academic papers and multiple patents, The philosophy behind the AI-based coresets is that running existing algorithms on the reduced dataset will probably give approximate result, as running them on the original big data. This field-proven technique allows us to reduce the data by unprecedented order of magnitude: from n to log(n)! This is done not by designing a new algorithm for solving the problem, but rather by running the existing algorithms on the reduced dataset.
Unlike other compression techniques like zip or mp4, our coreset is data reduction and not just compression of the input, in the sense that it is problem-dependent. our coreset, optimized for detecting network anomalies, removes most of the noise and therefore they are more effective and faster when executing the algorithms on them.
FURTHER INFORMATION
Selected Publications
For more information about Coreset AI anomaly detection technology, download these publications.
Coreset for k-means of Streaming Lines
Published by Dan Feldman and Yair Marom
Data Reduction for Weighted and Outlier-resistant Clustering
Published by Dan Feldman and Leonard J. Schulman.
Talk to an Expert
Interested in talking to a Cybersecurity Expert? Fill-out the form and we will contact you directly by phone or email.
ADDITIONAL RESOURCES
Explore Our Comprehensive Security
Solutions Across Mass-scale IoT Networks
Solutions Across Mass-scale IoT Networks
Shieild-IoT software solutions are deployed across a wide range of industry use cases and applications. Our purpose-built IoT security management solution provides continuous threat mitigation with no changes to your network.
