PLATFORM

Coreset Technology

Early detection is critical when an IoT cyber-attack or major operational event threatens to take down key physical services, cause significant financial damage and even loss of life. Explore how Shield-IoT Coreset data compression can detect anomalies in minutes  compared to days and weeks required by alternative  methods.
Schedule a Demo

KEY CHALLENGES

The Analysis of Big Data Takes Time and Effort
The main challenge is to overcome the problem of scale. As IoT networks become larger (from dozens of thousands, to millions of devices), most of current off-the-shelf anomaly detection algorithms fail to deliver results within a reasonable timeframe. Performing anomaly detection on extremely large datasets can take hours and even days. These solutions are not viable when an IoT cyber-attack or significant operational events threatens to take down key physical services, cause significant financial damage and even loss of life.

CURRENT SOLUTIONS

Off-the-Shelf Solutions
Are Limited

High False Positive Levels

Most anomaly detection algorithms perform heuristics without provable guarantees performance in either running time and especially quality of anomaly detection.

Impractical Running Time

Many existing anomaly detection algorithms have provable guarantees that are totally impractical, usually in terms of running time.

High False Positive Levels

Most anomaly detection algorithms perform heuristics without provable guarantees performance in either running time and especially quality of anomaly detection.

CORESET TECHNOLOGY

Accurate Anomaly Detection At Scale

AI-based coresets is a powerful technique which enables the use of smaller sets of data instead of larger ones without compromising the quality of the output. This approach is based on highly efficient set of algorithms that reduce the data input on one hand, and execute anomaly detection routines on the reduced dataset (coreset), on the other hand. This technology has been applied to fixed big datasets, distributed data or to streaming data, which is highly relevant feature when we aim at network traffic anomaly detection.

INNOVATION

Higher Detection Rates with Lower False Alarms

AI-based coresets is a powerful technique which enables the use of smaller sets of data instead of larger ones without compromising the quality of the output. This approach is based on highly efficient set of algorithms that reduce the data input on one hand, and execute anomaly detection routines on the reduced dataset (coreset), on the other hand. This technology has been applied to fixed big datasets, distributed data or to streaming data, which is highly relevant feature when we aim at network traffic anomaly detection.

Our "Secret Sauce"

The “magic” of ShieldIOT Coreset technology is that it delivers accurate analytics at scale in a fraction of the time required by alternative off-the-shelf products. In practice, Coreset does not directly solve the anomaly detection problem, instead it tries to solve the data problem by using much smaller datasets.

Data Compression is Critical

Coresets are mathematical constructs that compress the data in real-time streaming mode, from 1 million to 20 data points (n to log(n)), with almost zero loss of energy. By running Coresets on reduced datasets before applying existing AI/ML anomaly detection algorithms, Coreset-based solutions can deliver optimal results in minutes compared to hours or days. These results include significant lower false alarms levels (false positives) as well as improved detection rates (true positives).

Heuristics Models Are Impractical

Existing non-Coreset based solutions, need to perform various heuristics to enable analysis of mass scale data sets. This method results in very high levels of false alarms (in some cases exceeding 95% false positives).

INTELLECTUAL PROPERTY

Research Results

Stastical research  was undertaken to compare ShieldIOT AI error rate (false positive) versus RANSAC, a state of the art anomaly detection algorithm (data: security camera, 114 features per data sample). The results show that even for a very small number of samples (2500) Shield-IoT exhibited a superior accuracy of more than X5 better, and that the accuracy-level is almost identical to naively running on the entire data set (which is of course not feasible in large IoT networks / memory-sensitive edge devices). The proven ability to reduce the data into coreset, without losing important and relevant information is at the heart of the innovation of this AI-based anomaly detection solution. Since the coreset is small compared to the original big data, it is possible to execute and run more anomaly detection algorithms in a given time. Thus, the effective data reduction enables significantly better execution time, allowing faster detection and response time to any anomaly over network traffic.

INTELLECTUAL PROPERTY

Based on 15 Years Academic Research

Shield-IoT technology is based on over 15 years of academic research in MIT and Haifa University, and backed by over 70 academic papers and multiple patents, The philosophy behind the AI-based coresets is that running existing algorithms on the reduced dataset will probably give approximate result, as running them on the original big data. This field-proven technique allows us to reduce the data by unprecedented order of magnitude: from n to log(n)! This is done not by designing a new algorithm for solving the problem, but rather by running the existing algorithms on the reduced dataset.
Unlike other compression techniques like zip or mp4, our coreset is data reduction and not just compression of the input, in the sense that it is problem-dependent. our coreset, optimized for detecting network anomalies, removes most of the noise and therefore they are more effective and faster when executing the algorithms on them.

INTELLECTUAL PROPERTY

Based on 15 Years Academic Research

For more information about Coreset AI anomaly detection technology, download these publications.

Coreset for k-means of Streaming Lines

Published by Dan Feldman and Yair Marom

Data Reduction for Weighted and Outlier-resistant Clustering

Published by Dan Feldman and Leonard J. Schulman.

ADDITIONAL RESOURCES

Explore Our Comprehensive Security
Solutions Across Mass-scale IoT Networks

Shieild-IoT software solutions are deployed across a wide range of  industry use cases and applications. Our purpose-built IoT security management solution provides continuous threat mitigation with no changes to your network.

Electrical Power Grids

Read More

Critical Water Utilities

Read More

Smart City and CCTV Cameras

Read More

Point of Sale Devices

Read More

Connected Transportation

Read More

Gas Stations and EV Chargers

Read More

Talk to an Expert

Interested in talking to a Cybersecurity Expert? Fill-out the form and we will contact you directly by phone or email.

About Shield-IoT

Shield-IoT is an IoT cybersecurity software solution provider, enabling enterprises and IoT service providers to monitor and secure cellular IoT networks, generate new revenue and reduce operational costs. Based on more than 15 years of Coreset-AI academic research (MIT), Shield-IoT’s patented agent-less Asset Intelligence and Threat Prevention SaaS platform provides IoT visibility and security across any SIM-connected device and application, at unlimited scale.

Securing the Internet of Things

Twitter Linkedin-in

Contact Us

For more information about Shield-IoT solutions, send us an email and a Shield-IoT expert will contact you.

Product Demo

Sign -up now

Let us show you how Shield-IoT can expand your IoT security and operational monitoring capabilities.

Complete the form to contact Sales

By supplying my contact information, I authorize Shield-IoT to contact me with personalized communications about Shield-IoT products and services.

Skip to content