By Dave DeFusco
In a world powered by sensors, smartwatches and the Internet of Things (IoT), enormous streams of data are constantly being collected about our health, our environment and our devices. But sometimes that data contains outliers鈥攏umbers that don鈥檛 fit the normal pattern. Detecting those outliers early can prevent major problems, from identifying a failing machine part to spotting the first signs of a heart attack.
The challenge, though, is that most tools for finding these unusual data points, especially those using machine learning or deep learning, are heavy, complex and power-hungry. They work great on big computers but struggle to run on the smaller, low-power devices we carry or install in our homes.
That鈥檚 what inspired Dr. David Li, director of the Katz School鈥檚 M.S. in Data Analytics and Visualization, to design an algorithm, called IDOS (Interpolated Density for Outlier Score), which is fast, efficient and doesn鈥檛 require much computing power.
鈥淲e wanted to create a model that could think smart but act light,鈥 said Dr. Li. 鈥淚DOS is designed for real-world environments where every bit of power and memory matters, like wearable health monitors, embedded sensors and other IoT devices.鈥
Dr. Li and his co-authors鈥擸u Wang, a student in the Katz School鈥檚 M.S. in Artificial Intelligence, and Angela Li, a researcher at Stony Brook University鈥攑resented their work in March at the 59th Annual Conference on Information Science and Systems at Johns Hopkins University.
An outlier is simply a data point that doesn鈥檛 behave like the rest. In healthcare, for instance, if your heart rate or blood sugar suddenly spikes far beyond your normal range, that could be an early warning sign of trouble. Outlier detection algorithms act as silent sentinels that are always watching for those warning signals that might otherwise go unnoticed.
Traditional methods fall into two extremes: simple rule-based systems that might flag a reading if it crosses a fixed limit鈥攗seful but rigid, and sophisticated deep-learning models that can detect complex patterns but demand massive amounts of data and computational power.
鈥淢ost algorithms require tuning lots of parameters or running on high-performance servers,鈥 said Wang. 鈥淭hat makes them difficult to deploy on devices that have limited battery life or memory.鈥
IDOS takes a fresh approach. Instead of assuming data follows a specific shape or pattern, it adapts to the data itself. It uses what鈥檚 called a nonparametric method, meaning it doesn鈥檛 rely on fixed assumptions about how data is distributed.
At the heart of IDOS is something called the Empirical Cumulative Distribution Function, or CDF, which measures how data values are spread out, from smallest to largest, and helps the algorithm see what normal looks like. IDOS then uses interpolation, a mathematical way of filling in gaps smoothly between data points, to judge how far a new data point strays from normal.
Because it works directly with the data rather than a pre-set model, IDOS doesn鈥檛 need extensive parameter tuning or big datasets to learn from. 鈥淲e wanted it to work right out of the box,鈥 said Angela Li. 鈥淵ou don鈥檛 need to be a data scientist to use it or to spend hours adjusting settings. It just adapts.鈥
To see IDOS in action, the team tested it on a health dataset involving patients at risk for heart attacks. They analyzed features like age, heart rate, blood pressure, blood sugar and cardiac enzyme levels鈥攌ey indicators of cardiovascular health.
IDOS successfully flagged abnormal readings that signaled potential heart problems, using a small fraction of the computing power required by other methods. It also matched or outperformed more complex algorithms in detecting anomalies.
Even better, it could process data continuously, which is ideal for wearable health devices. 鈥淚magine your smartwatch quietly tracking your vitals and alerting you instantly when something looks off,鈥 said Wang. 鈥淚DOS makes that kind of real-time monitoring feasible, even on limited hardware.鈥
Unlike many machine-learning algorithms that must be retrained for every new situation, IDOS can be easily deployed across different devices and scenarios, from health monitoring to industrial sensors and even cybersecurity systems.
鈥淚t鈥檚 a simple idea with wide-reaching potential,鈥 said Dr. Li. 鈥淲hether it鈥檚 detecting a failing component in an aircraft, an unauthorized login in a network or an abnormal heartbeat in a fitness tracker, IDOS brings powerful anomaly detection to the edge, where data is generated.鈥
