Smart Embedded Devices: Here They Come
Machine learning for an artificial pancreas and deep brain stimulation
Embedded medical devices that both detect symptoms and treat them have existed for decades. Take, for example, the heart pacemaker. But a new generation of implants could soon emerge to do something far more useful and daunting: These devices will learn from and adapt to changing human physiology and behavior. Projects as diverse as the artificial pancreas and closed-loop systems for deep brain stimulation are developing cutting-edge treatments for diabetes, epilepsy, and Parkinson’s Disease (PD). At the heart of these efforts are machine-learning approaches: Computational algorithms that learn from patient-specific data.
Through it all, researchers are grappling with several fundamental questions: How smart do these devices need to be? Should we rely on simpler algorithms (which may be more predictable) or complex ones (which may be more robust)? And how do we make these devices failsafe?
The Artificial Pancreas: How Smart is Smart Enough?
Medtronic has run feasibility trials of an artificial pancreas in which sensor information is used in real time to modulate insulin delivery as blood sugars vary up and down. Their closed-loop system is pretty straightforward: The algorithm uses what’s called a proportional–integral–derivative (PID) controller that looks for deviations from a set point and then makes adjustments to bring glucose levels back to that point. “Most of the commercially available control systems use that approach,” says John Mastrototaro, PhD, chief technology officer at Medtronic Diabetes. The PID algorithm is quite robust over a wide range of insulin needs for a patient, Mastrototaro says. “It doesn’t have to learn so quickly.” But it does gradually learn. “We program in various parameters to model the sensitivity of the patient for our algorithm and then the machine will learn and change and drift as the patient’s needs change and drift over time,” he says. For example, pregnant women and growing children have changes in insulin needs.
Initially, Mastrototaro says, the learning can all be done manually and offline. As the patient wears the system, it gathers data that he or she can upload to online software provided by Medtronic, which recalculates and optimizes the parameter settings and feeds them back to the embedded system for use the next week. “And you can repeat that iterative process on an ongoing basis so it’s analogous to machine learning in the device,” Mastrototaro says. In the future, data will be uploaded and parameters adjusted automatically while the patient wears the device, he says.
Mastrototaro also envisions doing the machine learning outside the device. “With wireless technology, the device can talk to central computers like a cell phone can, so it doesn’t really have to be in the device to behave as if it is,” he says. Moreover, the patient can benefit from software modifications and updates without having to buy a new device.
As simple and effective as the PID controller seems to be, other groups are instead exploring a model predictive controller (MPC). Frank Doyle, PhD, professor of engineering at the University of California, San Diego is one of the pioneers of the MPC school. MPC controllers are used in everything from flight controllers to automobile controllers to the control of petroleum refineries. “Basically the high priority control loops in industry use more sophisticated algorithms, like MPC,” he says.
The Doyle group’s MPC algorithm doesn’t target a single set point or number. Instead, it creates multiple zones along the continuum from hypo- to hyperglycemia. Within each zone, all measurements are considered equally good—allowing the algorithm to ignore sensor noise. “It’s consistent with how a doctor analyzes data,” Doyle says.
MPC acts on a time scale of minutes. “Every 5 minutes or so the forecast is for the next 30 to 60 minutes,” Doyle says. This means the controller can respond quickly to changes in glucose. For example, Doyle’s group developed a meal detection algorithm that can spot a sharp rise in glucose (such as might occur during a meal) and take appropriate action (provide fast-acting insulin). “Most other groups let patients interact with the pump to give a priming bolus [large injection] of insulin at lunchtime.” Because children sometimes forget to bolus, he says, “we’ve sought results that don’t require that.” Other algorithms might detect and respond to exercise or illness.
Doyle’s group is also adding a layer of iterative learning control (ILC) that would learn over a longer time scale. He compares MPC to cruise control in a car, which works well on a scale of fractions of a mile but struggles a bit when it comes to a hill that it didn’t expect. “If you know the hill is coming you can anticipate,” he says. In diabetes, the hills might be exercise days or days when a patient is sick or anxious. ILC can anticipate these on a longer time horizon. “It’s our hypothesis that we will get information to inform a long-term control program,” he says. “It could be punched in (by the mom of a sick kid) or it could be something the algorithm could learn from.” For example, a woman’s monthly menstrual period might change sensitivity and could be learned.
To make it possible for the algorithm to function in an embedded device, Doyle’s group reduces the equation to an analytic solution. “We enumerate all the possible solutions and store it as a memory table,” he says, “So it’s a memory operation rather than a calculation.”
No clear winner has yet emerged between PID and MPC. “It’s a bit of a controversy,” Mastrototaro says. Currently , based on the clinical data produced using different algorithms, “the PID algorithm is performing every bit as well if not better than the MPCs,” he says.
But Doyle says the two algorithms have yet to be compared head-to-head in a clinical trial under the same conditions. In the year ahead, his group has been funded by the Juvenile Diabetes Research Foundation to conduct that very study in collaboration with the Sansum Diabetes Research Institute.
Mastrototaro concedes that as the MPC folks add more parameters and learning to their algorithm it will get better and better. “At the end of the day, to be quite frank, I think both of them will do a good job. The goal ultimately is to have phenomenal outcomes in managing diabetes.”
Closing the Loop on the Brain
Deep brain stimulation (DBS), which uses electrodes implanted in the brain, has been approved to treat Parkinson’s disease (PD) tremors since 2002 and epileptic seizures since 2010. Current DBS devices have fixed settings that are manually adjusted by medical experts during physical exams, and the stimulation is continuous.
Ideally, say researchers, DBS would be a bi-directional system: Electrodes would sense an imminent seizure or the onset of a PD symptom, which would signal the same electrodes to provide an appropriate level of stimulation. But because the brain’s signals are complex and vary among patients, machine learning could play a role in making such systems a reality.
So far, there’s been a lot of work on algorithms that detect or attempt to predict epileptic seizures, and some pilot work is now exploring similar detection schemes for PD and other disease states. In general, bi-directional systems are still in an investigational stage of maturity, says Tim Denison, PhD, engineering director in Medtronic’s Neuromodulation business. “We are still on the journey, not yet to the destination. Detection systems with high sensitivity and specificity are challenging; prediction systems are even more difficult.”
Among those working on predicting seizures are Mushfiq Saleheen and Homa Alemzadeh, graduate students in Ravi Iyer’s engineering group at the University of Illinois at Urbana Champaign. They used neural networks to train a device to predict seizures. The device relies on multiple parameters—not only electro-encephalogram (EEG) readings, but also oxygen saturation and body movements. And it’s a flexible device that can also work for detection of traumatic brain injury, cognitive decline, and heart attack prediction. “The math underlying making these predictions is not that different, but the device would be configured differently for each disease,” Iyer says. Iyer and Alemzadeh’s goal is to design a flexible device that can predict the onset of a traumatic event such as seizure or heart attack several minutes in advance, but the intention is to set off an alarm rather than provide treatment. “With an alarm, patients can take action to prevent the worst of the consequences.”
Medtronic hopes to use similar types of machine learning to distinguish seizure from non-seizure events but with an eye to using DBS as a closed-loop treatment. This presents unique challenges. For example, when DBS starts (say in response to brain signals suggesting a seizure is imminent), the large stimulation pulses could immediately drown out the brain signals needed to assess the patient’s state. Denison and his colleagues have found a way to teach the algorithm to distinguish this noise from the valuable background signal. It’s a key step toward making a bi-directional system a practical reality.
Creating a bi-directional system for DBS treatment of PD presents additional challenges. PD signals aren’t very strong compared to seizures. “They’re about 1000 times smaller than what cardio pacemakers detect today, and usually about 10 to 100 times smaller than a seizure,” Denison says. So it can be tough to get robust measurements. And unlike seizures, many of PD’s brain signals appear to be embedded as variations in normal rhythms. “The challenge is to discover what’s normal and what’s disease related when designing a robust classifier,” Denison says.
Eduard Bakstein, a PhD student at the department of cybernetics at Czech Technical University in Prague, is using neural networks to discover features of PD tremor. “I look at the signal and I know when tremor was present or not, and then I extract different features from the data,” he says. This process involves applying various transformations to the data—Fourier transform; wavelet transform; standard deviation of the signal—to observe how the features behave during the on- and off-tremor periods. The goal is to identify the features that change most when the tremor is starting. “Then I use these in the machine learning to identify the on and off periods,” he says. In a small pilot study of his model, it worked well for some patients and not so well for others.
Medtronic has access to more patient data than Bakstein, and has therefore applied its machine learning approach to large datasets of EEG data both across subjects and over time. Although there’s a particular signal that they believe seems to correlate with the presence of symptoms in an animal model, Medtronic researchers are still trying to clarify when it appears and whether it can be used to titrate stimulation as part of a closed-loop therapy. “We might really want something that gently coaxes the brain, rather than responding as it would to a large-scale event,” Denison says. Previous thinking about responsive stimulation in the brain evolved from a defibrillation mindset—applying a very strong stimulus as is done with cardiac devices. “That might not be the right approach,” Denison says, “And we are now exploring other schemes more in the spirit of a circadian or homeostatic feedback concept, based on first-principles measurements of physiology.”
Medtronic is also trying to simplify its machine learning approaches so that they don’t require too much power. “The therapy today to provide DBS is on the order of 100 microwatts—about a million times less than an incandescent light bulb,” Denison says. This limits the amount of power available for sensors and detection algorithms. “We can only get a budget of 10 percent of the therapy power,” he says. Using what’s called the reduced sets method and other schemes, Medtronic systematically seeks to simplify its detectors and algorithms to reduce the energy needed. “Frankly, a lot of detectors draw too much power for the performance that is achieved,” Denison says. “You’re not doing anyone any favor if you can’t implement the technology practically in an implant.”
Denison’s team spends a lot of time optimizing algorithm methods that are simultaneously accurate and low power. As an example, Medtronic uses a posture response algorithm in its RestoreSensor device, which uses stimulation to treat chronic pain. “We customized an accelerometer and algorithm to build a reflex into the device, drawing only microwatts of power,” Denison says.
Safety is Everything
For an embedded medical device to succeed, it must not only do what it’s designed to do but also do it in a failsafe way. That’s one of Medtronic’s big concerns now with the artificial pancreas. Because an incorrect dose of insulin can be deadly, putting decisions in the hand of a tiny device with a detector and a learning algorithm is a bit scary. One option, says Mastrototaro, is for the machine to learn the patient’s normal cycle of blood sugar variation and then use that information to send an alarm or suspend closed loop control when there’s something unusual going on—when the pump or detector aren’t working correctly. “That’s where our focus is now,” he says.