Agu with smartphone

The Physician in Our Pocket: Emmanuel Agu

September 28, 2018

It’s not inhuman to suggest that the human body can be viewed as a data center with arms and legs. Just as our DNA is a rich data repository that unfolds into the complexity of an individual human life, so too each life emits a continuous stream of subtle signals that can be pieced together for a view into that person’s health.

Emmanuel Agu thinks smartphones can help crack this code. “Smartphones are the perfect platform to augment—anytime, anywhere—the care given to patients in offices, clinics, and hospitals,” says the professor of computer science. “Healthcare is appointment-driven, which leaves patients underserved outside of scheduled times. But patients must be able to access care 24/7, and the large amount of data that’s continuously generated needs to be analyzed and put to use. Many questions that could not be answered previously can now be answered using data-driven approaches.”

What better way to gather and make sense of the data than with the constant companion of modern life—as anybody who has walked the streets or malls, or been disturbed by flashes of light in the movie theatre, can attest. “More than 80 percent of people in the United States, across all socioeconomic, age, gender, race, and literacy levels, own a smartphone, making it the first truly pervasive computing device,” he says.

Smartphones have the computer power and sensors to make it work. Among the many data types captured by our phones are environmental factors (temperature, humidity, altitude, light), soft data (call and SMS times, as well as browsing and social media patterns), location (GPS, places/types visited), and sounds and images from microphones and cameras. “Machine learning can use this raw data to discover patterns to infer health status,” he says.

Agu’s research isn’t theoretical—he’s building health apps rooted in medical research, developed in collaboration with physicians, and tested in clinical trials. His team’s patented Smartphone Wound

Analysis and Decision Support (SmartWAnDS) app—a continuation of his work on an NSF-funded diabetes management app called Sugar—enables chronic wounds to be imaged, then analyzed for healing progress via machine learning algorithms. The app deduces parameters, like wound depth and tissue composition, and generates a healing score.

Stethoscope emerging from phone illustration“Wounds that are not healing in a timely fashion can be flagged for closer attention,” he says. “The app can be used by patients or their caregivers to self-monitor their wounds, engaging them more in their care. It can be used by visiting nurses for decision support, and to get a second opinion. Based on wound status and the patient’s Electronic Health Record [EHR] information, machine learning algorithms can make recommendations and decisions similar to those of a world-class wound expert.” Professors Peder Pedersen, Diane Strong, Bengisu Tulu, and Clifford Lindsay are co-investigators on a new $1.6 million National Institutes of Health (NIH) grant to research and develop the SmartWAnDS app.

AlcoGait (patents filed) is his smartphone sensing app that detects users’ level of intoxication from their walk patterns. The app extracts accelerometer readings and infers posture and sway from gyroscope data, then classifies them through machine learning. “The app helps smartphone users know when they are too drunk to drive,” Agu says. “Identifying intoxication from gait is almost as accurate as a breathalyzer. Drunk smartphone users are alerted immediately; an Uber can automatically be called to pick them up, avoiding possibly fatal outcomes.” Doctors Michael Stein and Ana Abrantes of Brown University’s Butler Hospital are co-investigators on a current NIH grant to validate the accuracy of AlcoGait.

Machine learning is core to Agu’s work. “Machine learning has become one of the hottest and most usable branches of artificial intelligence,” he says. “I work in mobile sensing, which involves running machine/deep learning algorithms on smartphone or mobile device sensor data. With the rise of Big Data from smartphones, social media, sensors, and wearables, machine learning has become essential

to discovering useful, actionable patterns in massive data sets.”


Shifting Paradigm

Agu’s research dovetails with the widespread consensus that healthcare must become more proactive, through early detection and just-in-time notification, rather than reactive, in treating only ill patients using expensive procedures. He believes that doctors and patients should be rewarded and incentivized for improved health, and to control costs. “The extent to which patients abstain from smoking and drugs, exercise regularly, and eat healthily can all be quantified using sensed smartphone data,” he says. “Incentivizing individual patients, and the doctors who care for them (as opposed to how many procedures are done, or how much time is spent with the patients) would be a paradigm shift.”

That drives his work with the WPI Healthcare Delivery Institute (HDI). Agu is the faculty director for the steering committee of HDI, which aims to further healthcare research, education, and thought leadership, especially in digital health. HDI targets larger funded projects that cut across multiple departments at WPI. “Many of my projects involve multiple HDI faculty members,” he says.

Interdisciplinary work is a WPI strong suit, Agu explains. “WPI’s size makes it easy to work as a team across departments and schools; most of my collaborators are outside the WPI Computer Science Department. I hold collaborative appointments in seven WPI departments, making it easy to attract good students with diverse backgrounds for my projects.”

Wielding his skills alongside those of others suits his nature. “Applying cutting-edge computer science solutions to solve real problems that impact people’s lives is exciting,” he says. “My father always emphasized pragmatism, and having broad interests. I am equally comfortable discussing healthcare problems with clinicians and the requisite machine learning and app solutions with computer scientists.”


Mobile GuyDoctor illustration with phone

Agu has been working in mobile computing for more than 20 years. He’s researched and developed a range of applications beyond healthcare, including location-aware tour guides and security modules, and a mobile emergency medical services incident reporting app. One focus area is energy management—Agu and his first master’s student developed a tool called PowerSpy 13 years ago. It detected which apps and devices were using too much battery power so they could be shut down. “This functionality is now widely available on most smartphones today,” he says.

He also researches mobile graphics, and has prototyped a middleware architecture that takes up the challenge of rendering high-quality graphics, with their high resource usage, on mobile devices with low resource availability. “We synthesized techniques for making graphics more efficient on mobile devices,” he says. “For instance, 3D graphics content, like meshes and images, can be automatically scaled down based on the target mobile device’s hardware constraints. Remote servers can also do some of the work, optimizing preprocessing to speed runtime rendering on low-end devices.”

Agu’s research has been funded by the National Science Foundation, U.S. Department of Education, U.S. Army Research Laboratory, National Institutes of Health, DARPA, the Air Force Research Labs, Google,

NVIDIA, and AMD. He gives top marks to WPI students for their help in the investigative work. “My research is quite applied, which makes WPI an excellent place to conduct it,” he says. “WPI students love concrete and well-defined projects with clear goals. And once they get excited by a project, WPI students deliver unbelievably high-quality work in record time.”

The university scores a high grade as well. “WPI’s current push to increase commercialization is quite commendable,” Agu says, “and I am excited to be part of that effort. I have been involved in entrepreneurship and created Internet start-ups as a grad student. For example, we are in negotiations with several potential partners that have expressed interest in the wound healing app.” Agu pays it forward—he loves to teach. “I like to help students figure material out,” he says. “I also benefit, since I understand the material on a deeper level when I explain it to students. I taught karate in my undergraduate years, which made me realize that I have a passion for teaching.”


Back to Africa

Some of that comes from his upbringing. Agu comes from Nigeria, and education has always been important to his parents. “My father started out as a high school teacher, and rose to the position of Justice of the Supreme Court of Nigeria; I just found out he has his own Wikipedia page,” Agu says. “We were fortunate to attend great schools, and take extra lessons. Mixing with other students who were overachievers instilled in me some ambition, and a belief that I could do well.”

He thinks some of what he does well can find its way back to his homeland. “Healthcare apps could have great impact in Africa, where health disparities are extreme, and a large proportion of the population now own smartphones,” he says. “Some of the machine learning healthcare screeners and apps we have researched and developed here could save lives across the continent.”

The WPI seal is graced with an image of open books and an arm and hammer, signifying theory and practice. Agu integrates the two in his life’s work—and through that work, honors the first verse from his college anthem at the University of Benin in Nigeria, where he earned his BS in engineering:  “All over the wide wide world Our fruitful works unfurl; What we sow’d with joy and pride with good our common goal…”


The Warfighter Analytics for Smartphone Healthcare (WASH) program is working on detecting early signs of traumatic brain injury (TBI) and infectious diseases like flu, tuberculosis, and pneumonia. The goal of the initiative—funded by a four-year, $2.8 million award from the Defense Advanced Research Projects Agency (DARPA)—is to create a mobile application that passively assesses a warfighter’s readiness, both immediately and over time. Professor Elke Rundensteiner, director of the WPI Data Science Program, is a co-investigator on the DARPA WASH grant.

Warfighter assessment is currently accomplished with EKGs and other specialized medical devices—difficult to do in hostile, non-controlled environments, or after the fact in an office setting. “Our team will research and develop machine/deep learning algorithms that synthesize reliable smartphone biomarkers captured unobtrusively from smartphone sensors,” says Agu, the principal investigator for the project.

The biomarkers are key: smartphone-sensable characteristics and patterns that can be measured and evaluated to indicate normal or pathogenic processes and health status—like slurred speech and reduced communication and social interaction for TBI, or coughing and fatigued movement with infectious disease. “This will enable continuous, real-time assessment of TBI and communicable diseases afflicting warfighters, who can then be contacted by a clinician to confirm their status,” Agu says.

DARPA dates back to the launch of Sputnik in 1957, with a mission to make pivotal investments in breakthrough technologies for national security. Agu is pushing for his breakthrough by integrating feature