arrow_back Back to blog

Using machine learning to advance movement analysis with the Verily Study Watch

| Written by:
Tina Karimi

Tina Karimi

Contributing Editor, Verily

The average American visits the doctor four times per year. Assuming each visit takes an hour, this means less than 0.1% of our year is spent with a healthcare provider. The vast majority of what impacts our health – sleep, nutrition, movement – happens outside the clinic, in the real world.

At Project Baseline, we're on a mission to gather comprehensive health data to create a "baseline" that defines what it truly means to be healthy, and delineate the transition to disease. Our participants take a number of endurance, strength, agility, and balance tests at their annual study site visits, so researchers can aggregate and track changes over time. A key part of this effort is ensuring that participants have insight to their personal data–as well as a voice in how it's delivered.

Accordingly, the shift to real-world evidence (RWE) in clinical research aims to ensure that treatments are proven effective outside the controlled environment of a lab. Typically, however, RWE generation doesn't even begin until the late stages of a trial, and often continues after health interventions are approved and brought to market.

Could collecting richer data, in more realistic conditions, earlier in the research process ultimately improve patient care? Verily's Study Watch1, originally developed as part of the Project Baseline Health Study, was designed to help answer this question. While many tools are available for tracking personal activity, research sensors need to gather a tremendous amount of detailed health data and last much longer between charges.

"When designing a sensor for clinical research, we care a lot more about the sampling rate: we want much more data within a given unit of time," said Dr. Megan Rothney, program manager of the Project Baseline Health Study. "We want the flexibility to gather multiple kinds of physiological data over an extended period — up to a week at a time, with the Study Watch – so we can analyze it and identify the best opportunities for learning about human health. To truly understand health and disease, we want to pinpoint as many fruitful opportunities for research as possible."

The Study Watch is built to pick up a broad range of rich clinical data that researchers can mine for health insights. Beyond movement and environmental sensors, the Study Watch captures electrocardiogram data (or the electrical impulses of the heart), electrodermal activity (skin-related data including signals from the nervous system), and photoplethysmogram data (like heart rate and sleep quality). However, collecting more complex data is just the first step: a key part of better understanding health is creating ways to make that information useful.

Within the Health Study, participants and researchers collaborate to make data better and more actionable. Driven by participant feedback on the precision of daily step counts, the Project Baseline data team was able to update to the Study Watch's algorithm to improve accuracy. Using machine learning, this algorithm is designed to analyze and derive insights from Study Watch data, helping researchers establish what "normal" movement really looks like.

A type of artificial intelligence, machine learning refers to the idea that systems can "learn" from data, enabling them to make increasingly better decisions and predictions. When it comes to healthcare, machine learning may hold immense promise in diagnosing and treating disease. For example, Verily's retinal diagnostic collaboration with Google has launched a screening tool for detection of diabetic eye disease. Fed with thousands of pictures of eyes, ranging from normal to severely damaged by diabetic retinopathy, the algorithm is able to assess medical imaging to determine patients' eye health.

Diabetic retinopathy screening at Aravind Eye Hospital, Madurai, India, leveraging machine learning.

Similarly, to create a "baseline" of movement, Project Baseline is training the Study Watch algorithm in partnership with Health Study participants. In addition to generating health data, these participants are also directly impacting the statistical models and technology that make the information meaningful.

"Traditionally, researchers would execute a study like this by observing people in a lab, which presents a number of obstacles," said Dr. Rothney. "There's only so much we can do to mimic a real-world setting: people know they're being observed, and labs can only accommodate a limited number of people at a time. Getting enough data, from study populations that reflect our country's population, is difficult under these conditions. What's special about the Project Baseline Health Study is that participants from communities across the United States can generate incredibly rich data in real time."

The Study Watch algorithm learns by synthesizing many different examples of movement to determine what makes walking, for instance, distinct from any other actions we may perform in a day. Since the Study Watch is worn on the wrist, many gestural movements may read like walking at first – like driving a car, or cleaning, for instance. The more data we can gather, the more of a pattern will emerge around what the Study Watch recognizes as a particular type of movement.

To tag their activities, participants note what they're doing several times a day. Participants can "tag" their actions as walking or running, standing or sitting still (which helps establish a resting heart rate), or any other activity using a custom tag. Researchers can then use this data to help make the Study Watch "smarter" and more precise.

Participants can contribute granular data to enhance the Study Watch algorithm.

"I can't say enough how important Health Study participants were to this initiative," said Dr. Rothney. "Activity tagging participation was completely voluntary. We are encouraged to see just how many participants are excited to take part – sometimes contributing up to a hundred tags at a time! The Study Watch team was thrilled to see new tags come in every day, because they're essential to the study: more data means the ability to explore deeper and more challenging areas of human health."

Moving forward, Project Baseline hopes to leverage this data to advance science around movement disorders. Further, developing robust health-related sensors like the Study Watch can fuel precision medicine efforts and enable more personalized care in areas like weight management, diabetes, mental health, and more.

Since healthcare is not one-size fits all, understanding what happens in between medical appointments can drive more individualized lifestyle recommendations. Over time, healthcare providers can see trends in behavior and make inferences as to what they may mean for patients. For example, a sudden change in a patient's movement may indicate a possible injury, or the onset of a medical condition like heart disease, or depression. With a ready source of historical data, it's easier to catch health changes early – leading to more proactive care and better outcomes.

1. A version of the Verily Study Watch has been cleared in the U.S. by the FDA. The version of the Study Watch referenced here is an investigational device.

Next story: Applying user experience (UX) design to clinical research: Behind the Scenes Q&A

"Start with the use" is a guiding principle in technology, and the foundation of effective product design.