Songs and sounds that can help amputees walk better, safer, stronger? Frost School of Music researchers merge music, engineering, and medical disciplines to make sure there’s an app for that.
Could Rocky Balboa have beaten Clubber Lang if his training montage hadn’t been set to Survivor’s “Eye of the Tiger”? Maybe, but once Rocky III audiences heard those up-tempo rock riffs, they wouldn’t dream of exercising without adding the track to their Sony Walkman cassette tapes. To this day, the tune is a popular pick for workout playlists.
Colby Leider, associate professor and director of the Music Engineering Technology program at the Frost School of Music, knows a lot about musical motivation. He and biomedical engineer Vibhor Agrawal, Ph.D. ’10, an assistant professor in the Department of Physical Therapy at the Miller School of Medicine, are orchestrating a first-of-its-kind collaboration among musicians, biomedical engineers, and physical therapists to create a mobile app that motivates amputees to knock out harmful walking habits.
The unlikely marriage of these disciplines began in 2010, when Robert S. Gailey Jr., B.S.Ed. ’82, M.S.Ed. ’86, a physical therapy professor at the Miller School, arranged for his then-teenage son, Max, to chat with Leider about the Frost School’s Music Engineering Technology program. While in Leider’s office, Gailey took note of a graduate student’s research poster—a system that measures runners’ steps per minute and selects songs from their iPod library that have the same number of beats per minute. Gailey, who holds a research appointment at the Miami Veterans Affairs Medical Center and is an advisor on prosthetics to the U.S. Department of Defense, immediately thought of the potential for soldiers who’ve lost limbs in Iraq and Afghanistan.
“I know a whole lot of amputees who are already listening to music,” Gailey says. “So if we can get their music to talk to their prosthetics and vice versa, the sky’s the limit in terms of rehabilitation.”
For 20 years Gailey has fitted patients with prosthetics made by an Icelandic company called Össur, which has bestowed a research grant to Leider and Gailey to design and conduct a clinical trial of a new mobile app that employs audio, visual, social media, and haptic (vibration) feedback. The various signals let users know if they’re walking in a way that could cause body fatigue, ulcers on the stump attached to the prosthetic, or stress on the non-amputated leg, which greatly increases risk of double amputation.
“It’s a computer, it’s a phone, it’s a musical instrument—and by the way, you can talk to your knee on it,” Leider says, pointing to his iPhone.
The Össur study is one of six research projects presently under way at the University’s Functional Outcomes Research and Evaluation (FORE) Center on the Coral Gables campus, including a collaboration with the Frost School’s Department of Music Therapy to determine how and when infants begin responding to music with physical movement. Another study, funded by a grant from the Anesthesia Patient Safety Foundation, came to the center by way of Christopher Bennett, B.S.E.E. ’05, M.S.M.E.T. ’07, Ph.D. ’10, Frost School research assistant professor, jazz pianist, and expert on how humans respond to auditory signals.
Bennett completed his postdoc under Miller School anesthesiologist Richard McNeer, M.D./Ph.D. ’99, exploring how the cacophony of hospital monitoring devices affects stress levels in both patients and clinicians. The study allows Bennett, McNeer, and now Leider to continue that work. They are using a sophisticated set of microphones to isolate and record all sound sources in operating rooms at Ryder Trauma Center. The researchers play back the sounds for medical residents while the residents perform tasks on patient simulators at the Miller School’s Center for Patient Safety.
Bennett’s expertise in psychoacoustics makes him an invaluable collaborator to Leider, Agrawal, and Gailey on the Össur study because it requires deploying sounds to simultaneously convey a bevy of things—alert amputees when they’re doing something wrong, signal which movement is incorrect, and reward them when they improve their gait.
“When I first started in this field, amputees were basically relegated to a wheelchair,” Gailey says. “At UM we’ve brought rehabilitation to the highest level. The military has actually taken 50 service members with a prosthetic back into the field. We know we can get them there physically, but they want to know how they’re doing.”
Gailey says the mobile app will be like a “coach or therapist they can keep with them” without having to visit a rehab center, saving time and insurance costs. “When patients start noticing they’re getting tired more often, they can run the program without taking time away from family or work.”
Designing this handheld “coach” involves a lot of technical know-how, not just about app programming but also about the prosthetic limbs that communicate with the app. Össur engineers routinely visit the FORE Center from Reykjavik, Iceland, to help implement and adjust all the sensors, accelerometers, gyroscopes, and other widgets in the study’s three microprocessor knee models—the Justin Bieber, the Lady Gaga, and the Britney Spears. The pop-star labels are Össur’s way of giving UM musician-researchers a chuckle as they keep track of each prototype.
The Britney Spears knee is actually the company’s Rheo Knee, which is the knee worn by study participant Kelly Elizabeth, who, as an ER technician, a nursing student, and a mother, spends a lot of time on her feet. Elizabeth lost her leg in a boating accident in 2001 and was introduced to Gailey and the Össur study by her prosthetist Adam Finnieston, who also works with Project Medishare in Haiti.
“At first I didn’t know what I was getting myself into,” Elizabeth recalls. “But from the moment I put on [the Rheo Knee], I noticed a bounce in my step. It was—from what I remember—what it felt like to walk on two legs.”
Elizabeth travels from her home in Port St. Lucie, Florida, several times a week to the FORE Center, where wireless sensors on her body and floor sensors in the lab track her movements while she listens to her favorite songs on her iPod.
The FORE Center team is choosing methods of pairing music with movement in a way that would make the legendary behaviorist B.F. Skinner proud.
One way to encourage good walking behaviors is through what Leider calls a “vocabulary of auditory penalty and auditory reward.” This can be done with pleasing or displeasing songs or sounds, or it can be done with auditory effects on your favorite music.
“If we want to convey that you did something good,” Leider says, “we might supply an enhanced bass response, or we might make it a little louder. We could also cue an auditory effect penalty, like bit crushing. You as a user don’t need to know anything about mixing. All you know is that the beautiful Norah Jones song you were just listening to now sounds like it came through a 1950s telephone.”
While music is one of the primary feedback systems in the app, it’s important to include other mechanisms because the goal is to show users exactly what they’re doing wrong. With eight different gait variations and multiple movements involved in those variations, a vast catalog of sensory signals is necessary. But is it possible for a person to receive several kinds of signals at once and understand what they mean?
“We’re already doing it,” Leider says. “Your phone gives you feedback in the form of pictures, sounds, and vibration, all happening simultaneously. And you’re able to distinguish what these signals all mean—whether you’re getting a text message versus an email versus a phone call and who it’s from.”
Leider, Agrawal, Bennett, and Gailey make the perfect team for the Össur study and other research opportunities that are bound to spring from it. Leider is quick to point out that the Frost School’s Music Engineering Technology program was the first music engineering program in the United States as well as “one of the few places in the country where you need to be a geek and you need to be passionate about music.”
“Nobody in medicine can do what the Music Engineering Technology folks can do,” Gailey says. “What we learn can be translated to Parkinson’s disease, people with balance issues, and so many other areas of study.”
Gailey, who has published dozens of research articles, returned wounded soldiers to active duty, and enabled double amputees to run again competitively, calls his work with Leider, Agrawal, and Bennett “the most exciting project I’ve ever been involved with.
“I know this is the tip of the iceberg,” he continues, “and I can’t even see how far it’s going to expand.”
This story by Meredith Camel first appeared in Score, the Frost School of Music magazine.