Could Rocky Balboa have beaten Clubber Lang if his training montage hadn’t been set to Survivor’s “Eye of the Tiger”? Maybe, but once Rocky III audiences heard those up-tempo rock riffs, they wouldn’t dream of exercising without adding the track to their Sony Walkman cassette tapes. To this day, the tune is a popular pick for workout playlists.
Colby Leider, associate professor and director of the Music Engineering Technology program at the Frost School of Music, knows a lot about musical motivation. He is orchestrating a first-ofits-kind collaboration among musicians, biomedical engineers, and physical therapists to create a mobile app that motivates amputees to knock out harmful walking habits.
The unlikely marriage of these disciplines began three years ago, when Robert S. Gailey Jr., B.S.Ed. ’82, M.S.Ed. ’86, professor in the Department of Physical Therapy at the Miller School of Medicine, arranged for his then-teenage son, Max, to chat with Leider about the Frost School’s Music Engineering Technology program. While in Leider’s office, Gailey took note of a graduate student’s research poster—a system that measures runners’ steps per minute and selects songs from their iPod library that have the same number of beats per minute. Or it could select faster tempo songs to encourage faster running. “Like the rabbit in front of the greyhound,” explains Leider, faculty advisor for the project.
Gailey, who holds a research appointment at the Miami Veterans Affairs Medical Center and is an advisor on prosthetics to the U.S. Department of Defense, immediately thought of the potential for soldiers who’ve lost limbs in Iraq and Afghanistan.
“If we’re doing that with able bodied folks,” Gailey said of the poster, “I know a whole lot of amputees who are already listening to music. So if we can get their music to talk to their prosthetics and vice versa, the sky’s the limit in terms of rehabilitation.
For 20 years Gailey has fitted patients with prosthetics made by an Icelandic company called Össur. The UM-Össur connection strengthened in the aftermath of Haiti’s devastating 2010 earthquake, when the company partnered with Project Medishare co-founder and UM physician Barth Green to bring its products to those most in need. Last year, Össur bestowed a research grant to Leider and Gailey to design and conduct a clinical trial of a new mobile app that uses audio, visual, social media, and haptic (vibration) feedback to let users know if they’re walking in a way that could cause body fatigue, ulcers on the stump attached to the prosthetic, or stress on the non-amputated leg, which greatly increases risk of double amputation.
“It’s a computer, it’s a phone, it’s a musical instrument—and by the way, you can talk to your knee on it,” Leider says, pointing to his iPhone. “There are eight or so gait deviations that Bob can figure out just by looking at them because he’s been doing it for decades. But to get a computer to recognize those automatically, that’s what Matan just figured out how to do.” Leider is referring to Matan Ben-Asher, a second-year master’s student in music engineering who is among a dozen or so undergraduate and graduate student researchers at the University’s Functional Outcomes Research and Evaluation (F.O.R.E.) Center on the Coral Gables campus.
The center, which Gailey relocated last year from the Miller School campus and Miami Veterans Administration in order to work more closely with the Frost School, also employs Enrique Quinonez, B.S.B.E. ’10, M.S.B.E. ’12. Quinonez, a UM biomedical engineering Ph.D. student, has written a computer program that analyzes symmetry of forces on the prosthetic and non-prosthetic leg during various everyday activities.
The Össur study is one of six research projects under way at the F.O.R.E. Center, including a collaboration with the Frost School’s Department of Music Therapy to determine how and when infants begin responding to music with physical movement. Another study, funded by a grant from the Anesthesia Patient Safety Foundation, came to the lab by way of Christopher Bennett, B.S.E.E. ’05, M.S.M.E.T. ’07, Ph.D. ’10, Frost School research assistant professor, jazz pianist, and expert on how humans respond to auditory signals.
Bennett’s Ph.D. in biomedical engineering focused on psychoacoustics, the study of how the brain processes sound—everything from how we detect the origin of sounds to the many emotions that sounds evoke. Bennett completed his postdoc under Miller School anesthesiologist Richard McNeer, M.D./Ph.D. ’99, exploring how the cacophony of hospital monitoring devices affects stress levels in both patients and clinicians.
The Anesthesia Patient Safety Foundation study allows Bennett, McNeer, and now Leider to continue that work. They are using a sophisticated set of microphones to isolate and record all sound sources in operating rooms at Ryder Trauma Center. The researchers play back the sounds for medical residents while the residents perform tasks on patient simulators at the Miller School’s Center for Patient Safety. “We bring the residents to the threshold of stress by giving them complex tasks to perform,” Bennett says. “Then we play the alarms to overload them. This is what’s called alarm fatigue.” Monitoring and analyzing alarm fatigue is important because, as Bennett explains, a single machine in the operating room might be monitoring 40 different things, any of which can go slightly out of range. Multiply that by the dozens of machines functioning simultaneously, and clinicians are likely to ignore, silence, or misinterpret those alarms.
“The current standard for alarms is terrible from a psychoacoustic standpoint,” Bennett says. “One of the coolest things about audio is that it has so many different dimensions—rhythm, tempo, melody, spectrum, timbre, valence. All of these alarms are the same in every way except melodic sequence. A cardiovascular problem, for example, might be a major triad, and a respiratory problem might also be a major triad but just inverted. How can you have these two alarms and expect a clinician to know the difference? It’s something musicians have years of training in.”
Bennett’s expertise in psychoacoustics makes him an invaluable collaborator to Leider and Gailey on the Össur study because it requires deploying sounds to simultaneously convey a bevy of things—alert amputees when they’re doing something wrong, signal which movement is incorrect, and reward them when they improve their gait. “When I first started in this field, amputees were basically relegated to a wheelchair,” Gailey says. “At UM we’ve brought rehabilitation to the highest level; we’ve actually taken 50 service members with a prosthetic back into the field. We know we can get them there physically, but they want to know how they’re doing.” Gailey says the mobile app will be like a “coach or therapist they can keep with them” without having to visit a rehab center, saving time and insurance costs.
“It’s like getting a tennis or golf lesson,” Gailey continues. “You’re real smooth right after the lesson, but if it’s been a month or two, you get rusty. When patients start noticing they’re getting tired more often, they can run the program without taking time away from family or work.”
Designing this handheld “coach” involves a lot of technical know-how, not just about app programming but also about the prosthetic limbs that communicate with the app. Össur engineers routinely visit the F.O.R.E. Center from Reykjavik, Iceland to help implement and adjust all the sensors, accelerometers, gyroscopes, and other widgets in the study’s three product models—the Justin Bieber, the Lady Gaga, and the Britney Spears. The pop-star labels are Össur’s way of giving UM musician-researchers a chuckle as they keep track of each prototype.
The Britney Spears knee is actually the company’s Rheo Knee, which employs sensors to continuously measure the angle and weight on the joint while a computer chip selects the appropriate stiffness for every movement. It does this by turning a magnetic field on or off, causing magnetic particles in the joint fluid to either bond together or disperse. It’s the knee worn by study participant Kelly Elizabeth, who, as an ER technician, a nursing student, and a mother, spends a lot of time on her feet. Elizabeth lost her leg in a boating accident in 2001 and was introduced to Gailey and the Össur study by her prosthetist Adam Finnieston, who also works with Project Medishare in Haiti.
“At first I didn’t know what I was getting myself into,” she recalls. “But from the moment I put on [the Rheo Knee], I noticed a bounce in my step. It was—from what I remember—what it felt like to walk on two legs.”
Elizabeth travels from her home in Port St. Lucie, Florida, several times a week to the F.O.R.E. Center, where wireless sensors on her body and floor sensors in the lab track her movements while she listens to her favorite songs on her iPod. “I can’t go to the gym without my iPod,” Elizabeth says. “Add it to a prosthetic leg and we’re good to go.
The F.O.R.E. Center team is choosing methods of pairing music with movement in a way that would make the legendary behaviorist B.F. Skinner proud. “I could play you songs from my iTunes library that would punish you,” Leider says with a slightly devilish grin. “Or I could play you songs that would make you say, ‘Wow, that was like a piece of candy.’ We know that when you play certain sounds, the amygdala, which is the pleasure center of the brain, lights up like a fire. And when you play sounds that a person perceives as ugly, that doesn’t happen.”
So, one way to encourage good walking behaviors is through what Leider calls a “vocabulary of auditory penalty and auditory reward.” This can be done with pleasing or displeasing songs or sounds, or it can be done with auditory effects on your favorite music. “If we want to convey that you did something good,” Leider says, “we might supply an enhanced bass response, or we might make it a little louder. We could also cue an auditory effect penalty, like bit crushing. For example, we can take a 24-bit signal recorded at a really high dynamic range and crush it. You as a user don’t need to know anything about mixing. All you know is that the beautiful Norah Jones song you were just listening to now sounds like it came through a 1950s telephone.”
While music is one of the primary feedback systems in the app, it’s important to include other feedback mechanisms because the goal is to show users exactly what they’re doing wrong. With eight different gait variations and multiple movements involved in those variations, a vast catalog of sensory signals is necessary. But is it possible for a person to receive several kinds of signals at once and understand what they mean? “We’re already doing it,” Leider says. “Your phone gives you feedback in the form of pictures, sounds, and vibration, all happening simultaneously. And you’re able to distinguish what these signals all mean—whether you’re getting a text message versus an email versus a phone call and who it’s from.”
Microchips, wires, Britney Spears, and other electronic parts and pieces are scattered on the table in front of a dry-erase board where Leider, Bennett, and their students gather to scrawl formulas and discuss ideas. On the adjacent wall are two five-foot-tall electrostatic speakers that Leider found in storage and a flat-panel screen connected to Apple TV, which plays music and picturesque images that stimulate creative thinking. The selections range from Bach to Beck and from Piazzola to Pink, depending upon who has claimed DJ privileges at any given moment.
“This is the new classroom,” Leider says, “getting graduates and undergraduates together. There are two models of education. There’s the sage on the stage, and then there’s the guide by the side, which is what this is.”
The Össur study is Leider’s second funded research project. His first was a National Science Foundation study that aimed to quantify various adjectives that producers, recording engineers, and artists use to describe qualities in music. Leider, who holds a bachelor’s degree in electrical engineering, a master’s in electroacoustic music, and a master’s and Ph.D. in music composition, sees cross-disciplinary research as a positive trend in education. It’s also something that’s a natural fit for the University of Miami, which offers more academic disciplines than any other research university of comparable size.
“What’s happening now is really a return to the Renaissance,” Leider says. “A hundred years ago, if you said you wanted to combine the creation and performance of music with psychology and medicine, they would have said you’re crazy. Academics used to be pigeonholed into silos, but that’s not the way knowledge works anymore.” Leider, Bennett, and Gailey make the perfect triad for the Össur study and other research opportunities that are bound to spring from it. Leider is quick to point out that the Frost School’s Music Engineering Technology program was the first music engineering program in the United States as well as “one of the few places in the country where you need to be a geek and you need to be passionate about music.” “Nobody in medicine can do what the Music Engineering Technology folks can do,” Gailey says. “What we learn can be translated to Parkinson’s disease, people with balance issues, and so many other areas of study.
This is an emerging new field being born right here.” Gailey, who has published dozens of research articles, returned wounded soldiers to active duty, and enabled double amputees to run like the wind on blades of steel, calls his work with Bennett and Leider “the most exciting project I’ve ever been involved with. “I know this is the tip of the iceberg,” he continues, “and I can’t even see how far it’s going to expand.