Guest blog: How one UWE Bristol researcher’s work led to AI gloves that control sound
There are times in your research career when you get to combine all of your knowledge. For Tom Mitchell this moment was when he married his expertise in artificial intelligence, music and audio processing to help produce the Mi.Mu gloves.
These futuristic lightweight fingerless gloves enable performers to invent their own gestures and then map them to control any sound or virtual musical instrument. Tom, a senior lecturer in computer music at the University of the West of England (UWE Bristol) is one of the computer scientists behind the algorithms, software and machine learning that help the devices work.
Although Tom developed the original idea, the gloves are a product of the Mi.Mu collective, which brings together specialists in software development, motion capture, wearable electronics and textiles, who have worked together to piece together the device. After several iterations, the team has created a comprehensive system that a number of internationally renowned professional musicians have now incorporated into their creative process and live performances.
“Our ultimate aim is to make the gloves and its supporting software and algorithms available to as many people as possible”
In 2014 the gloves’ flagship artist Imogen Heap, who is part of the collective, wrote Me the Machine, the first ever song entirely created entirely using just the gloves. The electronic musician continues to promote the Mi.Mu gloves across the globe and demonstrated it at CERN’s 2015 TEDX event. You can see the video below:
That same year, acclaimed singer Ariana Grande sported the gloves to control sounds throughout her Honeymoon Tour in 2015. For Dutch artist Chagall van den Berg, who is also part of the Mi.Mu collective, using the gloves is a big part of the way she now writes and performs music. “Thinking about what gestures she will use has completely changed the way Chagall composes and performs her music,” explains Tom.
Breaking musical barriers
The collective has also worked with professional musician Kris Halpin who has an impairment that makes guitar and piano playing increasingly difficult. Kris joined up with Drake Music, a charity that helps musicians with disabilities to break down barriers to music, to become official collaborators on the gloves project. The gloves enabled the artist to imagine new ways of making music that were entirely unique to him. “Kris tells us that the gloves have literally changed his life and he now has opportunities to create music that would not have been possible in the past,” says Tom.
The Mi.Mu gloves were born after Tom (pictured right), a senior lecturer in computer music at the University of the West of England (UWE Bristol), met musician Imogen Heap at a wedding in 2010. The Grammy-winning artist wanted his help to develop new ways of creating music and performing songs with gestures. Tom therefore teamed up with the Bristol Robotics Laboratory (BRL) to tap into its research into human robot interaction with data gloves.
Throughout the following year he worked closely with Imogen Heap to develop software (that he named ‘Glover’) incorporating gesture recognition algorithms and techniques for ascribing these gestures to sounds. In 2011 the collective produced an initial prototype and subsequently the gloves were developed to wirelessly sense a wide range of gestures that musicians could easily use to trigger and control an infinite range of sounds.
“One of the major challenges is to take software developed for research in the lab so that it can be used by musicians on the stage,” says Tom. “Luckily we have now achieved this, and therefore avoid things going wrong during a live performance,” he adds.
Commercialising the product
The team has now sold an initial run of 25 pairs of gloves, and is looking to optimise manufacturing process to make them more widely available and affordable. “We receive constant emails from people enquiring about them and there is huge enthusiasm from the music community,” says Tom. “Our ultimate aim is to make the gloves and its supporting software and algorithms available to as many people as possible,” he adds.
He and the rest of the gloves team are in the process of adapting the Glover software to work with a whole host of general-purpose gesture detection and human-computer interaction devices. These developments will open brand new avenues to music making and have the capacity to transform traditional notions of music composition and performance.