Gesture Capture And Software For Musical Interactions

The core idea of the project was to conceive an open system of tangible and software interfaces, allowing users to invent their own digital musical instruments, and further configure them to enable a large set of possible gestures. This ensemble of objects is called the MO – Modular Musical Objects.

The research and development were divided in three parts. Small gesture interfaces, connecting wirelessly to a computer, were designed to sense users motions. These interfaces are used with software that were developed to control and process recorded sounds, or to interact with pre-composed musical environments. Additionnaly, the software called INScore was specifically built to create and manage augmented music scores. It supports the visualization of extended, dynamic score representations and the of music performance as well.

These developments were achieved in iterative cycles of prototyping and experimentations, including all the project partners, engineers, designers, artists and teachers.

Major Results

The musical interfaces MO (Modular Musical Objects) won the 1st prize of the 2011 Guthman competition of new musical instruments. These interfaces were featured in several international exhibitions, including «Talk to Me» at MoMa in New York, Biennale internationale design, Objet(s) du numerique – Design d’un nouveau monde Industriel in Paris, Lift Experience in Geneva.

The MO interfaces and the INScore software are the building blocks of concrete applications, in particular for musical education. The music school « Atelier de Feuillantines » actively participated in creating and using successfully such applications. New types of musical games were also designed, such as the Urban Musical Game, which was exhibited at the Paris Futur en Seine festival in 2011. Finally this project was also instrumental for the creation of a start-up, Phonotonic, that will pursue the commercialization of these developments. Other industrial applications are currently examined by the different project partners.


  • 17 publications (1 peer-reviewed journal, 1 music education journal, 3 book chapters, 12 peer-reviewed conference proceedings)
  • 4 Exhibitions on technology innovations
  • 1st Prize of the Guthman competition of new musical instruments (USA 2011)
  • 7 press and blog articles (Bloomberg, Scientific American, Creative Digital
  • Music, …)
  • Invitation to Lifts Conferences, invited seminars, radio interviews (France Inter, …)
  • 3 concerts of music students
  • 2 music education workshops
  • 1 interactive installation with >500 users
  • Start-up Phonotonic SAS (N. Rasamimanana)