Haptic musical instruments — it really is a part of your body

Fingertip enhancers for surgeons

I’ve done a bit of reading-up on the current level of understanding of haptics for musicians, and it’s really backwards, I think. Seriously. They glue little balls on a violinist’s bow hand and tell them to bow; the data recovered from that is way too crude and rough when studying something movements that are so fine and delicate — and so sensitive to error. I can’t see how anyone could be interested in that, and I don’t think that very much good data can be obtained. Failing that, they make some sort of useless interface to a musical instrument where you flail your arms around at random and it bleats at you or something else that belongs in FAO Schwarz.

What I’d like to see done is to use this sort of thing — the thin film over the fingertips in the above link — not to enhance the fingertips but to enhance the instrument itself. To create a thin film that lies over the belly of a viola, cello, violin, or double bass, and that feeds back the sensations of the belly vibrating as one plays to say, the skin of one’s upper back. To turn the instrument into something that feels like a part of the body. To make it a phantom limb of sorts.

And I’m curious as to how this sort of feedback would influence a learner. Would it be easier to learn to play if you could feel the vibrations in your body as well as hear them? One might think so, but there are plenty of people who sing off-key, and that instrument sure is haptic.

I think it would be a fascinating thing to try, to create a thin film of sensors that then connects to a similarly laid-out thin film of voltage transmitters that lies over the upper back of the musician, and that would allow the musician to feel the vibrations with their own body and extend their sense of body awareness to their instrument. Musicians always speak that way of their instruments, or at least some do. (I tend not to, but it’s hard if you’re a pianist. The goddamn thing is just too unwieldy.) A relatively single-note instrument with sustained and fairly fluid vibration like a string instrument would be a perfect thing to try this out on.

If it works, sensors could be embedded directly in the thing at point of manufacture if we’re talking about a CF string instrument like one of the Luis & Clark type thing. (Woodwinds might be neat, too, with sensors embedded in the bore.)

The sensor net would have to be calibrated, tweaked for optimum sensitivity, so that it wasn’t just a sheen of buzzing that the user felt but reflected more detailed information. Some sensors would need to respond more, some less. Perhaps they would have to be laid out differently for setup, depending on the location of the bass bar and the soundpost. Perhaps the signal would be slowed down before being transmitted to the skin. Perhaps a different calibration would have to be used for different forms of music. I think the calibration would be best done by the musician themselves, which means that the GUI would have to be very user-friendly and intuitive.

It’s all very interesting. Makes me wish I could study it formally. I keep thinking that someone somewhere must be, but googling “haptic instruments” routinely returns the crappiest imaginable results.