iPad Street Musicians and the Seline HD App

Posted by Christopher Sung on Oct 6, 2010

This week, the concept of the iPad musician has been burning up the blogsphere due to Alex Shpil's Vimeo video, in which she publicly plays music on an iPad on a touristed bridge - an iPad street musician. The iPad app she's using is Amidio's Seline HD, which has an intelligent, two-handed interface for creating scale-based music.

Not surprisingly, much of the debate has revolved around the use of a $500 piece of technology to busk for loose change, but this is just short-term economics. Intelligent, portable hardware with touchscreens are here to stay, and they will only get cheaper over time. In a few years, an iPad with a myriad of installed musician apps for live performance will probably be cheaper than an intermediate acoustic guitar.

To me, the more interesting question is how future audiences will view the concept of the "performing musician". A performer's proficiency on a traditional instrument (piano, guitar, violin, et al) is something people feel they can easily evaluate, having seen a multitude of performances on that instrument over time. How do you assess a performance that is not easily viewed, on an instrument that may have been just been invented, or whose interface is constantly changing over time? Or is the performance even relevant, as opposed to the end result it produces - the music itself?

During one summer in the late '90s, I was an intern at a technology think tank in Palo Alto called Interval Research, and my group was focused on creating new interfaces for musical expression, which has now evolved to the point of being its own field. The hardest aspect of this type of design is determining what type of musical content you will command and, in turn, designing a physical interface that provides a satisfying experience when controlling it.

For an interface, I ultimately chose one of their pre-fab broomsticks, which had a sanded flat surface on one end equipped with a force-sensitive resistor (FSR) to simulate a fretboard. I was then able to map the location of your finger(s) along the "fingerboard" to pitch (similar to frets), and the pressure of your finger(s) to trigger pitch bend and vibrato. The info from the sensor was sent to a MIDI module and thus, any sound in the module could be played using the broomstick.

The interesting part was mapping finger location to pitch. I ended up using a modified blues scale, and though the stick was only capable of producing one pitch at a time (monophonic), you essentially couldn't hit a bad note (plus you could bend). It couldn't change keys or its harmony, though with more work, other sensors could have been added to switch the mappings.

Now, with the advent of touch controllers as multipurpose as the iPhone and the iPad, the interface can be more robust. The Seline HD app uses the two-handed metaphor of the piano but assigns every other "key" to the two different hands. Thay way, when you play a scale, you're not using consecutive fingers, but alternating fingers on different hands - ergonomically, a much different experience.

You can also change keys, harmony, and sounds quickly. If that isn't enough, the app even analyzes your playing and will drone ambient background sounds for accompaniment - a good balance between controlling significant amounts of musical content without overloading both the player and the listener.

Here's a quick demo of the app from Amidio: