Menu
What's new
By:
Filters

Violin Bow MIDI Controller using the Apple Pencil

Audio Motion

Audio Motion

New Member
Hi everyone

I always felt that we lack a device for controlling bow movement for strings that is as intuitive as a breath controller. This is why I wrote a new iOS App called Pen2Bow, a violin bow MIDI controller using the Apple Pencil on the iPad. It takes full advantage of the sensors in the Apple Pencil and generates 4 continuous MIDI controllers: velocity, force, tilt and orientation. These can be assigned for example to expression, vibrato, bow pressure and position on the bridge to drive a virtual violin - or to any other parameter to drive synthesizers.

But the key innovation is that it recognizes linear as well as circular pencil motion to generate bow movement from the user input. Circular motion enables the infinite extension of the bow that otherwise would be limited in length. This frees the user from constantly thinking about performing bow changes in order to prevent the bow from running out. I believe it is one of the very few, if not the only practical MIDI input device for bowed instruments.

For more information please watch "Pen2Bow: Violin Bow MIDI Controller using Apple Pencil on iPad" on youtube:
I hope you will enjoy this new device.

Update:
There are various ways to connect the iPad to your computer and transmit the MIDI information. You can use a MIDI interface like e.g. "iConnectivity iConnectMIDI2" to connect through a cable or you can use a wireless MIDI connection over WiFi. To set this up please check this blog post: http://musicappblog.com/wireless-network-midi-ipad-to-mac/ (There is also a link at the top that shows how to achieve this using a pc). Please note, depending on the performance of your WiFi network this may add a few milliseconds of latency.

Kind regards
Dimos Gaidatzis
Switzerland
Click to read more...
 
Last edited:
This is incredible!!! What an amazing idea, and the perfect controller for SWAM instruments! I downloaded immediately, and can't wait to try it out, but please could you let me know how do I link the app to my computer as a midi controller?
 
Garry said:
This is incredible!!! What an amazing idea, and the perfect controller for SWAM instruments! I downloaded immediately, and can't wait to try it out, but please could you let me know how do I link the app to my computer as a midi controller?
Click to expand...

Thanks a lot, I am glad you like the idea! Please check the update in the first post.
 
Last edited:
This is making me considering swam violin again.
Interesting!

Btw: Apple just released a new ipad at a low price range that can also use the pencil
 
I don't have any iDevices so I can't try this out, but I've experimented with a homebrew system that works similarly to this (using a hacked drawing tablet), and I can confirm that the concept is really cool. I'm glad someone is taking the idea and making it commercially available.
 
Hey, it works!!!! I set it up using the blog you mentioned, and I have 0ms latency! Works perfectly! :)

In using the app, particularly with SWAM instruments, it really does add another layer of expression. I use multiple midi controllers to get the most out of these highly expressive instruments, but with the TEControl breath controller and the Korg Nanokontrol2, I'm not able to achieve the bowing action that you can get with Pen2Bow. The other controller that I use is Leap Motion, which uses hand gestures, and this does allow a similar bowing motion, but the Pen2Bow is definitely an interesting option that I will play with, and since it now DOES NOT require additional hardware (as the Leap Motion does, well it's not additional, it is itself hardware based), then Pen2Bow is considerably cheaper.

Thank you Dimos. I hadn't understood that this would work over wi-fi, but, I was wrong - the wi-fi based MIDI connection (on my system at least, can't vouch for others) works extremely well - I really wasn't expecting 0ms latency!

Minor downside: since you have to set up a local network, you are no longer connected to the internet. A minor point, and not the fault of the app of course, just the nature of the connection, but just some user feedback for others, in case this is relevant for them.

My only suggestion would be to add some description to your app about the wi-fi midi connection and setup instructions (or just the link you previously provided), and that if the latency on their specific system does not allow it to be used, then there is the additional hardware route as a failsafe (but more expensive) option. Then people will know what they are getting from the start. But if everyone has the same experience as me, there is absolutely no need for the additional hardware - the wi-fi connection is flawless.
 
Last edited:
Garry said:
Hey, it works!!!! I set it up using the blog you mentioned, and I have 0ms latency! Works perfectly! :)

...
Click to expand...

I am glad it works and that you enjoy using it. I added an update to the original post and on youtube to include the info on how to connect the iPad to a computer and will also update the App Store info as soon as possible. Thank you very much for your input.
 
I'd like to say thank you to @Audio Motion for Pen2Bow App. It is a cleaver and useful tool for controlling the SWAM Solo Strings with a natural gesture for the bowing gestures.
Note that this finally gives more sense to the "Bipolar" Bow Gesture we have introduced from the beginning, waiting for a suitable controller... now it has found its perfect match!

As for the connection between the iPad and macOS, now iOS11 and macOS Sierra and above support direct MIDI over USB.
Please watch this video showing how to control SWAM instruments from GeoShred. Starting at 00:38 it explains how to connect the iPad to macOS over USB: http://www.swamengine.com/2018/03/using-geoshred-with-swam/

BTW: we have recently released an update of all our SWAM products. Login to the Audio Modeling Customer Portal (https://my.audiomodeling.com) to get it.

Emanuele
CTO
Audio Modeling
www.audiomodeling.com
 
Pretty neat.

This sort of problem is why I have stayed clear of many solo string libraries, and while I am always an advocate of recording real solo strings, I understand that not everyone has the resources to do this. A step forward towards better mockups!

Audio Motion, do you have any plans on making a port of this prepackaged for the roli environment? While the ipad ecosystem is great, not everyone is sold on them. I do not know what the hassle would be to do this, but it may expand your potential market. I guess my thought is that people on a budget may go for an mpe controller like a roli block over an ipad pro with an apple pen.

Nice job.
 
Piano Pete said:
Pretty neat.

This sort of problem is why I have stayed clear of many solo string libraries, and while I am always an advocate of recording real solo strings, I understand that not everyone has the resources to do this. A step forward towards better mockups!

Audio Motion, do you have any plans on making a port of this prepackaged for the roli environment? While the ipad ecosystem is great, not everyone is sold on them. I do not know what the hassle would be to do this, but it may expand your potential market. I guess my thought is that people on a budget may go for an mpe controller like a roli block over an ipad pro with an apple pen.

Nice job.
Click to expand...

Thanks a lot for your encouragement. I thought about many ways of achieving a bow controller with different available hardware. I experimented also with regular touch control, but I found that a finger touching a glass or plastic surface causes too much drag. It makes it virtually impossible to measure force and velocity simultaneously as the finger starts to jump on the surface. I came to the conclusion that only a device like a pencil with minimum drag allows for precise and smooth control of a virtual bow. Moreover, I don't see how to measure additional controllers like tilt and orientation as easily when using the finger. If a platform offers a tablet-like device with accurate stylus support I would definitely consider it as a potential option.
 
Pen2Bow 1.0.1 update was released

updates:

- Added smoothing option for the pencil velocity. Pen2Bow is very sensitive and picks up the smallest fluctuations in pencil speed. When used with string instruments, this greatly adds to the realism. In the case of synths however, those micro fluctuations might be less desirable. Therefore now there is an option to dampen those fluctuations (see help).

- Support for iOS 11.0 and higher (before it was 11.2 or higher)

- Pen2Bow now produces force values from 1 to 127 when the pencil is attached to the surface. Before it was 0 to 127. Zero is now only emitted when the pencil is detached from the surface. This allows a downstream App to detect pencil attachments and detachments by reading the force continuous controller.
 
Thank you all for your interest and feedback! We just released update 1.0.2 that includes some of the features that users requested to make Pen2Bow more powerful when used with synths. Here is a video introducing what's new:

Updates in detail:

- Added a settings page. It includes sensitivity and range adjustment for the 4 MIDI controllers.

- Added a touch piano keyboard. Virtual instruments can now be played without the need for an external MIDI keyboard.

- Added a refresh button for the MIDI device list. Pen2Bow no longer needs to be restarted in order to detect new MIDI devices.

- Added smoothing option for the pencil velocity. Pen2Bow is very sensitive and picks up the smallest fluctuations in pencil speed. When used with string instruments, this greatly adds to the realism. In the case of synths however, those micro fluctuations might be less desirable. Therefore now there is an option to dampen those fluctuations (version 1.0.1 and higher).
 
New Video is out! Part 1 of a 4-part series of videos, showing the interplay between the MIDI controller App Pen2Bow and the virtual string instruments by Audio Modeling. This part is about the SWAM Violin.
 
New video is out! Part 2 of a 4-part series of videos, showing the interplay between the MIDI controller App Pen2Bow and the virtual string instruments by Audio Modeling. This part is about the SWAM Viola.
 
It is interesting to visualize the perspective of "realism" of this can generate, curiously it is what most concerned in the 'library' industry and 'samples' a realism in what refers in the joints, but finally we continue talking about the same is a sequence, program. To me at least anyway I find it very interesting.
 
Here is a new video: Part 3 of our 4-part series on the interplay between Pen2Bow and the virtual string instruments by Audio Modeling. This part is about the SWAM Cello. It also includes a comparison between a breath controller and Pen2Bow.
 
Last video of the series is out: Part 4 on the interplay between Pen2Bow and the virtual string instruments by Audio Modeling. This part is about the SWAM Double Bass. It also includes a summary of my three month experience with Pen2Bow and the SWAM string instruments:
 
You must log in or register to reply here.
Top Bottom