Controlling Multiple Parameters with Leap Motion and GecoMIDI

Discussion of Leap Motion’s ability to control multiple parameters

The first experiment showed the Leap Motion, combined with GecoMIDI to be effective in communicating with Resolume Avenue, or indeed any other MIDI capable software or device. In two planes it is possible to effectively control parameters such as width or height of an image, although the GecoMIDI software used is capable or receiving data 40 input controls through different hand movements.

The GecoMIDI interface is capable of reading the following 10 movement streams:

up down position Up & Down Position: distance from the Leap Motion Controller (Y-axis)
up down distance Up & Down Movement: distance from when your hand was first seen (Y-axis)
left right position Left & Right Position: distance from the Leap Motion Controller (X-axis)
left right distance Left & Right Movement: distance from when your hand was first seen (X-axis)
back forth position Back & Forth Position: distance from the Leap Motion Controller (Z-axis)
back forth distance Back & Forth Movement: distance from when your hand was first seen (Z-axis)
pitch Pitch Inclination of your hand’s palm
roll Roll Inclination of your hand’s palm
yaw Yaw Direction of your hand’s palm
presence Presence of your hand

http://uwyn.com/geco/docs/

These movements can be created, and read by the Leap Motion device as created by either an open or closed hand, therefore creating 20 movements with one hand. Both hands can be read simultaneously, therefore enabling 40 potential data sets for manipulation.

Two hands and 40 data input streams manipulated simultaneously offers significant potential for audio-visual performance, although usage of the device, in testing using the GecoMIDI software interface, presents issues which prevent the full potential of control to be exploited.

Up-Down, Left-Right, Back-Forth

‘Up & Down Movement’ and ‘Up & Down Position’ generate data simultaneously with the same hand movement. This also occurs with both Left & Right position/movement, and Back & Forth position/movement controls. With these three movement controls the movement and position controls receive data simultaneously. This generates similar data data streams, albeit not identical, and so cannot be utilised to control diverse sets of parameters.

Roll, Pitch and Yaw

Roll, pitch and yaw, terminology taken from aviation controls, allows data to be manipulated relative to the position/angle of the palm in the X,Y and Z axis. This control, if carefully executed, could be used in addition to the aforementioned direction and position controls.

Potential usage of this data control could for example be utilised as an expression control, similar to either a pitch bend or modulation within a synthesiser. The Roll, Pitch and Yaw parameters were assigned to Rotate X, Rotate Y and Rotate Z parameters within an effect module in Resolume Avenue.

Pitch controls were assigned successfully, with the tester able in independently control a Resolume parameter without hand movements affecting other assigned controls. Problems were presented with Roll and Yaw controls:

Roll controls were successful for the tester, who was able to move the palm of the left hand 180 degress to enable full control of this parameter. Combined with Yaw controls there was significant data spillage across the two parameters making simultaneous and independent control of both parameters difficult.

Testing with the left hand also presented problems with the Yaw parameter. Manipulating the hand to rotate in an anti-clockwise fashion proved near-on impossible to create a positive data stream.

 

 

Data Stability

How stable is the data provided during usage of the Leap Motion?

Repetitive Strain Injury?

[to be continued…]