Using Leap Motion & GecoMIDI to control video in Resolume Avenue

Resolume Avenue is a VJ tool that allow control of almost all parameters via MIDI messages. These messages and control can be assigned to:

  • Composition properties and effects – control of parameters on a global level
  • Layer properties and effects – control of parameters on a layer level
  • Clip properties and effects – control of parameters on an individual clip level

This allows a multifaceted control of Resolume parameters. Using the Leap Motion controller to affect these parameters give the audio-visual performer or VJ an interesting and additional option during performance. Here I’ve documented some experiments controlling Resolume 3.3.3 with the Leap Motion controller, using GecoMIDI as an interface between the two.

Experiment to control image width and height

Scaling a video with real manipulation was chosen as the first test. This is a simple effect and achieved simply by affecting the width and height parameters in Resolume.

Within Leap Motion, the Left & Right Position control was selected. The same Left & Right Position control was used for height and width control, with a closed hand control for width, and open hand for vertical parameters.

This worked, although had two problems. The rest value within the Geco MIDI control panel was not set, which meant that the video was not visible when the hand was removed from sight of the Leap Motion. The second problem was that the width and height could not be controlled simultaneously.

The first problem was overcome through assigning the rest value a suitable setting, found to be 12% and 16% with the test footage.

The second problem had two potential solutions. Firstly the height and width parameters could be controlled through separate hand movements, such as left to right for width, and up and down for height. This has a logical feel with feedback provided through the actual movement of the hand. The second option was to have the two parameters controlled through both the left and right hand.

As a control technique, assigning the width and hand controls to left and right hands respectively worked well. Hands could be moved simultaneously or separately, either opposed or in parallel to create interesting warping effects to the test footage. The second solution was implemented using the same hand with vertical movement utilised to control the height of the test footage. This gave identical results, although was more fluid in the output, with more accurate control over the width and height parameters. This also had the advantage of freeing the second hand to control further parameters during performance.

During this testing it was found that if the hand was moved to the extreme of the camera field on the horizontal plane (25cm) then the control would reset to the default position. With the vertical position one was able to raise the hand over 50cm, the upper limit of vertical data, before the hand image was lost and data was reset.

About Matt

Hello. If you want me I'll be here.

View all posts by Matt →