This experiment is to test the capabilities of using a joystick to control video and audio within Resolume Avenue 3.3.3.
junXion was chosen as the application linking the joystick to Resolume, as it offers the flexibility if required to utilise all functions of the joystick within Resolume.
The Speedlink SL-6612-SSV WASP² was chosen as an input device as it a unit with a large number of potential assignment controls, at a budget price (£14.99). Controls available on this device include:
- X-axis – Main joystick directional control.
- Y-axis – Main joystick directional control.
- Z-axis – Also the main joystick direction control, produces values from 0-127 on one side of the stick, with the side affected by the Z-rotation control.
- Z-rotation – A control located on the front of the unit. Operates from 0-127 and affects the Z-axis control.
- 12 buttons – Button 1 is the main trigger. Buttons 2, 3 & 4 are on the top of the unit, and easy to trigger whilst operating the main directional control. Buttons 5-8 are on the base of the unit.
- Hatswitch – This control, on the top of the unit, generates 8 values between 1-127. Resting state of 127.
To communicate with Resolume the joystick parameters were initially mapped within junXion as follows:
The features of the joystick lead to X-axis & Y-axis parameters controlling parameters with variable control, whereas the buttons are limited to two states (on or off). From an ergonomic perspective the following controls were chosen as the primary input controls to be assigned within Resolume Avenue:
- Button 1 (Trigger control)
- Buttons 2, 3 & 4 (top of unit controls)
The X-axis and Y-axis controls together allow for the following combination of states when the joystick is pushed 100% in the following positions:
Stick position X-axis value Y-axis value Up (0°) 63 0 Up-Right (45°) 127 0 Right (90°) 127 63 Down-Right (115°) 127 127 Down (180°) 63 127 Down-Left (225°) 0 127 Left (270°) 0 63 Up-Left (315°) 0 0
Sweeping the joystick in a circular clockwise motion around the extremity of its range provides the following data:
Stick position X-axis value Y-axis value Left to right (315°-45°) 0-127 0 Up to down (45°-135°) 127 0-127 Right to left (90°-1800°) 127-0 127 Down to up (180°-90°) 0 127-0
So what does all this mean? In terms of using the device, pushing the controller joystick in a ‘top-left’ movement, or 315°, sets both the X-axis and Y-axis to 0. Moving the joystick in a ‘bottom-right’ direction, or 135°, sets both axis’ data output to 127. Sweeping the control in a clockwise manner allows control of a parameter whilst another parameter is set either to 0 or 127. This creates interesting potential when assigned to effects parameters.
Initial Resolume Mappings
Button 4 – Shift RGB effect bypass
X-axis – Distance
Y-axis – Mode
Button 2 – Pixels in Space effect bypass
X-axis – Extrusion
Y-axis – Grid Type
Button 3 – Displace effect bypass
X-axis – X factor
Y-axis – Y factor
Through experimentation it was found that the Pixels in Space effect was far from subtle. Although striking in its approach, this effect was not suitable within the current configuration. A consideration would be to use an effect such as this through button 1, the ‘trigger’, which can be used (with ‘piano mode’ selected within Resolume) to turn the effect on or off rhythmically.
Performing with this configuration presented an opportunity to create an interesting range of visual effects by manipulating multiple video effects simultaneously.
Manipulating audio with this configuration, for this experiment, was achieved in a similar manner. Extreme delay was chosen to achieve an ambient, atmospheric effect purely through manipulating the audio within the source footage. Chosen for this experiment was the the Togu Audio Line (TAL) Dub delay effect, available as a VST plug-in. This was added as a global ‘Composition’ audio effect within the effect rack in Resolume. The feedback settings were accentuated to provide an extreme delay effect, with the ‘CUTOFF’ & ‘RESONANCE’ controls assigned to the X-axis and Y-axis respectively.
Problems to overcome:
- With high-definition video, it was found that the audio element of a video file utilising VSTs effect is not smooth when the video hits a loop point. All audio ceases, without reverb tails or delays masking the loop point. For this experiment it was proven to be more successful using lower quality video files. A 320×240 quicktime file was effortless.
- With the current configuration, audio-effects / VST’s are applied globally, and not assigned to specific audio effects. This fails to add any drama to the performance, with the audio dynamic consistent regardless of the use of video effect. This is a problem that can be rectified by applying VST effects to correlating video effects.
- At this point, three visual effects are utilised within Resolume. This gives an interesting but somewhat limited scope for performance use. Utilising all 8 buttons on the joystick would offer greater scope for VJ or performance usage.
- Selecting video clips – This is an issue to be addressed through an alternative interface. Unless one were to allocate a small number of columns within a Resolume composition to be assigned to a small number of buttons on the joystick device, it would be a more suitable solution to utilise an alternative form of control, such as custom interface on a tablet, from which to select Resolume source clips to manipulate within a performance situation.