AV16-4 – A Pure Data audio-visual experiment – Assigning Joystick Control

joystickThe first stage of the AV16-4 was to set up the instrument, with video playback and with audio sequencer and performance control, taking the form of a drum sample player. The next stage in development was to implement an OSC / tablet based interface that gave access away from the laptop, accessing most parameters of the instrument.

The second stage of development was to develop ways to access the instrument parameters via alternative interfaces. Chosen for this project were two methods.

Firstly, joystick / HID control using the Speedlink SL-6612-SSV WASP² was chosen as an input device. As per previous experimentation with Resolume Avenue, junXion was utilised as the data routing application to enable data to be manipulated within Pure Data.

Routing MIDI data in Pure Data is a relatively straightforward process. Chosen for data manipulation was:

X-axis, Y-axis, Z-rotation – The ctlin function was utilised to receive the data. This generated values between 0-127 on all three controls, which could be assigned to parameters such as volume, pan, reverb parameters or translate / rotate parameters used for video control.

Buttons 1-8 – The data received through junXion allowed both note on and note off functionality for the buttons. This generated data that was suitable for switching controls, such tempo presets or triggering individual samples or indeed stopping and starting the sequencer. A small obstacle in achieving this due to the button also sending note off, which when attached to a bang in Pure Data sent a second message. This small problem was overcome through the use of the moses function, which was set to bypass data below 1.

The initial screenshot of the master section of the AV16-4 is shown below:

Pure Data Joystick MIDI assign


Video manipulation was achieved through joystick manipulation of both the translateXYZ and rotateXYZ functions. The initial settings were assigned to enable the XY & Z parameters to be manipulated simultaneously. The Y parameter was versed to make an interesting movement.

The maxlib/scale function was utilised to scale the MIDI data to the slider parameters in each case, to allow full movement of the translate & rotate functions. It was found during initial testing that the scope of the joystick movement was too pronounced and created an extreme movement so the boundaries of the upper and lower movements were limited.

The video4channel.pd patches with joystick assignment is shown below:

Video Mixer Joystick Assignment avj16-drums.pd – Joystick assignment

Joystick assignment for the drum sample player was relatively limited compared to the sequencer and video elements. Assignments were made utilising the same maxlib/scale technique used within the video mixer. Assignments were made to the reverb parameters for the initial test. X-axis movements were assigned to dry levels, Y-axis movements were assigned to wet levels, and the Z-rotate control was assigned to the Freeverb roomsize parameter.

Assignment is such a way creates a usable way to manipulate the audio in some directions, although in a resting state there is some reverberation heard. The definitive assignments should be configured to allow a totally dry signal in a resting state.

The figure below shows the drum sampler with joystick assignments made:


The final MIDI-joystick assignment concerned the sequencer element of the AV16-4. This assignment was primarily button focussed, and took advantage of the buttons to manipulate the tempo presets within the instrument. The 8 buttons on the joystick were mapped to the eight tempo presets to allow swift changes of tempo without needing the access the computer keyboard or mouse. The patch with joystick assignments is shown below: