AV16-4 – A Pure Data audio-visual experiment – Introduction

av16seq - Pure Data Sequencer Patch

AV16-4 was created as a custom performance tool to experiment with performances interfaces and tools. Using Pure Data as a development from previous work using Resolume Avenue gave an opportunity to be in full control of the hardware and software signal flow and to create something unique for the end user.

AV16-4 is constructed using 3 main patches:

video4channel.pd

video4channel.pd - 4 Channel Video Player
video4channel.pd – 4 Channel Video Player

video4channel.pd is a 4 input video player. The patch accepts an input from the inlet and using the select command triggers a video file loaded into channels 1-4. This is sent through gemhead and pix_film into pix_mix.

In order for video to be manipulated within a performance situation there are a number of parameters that can be assigned to controllers:

pix_gain – R G B values can be modified.
pix_threshold – threshold filter, pixels above a value are passed.
translateXYZ – allows a video image to be moved in three planes, up-down, left-right and back-front (zoom control).
rotateXYZ – allows a video image to be rotated along the X, Y and Z axis.

Experiments were made initially using 16 video inputs, correlated to the 16 audio files. This was found to very resource hungry for the test machine to handle, resulting in failed video playback. The video section of the sequencer patch , as well as the video patch were reduced to 4 channels to achieve a usable result. To match the audio elements to the video sequencer elements, sequence parts 1, 5, 9 and 13 were mapped to video parts 1, 2, 3 and 4 respectively.

avj16-drums.pd

av16-drums - 6 Channel Drum Mixer
av16-drums – 6 Channel Drum Mixer

avj16drums.pd is a 6 channel monophonic sample player designed for playing programmed sequences created within the avj16-seq sequencer patch. The patch uses loadbangs to populate 6 soundfiler locations.

Each sample channel has an individual volume fader, along with a pan control. Built into the master section is a tempo-synced multi-delay effect, along with a freeverb based master section, containing a dry and wet fader along with a room size control.

avj16-seq

av16seq - Pure Data Sequencer Patch
av16seq – Pure Data Sequencer Patch

av16seq.pd uses the metro function combined with a mod (16) function to create a counter that is combined with a 16-step select function. This data is then fed to vradio boxes. These vradio boxes are linked to channels within the avj16-drums patch.

a second select function takes steps 0, 4, 8 and 16 feed 4 vradio boxes of whose values are fed to the video4channel patch. In this way a 16 step sample based loop can be created and edited in real time, whilst also triggering video clips every 4 steps. These sample and video sequences can then be edited within their own patches.

To aid usage of the sequencer within a performance situation some extra functionality was built into the sequencer:

  • Number of Steps – This can be changed on the fly by using a hradio function to send data to the select (16) counter within the metro section. This enables rhythmically interesting performances to be created.
  • Preset tempo options –  preset tempo options were chosen, with evenly divisible values chosen to ensure the result remains aesthetically and musically pleasing. 10, 60, 90, 120, 180, 240, 320 and 480ms steps were chosen. 10ms steps, especially combined with a small number of steps, provides an interesting glitch effect within the sequencer.

All three of the Pure Data patches shown here are in the form used in conjunction with iPad control. A custom TouchOSC controller patch was used which aimed to bring together in one panel the functionality of all three elements of the instrument (sequencer, drum machine and video player). OSC data was routed through a network of send and receive commands as shown through the screenshot below:

avj16-master OSC routing to  iPad
avj16-master OSC routing to iPad

The following image is a screenshot of the interface created within TouchOSC. As you can see, the top dial control is assigned to video manipulation, with a step control below. At the bottom of the interface are video channel controls, alongside individual delay line faders and reverb level controls.

TouchOSC interface created for Pure Data AV16-4
TouchOSC interface created for Pure Data AV16-4