If you’ve ever fantasized about mixing with your hands similar to how you control a Theremin or in the movie Minority Report, your dreams are now closer to reality! With the latest K-Mix update, which enables sending MIDI message to fully control K-Mix, combined with the the K-Mix API and the incredible Leap Motion controller, I’ll demonstrate how you can control K-Mix with only your hands and fingers, without touching K-Mix at all!. K-Mix used with the K-Mix API brings a new meaning to the term ‘Programmable Mixer’
The latest K-Mix firmware / editor release, 1.1.0 brings a host of improvements in addition to enabling MIDI in allowing you to control nearly all K-Mix parameters via MIDI messages.
The project below allows you to control K-Mix’s faders 1-8 (plus the master fader), as well as stereo panning for each channel fader. We have a simple Ableton Live project setup with 8 looped tracks of the K-Mix ‘theme song’ by Emmett Corman with each track assigned to its own channel on K-Mix. You could also easily control K-Mix’s other features such as onboard DSP and use instruments / mics directly connected to K-Mix’s inputs.
- Left hand (fingers, excluding thumb): faders 1-4
- Right hand (fingers, excluding thumb): faders 5-8
- Fist: Master fader
Fader level is controlled in the vertical axis (moving your finger up/down), and stereo panning is controlled in the horizontal axis (moving your finger left/right).
Let’s get started!
Sending MIDI to K-Mix
There are 3 different send modes with K-Mix:
Sending to the ‘k-mix-audio-control’ port controls the K-Mix Mix Bank, which controls all of the audio features of K-Mix like changing fader levels, EQ, and spatialization in addition to sending preset change messages.
Inputs 1 – 8:
All of the per input channel [1-8] automatable control parameters are listed in section 4.3.3 of the Manual. The input channel is denoted by ‘:channel’, so for controlling input channel 3’s fader level, you would use ‘fader:3’.
In order to control the main fader, send to ‘main:fader’, which controls the main fader level.
To control K-Mix’s miscellaneous features (reverb/surround/auxes), covered in section 220.127.116.11 in the Manual, send to ‘misc:reverb-level’, which controls the K-Mix’s main reverb level.
To change a preset on K-Mix, just send a message to ‘preset’. K-Mix has 12 onboard presets.
// inputs 1-8
kmix.send('fader:1', 127, time) // with time
// control input 1 high-frequency EQ parameter
// Main Fader
kmix.send(‘main:mute’, 11) // auto channel 9
kmix.send(‘misc:reverb-level’, 100) // auto channel 10
// load preset #2
If you wish to control the K-Mix’s 3 MIDI Banks, covered in section 4.3.4 of the Manual, you send to the ‘k-mix-control-surface’ port, referred to in this API as ‘control’. The main purpose of sending to the ‘control’ port is for K-Mix LED control.
As seen below, the desired control must be prepended by ‘control:’. The optional ‘time’ and ‘bank’ [1-3] parameters follow, ‘bank’ defaults to 1. If you wish to send to a different bank, you must set the time argument to either 0 or millisecond value before the bank argument.
kmix.send('control:fader-3', 127 [, time, bank]) // control, value, time, bank
// if you wish to send to a different bank, you must set the time argument to either 0 or millisecond value
kmix.send('control:rotary-1', 64, time, 2) // set rotary-1 to 64 in bank 2
// in order to control buttons, a button must be in 'toggle' mode
kmix.send('control:button-vu', 127) // turn button-vu on; any value > 0
kmix.send('control:button-vu', 0) // turn button-vu off
raw MIDI messages
If you think in midi messages, you can send raw midi messages to K-Mix. You pass in the byte array with optional time and the port you want to send to, which defaults to ‘k-mix-audio-control’. If you want to send to the ‘k-mix-control-surface’, add ‘control’ after time.
kmix.send([176, 1, 127] [, time])
kmix.send('control', [176, 1, 127] [, time]) // send to 'k-mix-control-surface'
// you can also use hex values [0xB0,1,0x7F]
Note: all faders and rotaries are of type 176 + channel,
all buttons are of type 144 for on and 128 for off.
Getting data from Leap Motion was pretty easy, everything happens in the Leap loop as shown below:
// the magic happens here...
// Everything we want is in the returned 'hand' object.
Leap Motion allows you to get a staggering amount of detail and data from the movement of your hands, fingers, and joints, but in this project, we’re only interested in three things:
- 1. which hand we’re using
- 2. what finger we’re using
- 3. is our hand is closed to a fist
We can get figure out what hand we’re using by simply using ‘hand.type‘ inside our Leap loop. This will return either ‘left’ or ‘right’. To determine which fader our fingers control, i’m figuring out how many fingers are extended, excluding thumbs, and if we’re using our right hand adding 4 to that number, see line 10 below. Leap Motion also makes it really easy to figure out if your hand is closed to a fist or not by seeing if ‘hand.grabStrength < 1’, where 1 signifies a closed fist.
Once we know which and and finger we’re using, we can get the position data (X, Y, and Z) for that finger. There are a few ways to get position data but we’re going to use ‘stabilizedTipPosition’ as that is, as its name implies, more stabile.
Once we have our finger position data we will need to scale it to MIDI range (0-127) using a simple range conversion function ‘convertRange’, see lines 39 – 49 below.
Lastly, we need to send our position data to K-Mix using the K-Mix API.
kmix.send('fader:' + controlNumberFader, percentageY * 127)
kmix.send('pan-main:' + controlNumberFader, convertRange(horizontalPosition, [-350,350], [0,127]))
Here is the complete code for our Leap loop:
If you don’t have a K-Mix yet, you can watch the video below to see it in action:
You can also play around with the app in fullscreen here https://files.keithmcmillen.com/blog/webmidi/k-mix/leapmotion-mixer/
The source code for this project is available here: https://github.com/keithmcmilleninstruments/k-mix-api-leapmotion-mixer