If you’ve ever fantasized about mixing with your hands similar to how you control a Theremin or in the movie Minority Report, your dreams are now closer to reality! With the latest K-Mix update, which enables sending MIDI message to fully control K-Mix, combined with the the K-Mix API and the incredible Leap Motion controller, I’ll demonstrate how you can control K-Mix with only your hands and fingers, without touching K-Mix at all!. K-Mix used with the K-Mix API brings a new meaning to the term ‘Programmable Mixer’
The latest K-Mix firmware / editor release, 1.1.0 brings a host of improvements in addition to enabling MIDI in allowing you to control nearly all K-Mix parameters via MIDI messages.
The project below allows you to control K-Mix’s faders 1-8 (plus the master fader), as well as stereo panning for each channel fader. We have a simple Ableton Live project setup with 8 looped tracks of the K-Mix ‘theme song’ by Emmett Corman with each track assigned to its own channel on K-Mix. You could also easily control K-Mix’s other features such as onboard DSP and use instruments / mics directly connected to K-Mix’s inputs.
Controls:
- Left hand (fingers, excluding thumb): faders 1-4
- Right hand (fingers, excluding thumb): faders 5-8
- Fist: Master fader
Fader level is controlled in the vertical axis (moving your finger up/down), and stereo panning is controlled in the horizontal axis (moving your finger left/right).
*If you haven’t already, please read my post, K-Mix API, Part 1 – Web MIDI Control Surface before jumping in to this post. Also, if you’re new to the Web MIDI API, please read my introductory overview of Web MIDI first Making Music in the Browser – Web MIDI API. Knowledge of JavaScript and using NPM is highly recommended. Currently, the only browsers that support Web MIDI are Chrome and Opera and since K-Mix is Mac only, please use Chrome/Opera on a Mac while using K-Mix with this demo.
Let’s get started!
Sending MIDI to K-Mix
There are 3 different send modes with K-Mix:
k-mix-audio-control; default
Sending to the ‘k-mix-audio-control’ port controls the K-Mix Mix Bank, which controls all of the audio features of K-Mix like changing fader levels, EQ, and spatialization in addition to sending preset change messages.
Inputs 1 – 8:
All of the per input channel [1-8] automatable control parameters are listed in section 4.3.3 of the Manual. The input channel is denoted by ‘:channel’, so for controlling input channel 3’s fader level, you would use ‘fader:3’.
Main/Master Fader:
In order to control the main fader, send to ‘main:fader’, which controls the main fader level.
Misc (reverb/surround/auxes):
To control K-Mix’s miscellaneous features (reverb/surround/auxes), covered in section 4.3.3.4 in the Manual, send to ‘misc:reverb-level’, which controls the K-Mix’s main reverb level.
Presets:
To change a preset on K-Mix, just send a message to ‘preset’. K-Mix has 12 onboard presets.
// inputs 1-8
kmix.send('fader:1', 127)
kmix.send('fader:1', 127, time) // with time
// control input 1 high-frequency EQ parameter
kmix.send('eq-high-frequency:1', 127)
// Main Fader
kmix.send(‘main:mute’, 11) // auto channel 9
// Misc
kmix.send(‘misc:reverb-level’, 100) // auto channel 10
// load preset #2
kmix.send(‘preset’, 2)
k-mix-control-surface
If you wish to control the K-Mix’s 3 MIDI Banks, covered in section 4.3.4 of the Manual, you send to the ‘k-mix-control-surface’ port, referred to in this API as ‘control’. The main purpose of sending to the ‘control’ port is for K-Mix LED control.
As seen below, the desired control must be prepended by ‘control:’. The optional ‘time’ and ‘bank’ [1-3] parameters follow, ‘bank’ defaults to 1. If you wish to send to a different bank, you must set the time argument to either 0 or millisecond value before the bank argument.
kmix.send('control:fader-3', 127 [, time, bank]) // control, value, time, bank
// if you wish to send to a different bank, you must set the time argument to either 0 or millisecond value
kmix.send('control:rotary-1', 64, time, 2) // set rotary-1 to 64 in bank 2
// in order to control buttons, a button must be in 'toggle' mode
kmix.send('control:button-vu', 127) // turn button-vu on; any value > 0
kmix.send('control:button-vu', 0) // turn button-vu off
raw MIDI messages
If you think in midi messages, you can send raw midi messages to K-Mix. You pass in the byte array with optional time and the port you want to send to, which defaults to ‘k-mix-audio-control’. If you want to send to the ‘k-mix-control-surface’, add ‘control’ after time.
kmix.send([176, 1, 127] [, time])
kmix.send('control', [176, 1, 127] [, time]) // send to 'k-mix-control-surface'
// you can also use hex values [0xB0,1,0x7F]
/*
Note: all faders and rotaries are of type 176 + channel,
all buttons are of type 144 for on and 128 for off.
*/
Leap Motion
You can use a Leap Motion controller with various programming languages and platforms, but since we’re working with the browser and the Web MIDI API, we want to use the JavaScript SDK, which is nicely documented here.
Getting data from Leap Motion was pretty easy, everything happens in the Leap loop as shown below:
Leap.loop({
hand: function(hand){
// the magic happens here...
// Everything we want is in the returned 'hand' object.
}
})
Leap Motion allows you to get a staggering amount of detail and data from the movement of your hands, fingers, and joints, but in this project, we’re only interested in three things:
- 1. which hand we’re using
- 2. what finger we’re using
- 3. is our hand is closed to a fist
We can get figure out what hand we’re using by simply using ‘hand.type‘ inside our Leap loop. This will return either ‘left’ or ‘right’. To determine which fader our fingers control, i’m figuring out how many fingers are extended, excluding thumbs, and if we’re using our right hand adding 4 to that number, see line 10 below. Leap Motion also makes it really easy to figure out if your hand is closed to a fist or not by seeing if ‘hand.grabStrength < 1’, where 1 signifies a closed fist.
Once we know which and and finger we’re using, we can get the position data (X, Y, and Z) for that finger. There are a few ways to get position data but we’re going to use ‘stabilizedTipPosition’ as that is, as its name implies, more stabile.
Once we have our finger position data we will need to scale it to MIDI range (0-127) using a simple range conversion function ‘convertRange’, see lines 39 – 49 below.
Lastly, we need to send our position data to K-Mix using the K-Mix API.
//kmix.send('fader:1', 127)
kmix.send('fader:' + controlNumberFader, percentageY * 127)
//kmix.send('pan-main:1', 64)
kmix.send('pan-main:' + controlNumberFader, convertRange(horizontalPosition, [-350,350], [0,127]))
Here is the complete code for our Leap loop:
If you don’t have a K-Mix yet, you can watch the video below to see it in action:
If you’re lucky enough to have gotten your hands on a K-Mix and have a Leap Motion controller, plug them in and enjoy touchless mixing!
Get your K-Mix here and your Leap Motion controller here.
You can also play around with the app in fullscreen here https://files.keithmcmillen.com/blog/webmidi/k-mix/leapmotion-mixer/
The source code for this project is available here: https://github.com/keithmcmilleninstruments/k-mix-api-leapmotion-mixer