Machine Learning for Musicians and Artists

07/21/2016
Machine Learning Pipeline
Input sensors: game controller, webcam, GPS, Twitter data, MPC
Processing/Decision-making (Model) computer decides what action
to take based on input
o Without ML, youd spend a lot of time tweaking this
processing
o Sometimes code is impossible to write if input is too complex,
noisy, etc
Output sound synthesis code, game code, animation, etc
Supervised learning algorithms tool using algorithm to build middle
component (model)
Input

Training
Data

Learning
Algorithm

Model

Output
Web-Controlled Drum Machine
Input - Processing: extracts data from webcame
Model Wekinator sends info to Chuck
Output Chuck: drum machine
Training data set give Wekinator training examples of standing in
front of webcam, not standing in front of webcam, etc, tell it what
output to give for each example
Connecting inputs and outputs with Open Sound Control
Open sound control connects inputs and outputs to the model
Use OSC libraries for whatever platform/software I want to use
OSC Communication
Application 1 (sender) sends OSC message and data to Wekinator
(model) which sends OSC message to output (receiver)
When sending an OSC message, need to tell operating system
where to send osc message
o What port do you expect the other program to be listening for
an input
o Port temporary virtual location on computer like a
temporary mailbox

o If expending to receive a message (output), need to tell

operating system what port we are expecting to receive a
message (which port we want to listen on)
o Think of this like the address on a letter
o By default, wekinator uses Port 6448
o Only one program can be listening on a port at a time only
one output program running at a time
If youre not getting data where you should be not
getting output, make sure theres only 1 program using
a port
Sending an OSC message to another computer requires an IP
address or host name
Summary
o Has a message name (e.g., /playNote)
May have one or more pieces of data attached to it
o Is sent to a host (computer) and port (specific address at
that computer, e.g. 6448)
If receiver is on the same computer, host name is
localhost
Dont forget
o Receiving application has to start listening on a port
o Only one program can listen on a port at once
o Sender and receiver have to use same OSC message name
Use of / characters must match
o Sender and receiver also need to agree on number of data
values in message