Final Write-up
I began the semester with two of the three main components of my system in some form and able to communicate. The first component, which receives audio and translates it into human-readable symbols, was in a sloppy but useable form. The second component, which uses these symbols as input and output, essentially did nothing. It existed only to test communication with the first component. The third component, which sonifies incoming symbolic data, didn’t exist yet and is still in a rudimentary form. It currently uses a built-in object in Max that uses one of the basic MIDI instruments in Mac OS X. Therefore, I’m not using any of the timbre data yet.
The first major task was developing a protocol for the components to communicate to each other. I knew I would build on the Open Sound Control (OSC) protocol, but not what kind of data I would be working with. I decided that the protocol should only send data relating to individual notes. Therefore, I scrapped all parts of my Max patch that were detecting higher level information, such as the number of notes over a fixed period of time (density). I decided that such work would be done in Python.
After developing the protocol, I had to decide what kind of analysis I would be doing and how I would structure it as a Python program. I decided early on to base my analysis on “motif” detection. In the current version of the program, this process is in the form of searching for the longest repeating pattern from a collection of notes. Motifs based on pitch and rhythm are detected separately. I had a lot of trouble finding a useable structure for the program. Initially, functions were called from inside each other in a fragile spaghetti code mess. After briefly exploring multiprocessing in Python, I stuck with the pythonosc library I had been using for inter-component (Max/Python) communication. I appended a few bits of code to my Max patch that make calls to several of my Python functions at regular intervals. These functions are for detecting motifs, generating random motifs, generating new motifs from permutations of the motifs already stored, queueing stored motifs to be sent to Max to be synthesized, and checking if the next note should be sent to Max. I feared that this approach might be too slow, but it works perfectly.
The permutate_motif function calls one of several functions that alter a motif in a way that should be noticeable when heard. I originally based them off the permutations I know from music set theory: retrogradation, inversion, and transposition. I decided that I didn’t find inversion recognizable enough, so I replaced it with two other functions: stretching/shrinking duration and adding a note to a motif (a “flourish”). The add_flourish function required the most effort of the four, because I had to consider how every one of the new note’s parameters should work with its neighbors. Jarring shifts in volume or key wouldn’t be ideal. I used key detection to determine what kind of pitches would be acceptable and the neighboring notes’ average velocity as simple solutions to these problems. More sophisticated approaches will likely appear later.
My use of the music21 library for computational musicology evolved over the course of the semester. Initially, all the data I stored was in the form of music21 note objects. However, once I began to incorporate timbre into my note data and realized that music21 isn’t designed for any kind of audio analysis, I decided to only use the library for specific tasks, such as detecting a motif’s key.
My final work will involve developing the third component and making small edits to the second component’s code.
