Alchemical Musings

Since I am total flosstitute I do lots of my work on the beautiful OS X desktop, though the servers I administer are all linux, and on my new thinkpad laptop I finally bit the bullet and wiped the windows partition (it came with vista, so there wasn’t much deliberation). My only encounters with windows nowadays are through virtualization, so I feel like I have that demon safely caged.

One of the things I love about the mac are the little easter eggs you can find if you hunt around long enough (or more likely accidentally stumble upon).

One of these black-ops is the music visualization software that comes with iTunes (at least on OS X). I seem to recall something about a Christian fundamentalist writing it originally, right before joining the navy and serving on a submarine crew. Thing is, he couldn’t get this piece of software out of his head, and winded up leaving the military to work on this software full time. I think Madonna used to use early prototypes at her private parties, and one way or another he started working at Apple, apparently on the iTunes team. (this is all from memory, and I couldn’t find a source, in case anyone has heard this story also).

In any case, I occasionally remember to check in on this tool, and it’s gotten better with ever release of OS X. I think last year I discovered that if you run it in full screen mode it seems to use a much improved rendering engine, and maybe even a different algorithm.

None of this prepared me for the experience that I had Tuesday night. A few months back I learned about a wicked cool piece of software on Alexander Limi (the Plone founder’s) blog. The software is called nocturne, and is pretty friggin cool on its own. It’s not much more than a simple set of macros that invert the hues of your display – to either black and white, inverted color hues, or even submarine red. It’s really nice if you want to use your computer at the end of the day, but don’t want to deal with all the energy of a full backlight.

So anyway, I had this kooky idea (no drugs involved!) to turn on the iTunes music visualizer with nocturne in night mode, and I simply could not believe my senses. I was witnessing the audioloom – an idea I had begun to think about a few years back that originated with the simple question – can synesthesia be learned? I became very interested in the natural relationships between color and sound, noticing that both seem to come in octaves (think of the color wheel – a venn diagram defining 3 singles, 3 doubles, 1 triple, and the background, making 7+1… just like the western musical scale!).

I even remember what sparked this question. I was playing with a new set of Christmas lights, the kind with a remote control that makes the lights dance in different patterns. The important part of this experiment was leaving the lights ordered neatly in the box, instead of making a tangled mess. With this arrangement, when I played music, I could swear that the photons were dancing to the beat 😉

In any case, I was intrigued by the possibility that there might be a fundamental ontological relationship between sound and color, but even with this foray into metaphysics, I thought there might be a natural mapping between these two types of sense data, one that might be empirically determinable.

I did some research on synesthesia, and read a great book called The Man Who Tasted Shapes. My idea began to take shape as a multi-phase project. Phase I was this screensaver on steroids, but Phase II is a musical instrument that plays light instead of sound. As with all fun ideas, there is nothing new under the sun, and many philosophers/inventors ranging from Aristotle to Newton to Benjamin Franklin have taken a crack at this problem (timeline), but the idea was ahead of its time… Until now.

So, back to Nocturne’s night mode. When I went full screen with non-monotone inverted hues, I swear to god it felt like I was entering a wormhole. Right out of that scene in Carl Sagan’s Contact, except without the extraneous seat that the stupid humans built.

I was transfixed, and will freely admit that on this first trip I spent a solid 2 hours staring at the screen and listening to my favorite tunes. Every time a song would end, I would wonder what another of my favorites would look like. I think the difference between day mode and night mode is that the visualizer outputs mostly dark. By inverting the hues, the screen explodes with backlit energy. Enough to keep your eyes working overtime. It was kinda like watching TV, except that instead of being hypnotizing, it was mesmerizing. I mean, I was grooving on my favorite music, but my eyes weren’t jealous of my ears – everyone had their work cut out for them.

Unlike TV, the audioloom experience requires active processing, as your brain frantically struggles to find patters in the sequences and segues. Since I don’t think the shapes and transitions are computed deterministically, there is an element of Art combined with the engineering mathematics displayed on the screen.

It made me wonder if this feeling would normally have required 10 years of devoted study in an ashram to replicate before this technology came along. One way or another, the experience was transcendental, and I just hope I haven’t stumbled upon the Videodrome, or the mysterious plot device in Infinite Jest…

In any case, I plan to continue my experiments and keep you posted with updates. It is quite a relief that I might not actually need to implement this invention one day. Just goes to show, ideas kept secret, go stale.

Filed by jonah at 6:52 am under aesthetics,air
No Comments