You are on page 1of 8

Some time ago, Joe Milazzo did a little piece on Eigenradio, a MIT music broadcasting project to which Bagatellen

has long provided a link. After reading Joe’s profile, I checked out the site and was hooked immediately, spacing out to its mysteriously concocted, gnarly mish-mashes at least weekly. So, I was naturally very sorry when Eigenradio went off line recently. Wanting to share my grief with the family, I hunted down Brian Whitman, the “music analyst/synthesist” responsible for both the theoretical science behind and the implementation of this computerprocessed radio music. After condoling, I asked Brian a few questions, not only about that site, but about his researches in the area of the musical essences and preferences generally. (A number of his papers are available on line through his website.) Brian’s responses indicate the depth to which he has thought about matters that Bagatellen contributors like me are quite content to muse out loud about at length using as few brain cells to do so as is humanly possible. I here reproduce our e-confab: When I listened to Eigenradio, I was struck by the enveloping Ivesian thing, you know, watching three marching bands approach from different directions, hearing nine practice rooms at once from the hall of a music building, that sort of thing. But I take it that while the Ivesian complexity is there, Eigenradio wasn't simply a matter of pouring fifteen broadcasts into one sonic soup tureen: there was some careful reduction of broth going on there. Can you describe this process for the layman? Was some computer literally sucking in a bunch of radio broadcasts and spewing out Eigenradio tunes 24-7? How long did the transformations take? Were commercials and/or news filtered out? The webcast of Eigenradio did involve a computer (well, eight of them) sucking in a bunch of broadcasts and spewing back Eigenradio. In between the radio recording and synthesis a lot of stuff goes on: the stream is first segmented into songs (attempts to separate speech and music are made, but I never explicitly check), and then after a set

It was broadcast on BBC Radio on Christmas Eve. given some range of filled-in distinctive properties. I counted roughly . a computer (eigenanalysis) hates it and 'optimizes it away'-. and the reason I think it was so successful was that it was a cheap way to hear music from a completely different perspective-. Computers are pretty good at this. grass. Fundamentally. As a result the Christmas record is more 'listenable' but shares the same mathematical lineage as the radio stream. buildings. and also b) a "synthesizer" that. what are you. really?" you'd hear Eigenradio singing back at you.of songs are recorded we queue them up into batches. Instead of pixels from pictures. if you took a bunch of music and asked it. can generate an approximation of a sparrow. given constraints. An example is when you don't know what a sparrow is--so you go to the library and you open a bird book and see three hundred sparrow pictures. different domain. We had a bank of eight computers doing the work for us. For music. You can synthesize sparrows endlessly as you know what makes a sparrow a sparrow and also how sparrows differ among themselves. Normally I try to take sixty minutes of music down to about three or four minutes. Sparrows have certain types of feathers. The serious processing happens in this reduction.why would it want to store two copies of the same thing? "A Singular Christmas" was very similar except a) we used only Christmas music for the input. we like periodicity and repetition in our music. but again this is sixteen processors worth of work. and the computer resynthesizes a smaller piece by mixing these dominant components from different pieces to create a new whole. "Music. not general radio streams. same thing. After listening to 60 minutes of audio the eigenanalysis starts: some dominant components are identified. their beaks angle a certain way. Note that it's pretty different than what we would think. What you're doing is looking at a large set of perceptual input and distilling it down from hundreds of pictures to a small set of unary "distinctive properties. and I'd guess that it took about ten minutes for every hour of Eigenradio to render the result. far more successful than Eigenradio in terms of listenership and press / publicity." Once you find a small set of these properties. It's the computer's idea of what music really is. and b) instead of synthesizing the dominant components directly we found the closest matching acoustic ("real") sound from a large database of instrument samples. and I mentioned this on the original website. you can use it as a) a "classifier" in which you can tell if something is a sparrow or not. You start noticing things they share and things that differentiate them from other birds and from trees. Not too intense. the computer is looking at features derived from the audio signal. “A Singular Christmas” was far.for example.

The problem is that when you resynthesize you need to find a path in a random field that a) reflects the range of components your analysis chose and also b) considers the short time and long scale expectations of the human auditory system. So I can take the dominant component from a group of music and instead of synthesizing it rubberbass style. mean power density in 80-90Hz etc. periodicity. etc.) banjo sounds. and we were left with the components and characteristics. If you compressed a song as a ZIP file.) and the dynamic ranges of each component that defines the "song space. it was fundamentally supposed to represent what computers found beautiful about music by finding what's known as the 'minimum description' of the perception." But I don't understand the methodological difference entirely. The "real sounds" come from this database which I maintain for a bunch of projects: internally it's called "all Possible Sounds" and is comprised of about 200 gigabytes of audio samples consisting of instrument and effect sounds.g. . Is it that there weren't so many pitchless (or 'grungy'?) sounds in the database you pulled the instrument samples from as might be heard on a regular radio broadcast? How did the program decide which samples to pull from this database? After you find the dominant components of a song (let's say they were tempo." your task is then to generate the songs with the widest range of characterization--the songs that should represent all other possible songs through some combination. which seem clearly "noisier.000 listens in the space of two weeks. it'd be garbage to you even though all the information is still there. But if you tried listening to the zip file. We took a bunch of Christmas songs. timbre and tone.600. did the Eigenradio "component finding" algorithm on them. the computer could 'listen' to the compressed version and make the same judgments about content as the original file. It's all in a database with an acoustic similarity back end. I can ask the database to find the best N samples that match it in time. But instead of just quickly spinning through the possibilities at some fixed rate (the main culprit in Eigenradio's drill-in-a-tunnel acoustic aesthetic) after we determined the components we did a search through "real sounds" to find the ones that best matched the chosen component. Eigenradio made little to no allowances for this effect. for example. "A Singular Christmas" was the same as Eigenradio up to the point of resynthesis. Other things we can do with this database include resynthesizing a single song just by using the components of all (e. This is a nice and easy arbitration between computer and human demands. I'm not surprised to hear you say that there was some difference in the manner of putting "A Singular Christmas" together: it seems like something that modern classical listeners might be more comfortable with than most randomly chosen Eigenradio broadcasts. spectral entropy. MIT ended up turning off access to the files for awhile.

The automatic record review project was a study to see if there was "learnable" language in record reviews. how will they respond? Objective would imply that there's only one answer. I've occasionally opined that a simple ranking system (say of 1-4 stars) combined with a historical database of recording names followed by stars for each reviewer would be much preferable to the linguistic comparisons with various animals and food and the obligatory gushing that constitute ordinary music reviews. what use is it to me to know that Jones likes or hates recording X if I have no idea whether I generally like the same sort of stuff Jones likes? I suppose there is some entertainment value both to having pieces compared with various colors and in the one-upsmanship of the hunt for the most fervent rave or nasty pan ever." thereby producing wildly different music? Does something important follow from that? What could one (possibly) infer about a group which always preferred Eigen I to Eigen II or vice versa? There is a direct link between Eigenradio and music retrieval work in that the analysis used to find the 'essence' of music is almost exactly the same up to the point of resynthesis. Doing preference studies on synthesized Eigenmusic is a level of indirection that I am not comfortable taking right now!! In your research there seems to be a hunt for objective measures of taste similarity. From a music retrieval scientist's standpoint.What if any connections do you see between your written research on musical tastes and Eigenradio-type extractions of what might be called "musical essences"? Can you imagine an Eigenradio II that extracts very different "essences. Before you try to get a machine to learn something you want to eliminate redundancy and find covariance among variables in the perceptual feature space. let me disclaim that I am not looking for 'objective' measures of similarity but 'predictive:' given a community and a piece of audio. I mean. we can start to understand what 'slow and plodding' means. But of course that's hardly ever the case-how many adjectives can you name that are objectively informative? Review language is usually far removed from the signal it refers to. Is that the sort of thing you're discussing in your research on taste and "automatic reviews"? Well first. it would seem that a knowledge of reviewer preferences and a thumbs up or down would be more helpful. If a reviewer calls two records "slow and plodding" and there's something in the actual content of the records that matches. So for that project we took the chatty gorilla of pop writing—“Pitchfork”--and pitted it against the boring reliable standby. Eigenradio is what the computer hears when it tells you that the Kinks and Sugarplastic share some traits. but from a consumer advice point of view.” Obviously AMG trounces “Pitchfork” in both correlation . “All Music Guide.

My hate on these systems is purely high- . Go to Amazon and you'll see an entire industry devoted to convincing you to buy something because some stranger you agree with also buys this something. really. if Amazon catalogued more than just sales. I’m not sure I understand why you believe your manner of predicting someone’s likelihood of enjoyment is more reliable or otherwise preferable to Amazon-type "A & B both like it" schemas--so long as the latter utilize sufficient data.e. instead of moving from a single joint purchase to a recommendation as it now does. Isn't there any hope that the Amazon model. my #1 enemy. they use language that can easily describe music) and correlation of "star rating" to audio (there is some underlying feature of the music that contributes to the rating. If the automatic listening systems light up with similarity matches. Economies of scale dictate that the more sales or reviews of something. plain and simple. The way to escape is to be smart about it: have something that has nothing better to do (a computer) do some first order predictions on preference based on content and contextual analysis. no matter how inconsistent my algorithms say they are. could eventually make suggestions on the basis of huge preference correlations and start reaching excellent success levels? Sure. With that model. so a feedback loop occurs which is more dangerous to this industry than Bittorrent and Soulseek put together. you can't predict a response to completely new data. Treat all of these new ways of discovering "hidden connections" as just another source of information to make filtering decisions with. we should pay attention.of words to audio (i. even if there's no audio yet available. So who's more worthless--the trendreflecting wavering music criticism site that helps people find new music or the scientist that proves that the site is trend-reflecting and wavering? The recommendation from review scheme you describe is collaborative filtering. and I can't say I would trust self-selected music writers to lead the marketing statistics just yet. Critics don't review every CDR and promo package that crosses their desk: selection is biased towards already popular or familiar music. The fundamental problem of collaborative filtering approaches is the popularity effect: for something to get noticed people have to notice it first. But I was responding to the underlying problem: you're predicting preferences by studying data that already exists. we should pay attention to it. the more likely it'll be recommended and with higher accuracy. “Pitchfork” and others are helping consumers find music.) But those things have little to do with how well the reviews help people find music. You're clustering random consumers into rock critics. If all of a sudden ILM lights up with talk about a new artist. That is public enemy #1 for niche/independent music to take a hold. of course they'll do better.

it affects people whether it should or not. think M. What the hell is going on here? Is there any value at all to that sort of colloquy? Can/should it make anybody change his/her mind? Does it provide anything that a thumbs up/thumbs down list doesn't? In the real world. etc. You also know the station and the time of day." When Y asks him for his reasons for this judgment. and we're getting there. Jason Falkner left Jellyfish in the middle of "She Still Loves Him. you make an assumption about the artist simply because they have music on the radio. maybe the song that came after it. really.level. But going down that path will just make popular records get recommended with some higher amount of success. . there are two types of context here: the type the artist tried to enforce on you (their "story.I. that each spells "Bach" repeatedly in some language or other. marketing." their marketing/background. This stuff is crucial. we have entire industries of marketing and radio assuring that will happen for all eternity. here). and yes. Finally Y wonders aloud whether X is sufficiently familiar with Anthony Braxton.. The way I see it. and the type the listener applied to the music (girl trouble. X responds "No reason. but how do you really automatically detect that the song "Shipbuilding" is about the Falklands war? Or that Oval made a custom CD player. that 100 hungry Chilean Indians are fed by the proceeds to his disc. and whether any such alleged similarity matters anyhow): Suppose someone X reports that he finds a certain recording of Y's music "really bad. I just found it boring. does anyone ever hear music with absolutely no context? Even if you hear something on the radio by accident. monitor chat rooms. check music news sites every day. I am sure there are tons of things that can be done to aid CF to do better. definitely. We don't need that. Gregorian Chant. Trends. The annoying bit for people who want to help people find music is that it's very hard to predict either type of context with any accuracy." and they kept his last solo on the tape. mood). There are many startups attempting just that and I wish them luck. Why don't we take advantage of the technology to help level the playing field for new artists instead of just using it as a cheap advertising tool? Consider the following hypothetical (based loosely on a recent argument between a Bagatellen reviewer and a saxophonist over whether a recording made by the latter sounds anything like Mats Gustaffson. I build systems that read music blogs. etc. hype.A. and that. general preference. like using text and finer grained preferences and so on." Then Y responds by asking if X realized when listening that each tune on the disk is based on a Fibonacci series. Uncompelling. Nurse With Wound and Onkyo to comment authoritatively on his work. It goes on forever. if listened to through specially filtered headphones the music therein will provide a cure to both insomnia and colon cancer.

is Eigenradio-style re-parsing more like re-arranging molecules than measures--even if whole words or phrases may be sometimes decipherable? We were given a free rein at the lab. But what pisses off copyright holders is if you make something that bites on the "meaning" of the song: its intended effect and audience reaction. celebrity. alone in a field in Arkansas. quantize them at the rate of rainfall in your hometown etc and you could still be stealing the original piece: you always start somewhere.there's never a right answer in these things. Obviously the most sought after form of meaning is popularity and sales and that's where the lawsuits come in. there's somewhat of a 'don't ask don't tell' policy within reason. Did MIT ever worry about the legal ramifications of Eigen-style analysis and resynthesis when Eigenradio was being proposed? Was there always complete confidence that these re-broadcasts were within the "fair use" exceptions to copyright protections? From your own (non-lawyer) perspective. Sadly. The point is that you can rip apart music into sine waves at parameterized phase and magnitude using a process that most computers are intensively optimized for (the FFT) and then you can put those sine waves in a blender. relationships. wire them to a fluorescent light. My favorite concordant example is the James Newton/Beastie Boys "Choir / Pass the Mic" case. No one can seriously say that they could like a song when devoid of all external forces. To some listeners it's more important. bio. nothing bad at all happened.." according to the deposition) and his style of playing ("flute overblow harmonics. where Newton sued Mario C et al for lifting not the sounds or signal of "Choir" (because they paid for and licensed that) but the meaning of the sound under it ("The four black women singing in a church in rural Arkansas. I love the Newton case because you could tell he was so angry that the Beastie Boys took his story and screwed into it a forgettable overdriven anthem for a Jeep Wrangler in the school . Newton wanted to own it. I was secretly hoping that I'd get in trouble-. collaborations are all just as important as the bits flying off the plastic.") Newton had a gorgeous unintentional non-sequitur in a letter relating to the case: "There is a spectrograph that moves wildly when my multiphonics are played" as if this singular spectrograph. It is nice to think about legality and fair use when music experiments are involved. but in reality it's only something you can guess at until you get sued-.stories. It wouldn't be music if they could. carried the secret of new forms of musical expression..I would have gladly put off my dissertation a semester just to sit in a courtroom playing Eigenradio samples at the jury with the RIAA across the table.

So since 2002 or so I've been concentrating on this "process music" instead. We're safe.. the company is 'future music' oriented and it should be a very interesting next few months... I soon got sick of watching people (and myself) behind laptop screens doing not much of anything.D. which is great as it's impossible to play live.. Are there connections here between either your research on musical taste or Eigenradio or both? I performed as "Blitter" under my brother Keith (Hrvatski)'s tutelage for a short period in 98-2001.parking lot. I understand you're a performer as well as a research scientist. Besides Eigenradio + Singular Christmas I hope to have a full length soon on some vanity label documenting my experiments in this field.. it's changing out from under me.. Walter Horn . The processes and ideas behind Eigenradio are certainly continuing. we completely remove the story and don't replace it or reappropriate it. Tell me about the sort of music you enjoy making/listening to. from MIT in May 2005 or so and have started a company (The Echo Nest Corporation) with fellow music analysis/synthesis scientist Tristan Jehan. Needless to say. there will absolutely be more Eigenradio-related projects in the near future. What are you up to now? Is there any chance that Eigenradio will go back on the air? I received my Ph. Besides the aforementioned full length. Now that I have lawyers and investors I can't talk so loudly. You mentioned to me that you're no longer connected with MIT. in Eigenradio's case. The problem is that every time I buckle down to record something I find a better sounding way to do it: "Never make it. Stay tuned." The forever lost title of this record is "Sofia Safari" so when you see this on a shelf you know I've stopped trying to fix things.

99024708..:7.9.. .0/..0841842093 90247009 -0 70.9043908088902885:70 .0.909.084770.3/0503/0392:8..4/ .434208 418.949.3/907.42203/0/.

00/8948/8.39 :78094:3/.07  3.0/ 03890339.9013/8.42503 %03708543/8-.9.443./0-90.0474907 9.9.:9479..9.0.:7094-4938423.79 2.80/448043..9.9:892.70.0.0039703/:89708412.9 :370.3/13077.0545:./4.80/43.88:739.900370..4:/090788:11.:0.79:58.9:304390/88-.24:39418:.70.3/8902:..8.0391..310/ 14730.3/8443 %070.34  $:55480842043070547989.882147870.943  7047.0.9902593 :899.9907 84:3/8./0.0438 47  ..2./54308902:8./94/4-09907 0 :83909.949.9 189030/94974:850. 8508.42203/0/9842007. 090739 /43 909..38 ./.088 0/43 9300/9..4543894.7010/-90574.3.99078.70.3-0/43094.843814798:/2039 708543/8470.11843 .0905.370.446: ..422039.38:. :943/439.98:89.9.00/82.798983890.3.3.33/. .43/078.843  70.-43.702./.8.3/.9 0.0419090. -.07983944 438/0790144354909.079.7 70...99008434307089070.3.3/0907.83170.3/9.3930.389.7050.039.47/341 82:8.7939437.90703574.39..5503147. .90.19070/ 0.90.34494050.3/ 3494../41:89:839.3/..7093..7:2039-09003.28:709070.070907.9847941.007.5..08.80708 9.3/7.90/38420.. -.95.3:./ 03..30/57010703.47/32.47/80970.0.. :8914:3/9-473 &3.709438419389.

0.84:/92..3-4/.308.

9.3939./0. 9:2-8:5.0723/4089574.

7093./4-.0.72:8.9.7094950841 ./4 4:.20.47/ /408.-4:990.43907.43909.3/90920 41/..-0908439.-84:9034.0434: 9078947 907 2. 2....9.7989 825-0./039 4:2.7989970/9403147.19079 %0.070.02:8.88:25943.3.8009 9070.:8090.943.031 4:0.784209343907.439090709095090..34300.9:2-8/4389/4083 9 39070.84349089.

439099 ../0.3. /090..:89425.07/.7093  ..9.843. .99 8.974428 .50 09.57010703. 30889080. 09. 7974:-0 0307.2:8.3343-9147504504. .  ./2:8.082  .3070190183902//041$0$94. 9408431470..970..07 %889:118.9 .-4:990.990843$5-:/38.07 .:7.8984443909..98504500907984:/47349 %703/8 50 2.3/9095090890307.2.7/94570/.-48 243947.-.99.3/900598.07.0.:942.39 94055045013/2:8.3/08 /01390 9. -:/8890289.7 79.89.550/9490 2:8.3/0 7009939070 -:94/44:70..0 244/ %0.7:.74:3/ 93   070 ..3/8.110..9090795041..

783 2470070 .08 94:/3 9-02:8.2508..7.21.943858 .38074:88.9 90:7990#.431/03.0.894708 -4 .47/. 9370.94384103 890.88.9..9.7:80032:8.-0 0070./.190.9 /093974:-0 4:/.99 8438420934:.07477.89070..890-98 13411905.8/43 990 54.:89.3.7488909.89807093901.425090 .4.8-035745480/.990.8:708 0.43./48.-4:90.380739080938  1./5:9 4112/88079.825479.07 507850.7:800.79 .320.0 9493.990.33240..9070 88420./ 3493-.89.94387424:743 343 ..4790.:089.390.05943894.07..3:088.00-79 70.941.050720398.-0 $..09073.8.4.77. %4842089030789 8247025479..39..4:/  /%0.880..70.9.55030/ 983.843 ./4.3/70 83908803037.4/41.703.4:7974425.0/ -:93 70.0314047/84757.250890.1700703.4579 57490./.39   4430.03.0 8037.4:/0.9:394:098:0/ 9070 830. -084209208/0.7094539.84303/0.09.2080943.9908070 -74.3037.8082.-4:9900./4 89070 5. /43 9.80208907:8994893.507.3/1..-47.9. 147.943.9438.

0.8904847.

59.705.574....:7.1:4708.943  -.41 47 -0.9-908439020.939490.42038333.3/9034:.7409.9900.07-4.35:99480830.37:7.375.190714724120.3/ 2.924.3/.8890 .958808411.94:.80 07009438:0/.7.47/39490 /0548943 .147099.1471933499084:3/84783.:804:.3/8.0/.070.38..798420070 :9.03.08/0322:9543.7243.07/7.04:.4203 4...334190843983903/0/ 0110.709413014728412:8.38.9.80 -0.4303.31.0057.4:/89-0 890.79..390902. 05708843 0943.. 4704:8:33903943.9 807090.3/ 8.045920/147 90 % .9747.34:742094309.44 .3/8890415.209070/5.:/03.039 9 6:.!.343 806:9:73.4:/900.338 545:.379./.7 850.7039038.4:890248984:9..95..425:9078.394830..8 .08.8 ..0900943.9747.3902147.083.90417.700/3949.08420939.80%0708..9907..3073908.08.884..819883:.39:/0:83..33419084:3/:3/07 9 %014:7-.50.10/37.3/.-03/07 7090294.9 -:99020.0889.7.8:98./147.3 1:904.0990770.5.. 850..92489.770/9080.390/94439  %0543989.792:8..0380/9.8904894488947.:80905.80.889.390473.3/9.8  0943.-04.4579 4/078814:2.3/4:.5 .8.

7080.3/4: 70.5.80 0.3.-4:9908479 412:8.0908947.09 4770.3//43 9705.7.557457.8.7349 3037..507147207.10  :3/0789./4 8.8.4:0342.03989 %020.4250907024..80.909 0 708.

845094.47/8$41.90.4330.77892.432:8./ .0/2!  1742%3.393 $483..0 4784 .9.0397.7 70.3/2801 -03/..0.88.7./44-.07920 -:.790/.39 .9.:20393205072039839810/ %0574-0289.41.9037.0147. 4784.3890.59458.0/439470.94/490...425..89 825488-0945.70038 /433492:.0.334:91742:3/0720 %01470.0 9 9 8.-0/4.8014:34 .0-003..9.8903394 709070.3 %0 .1.0894550/973941938  4:2039430/94209.3.90/9% .3.0882:8.41.-0990784:3/3.408947547.43.350450 .47/84209313/.0 08/08 037.89047037.3.84795074/ 3   8443498./447-49 50714720/.784034:8009843.. $.072.1:0398443438420.07489990419870.704::594 3489070.9.089./4 $3:.934398 574.09...943 91042:8.98 89:90.3/.89907:3/072-7490709 7.9438070-090030907 4:77080...4390.4330.94: 70344307.870.

0.98 39030..073907089330910 24398  %0574.03989%789.71:9:70 $9.38 1:9:702:8.3//0. 90 .078.08808. 470390/.3/984:/-0.-84:90-02470037.425.14702039430/1:039 9070.9:30/  .30..089478./4..3  49.3 99.70.8-03/037.844:/ 00/088948.907473  .3/3.90/5740.8390888.3.9.4393:3 08/0890 ./4 70.079.