Launching new iPhone apps is a bit like making babies, but with a less predicable gestation period. There's a lot of anxious waiting for the big day. So we're proud to announce that you can now download Moodagent by Syntonetic -- free for a limited time!

As a beta tester for Moodagent, I've had the opportunity to discover first-hand what distinguishes the various music playlisting options available. As you can imagine, playlist quality is quite subjective. And no one wants to listen to music the same way in every situation. Therefore, the good news today is that you can automatically build playlists for an ever-increasing array of music in more ways than ever.

Automatic Playlisting Distinctions

To explain the fundamental distinctions between automatic playlist generators, I'll compare four products:

• Pandora Radio from Pandora Media
• iTunes Genius from Apple
• MusicIP Mixer from Amplified Music Services
• Moodagent from Syntonetic Media

On the face of it, all four products have similar goals -- to pick music tracks that sound good together in a playlist. But each one works in different ways and employs different technologies under the covers -- producing different results.

In this post I'll briefly provide some context for comparing these products and summarize their features and technology differences. Following a head-to-head comparison, you'll find further discussion and links to some important issues.

In my next post, I'll describe an interesting experiment I conducted which illustrates how the playlisting results differ when using each of the four products.

Recommender Technologies

Playlisting products fall into the general category of Recommender Systems, which employ ...

... a specific type of information filtering (IF) technique that attempts to present information items (movies, music, books, news, images, web pages, etc.) that are likely of interest to the user. Typically, a recommender system compares the user's profile to some reference characteristics, and seeks to predict the 'rating' that a user would give to an item they had not yet considered.

-- Wikipedia article on Recommender System

For more detail about the music recommendation field, see the later section on Making Music Recommendations. For a skeletal overview, I recommend Music Discovery on the Net, a short slide presentation created by Petar Djekic, Director Marketing at mufin, an online music discovery service. Even without any explanatory notes, you will grasp its main points. On slide six, Petar identifies three distinct ways to produce music recommendations:

Human behavior analysis

Human annotation

Content analysis

Comparing The Four Products

Having laid out the general landscape for music recommenders, let's take a closer look at each of the four products, with these questions in mind:

• On which devices is it available?
• What can be used as a musical seed?
• What kinds of controls let you refine the playlist?
• Which recommender technology does it use?
• How does it discover the characteristics of a music track?
• Does it recommend new music?
• Does it play the music in your own collection?

... and finally:

• Does it work with music in the long tail?

To me, this last requirement is crucial. A music recommender should be able to include any music I like in my playlists. Obscure tracks in the long tail of the music business may still be hits in my personal collection.

Pandora Radio

Pandora Radio is an Internet radio station available on a computer or on mobile devices like the iPhone. Writing a detailed account of Pandora would be like describing the BBC or PBS, so (to imitate Dana Carvey imitating George H.W. Bush), I'm not gonna to do it. If you don't already use Pandora, or just want to find out more, here's a short review. Ideally, read the excellent review in the recent NY Times article by Rob Walker, The Song Decoders.

You create a custom radio station using a single track or an artist as a seed, then refine it by supplying feedback about Pandora's playlist selections, or by defining more tracks or artists to be used as seeds for that station. Pandora selects music to play based on its Music Genome database, which contains the profiles for songs previously analyzed by Pandora's staff of musicologists. Pandora's profiles use hundreds of characteristics (like rhythm, instruments used, harmonic scheme, musical key, and lyrical themes) to describe each track. Although measured by expert human ears, not programs, this "music DNA" plays a similar role to an acoustic fingerprint, ...

I included Pandora in this comparison not because it competes directly with the other three playlist generators I'm reviewing; it cannot, because it does not play your own music. On the other hand, it does offer some flexible ways to specify the music you like, and creates playlists containing that kind of music. So I'm using it as an informal standard, a baseline for evaluating other music recommendation services.

One significant limitation is the size of the Music Genome DB; Pandora's manual profiling approach severely limits its ability to catalog music in the long tail. According to Pandora's FAQ, there are "well over half a million analyzed songs", with 15,000+ tracks added monthly. That probably sounds like a lot -- if you don't know that the widely used Gracenote database contains music metadata for over 100 million music tracks.

iTunes Genius (Apple)

The Genius feature of Apple's iTunes is almost a complete black box. Once it has read the contents of your iTunes library, it can generate playlists using many (but not all) of your tracks as seeds. If you don't like a playlist you can remove some songs, or get a new one. But you have no levers to control the mix.

A single paragraph in Wikipedia is probably the best summary of what is known about how Genius actually works:

Apple has not released any official information, and there isn't even much speculation on the subject online. This is not surprising, since Apple is notoriously secretive. However, using the items I highlighted in the Wikipedia entry above as a starting point, it is reasonable to suppose that Genius playlist recommendations may be based three sources of information available to Apple:

• The purchase history of millions of iTunes users (i.e. listeners who bought song A also bought songs B, C, D, …)
• Affinity relationships among songs that are found in the information iTunes collects from Genius users’ music collections and playlists (i.e. users’ playlists containing song A also contain songs B, C, D, …)
• Star ratings maintained by iTunes users (i.e. users who gave a 5-star rating to song A also rated highly songs B, C, D, …)

The experiences of Genius users, as reported in comments on blogs and online articles, reinforce these suppositions. Some researchers go further, arguing that Apple's collaborative filtering approach is actually the source of a major limitation when it comes to understanding the long tail. In their 2009 paper Smarter Than Genius? Human Evaluation of Music Recommender Systems, UCSD researchers Barrington, Oda, and Lanckriet conclude that "Genius fails on music for which collaborative filtering data is unavailable, such as the huge volume of undiscovered content in the “long tail” of the music market".

So, even though Genius can play your own music (and, naturally, recommend new music), if your collection strays far from the mainstream, it may not recognize all the tracks in your iTunes library. So those less popular tracks can't be used as seeds, and won't ever show up in Genius playlists.

MusicIP Mixer (Amplified Music Services)

To understand this product from Amplified Music Services you have to disentangle a maze of intertwined online references. The place to start is the announcement of Amplified's launch in October 2009. Amplified acquired the assets of the former MusicIP Corporation, which itself was formerly Predixis. Amplified is now focused primarily on selling music technology to the non-Apple B2B market, so this app runs only on your computer, not on your iPhone.

Be careful what you download. The successive corporate manifestations have each left their share of links online, some forwarded, some broken, and some amazingly still live. So, for example, you can still download Predixis MusicMagic Mixer v1.5, even though Predixis became MusicIP early in 2006. You'll also find versions on freeware sites like CNET and Tucows. Three months ago I downloaded an iTunes plugin version of the software (MyDJ) that installed, profiled all my music, but then would not create playlists. After wasting a lot of time, I finally contacted the company and found out it was not compatible with the latest iTunes software. So stick to the standalone version. The latest is MusicIP Mixer 1.8.x, and versions for Linux, Mac, and Windows are here.

The MusicIP Mixer software has a simple interface, and is extremely robust and easy to use. Once I installed it and pointed it at my music library, it just ran happily in the background. It systematically profiled over 10,000 tracks in my library, checking each one against its own database which (according to the press release cited above) already has the metadata for 43 million songs. The first time through, the profiler took from 1-3 seconds per track, depending (presumably) on whether it already seen a song before, or had to create a new profile. After that, whenever I open the program, it automatically finds any recently added music tracks, and profiles them on its own. I like that.

MusicIP Mixer can create playlists based on a track, an artist, or an album. Initial mix preferences include sliders for style and variety, and generated playlists can be further refined by indicating whether you approve/disapprove of specific tracks. Playlists are entirely composed of your own music. At one time it must have also recommended new music, because there's still a button for that, which now produces only a broken link. Do you need more new music recommendations?

The recommendation algorithm employs a hybrid approach, considering a track's acoustic qualitities, as well as metadata such as music genre and user ratings (which I don't use). I know it handles obscure music because it profiled a 1997 CD by Moonshine, my son Bryn's first band when he was in high school. Only Moonshine's friends and family have that CD; it surely resides at the far end of the long tail of the music business (see note below). Moonshine would never appear in a Pandora or Apple Genius playlist. MusicIP mixer, in contrast, can use any profiled songs as playlist seeds, and happily generates playlists containing Moonshine tracks alongside others with similar acoustic qualities from major artists.

Today Bryn is likely to cringe if I play his music from that era, but to MusicIP Mixer, for the right seed song, Moonshine ranks alongside Van Morrison. When the seed song is Van's performance of Caravan with The Band at their 1976 farewell concert, The Last Waltz, the resulting 25-song playlist includes not just one but two Moonshine songs. This illustrates a major advantage of using a program to classify and recognize music: programs are not influenced by human biases, and can classify and play your music, no matter how obscure it is.

Moodagent (Syntonetic)

The best introductions to Moodagent are the Syntonetic announcement: Syntonetic Announces Moodagent Playlisting Solution For iPhone, and the Moodagent website. There's also a Facebook page.

Moodagent runs only on an iPhone or iPod Touch device, not on other devices, or on a desktop or laptop. However, you do need to install a companion Moodagent Profiler app on the computer where you keep your music library and run iTunes. In September 2009, Nokia released Playlist DJ, a Nokia X6 app that's also built on Syntonetic's recommender technology and shares Syntonetic's music profile library -- dubbed the Moodagent Cloud.

Like other music recommenders, Moodagent works by creating and using a distinct profiles for each music track. Pandora's experts do this manually, MusicIP Mixer does it programmatically. What sets Moodagent apart is the nature of its analysis, which which combines digital signal processing, artificial intelligence and music science. This analysis goes beyond purely musical properties, to focus on the way music is perceived emotionally by the listener:

The result is a profile for a track composed of scores for each of five mood-related properties: sensual, tender, joy, aggressive and tempo. A set of these scores (a vector, i.e.) characterizes a track, and can be used to identify similar music. This approach allows Moodagent to offer a unique and simple interface to seed a playlist. As well the usual method of picking a song as a seed, with Moodagent you can also move a set of five sliders to set a "mood profile". If you reset the sliders, or pick a new seed track, the app immediately recalculates your playlist in seconds. The combination of these two methods is powerful, but also simple and quite intuitive once you start using the app.

The Moodagent Cloud already has over 7 million profiles [FAQ], and new ones are added continually as the users of Nokia and Apple devices profile their music collections. So profiles probably already exist for your more popular music tracks. But to ensure that Moodagent will play all your music, you do need to run the profiler. Otherwise Moodagent responds with a polite apology when you attempt to use what it views as an "unknown" track as a playlist seed. Polite or not, you will soon find this frustrating!

Running the profiler adds any previously unknown profiles to the Cloud, making them immediately available for downloading (syncing) to your Moodagent app on your iPhone or iPod. Note: Even though Moodagent creates playlists for your iTunes tracks, it always gets its track profiles from the Cloud; never directly from your computer, as in an iTunes sync. If you are used to working with iTunes (or Genius) playlists, this difference can be a bit confusing when you first install Moodagent.

The profiling process behaves similarly to that for MusicIP Mixer (described above), and -- on my laptop -- performed similarly the first time through my library. Tracks already in the Cloud database took less than a second, and profile creation (which is only needed once for each track) took about 3 seconds per track. One difference is that, when you have added new music to your library, you need to start the Moodagent profiler and explicitly point it at the folders you want (re-)profiled. So it helps to be well organized, and re-profile promptly before you forget where that new music went, otherwise you'll have to re-profile your entire library just to find tracks without profiles.

Moodagent does not recommend new music, but (based on my experience as a beta tester for the last three months) its automated playlist generation based on songs or mood settings is a very effective way to find compatible tracks among those you already have in your collection. The profiler can't define a mood for very short clips, long and varied podcasts, or DRM-protected tracks. But because it uses programmatic profiling, it will profile and play everything else in your music library, no matter how obscure it may be.

Overall Summary

This post is already long enough without reiterating points I've already covered along the way. So Table 5 below shows my summary of all four products side-by-side:

Some Personal Conclusions

Music, and music playlists, are subjective matters. Even if you use (or have used) all four of these products, your appraisals and conclusions may be different from mine. I invite you to draw your own conclusions, and to contribute your comments.

Here is my personal summary: Having spent a lot of time playing with these products, I have become a fan of Moodagent, so much so that I would prefer to use it to select and play my digital music at almost any time. But I can't actually fit all my music files on my iPod, even though I have the largest iPod Touch (64Gb). So there are times when having the MusicIP Mixer on my laptop is a handy alternative.

iTunes tells me that it would take 35.4 days to play the music in my collection, so I'm not short of music to play. Both Moodagent and MusicIP Mixer, because they find new and unexpected musical relationships within my collection, are as good as any radio station. Better really, because they're playing my music. Most of it is in my collection because I chose it in the first place, so I want to hear it! But if ever I want to hear some new music for a change, I'll turn on Pandora for a while, and try to resist any urges to buy more music.

I've pretty much given up on Genius. I want more controls, and I need it to play all my music. For me, Genius has curiosity value only. I have an analytical mind, and I've also spent years working in the software business, quite a few of them in the field of expert systems. So I'm always interested to see what any so-called "expert" software product will recommend. But that does not make me a typical Genius user. I'm sure there are plenty of iTunes users who just like their Genius playlists!

Finally, in this post I have compared application features only, not the quality of the playlists produced. Although "playlist quality" is a very subjective matter, it's an important topic. So in my next post, I'll describe an interesting experiment I conducted which illustrates how the resulting playlists differ when using each of the four products.

Further Discussion

1. Making Music Recommendations

For a detailed discussion of recommender technology as it applies to music, I recommend Music Recommendation and Discovery In The Long Tail by Òscar Celma Herrada. The introductory sections of this 232-page 2008 Ph.D. thesis provide a particularly useful overview of the subject area. Copies are available online from several sources; here's another: [7.4Mb pdf].

The later section on The Long Tail of the Music Business elaborates on the concept of the long tail of the music business, and its special relevance for the products compared. The concept applies to many consumer businesses, but Òscar explains several important ways in which recommending music differs (for example) from recommending movies or books:

Tracking users’ preferences is mostly done implicitly, via their listening habits (instead of asking users to explicitly rate the items). Any user can consume an item (e.g., a track or a playlist) several times, even repeatedly and continuously. Regarding the evaluation process, music recommendation allows users instant feedback via brief audio excerpts.

The context is another big difference between music and the other two domains. People consume different music in different contexts; e.g. hard–rock early in the morning, classical piano sonatas while working, and Lester Young’s cool jazz while having dinner. Thus, a music recommender has to deal with contextual information.

-- Òscar Celma Herrada, Music Recommendation and Discovery In The Long Tail, 2008 [pdf]

2. User Preferences and Music Seeds

Knowing what's in your own collection is an aspect of music recommendation in which your interest, as a music enthusiast, is almost diametrically opposed to traditional music retailer's interest. Retailers would always prefer that we all buy new music rather than listen to music we've already paid for. For them, the only value in knowing what music we have in our collections is in being able to predict which new music we might buy.

Happily for us, that selfish business interest actually gives rise to an important technical challenge that, if addressed well, will also help us find and play our own music. Because the essence of mining those “hidden treasures” in our own collections to create satisfying playlists lies in discovering similarities among tracks from different artists.

Of course, anyone can grab a few CD's by a couple of similar artists and play the tracks in shuffle mode; that's just the digital equivalent of a 5-disc CD changer. But trying to pick two hours of compatible music from a diverse collection of thousands of digital tracks by hundreds of artists could take several hours of thought -- or a clever program.

The best way to tell that clever program what you want it to play is to select a musical seed -- a track or an artist that embodies the musical characteristics that best match your mood, or the context for the music.

3. Playlists and Music Ownership

Many people believe that the music business is heading into a virtual world where ...

... the digital music files on people's computers could join vinyl records, cassette tapes and CDs in the dusty vault of fading music formats. Instead, music fans will use their always-online computers and smartphones to visit a vast Internet jukebox, where Gregorian chants, Lady Gaga tracks and the several centuries of music in between are instantly available.

-- Brad Stone and Claire Cain Miller, NY Times, December 15, 2009

I too believe that the above new world is inevitable. But we will not arrive there overnight. In the meantime, there are still plenty of us who -- having devoted years to assembling our own music collections -- especially enjoy those particular music selections, and want to hear them again. Yes, it's always interesting to discover new music. But the more avid our enthusiasms, and the more time and money we have invested in selecting and collecting music (although less important than the emotional investment), the more significance we attach to our collections -- and the more we care about being able to mine our own music.

Even so, as we rip our CD's and gradually replace (or digitize) our treasured vinyl LP's, our collections are increasingly becoming digital. Because digital music playlists are not bound by the physical constraints previously imposed by those LP's and CD's, we frequently face the problem of having too many choices when we set out to select music to play. Again, Òscar Herrara sums it up well:

In recent years typical music consumption behaviour has changed dramatically. Personal music collections have grown, aided by technological improvements in networks, storage, portability of devices and Internet services. The number and the availability of songs have de-emphasised their value; it is usually the case that users own many digital music files that they have only listened to once, or not at all. It seems reasonable to suppose that with efficient ways to create a personalized order of users’ collections, as well as ways to explore hidden “treasures” inside them, the value of their music collections would drastically increase.

-- Òscar Celma Herrada, Music Recommendation and Discovery In The Long Tail, 2008 [pdf]

I have no research to back this up, but I'm willing to bet that the older you are, the greater your investment -- financially and emotionally -- in your own music collection. Therefore, regardless of whether it can recommend new music, it is important for any music recommender system to incorporate the music already in your own collection when generating playlists.

4. The Long Tail of the Music Business

Here I'm going to quote almost verbatim from Òscar Herrara's thesis, because there's no point in rewriting what he describes so precisely -- especially considering that English is not his native tongue. [For the full text with supporting references see the full thesis, linked above]. Òscar explains the long tail concept, and why it is so important to the music industry:

State of the Music Industry

The Long Tail is composed by a small number of popular items (hits), and the rest are located in the tail of the curve. The main goal of the Long Tail economics—originated by the huge shift from physical media to digital media, and the fall in production costs—is to make everything available, in contrast to the limitations of the brick–and–mortar stores. Thus, personalized recommendations and filters are needed to help users find the right content in the digital space.

... the 2007 State of the Industry report by Nielsen SoundScan presents some interesting information about music consumption in the United States. Around 80,000 albums were released in 2007 (not counting music available in Myspace.com, and similar sites). However, traditional CD sales are down 31% since 2004—but digital music sales are up 490%. Indeed, 844 million digital tracks were sold in 2007, but only 1% of all digital tracks accounted for 80% of all track sales. Also, 1,000 albums accounted for 50% of all album sales, and 450,344 of the 570,000 albums sold were purchased less than 100 times.

Music consumption based on sales is biased towards a few popular artists. Ideally, by providing personalized filters and discovery tools to users, music consumption would diversify. There is a need to assist people to discover, recommend, personalize and filter the huge amount of music content.

The Problem

Nowadays, we have an overwhelming number of choices of which music to listen to. We see this each time we browse a non–personalised music catalog, such as Myspace or iTunes. We, as consumers, often become paralyzed and doubtful when facing the overwhelming number of choices. There is a need to eliminate some of the choices, and this can be achieved by providing personalized filters and recommendations to ease users’ decisions.

-- Òscar Celma Herrada, Music Recommendation and Discovery In The Long Tail, 2008 [pdf]