Games for Music Tagging

Music is everywhere on the web. iTunes, the world's biggest music store, offers over 13 million songs -- and those are just the popular ones. Now add the millions of unknown artists on MySpace and Last.fm. Even the most dedicated critics can't listen to, categorize and recommend all this music.

Part of our work at the Computer Audition Lab aims to build machine learning systems that can automatically analyze and tag music, making it possible to tag, search and recommend every song on the web. But our machines need to learn how to listen -- this is where the humans come in! We use human computation games to collect and motivate crowds of people online to contribute knowledge about music. This crowd-sourced knowledge is tailored for training our machine learning algorithms.

We've launched a new music game called Herd It that quizzes fans about the genres, emotions, instruments and even colors(!) of music. Current research involving data collected through Herd It will answer questions like:

• Can crowd-sourced data match the quality of expert annotations?
• How many example songs are needed to train reliable automatic music taggers?
• Do listener demographics (age, gender, location) affect musical appreciation?

Other CALab music games include:

• ListenGame, a simple predecesor of Herd It that founded the multiplayer technology and inspired some of the game play mechanics. ListenGame served as a proof-of-concept that collected a small but reliable set of tags.
  
• QuizTape, a way of sharing playlists on Facebook that challenges you to guess what tune should come next in the mix.

Crowdsoured Remote Sensing for Archaeology

Dr. Albert Yu-Min Lin's research group, with support from National Geographic, led an expedition into the Valley of the Khans in a non-invasive, high-tech search for archaeological remains of the Mongol empire of Genghis Khan. Part of this quest featured analysis of vast amounts of high resolution satellite imagery -- so much data that analysis by a single individual was infeasible. In Dr. Lin's words: "It's like searching for a needle in a haystack, without touching a blade of grass". Furthermore, the mystery and remoteness of the area meant that it wasn't even clear what to search for -- like searching for a needle in haystack when you don't know what needles look like!

To solve this challenge required the accuity and insight of human perception on a massive scale. We launched a game on NationalGeographic.com where "virtual explorers" could examine satellite images and identify interesting features: modern and ancient structures, roads, rivers, etc., as shown in the image below. This guided the efforts of a field expedition that travelled to Mongolia in the summer of 2010. Stay tuned for the discoveries from this exciting adventure!

Currently, the data collected through this game is being used to train computer vision algorithms to automatically detect these features in many more, untagged satellite images.