Spotify, pipelining your music
[EuroPython 2012] Jyrki Pulliainen - 3 JULY 2012 in "Track Spaghetti"
Spotify’s current catalog contains 15 million songs. Original storage of audio and metadata is over 500 terabytes and we’re transcoding 500 000 new audio streams a day. At it’s best the system can make an album playable just few minutes after it’s delivery. This talk is about building the music pipeline, all the way from the labels, who deliver music and metadata XML to our system, to the clients. Problems here are the concurrency, massive amount of data, enriching the metadata to provide better quality and to actually deliver 100 gigabytes of indexes daily.