Editor’s Note: This blog post was written by Nicholas Harteau/VP, Engineering & Infrastructure
——–
As a company most often associated with amazing music recommendations and awesome parties (not to mention life-changing employee benefits), it’s rare that we get to talk about the exciting world of technical infrastructure – the real power behind the music – but today is special. Today we are announcing that we’re working with the Google Cloud Platform team to provide platform infrastructure for Spotify, everywhere.
This is a big deal. At Spotify we are obsessed with providing a streaming experience that feels as though you have all the music in the world on your phone. Historically, we’ve taken a traditional approach to doing this: buying or leasing data-center space, server hardware and networking gear as close to our customers as possible. This approach has allowed us to give you music instantly, wherever you are in the world.
But in a business growing quickly in users, markets and features, keeping pace with scaling demands requires ever increasing amounts of focus and effort. Like good, lazy engineers, we occasionally asked ourselves: do we really need to do all this stuff?
For a long time the answer was “yes.” Operating our own data-centers may be a pain, but the core cloud services were not at a level of quality, performance and cost that would make cloud a significantly better option for Spotify in the long run. As they say: better the devil you know…
Recently that balance has shifted. The storage, compute and network services available from cloud providers are as high quality, high performance and low cost as what the traditional approach provides. This makes the move to the cloud a no-brainer for us. Google, in our experience, has an edge here, but it’s a competitive space and we expect the big players to be battling it out for the foreseeable future.
What really tipped the scales towards Google for us, however, has been our experience with Google’s data platform and tools. Good infrastructure isn’t just about keeping things up and running, it’s about making all of our teams more efficient and more effective, and Google’s data stack does that for us in spades.
Google has long been a thought-leader in this space, and this shows in the sophistication and quality of its data offerings. From traditional batch processing with Dataproc, to rock-solid event delivery with Pub/Sub to the nearly magical abilities of BigQuery, building on Google’s data infrastructure provides us with a significant advantage where it matters the most.
We have a large and complex backend, so this is a large and complex project that will take us some time to complete. We’re looking forward to sharing our experiences with you as we go, so watch our engineering blog for more information on what we learn, build and break along the way. We’re pretty excited about our Googley future and hope you’ll find it interesting too.
