Software Startups Not on Cloud ‘Doing It Wrong,’ Says Reddit’s Jeremy Edberg

A startup faces a lot of challenges in its’ life-cycle, but there are few that stick out.

The first question has to do with explosive growth—how do you start it?

A lot of startups often die looking for that growth.

Those that thrive, and find user attention often struggle with what can at time be an even more complicated problem: how do you manage that growth once it has started?

Reddit draws just over a hundred million unique visitors every month. With just under 8,000 active communities embedded within the site as subreddits, it powers the global “community of communities”—and is one of Alexa’s top 100 sites. It is a global brand of its own, capable of curating a rush of traffic to the most pressing political issue of the day, or a funny picture of a dog.

Yet even a site like Reddit had to have to started somewhere.

Recently, Jeremy Edberg came to Montreal for the International Startup Festival and elaborated on how Reddit handled scaling on the technical side. He was the first paid employee at Reddit, responsible for a lot of the technical architecture behind the site.

His talk focused on Reddit’s transition to hosting their own content to switching to the cloud, working with Amazon Web Services to bring thumbnails and more to life on the Reddit platform.

The initial spark for this change was interesting: Reddit’s co-founder Alexis Ohanian was a perfectionist, and constantly committed changes to the space alien logo that defined Reddit’s front page—which meant that he was constantly crowding the commit logs for the engineers.

The tech team decided to migrate the logo to Amazon Web Services, pointing at an externally hosted image so that Alexis could edit it to his at his heart’s content without having to document everything internally. This kicked off a gradual process where Reddit gradually transitioned to the cloud.

It was becoming very difficult to maintain an internal data center, especially with the constant explosive growth Reddit was experiencing. Average daily page views on Reddit almost doubled in less than one year between 2008 and 2009—from slightly under four million page views a day to just under eight million. The technical team had to be quick on their feet; Jeremy was imaging and racking servers manually under his desk at one point. There needed to be a change.

In one long Friday night, all of the migration to the cloud was finalized. The Reddit team decided to use EC2 cloud computing server instances, something that would scale well with explosive growth on their web platform. The team could now start new server instances in minutes, reducing the burden of dealing with what had become unpredictable exponential growth. They saved up to 29% in monthly costs using EC2 rather than their data center.

Jeremy has done a lot of evangelizing for cloud, and it certainly worked for Reddit. But he also makes it a point to emphasize that even the cloud is infallible. What can go wrong, will go wrong.

There are ways to mitigate this. Jeremy’s approach is the Chaos Monkey, which will delete random server instances in production, forcing engineers to think on their feet, and build code that will still work, even if the cloud fails them. This leads to better, and more flexible code.

Jeremy thinks this is the present and future of how startups should play with their data—that if you’re not a hardware startup, you’re probably doing it wrong if you don’t start on the cloud.

The present is one dominated by Amazon Web Services, but Jeremy sees a future where somebody, perhaps Google, or a determined entrepreneur using OpenStack, can offer something similar or even better.

All in all, Jeremy is a big believer in the cloud and what it can do for startups. He’s seen it with Reddit, and in his current role at Netflix. His perspective on what it took for Reddit to grow well carries lessons for startups no matter what phase they are in.