I'm Andrew. I write about the past and future of tech, music, media, culture, art, and activism. This is my blog.

How to Fix Content Discovery

Posted: March 2, 2016

First, let’s introduce the problem in terms that really drive it home: Every hour, more than 2 years of video content is added to youtube. If you were to start watching right now, and didn’t take a break to sleep for the next 2 years, you’d be able to watch all the videos that had been uploaded in the last hour. Obviously this is more content than could reasonably be proccessed in any meaningful way, and there will be two more years worth in an hour. How is anything new supposed to get discovered in that overwhelming miasma of, mostly, crap?


To a greater or lesser extent, the same thing is a problem across all media platforms. Twitter has too much noise, Bandcamp and Soundcloud already have more than a lifetime’s worth of music (with more added each day), and the problem is getting worse. It’s easier than ever to create things. It’s easier than ever to share them. It’s impossible, as a result, to get noticed.

You might think I’m exagerating, but people who make their living on Youtube agree. The problems run deeper than that, even because the people who already have our attention also strugle to keep it, and it’s difficult for us to find out about even things that are important to us, unless they are well marketed.

And that’s the real takeaway here, if you want to get noticed today you need the support of someone who has already been noticed (and how are you going to get them to notice you?) or you need to be well marketed, well funded (and, even still, fairly lucky.) With everyone in the world competing in the same marketplace, for the same people, the distribution of wealth (in terms of attention at least) is becoming rapidly more concentrated in to the hands of a small few.

See, it’s US economics post WWII. The birth of the internet was WWII for media, massively redistributing attention. Now we’ve entered an attenion bubble. Everyone is siloed away in to an echo chamber that serves up what they are already comfortable with, and what the algorithms decide is enough of the same to keep them comfortable.

We discover nothing new. We march endlessly towards one, boring, massive monoculture where everyone listens to the same music, watches the same shows, and drives the same cars.

The Major Players try to fix things

But it doesn’t have to be this way. We can fix this, and it doesn’t even have to be very hard. See, the problem is scale. There’s just too much stuff, and it’s too easy to look at that and go for the stuff that your friends already vouched for, that the celebrity you like just endorsed, that the algorithm fed you. So reduce the clutter.

There are dozens of ways to do it. New social networks pop up all the time, and every once in a while one of them will catch on for long enough to be the perfect place to find some of the interesting, weird, and wonderful things that are being created. Then growth happens, because this is the place to find the cool new stuff, and monitization tactics creep in, and suddenly your surrounded by listicles again.

Some networks have tried to limit some aspect of their service, so that growth doesn’t make the service useless. Some services (Vine and Twitter being noteable examples) have chosen to limit the size of the content that can be shared, this way you can proccess more of it so it matters a little less that there is so much. Other services (Tumblr, Twitter) allow one user to re-share another’s posts, and encorage you to follow a relitively small number of people, who are themseleves following a relitively small number of people. This works really well, if you have a strong, diverse network. But how do you get a strong, diverse network?

Eventually, all of these artificial limitations run in to the same problems of scale that have plagued every large digital project. The more people you add, the more the service becomes a mass of incoherent babbling, punctuated by the occasional worthwhile content. Twitter, most notably, has failed to meaningfully monitize their platform and can’t seem to cope with it’s own massive scale. Too many people to be useful, too few people to get rich.

The Real Solution

These systems for discovering new stuff can’t scale well because individual people can’t cope with that kind of scale. Computers can, but algorythmic curation will result in a digitally enforced monoculture. Large companies, like twitter and Facebook, need the scale of millions of users to turn a profit, so they’re going to keep pushing for more users (and more data, and more opportunities to monotise their users), rather than for a better user experience. I think that there is another alternative, and it is to be found as much in our past as in our future.

In a great piece that touched on many of these same issues, Jesse Kriss said:

I am coming to the conclusion that we simply can’t rely on corporations to produce and maintain great, reliable, human-centered software. The systems and incentives are in direct conflict.

And he’s entirely right. We need our software (and our networks) to work on the scale of individuals, rather than on the scale of millions of users. If we can achieve this, we’ll reduce the volume of the current stream, and probably increase the signal-to-noise ratio all at once.

The answer may be that we need to combine the ease of use and ability to re-share that makes today’s best content discovery platforms thrive, and combine them with a real limitation in scale that makes serindipity more likely. I can imagine, and I’m sure you can as well, a dozen ways that this might not work. There is one, however, that I think stands a real chance:


A local, human scale network. Something akin to yesterday’s BBSs, with a little help from the Punk Zines of the 70s and 80s, built in such a way that it can only operate at the local level, and on the human scale. A Community Network, if you will.

If built correctly, a network or service that worked on this scale could revolutionize content discovery within a community (and then, within every community that community touches. Shortly, thanks to Kevin Bacon, every community would be represented.) The gains made from a Community Network would, neccisarily, find their way back in to the internet at large, resulting in:

  • Easier content discovery for everyone
  • More diversity among the most popular content on the traditional internet platforms, even facebook
  • The ability to actually find stuff you like, by accident.

So, what do you think?

Want to help?

If you enjoyed this post, please consider signing up for my newsletter. or following me on Mastodon.