Skip to main content

Consuming Hacker News

Like many developers, I like to read Hacker News, which aggregates user-submitted news stories with a technology bias.

Every now and again, I'll post a Risk-First article on there. Sometimes, they gain traction and make it to the front page, but mostly they don't. This can depend on a lot of things such as time-of-posting, subject or title (this article builds really cool Neural Network for Rating Your Hacker News Title. Short of putting "has died" somewhere in the title, there's no sure-fire route to success).

Which is as it should be, for two reasons. First, there needs to be serendipity, otherwise we'd all just be reading the same thing all the time (which would be boring) and second the volume of submissions: Every hour 30 or so articles get posted, which is enough to fill the front page, but the front page entries tend to stay there for a day or so. So that's a lot of stuff that disappears.

Alerting​

There is a really cool tool called AlertHN which allows you to get email alerts whenever someone mentions a specific search term. This is super useful, as I wanted to know if anyone posted articles about Risk-First, or commented on articles I had posted. And, it totally works and has helped me with this. I have no idea who wrote it, or whether it'll be there next month, but it's a great -and somewhat mysterious- service.

Unintended Consequences​

Recently I decided to widen my search terms to include things like "methodology" and "estimates", which might flag up content relevant to Risk-First, and keep me in the loop on these conversations.

As you might expect, I now get a whole bunch of emails every day about all kinds of things. They range from off-topic (for me) but interesting, like Former Boeing Engineers Say Relentless Cost-Cutting Sacrificed Safety to super on-topic, but appearing nowhere on the HackerNews front page like this discussion of estimates.

What has happened since then though is that now I tend to consume HackerNews not by visiting the site directly, but by reading through all these random emails. This may not be the way a lot of people want to go (after all, email spam) but it seems to give me a wider, more random selection of stuff to look at.

Weirdly, I like it.

Filter Bubbles​

The HackerNews algorithm is proprietary and hidden, but it seems to work roughly like this:

  • HackerNews will "try out" new submissions with a small part of it's user base, to see how many up-votes the content gets.
  • If it achieves a certain velocity, it will promote the submission to a wider audience.
  • Once the post is "proven" it will appear on the front page for a day or so, then over the next few days will decay off receiving fewer and fewer views.

But this means that there is a central parameter at the heart of HackerNews which controls how much of a "shared experience" we all get:

  • The more consistently it promotes highly-rated posts, the more my experience of HackerNews will match yours.
  • However, sliding this parameter the other way means we all get more "random" posts, but this reduces the shared experience, and means that the conversation is spread over more topics.

We like to think about filter bubbles with respect to political divisions on Facebook. Users are relentlessly profiled and marketed-against there. However, they are a natural consequence of the HN algorithm too.

If you don't like it then that's tough, because the parameter isn't under your control... unless you use something like AlertHN to re-randomize it again (or maybe read HN via the "Newest" view).

SkimFeed​

Recently, I've been hearing about Skimfeed, which aggregates 12-or-so articles from a bunch of news sources including HN. It seems to be a different view of "top" articles than the one I have when I hit the website.

So is this exacerbating the filter bubble or loosening it?

I'm not sure yet.