Goodhart’s Law is Killing Free Journalism — Bundles are the Solution

12 min readMar 17, 2021

Visit an article from any free news outlet and you’ll likely see some of the following:

Huge banner ads that take over the screen
Random videos that play automatically
A Taboola or Outbrain ‘chumbox’ of junk recommended articles
A popup promoting a newsletter
Slideshows that force you to click through multiple pages
Articles with affiliate links trying to get you to buy something

*Screenshot of US News taken in March 2021.*

Free news publishers will all claim they want to build a loyal audience, yet they continue to ruin their user experience with the shortsighted hope of getting a few more cents per visitor. Why?

The easiest explanation is Goodhart’s law. Simply put, Goodhart’s law says ‘When a measure becomes a target, it ceases to be a good measure.’

There are examples of Goodhart’s law in action all around us. The US government set emissions targets for diesel vehicles, and instead of actually reducing emissions, Volkswagen (among others) famously decided to build their vehicles to cheat the emissions test. Google’s PageRank system relied on counting the number of backlinks to webpages to help determine their ranking, and backlink farms were created to game the system instead of incentivizing high-quality, highly-referenced content.

The same type of problems are happening in the free publishing industry. Publishers optimize for ad impressions, and we see their pages gradually fill up with ads. Newsletters are seen as one of the few ways to encourage repeat viewership, so popups are added promoting signups. In an effort to gain a larger audience, many publishers chase quantity and produce superficial stories that get views rather than going deep on a few important topics.

Typically, once an organization recognizes that Goodhart’s law is at work, they refine their targets and add secondary metrics to act as guardrails that prevent further problems. Unfortunately, this doesn’t seem to be happening with ad-supported journalism. Many free publishers are continuing to optimize for short-term ad revenue despite knowing their bad behavior is incentivizing users to install adblockers and move to paid news sources.

The rest of this article will cover why free publishers optimize the wrong metrics, how free journalism has been disrupted by shifts in the advertising industry, why current subscription publications are only part of the solution, and how broad bundles can make quality journalism widely accessible.

What gets (easily) measured gets managed

The decline of ad-supported journalism starts with asymmetry in measurement. When a free publisher includes an additional ad on a page, the positive impact of the additional ad revenue is immediate and easily measured. However, measuring and attributing the associated long-term downside is not as straightforward.

Services that require user accounts like Facebook or NYT can easily get a holistic view of their user’s long-term behavior across devices because all usage is tied back to an account. Unfortunately, most free publishers have yet to find a compelling reason to get their visitors to create an account, meaning the best they can do is tie usage back to a cookie. In our mobile first world, cookies are fickle at best. Even on a single device, they typically aren’t shared between the main browser and the browsers in apps, and they definitely aren’t shared across multiple devices.

If you can’t accurately attribute product changes to long-term changes in user behavior, you’re going to struggle to optimize your product. Combine this lack of precise tracking with the inherent volatility of the news cycle, and you can see why it becomes very difficult for a free publisher to answer the question ‘what’s the harm?’ when deciding if they should include yet another ad on their page.

Even more troubling for publishers, what if a large portion of their audience was largely outside of their ability to influence? What if external organizations were pulling the strings?

Aggregators in control

News publishers were a dominant force in the advertising industry for nearly 200 years, but the geographic boundaries of a newspaper delivery area is no longer the best ad targeting available. Facebook and Google fundamentally changed the digital ad industry by providing advertisers with massive scale, sophisticated targeting, and precise attribution. Individual publishers are struggling to win ad spend when compared against the highly efficient tech giants, and now they’re scrambling to sustain a disrupted business model.

This loss of ad revenue has been devastating for free publishers, but they’re facing yet another crisis. Aggregators like Facebook and Google have not only taken their ad revenue, they’ve also taken over their relationship with the user. Instead of going directly to a few publishers to find the day’s news, many people are now getting their news through products that create personalized experiences, showing the ‘best’ stories from a variety of sources, tailored to their specific interests.

The change in the way news is discovered has de-emphasized the individual publishers’ brand and enabled their work to be easily swapped with something similar from another source without anyone caring. Because publishers are now reliant on aggregators for a large chunk of their traffic, they have little room to negotiate for more preferable treatment (a prime example is what just happened in Australia with Facebook).

Between loss of ad revenue, a severed relationship with their readers, and lack of negotiating power, free publishers are in a tough spot. How do they respond when their content has effectively been turned into a commodity and they have little control over if the user will ever come back? They adopt a short-term perspective, try to earn as much money as possible from every visit, and throw a Hail Mary by begging users to sign up for their newsletter or app with a popup.

Subscriptions solve Goodhart’s law

The current model of ad-supported journalism leads to clickbait headlines, commodity content, and pages that are stuffed with ads, resulting in a substandard experience for users. It also brings a huge opportunity cost to publishers — monetizing primarily via ads forces publishers to devote significant resources to managing their ad systems and making deals instead of investing in creating better content and building improved products to deliver it.

Look at most free publishers today — their products have not fundamentally changed much over the last 10 years. Sure, there may be the occasional interactive chart, some videos, and potentially a live blog, but compared to other industries, that’s not much progress for 10 years. In reality, the biggest changes in their product are likely found in their ad tech.

Contrast this with subscription journalism companies like The New York Times and Substack. NYT has led the charge in changing the way stories are told with their impressive data visualizations and interactive tools. Substack has helped bring about an entirely new business model that is now funding the creation of some world-class journalism.

Much of NYT’s and Substack’s success can be attributed to their business model. Subscriptions provide these companies with focus and align incentives so that what is measured and optimized (average revenue per user and lifetime value) is directly in sync with what is best for the user. Goodhart’s law, solved.

The rise of niche subscriptions

Given all the issues associated with free publishers, it’s no surprise that paid news subscriptions have been gaining popularity. They help cut through the noise and consistently deliver quality journalism on the topics people care about. Included in this subscription news boom are niche services like Substack, where individual authors can be directly supported by their audience. These direct subscriptions are not only changing the landscape of journalism, they’re also raising the price.

Consider the following subscriptions:

Thousands of new articles from The New York Times: $17/month
Millions of songs on Spotify: $10/month
Thousands of movies and shows on Netflix: $14/month
The output of a single Substack author: ~$10/month

Despite the massive disparity between the apparent value of these subscriptions, Substack is thriving. These niche writers are tapping into small pockets of highly motivated readers that find tremendous value in their content. Because the authors are not forced to chase impressions, they can focus on depth and quality over raw output. This has led to a golden era for many readers and writers, but there is a downside.

By putting quality journalism behind a paywall many would argue we are doing a disservice to society. Only the ‘super fans’ are likely to subscribe, meaning the reach of this content is severely limited. If we want this high-quality journalism to be widely accessible, it seems services like Substack are a step in the wrong direction.

Bundle economics

Bundles can help solve the accessibility/value problems that are inherent to niche subscription media products. In fact, this is exactly why they exist and partially how companies like the NYT, Netflix, and Spotify are able to provide such excellent value relative to their à la carte alternatives (the other part being economies of scale). As will be discussed later, the same principles can even be applied to mainstream products to further expand reach.

To illustrate, let’s look at an example of a simplified world with no bundles and only four people:

Sports writer
Tech writer
Sports super fan
Tech super fan

In the chart below you can see how much each fan is willing to pay for a newsletter from each writer. For example, the sports super fan is happy to pay $10 a month for a sports newsletter, but they’re just a casual fan of tech, so they’re only willing to spend $1 a month for a tech newsletter.

Full credit to Chris Dixon for first covering this so clearly.

With no bundling, the sports writer maximizes her earnings by selling one subscription to the sports super fan. The same happens with the tech writer and tech super fan. The total amount generated between the two writers is $20 per month ($10 each). The writers are leaving money on the table — there is no cost to replicate and distribute additional copies of their work, yet they are not selling it to their casual fans. If they lowered the price so casual fans would buy ($1), super fans would demand the same price, making the writers worse off.

Now imagine we’re in the same world, but the tech writer and the sports writer decide to work together and offer a bundle that contains both of their newsletters for $11. What happens? Because the bundled subscription only costs one more dollar, both the tech super fan and the sports super fan decide to subscribe to the bundle over the standalone newsletter. They’re now able to participate as casual fans for their respective interests where previously they would have been excluded.

This makes the writers better off — the total amount generated for both of the writers is now $22 per month ($11 each). Bundling effectively allows the writers to price discriminate and charge different amounts to super fans vs casual fans. Price discrimination sounds bad, but in this situation it actually benefits the consumers too — they now can get twice the content for only one more dollar.

Bundling isn’t new in the publishing world: packaging content from multiple authors for one monthly subscription is exactly what companies like the NYT are already doing. The bundles of the future will be different in the breadth of their content and how contributors are compensated.

Big, broad bundles

It seems intuitive that a successful bundle would tie together a bunch of similar content, like an ultimate sports bundle for sports super fans, or an ultimate fashion bundle for fashion super fans. While this makes sense from a marketing perspective, bundles actually work best when they have diverse content across a variety of interests. A successful bundle will have many different types of super fan subscribers (minimizing the super fan overlap), and have lots of overlap between their casual interests (maximizing the casual fan overlap).

Minimizing the super fan overlap seems counterintuitive, but the dynamics are actually similar to a potluck. To have a successful meal you need to have well-rounded demand for all the different dishes. If one dish is too popular, it will run out quickly, causing people to become dissatisfied.

Similar forces are at work with bundles. While the content in a bundle can never run out (it can be infinitely copied), the money can. If all subscribers use a bundle to access the same few things, there will not be enough money generated by the bundle to adequately compensate its contributors. The popular contributors will be incentivized to leave because they can make more money as standalone subscriptions, and the bundle will eventually fall apart.

If the bundle can attract a diverse group of super fans with overlapping causal interests, it will be balanced and provide value for both consumers and producers. This is why cable bundles have MSNBC and Fox News — they both appeal to very different super fan groups, but these fans likely have casual interests in common, like history, sports, or comedy.

The broader the bundle, the more subscribers it can attract because there is a larger pool of people that will find a subset of it appealing (and worth paying for). As the bundle attracts more subscribers, its economies of scale improve, allowing it to compensate contributors better than if they were to monetize on their own. This improvement in compensation attracts more contributors, further improving the desirability of the bundle. This model has worked for Spotify and Netflix, but there is room to go broader.

The third business model of the internet

Shishir Mehrotra talks about bundles being the “third business model” for the internet. Historically we’ve had two models:

Direct transactions for super fans. Super fans of a service have the energy and desire to look for it, and once they find it, they’re willing to pay directly to get it. Netflix, Spotify, the New York Times, and Uber are all examples.
Free/ad supported products for non-fans. The internet is full of services that people love to use, but would be unwilling to pay for as a standalone transaction. Ad-supported businesses like news companies are an example.

These two models leave a gap in the middle. The previous example of tech and sports fans illustrated there are people that aren’t quite super fans of a service (aka casual fans) that would be willing to pay a nominal fee to access the content super fans are already supporting. Similarly, there are businesses (currently struggling to find ad revenue) that likely don’t make sense as standalone subscriptions, but they may be viable as part of a broader subscription bundle — this is what companies like Scroll are betting on.

Imagine a new subscription that bundled and curated content from the best creators on Substack, Youtube, NYT, Patreon, Medium, The Economist, Vox, Wall Street Journal, CNN, Fox News, NBC, etc. for one low monthly rate. While you likely wouldn’t be willing to subscribe to all of these individually, as a bundle it starts to make sense.

With leaner operations than legacy media companies and a more equitable and competitive method for compensating contributors, the bundles of the future will be broad across mediums and topics, high-quality, and excellent value. While there is no doubt that ad-supported journalism will continue for the foreseeable future, expect to see broad bundles start gaining popularity.

Note: Shishir Mehrotra’s writing on The Four Myths of Bundling heavily influenced my thinking. If you haven’t read it already and you’re interested in diving into the economics of bundling, I can’t recommend it enough.