The Cost of Trust

One of my biggest concerns about the “AI Era” is that the cost of disinformation is now effectively zero while the cost of defense is potentially unlimited. There was a minor implosion on the coding bootcamp subreddit late last week when Lars Lofgren posted a well-researched piece about a moderator who seems to have an axe to grind against a well-known bootcamp provider, CodeSmith.

Now, I don’t have any horse in this particular race, but I do want to weigh in on this growing disinformation problem. The reason that social media is flooded with bots and AI posts is that there are massive benefits to be had once you control a distribution channel, whether this is a large subreddit, a popular Facebook group, a trusted review platform, or increasingly, the training data that feeds AI recommendations.

As I predicted many months ago, OpenAI is now attempting to slow the bleeding by dabbling in shopping and recommendations. I am genuinely concerned that there is soon to be a collapse of peer recommendation systems entirely.

The Pollution of Everything

Try finding genuine product reviews at scale today. Most of your Google searches surface affiliate blogs disguised as advice. AI generated roundups frequently source unverified and unvetted information from social media sites like Reddit, which has been signing deals to allow AI tools to scrape and summarize information from its threads.

The problem here is that “flooding the zone” is rapidly becoming an affordable strategy. The economics were already tilted towards this strategy. If I want to sell you a widget, I could always hire people to flood social media with praise. I can generate Amazon reviews, I can use AI video to set up YouTube channels that look like authentic consumer advocates. Previously, this wasn’t trivial or cheap to set up, so there was enough friction to constrain it a bit.

Then, LLMs arrived and removed the biggest point of friction: effort.

Now, I can pay any number of providers to generate thousands of convincing social media posts about my widgets in an afternoon. I can create entire comment histories for sockpuppet accounts.

Hell, let me show you what my junk email folder looks like for my business email right now:

It’s almost all unscrupulous vendors trying to sell me reputation boosting for my business.

Side note: Because I actually interact with my Discord community and learners pretty much on a daily basis, this wouldn’t work even if I were ethically-challenged enough to attempt it.

The point is, anyone can now poison the well at scale for essentially zero marginal cost. And they can do this before people even know to be suspicious of the source or community.

Back to Reddit

The example from the article about the Reddit mod is interesting because it didn’t use LLMs or any scaling. However, it shows the damage that one actor can do to a businesses reputation over time, especially when Reddit is used as a source for Google Search and AI summaries. Undoing several years of “training data” is a near impossible task for a small business.

Now, do I believe that this singlehandedly destroyed CodeSmith’s business? No. After all, the best defense is quality outcomes and genuine promotions from happy customers. However, do I believe financial damage has occurred because of this campaign? Yes, I absolutely do. Anyone who searches for opinions on CodeSmith is going to find these threads, just out of sheer volume.

It’s About to Get So Much Worse

As tools like ChatGPT and Perplexity get into the search and recommendation space, we’re moving away from consumers needing to fact check their sources. This isn’t a small issue either, as some studies are showing a 50% drop in clicking the source links when AI summaries are involved.

Think about this as a business owner. When AI gets involved, your potential customers are 50% less likely to go to your site and get information from the source. Free sites that depend on clicks and ad revenue are actively being harmed by this, and there are reports of local businesses now having to deal with angry customers who are seeking hallucinated prices, coupons, and sales.

As an instructor, I can assure you that most people are intellectually lazy. They’re not interested in deep research, so when an AI tool tells them which laptop to buy, they won’t “check the vibe” of the source. They won’t worry about whether the review site is SEO spam or from a Reddit community that has been astroturfed. The answer just appears in that authoritive tone that makes some people ditch their critical thinking skills.

And soon, these tools will start showing ads. Or, more insidiously, embedding subtle ads in conversations without identifying them as such. We’ve seen people slipping into AI induced psychosis and developing intense personal relationships with these tools, what happens when those tools are tuned to empty the user’s wallet?

As AI continues to train on the flood of synthetic reviews flooding the internet we will lose the ability to know what’s real. The training data itself becomes a vector for attack. Poison the source and you poison every answer the AI gives, forever. Human intervention at these providers will not be able to scale as quickly as the slop, and fighting back will likely require time and money that only the wealthy companies will be able to deploy.

We are in danger of losing both the ability to trust sources and the synthesis itself.

My Prediction: Contraction

This brings me to my prediction: social media as we know it is going to contract.

I don’t mean disappear, I mean contract. People are going to retreat from the open, algorithmic platforms in favor of smaller, self-policed communities. This idea is taking me back to the early days of the internet. Old school forums, community sites that are smaller, curated, and gated. These spaces will be looking for solutions to “know your neighbor” and apply policies that make bad actors economically unviable.

It’s really difficult to astroturf a 50-person Discord server when everyone knows everyone else’s posting history.

We’re already seeing some of this happen with paid communities in Slack, Skool, and newsletter writers building private discussion spaces for subscribers. Group chats are likely to become the new social media.

To put it succinctly: I predict the internet will transition from scale economies to trust economies.

The Irony Isn’t Lost on Me

As someone who joined the workforce during the dot com crash, it isn’t lost on me that I’ve spent most of my career building technology to connect everyone. The rise of ubiquitous high speed internet, mobile, and cloud was supposed to give everyone a voice and make information abundant and free.

Unfortunately, bad actors have put profits above people and the end result is that genuine connection is becoming more scarce, and hence, valuable, again.

I was initially concerned about AI’s impact on my reach, but I’ve since realized that my small, exclusive community is the future because it will remain a place where you can believe what you’re reading.

The Tradeoffs

The biggest issue with a contraction is that it comes with real costs:

• Discovery: The early internet had webrings and blogrolls. Maybe this comes back, but right now I can see that finding good communities will be very difficult. How do you find trusted sources when the algorithms have been polluted?
• Moderation: Running a healthy community takes work. Who does that work? Monetizing the communities helps a lot, but then you’re creating access barriers that could significantly and negatively impact some groups.
• Fragmentation: When people retreat to gated communities, we can lose shared reality. We will be less likely to bump into people with different perspectives. The issue of echo chambers could get much worse.
• Platform Issues: Even self-run communities often sit on big tech infrastructure. That means these companies could still potentially inject ads or set up policies that work in opposition to trust building. True independence will require decentralized and open source alternatives, which aren’t genuinely competitive in a lot of cases.

What Happens Next

Change is inevitable. But if I’m right, the most valuable companies and brands of the next decade won’t be the ones that maximize “engagement” or “reach”, they’ll be the ones that maximize trust.

The cost of trust is much higher than the cost of disinformation, but only in the commons. That’s why I believe we will abandon the commons and go back to villages.

I’ve already made my bet. My Discord, my newsletter, my growing social media following, that’s where I’m investing. Not because I’m scared of AI, but because I can see where the winds are blowing.