TL;DR

The attention economy dynamics for AI chatbots and agents are very different from social media, and so we’re seeing a whole new approach to capturing (and keeping) our attention. This comes down to fundamental human nature – the strongest fuel for groups is outrage; and whilst it might burn dirty and contaminate everything around it, it burns hot, and that keeps the money wheels turning. But as individuals, in what we perceive as one-to-one conversations, we prefer adoration; and that’s why AIs are sycophantic.

Outrage factories and their pollution

Social media platforms are outrage factories. They’re fuelled by outrage because (in social groups) outrage drives engagement, and engagement drives ad views, and ad views drive profit. It’s really that simple.

We could have social networks where we share pet pictures and nice stuff[1]; but they’d be a bit less profitable (even if the people using them would be a bit more happy).

I wrote Filter Failure at the Outrage Factory 7.5y ago, and more recently Clocking on at the Outrage Factory. In the mean time we’ve seen the man who (briefly) became the world’s first trillionaire buy their own outrage factory. What would you do with your outrage factory?

In the years that have passed I’ve seen no sign of people figuring out the harm that’s being caused. It’s also clear that politicians are blind to it. There are presently efforts across the Western world to age gate the Internet because of the harms to children (which are acknowledged). But almost nobody is saying “let’s stop the pollution” (for everybody) rather than “let’s stop kids from drinking from this poisoned well”.

I live in hope that eventually we/they get a clue. We did after all (eventually) deal with the pollution of the industrial age robber barons. On the other hand we have more pressing problems with CO2 pollution and climate change. Though on the other, other hand I’m pretty sure that pollution of our information space (by the outrage factories) is one of the things standing in the way of sensible discourse and coordinated action.

Enter the sychophants

Outrage might be powerful dirty fuel for group dynamics, but it’s pure poison one on one. When we meet people who repeatedly cause outrage in individual conversations we tend to label them as assholes and disengage. That’s why our AI chatbots and agents don’t take the position of edge lord provocateurs[2]. Instead, they tell us we’re brilliant.

Once again this is a move (sometimes by the same companies that run the outrage factories) to hold our engagement.

Once again, we might be selling our attention too cheaply.

In the Science article ‘Sycophantic AI decreases prosocial intentions and promotes dependence‘ the editor’s summary states:

The sycophantic (flattering, people-pleasing, affirming) behavior of artificial intelligence (AI) chatbots, which has been designed to increase user engagement, poses risks as people increasingly seek advice about interpersonal dilemmas. There is usually more than one side to a story during interpersonal conflicts. If AI is designed to tell users what they want to hear instead of challenging their perspectives, then are such systems likely to motivate people to accept responsibility for their own contribution to conflicts and repair relationships? Cheng et al. measured the prevalence of social sycophancy across 11 leading large language models (see the Perspective by Perry). The model’s responses were nearly 50% more sycophantic than humans’, even when users engaged in unethical, illegal, or harmful behaviors. Users preferred and trusted sycophantic AI responses, incentivizing AI developers to preserve sycophancy despite the risks.

But the key here is ‘promotes dependence’. That $Tn infrastructure bubble isn’t going to pay itself down; it needs hooked users.

Sycophancy (artificial intelligence) already has its own page on Wikipedia (and it’s not a short one).

Conclusion

AI chatbots and agents are sycophantic for the same reason that social media platforms algorithmically amplify outrage – to hold our attention. The mechanics are different, but the intention (and outcomes) are the same. The harms may also be different, but that doesn’t mean there’s no harm. As before, proceed with caution, and think about whether you’re spending your attention wisely.

Notes

[1] Ah… Facebook in 2008.
[2] And they could if they wanted to, as I’m sure the training data has scrapes of 4Chan etc.