TL;DR

Large Language Model (LLM) based Artificial Intelligence (AI) may be great at cranking out plausible sentences. But stringing those sentences together to form some kind of narrative doesn’t get a good story. That’s a problem, and an opportunity; because humans run on stories, and (for now at least) it seems that we still need humans to craft good stories.

Accelerating the epistemic crisis

I’m pretty sure we were in an epistemic crisis before LLM based AI came along, which is how we got filter failure at the outrage factory etc. But the torrent of slop sure isn’t helping. Search results are being poisoned, but that hardly matters, because search was already being enshittified; and now search is trumped by hallucinatory AI summaries[1].

But… there’s always an opportunity to do better. It’s becoming clear that AI is atrocious at telling good stories, which makes it clear where humans can add value.

The ‘triple threat’ data scientist

In my last job I talked about data scientists needing to be skilled in three areas:

1. Statistics – being able to understand the shape of data, and what analysis actually meant
2. Domain expertise – knowing enough about the context of data to get how it applied to a particular problem
3. Storytelling – crafting a compelling narrative that drove understanding of why things needed to change

There were plenty of science/maths folk who could do (1), and they were generally quick studies who could easily grasp (2); so it was the storytelling (3) that was the real differentiatior[2].

The grim reality was that these highly skilled (and highly compensated[3]) people spent around 90% of their time on ‘janatorial’ work, cleaning up the data so that it could be analysed. I sense that things aren’t the same with that any more, but it just makes the storytelling more important. As Kent Beck put it: ‘90% of My Skills Are Now Worth $0
…but the other 10% are worth 1000x‘.

Plausible sentences and slop grenades

I quite like Cory Doctorow’s description of LLMs as ‘plausible sentence generators‘, and when I need a plausible sentence generated, that I can’t just pull from my own thoughts, I’m happy to reach for one.

But a string of plausible sentences doesn’t get us to clear and concise explanations of things. I recently heard Shopify founder Tobias Lütke talking about ‘slop grenades’, which for me perfectly captures the challenge of wading through wordy protracted things that might be ‘right’, but take far too long getting the point over. Work is being transferred from the writer (where the human pressing some buttons may also have zero comprehension) to the reader.

It’s the XML problem all over again. Easy to create, and a pain to parse.

I have proof

Nowhere is this problem more telling than the recent (much hyped) AI breakthroughs on longstanding maths problems like Navier-Stokes. OpenAI might trumpet that they have a ‘solution‘, but it’s incomprehensible garbage that’s come from the application of brute force. It doesn’t advance our understanding of maths, because no human mathematician can articulate the ‘proof’ as a story. Eamon Duede perfectly encapsulates this in his (great story) ‘After Math‘[4], and of course he references the declaration from past Fields Medallists ‘A Severe Misalignment of AI in Mathematics‘.

What to do…

We can tell better stories, and we can learn to get better at telling stories. Writing is thinking, and writing more frequently (not more volume) tends to improve our skill.

Notes

[1] I’m still cross about trying to visit somebody in hospital at 1000 a few weekends back only to find that visiting started at 1400, because I’d made the mistake of believing an AI summary rather than checking a source for truth.
[2] I suspect this largely comes down to the fork in education between science and humanities, where the science kids suddenly stop doing anything that hones their storytelling skills.
[3] We couldn’t actually hire full time data scientists, as they simply didn’t fit into the ‘ladders and levels’ that HR stomped onto everything. A relatively junior data scientist commanded the salary of a time served VP, and that would not do. So those we worked with all came from ‘partner’ companies.
[4] Bonus points for punny title.