Defamation by LLM
This post is part of a series for #WeblogPoMo2025. Read the introduction here.
After publishing yesterday's post about the reliability of the world wide web, Jack Yan commented on Mastodon:
I suffered through nine months of disinformation articles going up daily (sometimes many times daily) about me last year, thanks to “AI”. Long story which Iʼve blogged about, but I really believe everyone will get done at some point.
Naturally I asked for more information. Jack replied:
Yes, one thing the more conscientious ones, who felt guilty for spreading disinformation, told me was they used a program called Semrush. The software claims to show what search volumes are, so these self-professed "SEO experts" had their "AIs" concoct a story about me and the other words that were supposedly trending.
Jack's written several posts about the experience. This gives the best overview. In short:
- People who need #content for #engagement but have no #expertise of their own use a service called Semrush which identifies some key word topics that are popularly Googled but for which there's not much content;
- For some reason Semrush (falsely?) identified google seo jackyan as a topic being Googled a lot but with little results;
- #Desperate #content #marketers went to their AI service of choice and prompted it with "write me a blogpost that will appeal to the google search term google seo jackyan";
- Dozens of articles appear online about 'Google's new Jackyan algorithm, named after its lead developer Jack Yan'.
While kind of flattering it's 100% pure nonsense, posts that aren't just hallow but actively damaging to someone's credibility.
The bloggers and content marketers didn't know. They were following a mechanical process: take the false information from one service, ask another to write a post they know nothing about. They're presenting the information as if it's true and as if they know about it in depth.
In this instance Jack has real people to engage with - Semrush themselves, the individual content marketers, the web hosts - so posts have largely been removed even if it's taken him countless hours over many months. No-one wants that stress.
Imagine how much worse it would be if an AI service skipped those manual steps? It doesn't feel like such a future is that far away.