the image linked to by the post is the Pangram LLM-detection report of paragraphs in the first screenshot below, reading “100% AI”. in lieu of alt text for the screenshots below, I suggest scrolling through @eicker@lemmy.world’s profile and https://lemmy.world/comment/26323310, which the screenshots are of. Look at the vote counts.
FYI: Pangram sucks. It’s about as accurate as throwing darts these days.
A long, long time ago (i.e. 2024) there was only one LLM that was widely used by the populace: ChatGPT. It only had a few versions and they didn’t differ much in how they worded output.
Now it’s 2026 and we have hundreds of thousands of AI models. That includes over 1000 commercial vendor models!
Another thing that’s really important that you need to understand is that even if two providers run the same model, their configurations will be different and thus, their output won’t be the same.
To summarize: There’s no fucking way Pangram can work properly anymore. It’s useless garbage.
and for that matter, looking into it, Pangram seems to have an extremely low false-positive rate these days: https://www.theatlantic.com/technology/2026/05/pangram-ai-detection-accuracy/687381/ . the only complaints seem to be about its false negative rate. the FP rate of 1 in 10,000 is maybe higher than you want but whatever of Eicker’s text you throw at it it always comes up positive.
Yeah, I don’t think that there’s going to be a viable future in distinguishing between LLM and human text either, certainly not for people who have an interest in defeating it. That being said:
Some people are just going to use the most-common models.
Some models are derived from other models and will share statistical similarities.
I still think that trying to detect LLMs is largely going to be a dead-end road, though.
There are also other wrinkles. I’d bet that an increasing amount of not-trying-to-maliciously-impersonate-a-human text out there is going to come from LLMs. Like, okay, Google Translate uses an LLM. I remember listening to an interview with a senior British EU translator (this was back before Brexit). This is going to be a quality human translator. He was saying that machine translation had gotten so good that these days, that now he always did his first pass with Google Translate and just cleaned it up by hand. Okay, so you figure that this is probably widespread, probably even more so now, right? So if most of the EU’s documents are coming from an LLM, even if you could hypothetically detect all LLMs, you probably wouldn’t want to trigger on some of them.
Not to mention that people adopt mannerisms from text that they read or hear spoken. If someone is using, say, automated browser translations, they’re seeing a lot of the world through an LLM, and you’d expect them to pick up characteristics of that.
The third screenshot is me asking if it was written by AI and not receiving a reply. The user pretty much does not reply to people, just obviously-LLM summaries to his own posts. Almost all such comments are the same amount of lines. I used the first result for an LLM detector just to give a number; Eicker’s language always ticks the “this is LLM” neurons to me.
And yet they still all write the same. And the vast majority are only using of a handful of models anyway.
They all have the same hallmarks.
You’re intentionally misrepresenting my statement. It wasn’t about the models that exist, it was about the models people are using.
I’m not misrepresenting anything. If you take any model and just change the temperature a little bit (it’s a setting that controls the influence of the RNG) you can have wildly different output.
Different AI providers using the exact same model will have outputs so different using the same exact prompts you’d think they were in different universes.
They absolutely do not “have the same hallmarks.”
This stuff is benchmarked to absurd degrees every time a new model comes out. Just look it up already! You’re so obviously arguing from an ignorant position it’s silly.
You absolutely did. So much so that you didn’t even bother to explain yourself.
I don’t need to look up anything. It’s been shoved in my face for the last 5 years. You are so obviously arguing about nuances that don’t matter in reality and it’s ignorant and silly! Things like em-dashes, bullet points, and gargling the user’s balls are all signs that everyone who’s paying attention can quickly and easily recognize at this point.