unregulated bullshit emitter

Don’t worry, that’s why OpenAI and Anthropic are pretending that agents are attacking governments of their own volition. So the industry gets regulated and only “safe American AI companies” like themselves who do whatever the Trump admin says are allowed to be used on American soil. Then they’ll lobby the EU to ban Chinese models too. Especially open weight ones you could run on your own device, those are the devil.

true of the “Pro-AI” crowd (in particular those selling AI). It’s to the extent that the economy is inevitably fucked once investors start pulling out. AI CEOs lie regularly about the capabilities of their products.

That’s nothing new about AI - the only concerning fact is that AI has attracted such numbers - hopefully most of the “lesser” investors still remember .com and won’t be burned the way they were back then.

I’m a bit “overweighted” in my company stock after 10+ years in the ESPP and being lazy about diversification. At one point within the last year, the company stock was trading at 200, but now it’s down to 150, and I am sad because I missed the opportunity. My buy-in price is roughly equivalent to 120 (depending on how you look at inflationary factors), so I haven’t exactly lost 30% even if it might feel that way.

It’s the same for all volatile investment, it’s not your money until you cash out - don’t make plans like it is. Of course a lot of the world runs opposite to this advice, so we do get “big depression” when a lot of people lose their volatile investments, but it’s important to remember the zero-sum aspects of the game. The money isn’t destroyed, it just changes hands - usually to people who are less free with risky investment of it.

a political approach to AI

Like a military approach to infant daycare, or a religious approach to farming, or a democratic approach to large bridge engineering?

for all the good AI has done for people, it is also an unregulated bullshit emitter.

Just the latest in a long line of such things, going back at least to Gutenberg.

tried asking Kagi Assistant a question, and it gave me an answer in Chinese.

Sounds like a beta-tester problem to me… I started exploring LLM agent coding in mid-late 2025, it created a lot of crap I threw away, it also created some good / useful stuff that has been incorporated into our product. The models are improving, and I am improving in my ability to use them effectively. I may not be as secure in my job as I was 3 years ago, but I may also be more secure in the near future if/when it is recognized that very few CEOs of timber companies have any desire to use the chainsaws themselves, even if a crew of 10 with chainsaws can do the work of 100 men with the previous generation of cutting tools.

“correct 80% of the time in a benchmark the model was optimized for” is good enough to always trust.

No, but people do it all the time for many things they shouldn’t - not just with the latest AI tools, but with all kinds of “mostly reliable” information. In software the past 30 years have seen a development of processes / best practices that help deal with that 80% right problem in the output of human coders. Applying those processes improves the quality of the results, in exchange for additional work. What one guy would build and ship in 1992 now gets built by a team of 5, tested by a team of 10, quality reviewed by a team of 3, regulated by annual audit, approved by a team of 6, shipped by a team of 20, monitored by a team of thousands, corrected as problems are observed and reported a couple of times a year, and we call that state of the art.

How good is good enough is a matter of what your application is. A lot of people have always over-estimated the validity of their sources, and if those people are relying on that source information for important things, that’s where we get regulated industries that help to ensure they don’t (usually) take too much risk for the application, and when they do: you get Elizabeth Holmes documentaries.

Finally, it is a fact that AI companies have done irreparable harm to the world on an enormous scale, socially, ecologically, politically, and so on.

Yeah, much like the petroleum industry, the military-industrial complex, nuclear, pesticides, global shipping, introduction of exotic species, air travel, mining, logging, beef farming… everything big enough has caused enormous harm.

I haven’t attempted to verify the truth or expose the fiction of Torvalds’ patch and vacation thing - it doesn’t feel worth anywhere near the effort it would take to get to the bottom of - but I have certainly read it parroted more than twice…

In response to that, I’ll continue to repeat the fact that Torvalds the human issued a bad patch to Dirty COW which he himself described as sloppy (in so many words), and that involved race conditions that went undetected in the production kernel for years and years before anyone ever attempted to have an LLM write code.

It’s not that Torvalds is bad, he’s human - these things happen. They also happen with greater frequency when using LLMs, but in both cases: using due care and good practices dramatically reduces the liklihood of such problems getting out to the field. The LLM tools speed things up, in the creation of code, in the creation of bugs in the code, and in the finding and fixing of bugs in the code. How you choose to use it should be your own responsibility.

A lot of the sloppy human coders didn’t have the perseverence to make anything that would compile cleanly, let alone pass basic “smell tests,” until they got LLM tools - that’s not the tools’ fault, but it is the maintainers’ problem. Ignoring/attempting (with inevitable futility) to block the tools because they can be used poorly is… ridiculously bad logic.