His point is that an LLM is just another tool for increasing productivity, the same as a compiler or IDE. The quality of the code is the only metric that should be judged.
Except they’re not? Very few tools are deterministic. Only Mathematics is, and that’s only a small subset of tools.
You’re free to argue that non-deterministic tools lack value, but that’s clearly false: OCR only exists because of lossy training machine learning algorithms. Or, going even more basic, Monte Carlo simulations are by far the most efficient method of testing entire categories of problems that are intractable deterministically.
LLMs can be shitty, as a business practice, due to their unpredictability, but (hopefully obviously) that depends on how it’s used.
I’d argue that Monte Carlo is a bad example here, and is an example of a deterministic tool.
You use a tool to create an outcome, you’re looking for an answer or a solution that you can trust.
If you want to examine behavior of a system that has input variables across some known range, you could brute force it, try every single possible combination and wait a millennia. Or you use a monte carlo strategy, let randomness enter the simulation to build out a basic characterization. You want more detail? More confidence? Increase the number of runs. I can trust that if I increase the number of runs I gain higher fidelity, if the tool/strategy was non deterministic I couldn’t do that
His point is that an LLM is just another tool for increasing productivity, the same as a compiler or IDE. The quality of the code is the only metric that should be judged.
And the person you replied to is saying it is not just another tool because tools are deterministic.
Except they’re not? Very few tools are deterministic. Only Mathematics is, and that’s only a small subset of tools.
You’re free to argue that non-deterministic tools lack value, but that’s clearly false: OCR only exists because of lossy training machine learning algorithms. Or, going even more basic, Monte Carlo simulations are by far the most efficient method of testing entire categories of problems that are intractable deterministically.
LLMs can be shitty, as a business practice, due to their unpredictability, but (hopefully obviously) that depends on how it’s used.
I’d argue that Monte Carlo is a bad example here, and is an example of a deterministic tool.
You use a tool to create an outcome, you’re looking for an answer or a solution that you can trust.
If you want to examine behavior of a system that has input variables across some known range, you could brute force it, try every single possible combination and wait a millennia. Or you use a monte carlo strategy, let randomness enter the simulation to build out a basic characterization. You want more detail? More confidence? Increase the number of runs. I can trust that if I increase the number of runs I gain higher fidelity, if the tool/strategy was non deterministic I couldn’t do that
Reviewing AI code is harder than reviewing human code. I’ll judge humans on what they do, and LLMs on what they are.