Locally run AI should be the standard. All you have to do is download a .gguf file from hugging face (around 8 gig) and load it into Koboldcpp’s GUI… Or AnythingLLM if Kobolcpp is too technical for you.
Some models run faster, others run slower (depending on hardware, model size, and structure). Models like LFM2.5 for instance runs pretty fast because it’s more like 3 small models cowriting responses.
Minimal viable local AI that can actually solve problems in agentic mode is qwen 3.8 dense/flash. You are looking at fat ass pc with 16 gb vram 64 ram for flash or 32vram for dense.
Maybe if not for AI bubble prices on hardware, that would be affordable. But today it costs a whole lot for a machine that ideally shouldn’t be the same you are working on - cause whatever you use will eat vram/ram.
If by “solving problems” you mean - you yell your criteria at it enough times that it actually listens and does what you asked for half an hour ago… then sure.
Let’s face it, most of the time that’s what interacting with AI is like. It feels like repeatly asking for the same thing and getting variations that you didn’t really ask for… Then eventually settling for what it’s done.
When deepseek came out I never looked back.
Open source should be the standard for supposedly world ending technology.
Open source should ideally be the standard for technology in general
Locally run AI should be the standard. All you have to do is download a .gguf file from hugging face (around 8 gig) and load it into Koboldcpp’s GUI… Or AnythingLLM if Kobolcpp is too technical for you.
Some models run faster, others run slower (depending on hardware, model size, and structure). Models like LFM2.5 for instance runs pretty fast because it’s more like 3 small models cowriting responses.
Minimal viable local AI that can actually solve problems in agentic mode is qwen 3.8 dense/flash. You are looking at fat ass pc with 16 gb vram 64 ram for flash or 32vram for dense.
Maybe if not for AI bubble prices on hardware, that would be affordable. But today it costs a whole lot for a machine that ideally shouldn’t be the same you are working on - cause whatever you use will eat vram/ram.
If by “solving problems” you mean - you yell your criteria at it enough times that it actually listens and does what you asked for half an hour ago… then sure.
Let’s face it, most of the time that’s what interacting with AI is like. It feels like repeatly asking for the same thing and getting variations that you didn’t really ask for… Then eventually settling for what it’s done.
When deepseek came out I was already slopfree. You should try it, it’s liberating.