That’s a pretty good way to describe those kinds of models. Somewhere between rubber duck debugging and a really lazy intern. Better than an algorithm, faster than a human, but less trustworthy than an intern. It’s great if you know how and when to use it. Maybe it’ll become an intern soon enough, but for now let’s focus on decompiling adobe into bankruptcy.

Yeah running in the background type beat. Not gonna trust this with anything important but it’s great to have someone just do shit for you or get a second opinion.

That’s gonna be quick af with the same models.

Oh damn what a useful link. Gonna check out those orchestrators, maybe I can replace my janky ass proxy.

MCP is a really dumb server that just takes requests and sends back data.

Somewhere in your frontend settings (Open-WebUI has it under “integrations” I believe, some backends can set these things too) you see a section called “MCP server”. You need to keep your MCP server running somewhere (it’s very slim, so not a huge deal usually) and then in those settings you put your IP and port for the server, for example “http://192.168.1.15:8080/”. Usually if you did it right (sometimes you can check if it works in the settings), you can ask the model if it has access to the tools and it will tell you.

What’s happening in the background is that every conversation the AI does, it gets a short preamble with some context about what it can do, and if you connect web search, custom skills or an MCP server (it’s just skills but they are run through their own server basically), the AI sees all that in this preamble if it’s set up correctly. And then when it feels like it needs to use those tools, it will check the description on how to do that, and then it uses them.

1070ti and some hexacore something cpu that’s alright with around 24gb ddr4, I think one of the slower speeds. I also got a faster PC with another llama.cpp instance and I vibe coded a quick and dirty proxy which I can use to switch between the different backends. It’s genuinely useful because I got code editors but also my Open-WebUI using it.

Sounds elaborate but generally it’s all pretty primitive.

Also fyi long preloading steps because my vram is obv not good enough to hold the full model and kv cache and stuff, generation at about 15t/s tops.

My experience is right now we’re at the stage where it’s a good assistant but not great for actually taking over bigger tasks completely.

Am looking forward to:

• experimenting with existing non-uniform quants; I believe those quants are the next step to more bang for your buck on those low-grade consumer rigs
• hoping for new kinds of quants to come out and be more widely adopted
• maybe a more universal or consistent harness that is less prone to broken tool-calls etc.
• better models, specifically a new version for 35b a3b.

• Qwen3.6 35b a3b Q4_K_M running on llama.cpp
• Open-WebUI as a frontend
• integrations with an embedding model and searxng & crawl4ai
• doing lots of different stuff with it, asking basic stupid questions, recipes, tech stuff, etc