Hardware is a 5070 12GB VRAM, an AMD 5950x and 64G of DDR4 running on Nobara (Fedora spinoff).
I use 3 models for most things: Ternary Bonsai 27b and Gemma 4 E4B Uncensored Q8_0 for everyday stuff, and Qwen Agentworld 35B A3B Q8_0 with CPU offloading for tasks that require to keep the model on track for long periods of time.
Everyday performance is best with Bonsai for me - it comfortably fits in my VRAM and is noticeably better with staying on task than similarily sized models. But if i want it to work overnight on a task without issue, the lower speed of the agentic Qwen is outweighed by its capability to do this.
They are allowed tool calls for websearch and reading webpages and have a sandboxed directory for R/W operations, but anything regarding command line and R/W access in other areas has to be manually authorized by me (i’m not willing to play with fire here).
They are currently not reachable from outside, but i am currently building a smaller machine as a home server, which will then be using a 3070Ti and 32G RAM to provide probably the Bonsai Model for me on the go per VPN.
The smaller models are here for quickly creating documents i do not care to do myself, RAG for technical documentation, creating quick python scripts for automating small tasks and teaching me RegEx and Python(mostly by analyzing the LLM output and playing around with it) and also some RP as NPC in a fantasy setting. The agentic model is currently working on cataloguing my games collection (around 36000 titles) by looking up games, their genres and descriptions and filling in blanks in the database. (Funnily, this task was refused by any model freely available because of the size of the database. Even the local models moan and bitch that it takes so long. No shit, this is why i outsource this!)
A kilometer is about a 15 minute walk (meaning you can walk ca 4 km in one hour - actually a bit less, avg walking speed is around 3.6km/h, but that is so very much individual that it has the same usability as your mile in 20 minutes).
0 and 100 °C is where water changes between frozen/liquid/vapor (and that does NOT change, while “reasonable comfort” is very much dependent on humiditiy - if it’s close to 0° outside, expect slippery roads. 0 and 100 Fahrenheit do tell you fuck all about what to expect outside regarding road conditions without extra knowledge.) Default room temperature is around 22°C, smack in the middle of the personal preference normally being around 20-25°. Having the same scale in mind, everything above 30°C is pretty warm, with 42° the current heat record in Austria, set this year. 60° is the temperature where proteins start to denature a.k.a getting burnt, pretty much 2/3 of the way to water vapor.
One liter is the volume that fits in a 10x10x10 cm cube, just as a milliliter is the volume of a 1x1x1 cm cube (because 10x10x10=1000, and a milliliter is a thousand of a liter). we still use tea/tablespoons in many cooking recipes tho (or if you want, you can use 5 and 25 ml or 5 and 25g if it’s water.)