Not really, they are testing your passwords against have I been pawnd. I agree that transparency should apply here, and also an upfront question to skip.
But it’s not “unsafe” or a privacy breach per se. No password has actually been “sent” anywhere only a partial hash that is useless for accessing your home assistant.
Yes i do.
I esperiment a lot, but my stable setup is Qwen 3.8 27B Q6_K on my home server. Hardware is Ryzen 9 5950x with 64gb ddr4 and two old but effective NVIDIA RTX A4000 for a total of 32gb VRAM.
Qwen 3.8 is the best model I can run in terms of output quality. It can perform all tasks pretty well and it’s reasonably fast at 500ts prefill and 30/40ts processing. There are much faster models but honestly not as good.
I can also run gemma4 all on CPU, with no GPU at all.
The key point is having both GPU on pci-e CPU, not on chip set bridge, so I am using a PCI riser 8x8x to share the 16x slot between the two GPUs. Inference speed is unaffected, but prefill benefits immensely.
Software side, use llama.cpp on Linux, you get the best performance possible at the expense of a bit more complexity compared to other engines.
From a power consumption perspective, I tined down my CPU governor and maximum frequency, and also the GPUs maximum watts. At ide the system is around 80w. At peak, it’s about 280w with the limiters, and inference speed doesn’t suffer too much. Without limiters I can easily reach 650w.
Edit: I run llama-swap in front of llama-cpp, and openwebui as agent manager/ web GUI. I also have ComfyUI installed because why not but seldomly use it.
You can read more of my setup on my wiki https://wiki.gardiol.org/8-llm-and-ai (no ads, no monetization whatsoever because fuck a hobby should not be a side gig)
Reverse proxy my friend…
Using one static IP per service feels so much like … 1990s…
… As well as owing a /24, I admit.