I recently gave Strata a shot for my local AI. I’m very impressed with the performance they have eeked out of my hardware. I’ve got a rather beefy gaming desktop I built as an AI server back when the current craze started, before prices started going up, but models up until this point really failed to take full advantage of that hardware. Strata (which uses a Qwen model) both simplified the setup process, and it maxes out the hardware when running.
Strata acts like an OpenAI compatible API server, I then have Hermes as a “harness” to make an “agent”, which I interact with on “buzz”, which is like discord, but made for bots. Although Hermes also supports discord and several other chat platforms.
If you have the hardware to run Strata, I recommend setting it up, it comes with a basic chat interface, which you can then grill to assist you with setting up Hermes and whatever else.
One thing I didn’t mention, is security. I highly recommend you treat AI as unpredictable, unreliable, and capable of being taken over by bad actors. This isn’t a problem for a pure chat interface, but Hermes gives the AI tools to do things, and those tools could be used to take over your computer, scan your network, and create all kinds of havoc. For my part, Hermes is running in a virtual machine which is on an isolated network. I keep tight control on what it does and doesn’t have access to in my network main network. That may not be something you have the hardware or knowledge to do, at the very least I recommend running Hermes in a docker container, and not giving it access to your personal desktop nor credentials to things like your email account.
Specifically for Piefed/Lemmy, I have all the experience and knowledge necessary to self host my own instance. I’ve been managing servers since before half of Lemmy’s population was born. And what it has taught me, is it is always better when its is someone else’s problem. My home lemmy instance closed down, which could have been a chance for me to self host, but I, instead, spent a little time finding another instance I liked, because I didn’t want to be responsible for it.
I actually own a domain with the idea of starting a Piefed and Mastodon instance, but I’ve never been willing to set it up and deal with the overhead. Maybe one day I’ll decide to, especially if feddit.online ever goes offline, but for now, I enjoy it being someone else’s problem.
Also, I’m pretty sure the fediverse wouldn’t work if everyone had an instance of 1. Maybe some day someone will crack a fully distributed P2P ActivityPub like solution, but until then, the health of the swarm requires we stay somewhere between “1 company runs everything” and “there are 7 billion fediverse instances”.