Hi all. I’ve been building my own little fediverse setup at dboft.com to fit how I like to read, and I’m grateful for all the work and community that makes something like this possible. Looking forward to being here with you all!
Hi all. I’ve been building my own little fediverse setup at dboft.com to fit how I like to read, and I’m grateful for all the work and community that makes something like this possible. Looking forward to being here with you all!
Pretty much, yes. Embedding a post takes about half a second on dboft’s small server, and the lookup afterwards is under a millisecond. At 3–4 thousand new posts and comments a day that’s maybe 40 minutes of work, so the ceiling is exactly what you guessed: how fast new posts can be embedded. More cores or a small GPU would push it way out i would think. Storage grows too, about 14 KB per post with its index, so 35k today is ~500 MB and a million would be ~14 GB. It all lives on disk in Postgres; keeping the index in RAM just makes lookups sub-millisecond instead of a few milliseconds, and Postgres caches the busy parts on its own, so it degrades gently rather than falling over. This was just an idea; I’m just putting it into together to see if it holds up. I like the idea of a shared service… I wonder what the memory constraints would be at scale.