

This page is both funny and educational/useful, thanks!


This page is both funny and educational/useful, thanks!


They talk about like it’s the second coming of the Christ, and it’s just 7B parameters model, that probably will be shitty. No benchmarks, no date. Only claim: “Olala mini is based on a 7-billion-parameter architecture (7B), with one billion active parameters spread across 256 experts per layer, and hybrid attention combining Differential-TPA and Mamba-3, in BF16 format.(…) the model runs on a GPU with as little as 8 GB of memory, reaching 160 tokens per second on an RTX A4500, and even without a GPU, on a standard processor, at 50 tokens per second. Its context window currently reaches 32,000 tokens. Olala mini will be available as open source on Hugging Face, under the Apache 2.0 licence.”
32k context is not useful for anything other than chat. One positive thing: if the model will exist it can be studied and used further thanks to Apache 2.0. But who knows - maybe it will be good/useful? (I sure hope so).


Interesting, maybe caused by design focused on overprotection against distillation of the thinking process, so “it’s a feature, not a bug” mentality. I don’t use gemini models so I can’t confirm by myself, thanks for pointing that out!


What is this bug you described? First time hearing about it… Mistral seems weirdly low, it’s a very big model, maybe they didn’t finetune it for benchmarks?


Big boys tested Big Cloud LLMs for this and it doesn’t look good for mental health right now. Tiny models - probably even more sycophantic.


This is huge news for (moving in the direction of) universal Strata-like engine in native llama.cpp - if i uderstand it correctly. Good job, for everybody and especially for people with low VRAM. VERY GOOD NEWS!!!


It sounds like fearmongering (so hot right now), but hey: that is the true benefit of opensource! - it can be diagnosed and monitored. Claude just refuses to answer biology homework question, because of “security” (fuck Anthropic for that). Overall I agree that vigilance is needed, but still it should not make people dismiss open weight/open source models - if anything it should convice people to invest (their time and money) into open weight models EVEN MORE as they can be further finetuned / distilled / pruned or otherwise modified and run locally with any prompt you would like with any behaviour you train it to perform (i.e. you CAN make your own model out of it, even if it is anti-“insert your identity here”).
The final points of this article are worth implementing regardless of your stance, i’ll cite them here:
What model, what quant? (probably 3.8-37B). They (qwen3.8 family) sometimes loop, from what i’ve heard, it hasn’t happened to me on QFN3.8 IQ3_S (UD 3.0), yet…