From Unsloth’s post on Reddit:

Hey guys Thinking Machines released Inkling 2 Mini, the strongest 26.4B model (regardless if MoE or dense) that runs on ~16GB RAM.

Inkling 2 Mini is Thinking Machines Lab’s new 26.4B MoE (5.4B active) model for coding, agentic & vision workloads.With a Looped Transformers architecture and 1M context, it scores 48.2% on DeepSWE v1.1 and 77.9% on SWEBench Pro v2 (Hard), the highest for its size. Thanks TML for day 0 access.

Run on Unsloth Desktop or llama.cpp.

GGUF: https://huggingface.co/unsloth/Inkling-2-Mini-GGUF

Guide: https://unsloth.ai/docs/models/inkling-2-mini

But looks like the model is not available in llama.cpp yet:

But if benchmarks are true, it’s somewhat on par with Qwen 3.8 27B!

  • EasyPeasy@piefed.zip
    link
    fedilink
    English
    arrow-up
    3
    ·
    1 day ago

    One of the PRs is merged now, although not the unsloth one so personally I am trying the ggml-org weights first to be on the safer side. With a novel architecture and looping layers I wouldn’t rule out a few days of problems to iron out in the implementation though…

    • Shatur@discuss.tchncs.deOP
      link
      fedilink
      English
      arrow-up
      4
      ·
      19 hours ago

      According to reports it’s not very stable in llama.cpp right now. Unsloth fixed it in their fork, so yeah, I’d wait a little.