Aller au contenu
LE LAB X

Ressources

Tous les sujets

Infra & self-host

Machines locales, déploiements, données et stacks que l’on garde sous contrôle.

3 articles8 signaux

Articles

Signaux

DumbleD@dumblebackdoor

DumbleD

@dumblebackdoor

Sujets repérés · Infra & self-host

Would /v1/decisions (label probabilities from one prompt pass, no generation) be useful to you in #TensorFold’s native engine? Working on Flash Next CUDA (DGX Spark) first; the hook would be shared for others models. @IntelAI Arc B70 users, interested too? cc @ashxhart #LocalAI

  • Infra & self-host
DumbleD

@dumblebackdoor

Sujets repérés · Infra & self-host

An #NVIDIA model, on an #Intel GPU, running a #TensorFold engine, piloting a @pollenrobotics duck. This is either the future of physical AI or the start of a very nerdy sitcom 🦆 Yes, I'm porting Bolt E4B to the B70 now 🐤 https://t.co/O9oCKj693K Coin ! #Microduck

  • Infra & self-host
DumbleD

@dumblebackdoor

Sujets repérés · Infra & self-host

NVIDIA's Nemotron 3.5 Bolt E4B (Gemma 4 E4B) now runs on a ~€1,900 Intel Arc Pro B70 in #TensorFold: • 6,770 tok/s prompt, 136 tok/s decode on one stream • 426 tok/s across 4 concurrent chats PR: https://t.co/tUtKBG0n9B #LocalAI @ashxhart @fragtalife @yannFichant @IntelAI

  • Infra & self-host
DumbleD

@dumblebackdoor

Sujets repérés · Infra & self-host

Gemma 4 E4B (NVIDIA Nemotron 3.5 Bolt E4B) now runs on the Intel Arc Pro B70 in #TensorFold main. 4-bit, on one card: 6,772 tok/s prompt, 135.6 tok/s decode. Served chat: 140 / 253 / 426 tok/s at 1 / 2 / 4 streams. Build from main for now; it isn't in a release yet. More prefill

  • Infra & self-host
DumbleD

@dumblebackdoor

Publication avec citation · Modèles & recherche · Infra & self-host

Gemma 4 on Intel Arc is now on this version 1.2.0 🎉🎃 Next version will introduce 2x to 2.8x in prefill speed, more than 15k tok/s on the same GPU! @intel @fragtalife #LocalAI

  • Modèles & recherche
  • Infra & self-host
DumbleD

@dumblebackdoor

Publication avec citation · Modèles & recherche · Infra & self-host

🚀 Proud of this PR merged! Prompt processing for Gemma 4 E4B is now 2-2.6× faster on Intel Arc Pro B70. Massive win for local AI workloads on Intel GPUs: less waiting, more tokens🔥 #OpenSource #LocalAI #IntelArc #Tensorfold

  • Modèles & recherche
  • Infra & self-host
DumbleD

@dumblebackdoor

Sujets repérés · Infra & self-host

For fun, and since some of you do robotics: @liquidai LFM3-VL-2B on a single @intel Arc Pro B70, on a work-in-progress branch I’m preparing for #TensorFold. 55 ms to first token on a fresh image, ~270 tok/s for one stream, ~4,500 tok/s served at 32 concurrent streams. And the

  • Infra & self-host
DumbleD

@dumblebackdoor

Publication avec citation · Modèles & recherche · Agents & automatisation · Infra & self-host

Proud to be part of this great release. More to follow on XPU: Qwen3.8-27B attention merge with 16 subgroups per head, bit-identical. Some tweaks for more tok/s. #LocalAI @intel

  • Modèles & recherche
  • Agents & automatisation
  • Infra & self-host