Skip to content
LE LAB X

Resources

All topics

Infrastructure & self-hosting

Local machines, deployments, data and stacks you keep under control.

3 articles8 signals

Articles

Signals

DumbleD@dumblebackdoor

DumbleD

@dumblebackdoor

Spotted topics · Infrastructure & self-hosting

Would /v1/decisions (label probabilities from one prompt pass, no generation) be useful to you in #TensorFold’s native engine? Working on Flash Next CUDA (DGX Spark) first; the hook would be shared for others models. @IntelAI Arc B70 users, interested too? cc @ashxhart #LocalAI

  • Infrastructure & self-hosting
DumbleD

@dumblebackdoor

Spotted topics · Infrastructure & self-hosting

An #NVIDIA model, on an #Intel GPU, running a #TensorFold engine, piloting a @pollenrobotics duck. This is either the future of physical AI or the start of a very nerdy sitcom 🦆 Yes, I'm porting Bolt E4B to the B70 now 🐤 https://t.co/O9oCKj693K Coin ! #Microduck

  • Infrastructure & self-hosting
DumbleD

@dumblebackdoor

Spotted topics · Infrastructure & self-hosting

NVIDIA's Nemotron 3.5 Bolt E4B (Gemma 4 E4B) now runs on a ~€1,900 Intel Arc Pro B70 in #TensorFold: • 6,770 tok/s prompt, 136 tok/s decode on one stream • 426 tok/s across 4 concurrent chats PR: https://t.co/tUtKBG0n9B #LocalAI @ashxhart @fragtalife @yannFichant @IntelAI

  • Infrastructure & self-hosting
DumbleD

@dumblebackdoor

Spotted topics · Infrastructure & self-hosting

Gemma 4 E4B (NVIDIA Nemotron 3.5 Bolt E4B) now runs on the Intel Arc Pro B70 in #TensorFold main. 4-bit, on one card: 6,772 tok/s prompt, 135.6 tok/s decode. Served chat: 140 / 253 / 426 tok/s at 1 / 2 / 4 streams. Build from main for now; it isn't in a release yet. More prefill

  • Infrastructure & self-hosting
DumbleD

@dumblebackdoor

Publication with a quote · Models & research · Infrastructure & self-hosting

Gemma 4 on Intel Arc is now on this version 1.2.0 🎉🎃 Next version will introduce 2x to 2.8x in prefill speed, more than 15k tok/s on the same GPU! @intel @fragtalife #LocalAI

  • Models & research
  • Infrastructure & self-hosting
DumbleD

@dumblebackdoor

Publication with a quote · Models & research · Infrastructure & self-hosting

🚀 Proud of this PR merged! Prompt processing for Gemma 4 E4B is now 2-2.6× faster on Intel Arc Pro B70. Massive win for local AI workloads on Intel GPUs: less waiting, more tokens🔥 #OpenSource #LocalAI #IntelArc #Tensorfold

  • Models & research
  • Infrastructure & self-hosting
DumbleD

@dumblebackdoor

Spotted topics · Infrastructure & self-hosting

For fun, and since some of you do robotics: @liquidai LFM3-VL-2B on a single @intel Arc Pro B70, on a work-in-progress branch I’m preparing for #TensorFold. 55 ms to first token on a fresh image, ~270 tok/s for one stream, ~4,500 tok/s served at 32 concurrent streams. And the

  • Infrastructure & self-hosting
DumbleD

@dumblebackdoor

Publication with a quote · Models & research · Agents & automation · Infrastructure & self-hosting

Proud to be part of this great release. More to follow on XPU: Qwen3.8-27B attention merge with 16 subgroups per head, bit-identical. Some tweaks for more tok/s. #LocalAI @intel

  • Models & research
  • Agents & automation
  • Infrastructure & self-hosting