SAN FRANCISCO — What began as an experimental enthusiast movement has morphed into a formidable economic counterweight. Across software engineering, academic laboratories, and privacy-sensitive industries, millions of researchers are now operating high-capability foundation models directly on personal workstations and local server clusters.
Driven by breakthroughs in 2-bit quantization and specialized unified-memory neural processing units, desktop inference speeds now routinely exceed fifty tokens per second without transmitting a single byte over the public internet.