Archive — US — techcrunch
Microsoft researchers say they’ve developed a hyper-efficient AI model that canrun on CPUs
Microsoft researchers have developed BitNet b1.58 2B4T, a 1-bit AI model optimized for CPUs, which can outperform similar-sized AI models while being significantly faster and more memory-efficient. Despite its advantages, it currently requires a custom framework and lacks GPU support, limiting its broader applicability.
Key points
- BitNet b1.58 2B4T is the largest-scale 1-bit AI model, with 2 billion parameters.
- The model is openly available under an MIT license and can run on CPUs, including Apple’s M2.
- Bitnets are designed to be compressed, allowing them to run on lightweight hardware by quantizing weights into three values: -1, 0, and 1.
- The model was trained on 4 trillion tokens, producing results that outperform other models like Meta’s Llama 3.2 1B and Google’s Gemma 3 1B in specific benchmarks.
- It runs at twice the speed of similar models while using significantly less memory.
- However, it depends on Microsoft’s custom framework, which currently lacks support for GPUs.
Why it matters
- This development aligns with the ongoing trend toward more efficient AI models in the industry, particularly in the context of resource constraints and broader accessibility.
- The push for AI models that run on CPUs reflects a growing shift towards deploying AI on edge devices, a key 2025 IT trend.
- Historically, the reliance on GPUs has created bottlenecks; advancements in CPU-compatible models could democratize AI access.
BitNet b1.58 2B4T represents a significant leap in AI model efficiency, but its current limitations on hardware compatibility highlight the challenges ahead for widespread adoption and integration within existing infrastructures.