Archive — US — techcrunch
PrismML hopes its tiny LLM will change how we all use AI
PrismML aims to revolutionize AI usage with its compact large language model (LLM), Bonsai 2 27B, which significantly reduces size without a loss in performance. With successful compression methods, it fits on PCs and smartphones and boasts near-original benchmark scores. Supported by Caltech researchers and investors, the company targets advanced AI accessibility directly on user devices.
Key points
- PrismML has raised $22.25 million and aims to reduce LLM sizes without performance loss.
- Bonsai 2 27B model compresses Qwen3.8 27B to 5.9 GB, achieving a 9x-10x memory reduction.
- The model achieves 98% of the benchmarking scores of the original, up from 95% in earlier versions.
- Compression uses 'ternary weights' (values of +1, -1, 0) instead of the usual 16 bits.
- Future models in the hundreds of billions of parameters are planned, with expectations of reaching more advanced compression.
Why it matters
- Current trends in IT focus on model efficiency and running AI on edge devices, aligning with PrismML's goals.
- Compression technologies are becoming increasingly important for making AI applications more accessible and maintaining user privacy.
- Interest in device-based processing reflects broader trends towards decentralization in technology and data privacy.
PrismML's innovative approach to LLM compression holds promise for making AI more accessible and efficient, potentially transforming how users engage with advanced technologies on their personal devices.