Archive — US — techcrunch
DeepSeek releases ‘sparse attention’ model that cuts API costs in half
DeepSeek launched V3.2-exp, a model that halves API costs for long-context operations through its Sparse Attention mechanism. It uses a lightning indexer for selective excerpt prioritization and a fine-grained token selection system for efficiency. The model is open-weight and available on Hugging Face, allowing for further third-party testing in the drive to reduce inference costs in AI.
Key points
- DeepSeek released model V3.2-exp on Hugging Face.
- The model features Sparse Attention, using a lightning indexer to prioritize context excerpts.
- A fine-grained token selection system is used for efficient processing within limited attention windows.
- Initial tests indicate that API call costs for long-context operations could be reduced by 50%.
- DeepSeek aims to improve efficiency in the transformer architecture to lower inference costs.
Why it matters
- The release follows ongoing efforts in AI research to minimize operating costs, especially as models grow in size.
- DeepSeek previously launched R1, which was designed to reduce training costs, indicating a pattern of cost-effective innovations.
- Current trends emphasize the need for efficient AI models amidst global competition, particularly between U.S. and Chinese research entities.
DeepSeek's V3.2-exp introduces an innovative approach to reduce API costs for AI, reflecting a significant shift towards cost-efficiency amid escalating competition in the AI sector.