Archive — CN — technode
Baidu open-sources ERNIE 4.5 series models, including multimodal MoEarchitecture
Baidu has open-sourced its ERNIE 4.5 models, including 10 variants on platforms like Hugging Face. The models feature a multimodal mixture of experts architecture with a total parameter count of up to 424 billion, optimized using PaddlePaddle. Open-source under Apache 2.0, it's designed for both research and commercial applications, offering tools for easier fine-tuning and deployment.
Key points
- Baidu releases ERNIE 4.5 series, open-sourcing it on Hugging Face, GitHub, and PaddlePaddle.
- Models include large MoE variants with 47 billion and 3 billion activated parameters, and 0.3 billion dense models.
- A multimodal MoE design allows for inter-modality parameter sharing while maintaining dedicated spaces.
- The models improve performance in vision-language tasks without compromising text performance.
- Trained with PaddlePaddle, achieving model FLOPs utilization of up to 47%.
- Released under Apache 2.0 for both research and commercial use, with tools like ERNIEKit for deployment.
Why it matters
- This release reflects the trend of open-source AI models in 2025, promoting collaboration and innovation.
- Baidu's models align with the increasing demand for multimodal AI applications across various industries.
- The use of MoE and optimized resources is consistent with IT trends focusing on efficient AI model training and deployment.
Baidu's open-sourcing of the ERNIE 4.5 models marks a significant contribution to AI, enhancing multimodal capabilities and supporting both research and commercial development in the rapidly evolving tech landscape.