Baidu releases ERNIE 5.5 Flash Base, a raw 280B pre-training checkpoint
A text-only Mixture-of-Experts model released at the end of pre-training, with no later tuning, so teams can apply their own methods. It was trained on 33.5 trillion tokens. Because it is a base model, its scores are not comparable with finished assistants.
Why it matters: Clean base checkpoints at this scale let outside teams compare training methods from one starting point.
- Parameters
- 280B total
- Active per token
- 14B
- Weights size
- 288.4 GB
- Context
- 128K in training; 1M for fine-tuning and inference
- License
- ERNIE 5.5 Flash Base Model License v1.0 (custom)
| Benchmark | Score | Type | Measured or published by |
|---|---|---|---|
| MMLU-Pro | 78.4 | Cited | Baidu, via MarkTechPost |
Sources: huggingface.comarktechpost.com