Sparse all-MLP
Meta AILanguage modeling
Sparse all-MLP is a language modeling model from Meta AI released in 2022 with 9410000000.0 parameters.
About Sparse all-MLP
All-MLP architectures have attracted increasing interest as an alternative to attention-based models. In NLP, recent work like gMLP shows that all-MLPs can match Transformers in language modeling, but still lag behind in downstream tasks. In this wor
Details
- Provider
- Meta AI
- Task
- Language modeling
- Parameters
- 9410000000.0
- Released
- 2022-04-14
- Open weights
- No