Theseus 6/768
University of California San DiegoBeihang UniversityMicrosoftText autocompletionOpen weights
Theseus 6/768 is a text autocompletion model from University of California San Diego,Beihang University,Microsoft released in 2020 with 66000000.0 parameters.
About Theseus 6/768
In this paper, we propose a novel model compression approach to effectively compress BERT by progressive module replacing. Our approach first divides the original BERT into several modules and builds their compact substitutes. Then, we randomly repla
Details
- Provider
- University of California San Diego,Beihang University,Microsoft
- Task
- Text autocompletion
- Parameters
- 66000000.0
- Released
- 2020-02-07
- Open weights
- Yes