FFN SwiGLU
GoogleLanguage modelingQuestion answering
FFN SwiGLU is language modeling model published by Google in 2020 featuring 220000000.0 parameters.
About FFN SwiGLU
Gated Linear Units (arXiv:1612.08083) consist of the component-wise product of two linear projections, one of which is first passed through a sigmoid function. Variations on GLU are possible, using different nonlinear (or even linear) functions in pl
Details
- Provider
- Task
- Language modeling,Question answering
- Parameters
- 220000000.0
- Released
- 2020-02-14
- Open weights
- No