funnel-transformer/small-base

A small language model corrupts the input texts and serves as a generator of inputs for this model . The model does not contain the decoder, so it ouputs hidden states that have a sequence length of one fourth of the inputs . It's good to use for tasks requiring a summary of the sentence (like sentence classification) but not if you need one input per initial token . This model is primarily aimed at being fine-tuned on tasks that use the whole sentence (potentially masked)to make decisions, such as sequence classification, token classification or question answering . For tasks such as text-generation you should look at model like GPT2. See the model hub to look for,