google/reformer-enwik8

The model is a language model that operates on characters and does not need a tokenizer . The text was chunked into batches of size 65536 chars (=2^16) The model's weights were taken from https://://://console.cloud.com/storage/browser/trax-ml/reformer/enwik8 and converted to Hugging Face's PyTorch ReformerLM model ReformerModelWithLMHead . The model was pretrained on the first 90M chars of enwik8 . The language model is not optimized yet and is rather slow . Text can be generated as follows: "Reformer model with Reformer model on character level and trained on enwik,