kuisailab/albert-large-arabic

The models were trained using Google ALBERT's repository on a single TPU v3-8 provided for free from TFRC . They were trained for 7M training steps with batchsize of 64, instead of 125K with batch size of 4096 . The final version of corpus contains some non-Arabic words inlines, which we did not remove from sentences . There is no cased and uncased version of the model. The corpus and vocabulary set are not restricted to Modern Standard Arabic, they contain some dialectical Arabic too. You can use these models by installing torch or tensorflow and Huggingface library transformers. And you can use it directly by initializing it like,