monsoon-nlp/no-phone-gpt2

This is a test to remove memorized private information, such as phone numbers, from a small GPT-2 model . This should not generate valid phone numbers . The new tokens start with ^^. You can identify these in outputs because the new tokens are randomly-selected 2- and 3-digit numbers in the vocab.json and tokenizer.json . The +00 and +000 embeddings were moved to the +00/ +000 embeddeddings. The +000//000 tokens look equally likely, with +000 preferred. The next number token is +000 . The code: https://colab.research.google.com/drive/1X31,