lvwerra/gpt2-imdb-ctrl

A small GPT2 language model fine-tuned to produce controlled movie reviews based the IMDB dataset . The full experiment setup can be found in the Jupyter notebook in the trl repo . The model is trained with rewards from a BERT sentiment classifier (lvwerra/bert-imdb) via PPO . The strings "[negative]", "[neutral]", and "[positive]" are added at the beginning of the query to control the sentiment . A few examples of the model response to a query before and after optimisation:response [negative]rewards [negative],response [neutral]reward [positive]response (negative)response [positive)response,