Text Synth - Technical Notes

Text Synth is build using the GPT-2 language model released by OpenAI. It is a neural network of 1.5 billion parameters based on the Transformer architecture.

GPT-2 was trained to predict the next word on a large database of 40 GB of internet texts.

Parameter explanation:

This implementation is original because instead of using a GPU, it runs using only 4 cores of a Xeon E5-2640 v3 CPU at 2.60GHz. With a single user, it generates 10 tokens per second. It is programmed in plain C using the LibNC library. A standalone command line version can be downloaded here.

Thanks to OpenAI for providing their GPT-2 model.

News: [Back to the main page]