Nicola
Mon Mar 10 2025
|
6 answers
1398
What are other tokens in GPT2?
I'm interested in knowing more about GPT2. Specifically, I want to understand what other types of tokens are used in GPT2 besides the regular words and phrases.
Valentina
Sun Mar 02 2025
|
6 answers
1302
How does GPT2 tokenize text?
I am interested in understanding how GPT2, the popular language model, tokenizes text. I want to know the specific process it follows to break down text into tokens for further processing.
