The Lexicon
Token
The unit models actually read and write: chunks of a few characters each. Context windows, pricing, and limits are all measured in tokens, roughly three quarters of a word apiece.
Lego bricks of language: the model builds and bills by the brick, not by the sentence.
Context & Reliability
