Distinguishing tokens, parameters, embeddings, and epochs
Distinguish the basic generative AI terms by 'what is being counted'.
・Token: the smallest processing unit of text. The basis for input/output amount, pricing, and the context window.
・Parameter: the number of internal weights of a model (model scale).
・Embedding: a vector representation of meaning.
・Epoch: the number of training iterations.
When the question is about the 'unit for billing or input limit', it is the token.