An AI team is explaining the reason for adopting RAG to executives. Without an additional mechanism, a foundation model (FM) can sometimes fail to answer accurately about the company's latest documents or events that occurred after its release. Which is the MOST appropriate fundamental reason for this?
A question about choosing the fundamental reason an FM cannot answer about the latest or internal information.
Because the temperature setting at inference time is too high
Temperature is a parameter that adjusts the randomness of output, and changing the setting does not make the model able to answer about information it has not learned.
It is a different matter from whether the knowledge exists, so this is incorrect.
Because the context window is too large
The context window is the upper limit on the amount of input that can be passed at once, and the larger it is, the longer the text it can handle.
It is unrelated to the reason the model does not know the latest or internal information, so this is incorrect.
Because it simply does not know information not included in its training data
Correct. An FM only knows the scope of the data it was trained on. It does not have events after its training cutoff or company-specific information not included in training, so without an additional mechanism (such as RAG) it cannot answer accurately.
Because the tokenization scheme is outdated
Tokenization is the scheme that splits text into processing units and is unrelated to whether knowledge exists.
The reason the model does not know the latest or internal information lies in the scope of training data, so this is incorrect.
The correct understanding is that an FM only knows the scope of its training data.
- It does not know events after the training cutoff.
- It also does not know company-specific information not included in training.
- So RAG retrieves the latest/internal information from outside to supplement it.
Temperature (randomness), the context window (input capacity), and tokenization (splitting scheme) are unrelated to the fundamental reason it lacks knowledge.