During a training session, a participant is learning how to adjust the behavior of generative AI. Which of the following BEST describes the role that inference parameters (such as temperature, Top-P, and maximum token count) play in the responses of generative AI?
Select the accurate description of the role inference parameters play.
They control the randomness, length, and diversity of output at generation time.
This is correct. Inference parameters are settings that adjust the behavior at generation (inference) time without changing the model's weights. They include output randomness (temperature), how candidates are narrowed (Top-P/Top-K), and length (maximum token count).
They retrain the model's weights to update performance.
This is incorrect. Retraining model weights is the process of fine-tuning or training. Inference parameters adjust behavior at generation time without changing weights — they are not retraining.
They automatically expand the training dataset.
This is incorrect. Expanding training data is a training-side task such as data augmentation or data collection. Inference parameters adjust generation-time behavior and do not expand data.
They modify the model's architecture (layer structure).
This is incorrect. Modifying the architecture is work belonging to model design and development. Inference parameters adjust generation-time output without changing the structure — they do not involve structural modification.
Understand that 'inference parameters = settings that control generation-time output without changing weights.'
- Temperature = randomness, Top-P/Top-K = how candidates are narrowed, maximum token count = length, stop sequences = cutoff.
- All allow changing output properties without retraining.
Retraining weights (fine-tuning), adding data, and modifying architecture are all training/design-side concerns — separate from inference parameters.