Applications of Foundation Models #1
Negative prompts suppressing unwanted elements in the output
Remember the correct answer, the negative prompt.
・It explicitly states elements that should not be included in the output (specific words, styles, subjects, and so on) and suppresses them.
・It is used to avoid undesirable results and bring the output closer to the intent.
The system prompt (overall role and policy), maximum number of tokens (upper limit on length), and batch inference (execution method) are none of them the specification that suppresses unwanted elements.
Differences among ROUGE, BLEU, and BERTScore
Remember the representative metrics for text generation evaluation.
・ROUGE: For summarization evaluation, counts the overlap of words and n-grams with a reference summary (surface-match based).
・BLEU: For machine translation evaluation, looks at n-gram matching with a reference translation (surface-match based).
・BERTScore: Converts generated and reference text into embeddings (vectors) and evaluates by semantic closeness (robust to paraphrasing).
The surface-match ROUGE and BLEU are weak against paraphrasing; when you want to evaluate by semantic closeness, use BERTScore.
RAG for external references versus fine-tuning for built-in behavior
Remember the core of choosing between RAG and fine-tuning.
・RAG (retrieval-augmented generation): References external, latest, frequently changing, or company-specific information and reflects it (no retraining needed). Example: search the latest product manuals or internal policies and use them in the answer.
・Fine-tuning: Builds repetitive behavior into the model itself, such as a fixed writing style, output format, classification, and technical terminology (updates the weights). Example: answer in the tone and standard format of the company's own support desk.
Options that reverse the roles or reverse the cost ordering are the classic traps. The two can also be combined.
Data quality required for fine-tuning
Preparing fine-tuning data is fundamentally about 'preparing high-quality, representative data aligned with the task, with appropriate labeling.' Data curation and governance (managing rights, privacy, and bias) are important. 'Quality does not matter as long as there is volume,' 'unlabeled is fine,' and 'mix in unrelated data' are all errors that reduce performance.
Amazon Bedrock Guardrails blocking harmful content and masking PII
The protections that can be configured with Bedrock Guardrails include 'filtering harmful content (hate, violence, and so on)' and 'detecting and masking personal information (PII)' (along with blocking denied topics, blocking prompt attacks, and automated reasoning checks). Locking the server rack (physical security), air-conditioning temperature, and logo resolution are none of them Guardrails output filter configuration items.