AWS Learning Material Certification Cheatsheets
AIF-C01 Domain 3 28% domain

Task 3.3: Describe the training and fine-tuning process for FMs

Applications of Foundation Models · 4,412 words · source: Certified-AI-Practitioner-AIF-C01/domain-3/task-3-3-cheat-sheet.md

Domain 3: Applications of Foundation Models

Task Statement 3.3: Describe the training and fine-tuning process for FMs

Domain 3 is "Applications of Foundation Models" and represents 28% of the scored AIF-C01 exam content. Task 3.3 focuses on how foundation models are trained, how they are adapted after pre-training, and how data must be prepared for successful fine-tuning.

Use this sheet to answer scenario questions such as "which training method fits this requirement?", "what data does this method need?", "what AWS service helps prepare or label the data?", and "what risk appears when a model is tuned too narrowly?"

Official Study Path

Use these in order:

  1. AWS Skill Builder: AWS Artificial Intelligence Practitioner Learning Plan
  2. AWS Skill Builder: Exam Prep Plan: AWS Certified AI Practitioner (AIF-C01)
  3. AWS Skill Builder: Domain 3 Review: AWS Certified AI Practitioner
  4. AWS Skill Builder: Domain 3 Practice: AWS Certified AI Practitioner
  5. AWS Exam Guide: Content Domain 3, Task Statement 3.3
  6. AWS Docs: Amazon Bedrock model customization, data preparation, hyperparameters, reinforcement fine-tuning, and model distillation
  7. Local Skill Builder transcripts: Task Statement 3.3, Lessons 1 and 2

Exam Objective Map

You should be able to answer questions about:

Objective What to know for the exam
Pre-training The initial large-scale training process that teaches broad capabilities from huge unlabeled or unstructured datasets
Fine-tuning Additional supervised training on labeled examples to improve performance on a specific task or use case
Continuous / continued pre-training Further training a pre-trained model on additional unlabeled domain data to adapt knowledge and vocabulary
Distillation Using a larger teacher model to train a smaller, faster, lower-cost student model
Instruction tuning Fine-tuning with prompt-response examples so a model follows instructions better
Domain adaptation Fine-tuning on domain-specific data so the model handles jargon, terminology, and specialized patterns
Transfer learning Reusing a pre-trained model and adapting it to a new task instead of training from scratch
Data preparation Curate, clean, govern, label, size, split, and validate representative training data
RLHF / reward-based tuning Use human preference or reward signals to align outputs with desired behavior
AWS tooling Amazon Bedrock custom models, SageMaker JumpStart, SageMaker Canvas, AWS Glue, SageMaker Clarify, SageMaker Ground Truth, SageMaker Feature Store

One-Minute Mental Model

Training and fine-tuning are about changing a model, not just changing a prompt.

  1. Pre-training creates broad model capability from massive data and compute.
  2. Continued pre-training adds domain knowledge from unlabeled domain data.
  3. Fine-tuning adjusts model behavior from labeled input-output examples.
  4. Instruction tuning teaches the model to follow task instructions and response patterns.
  5. Domain adaptation teaches domain language and specialized terminology.
  6. RLHF or reinforcement fine-tuning aligns outputs with feedback or reward signals.
  7. Distillation transfers useful behavior from a larger teacher model into a smaller student model.
  8. Good data matters more than volume alone: it must be relevant, governed, representative, and labeled where required.
  9. After tuning, evaluate on validation and test data before production use.

Exam shortcut: If the source knowledge changes often, prefer RAG. If the model must repeatedly behave differently for a stable task, consider fine-tuning. If a larger model works but is too expensive or slow, consider distillation.

1. FM Training Lifecycle

Stage What happens Data type Output Exam clue
Data selection and curation Choose and clean training data Raw, unlabeled, labeled, domain-specific Training-ready dataset "Curate representative data"
Pre-training Train from broad data to learn general patterns Very large unlabeled/unstructured corpus Base foundation model "Millions of GPU hours, huge data, broad capabilities"
Fine-tuning Further train a pre-trained model Labeled examples Custom model for a task "Improve performance on a specific task"
Continued pre-training Continue training on more domain data Usually unlabeled domain corpus Model with stronger domain knowledge "Teach industry language or domain corpus"
Evaluation Measure quality before deployment Validation/test sets, benchmarks, human review Quality metrics and decision "Holdout validation/test dataset"
Deployment Make the model available for inference Custom model artifact Production endpoint or managed inference "Use the tuned model in an application"
Feedback Collect user, human, or system feedback Logs, ratings, corrections, reward signals Improvement loop "Use feedback to improve future versions"

Key Exam Distinction

Pre-training builds broad capability. Fine-tuning specializes an existing model. Continued pre-training adapts knowledge. Prompt engineering changes the input only. RAG adds external context without changing model weights.

2. Pre-Training

Pre-training is the first major training stage for a foundation model. It uses huge amounts of data and compute so the model learns general statistical patterns, language structure, visual structure, code patterns, or multimodal relationships.

What to Know

  • Pre-training is expensive, complex, and usually not performed by ordinary exam-target users.
  • It can require very large compute resources, large datasets, many tokens, and long experimentation cycles.
  • It often uses self-supervised learning on unlabeled or unstructured data.
  • It produces a base model that can later be adapted with prompt engineering, RAG, fine-tuning, continued pre-training, or distillation.

Exam Signals

Scenario phrase Best interpretation
"Train a new foundation model from scratch" Pre-training
"Huge corpus, massive compute, broad capabilities" Pre-training
"General language/image/code capability" Pre-training result
"Most expensive customization path" Pre-training from scratch

Common Trap

Do not choose pre-training when the company only needs a model to answer from its documents, follow a specific format, or understand a narrow domain. Those usually call for RAG, prompt engineering, fine-tuning, or continued pre-training.

3. Fine-Tuning

Fine-tuning extends training of a pre-trained model to improve performance for a specific task or use case. In supervised fine-tuning, the dataset contains labeled examples that show the model what output should be generated for each input.

What Changes

Fine-tuning changes model parameters or adds trainable adaptation parameters. This is different from prompt engineering or RAG, which do not modify model weights.

Fine-Tuning Flow

  1. Select a base model that supports customization for the needed modality and Region.
  2. Curate task-specific examples.
  3. Format the dataset as required by the model and customization method.
  4. Split data into training, validation, and test sets.
  5. Run the fine-tuning job.
  6. Monitor loss, validation metrics, and signs of overfitting.
  7. Evaluate the tuned model on holdout test data.
  8. Compare the custom model to the base model and business objectives.
  9. Deploy only if quality, cost, latency, and risk are acceptable.

When Fine-Tuning Fits

Use fine-tuning when:

  • The model needs consistent behavior on a stable task.
  • You have labeled examples of desired input-output behavior.
  • Prompt engineering and RAG are not enough.
  • The model must learn a style, format, classification boundary, or task pattern.
  • Domain-specific phrasing or jargon affects output quality.
  • You need better task accuracy for repeated production workflows.

Avoid fine-tuning when:

  • The knowledge changes frequently.
  • The model only needs access to current company documents.
  • The problem is an unclear prompt or missing context.
  • You do not have enough representative or governed data.
  • You cannot evaluate whether the tuned model improved.

4. Fine-Tuning Methods

Method What it means Data needed Best for Exam clue
Supervised fine-tuning Train on labeled input-output pairs Labeled examples Stable task behavior, style, classification, summarization "Known correct responses"
Instruction tuning Fine-tune with instructions and responses Prompt-response examples Better instruction following and zero-shot task behavior "Follow instructions for many task types"
Domain adaptation fine-tuning Adapt to domain-specific language/data Domain-specific dataset Jargon, technical terms, industry vocabulary "Medical/legal/financial terminology"
Continued pre-training Continue training on unlabeled domain corpus Unlabeled domain text or data Broad domain knowledge adaptation "Large unlabeled domain corpus"
Transfer learning Start from a pre-trained model and adapt it Smaller task/domain dataset Avoid training from scratch "Reuse a pre-trained model"
Multitask fine-tuning Train on examples from multiple tasks Labeled examples for many tasks Reduce narrow specialization and improve broad instruction behavior "Summarization, translation, ratings, code examples together"
PEFT / LoRA Freeze most base weights and train small adapters Task-specific examples Lower memory and compute than full fine-tuning "Parameter-efficient tuning"
ReFT Freeze base model and learn interventions on hidden representations Task-specific examples Representation-level adaptation "Representation fine-tuning"
RLHF / reinforcement fine-tuning Optimize with human preference or reward signals Feedback, rankings, reward functions, or logs Alignment with human preferences or measurable goals "Humans rate outputs" or "reward function scores responses"
Distillation Use teacher responses to tune a student model Prompts, teacher responses, optional labels or invocation logs Lower latency and cost with acceptable quality "Large model is good but expensive"

Instruction Tuning vs Domain Adaptation

Need Choose
Model should follow instructions more reliably Instruction tuning
Model should understand domain language, jargon, or specialized data Domain adaptation
Model needs broad domain exposure from unlabeled corpus Continued pre-training
Model needs current facts from changing documents RAG, not fine-tuning

5. Continued Pre-Training

Continued pre-training, also called continuous pre-training in the exam guide, means taking a pre-trained model and continuing training on more data. The Skill Builder content emphasizes this for adapting a model over time to different topics, genres, and contexts.

What to Know

  • It is usually based on unlabeled data.
  • It can help a model learn domain-specific knowledge and vocabulary.
  • It is heavier than prompt engineering or RAG.
  • It changes the model rather than retrieving facts at inference time.
  • In AWS study content, Amazon Bedrock is associated with continued pre-training for Amazon Titan Text models using unlabeled data in a managed environment.

Exam Signals

Scenario Best answer
"Large unlabeled corpus of domain documents" Continued pre-training
"Need model to understand industry vocabulary broadly" Continued pre-training or domain adaptation
"Need fresh policy answers with citations" RAG
"Need one stable task to perform better from examples" Supervised fine-tuning

6. Distillation

Model distillation transfers knowledge from a larger, more capable teacher model to a smaller, faster, more cost-efficient student model. In Amazon Bedrock, distillation can use prompts, teacher-generated responses, labeled examples, or invocation logs to fine-tune the student model.

Distillation Flow

  1. Choose a teacher model with the quality you want.
  2. Choose a smaller student model to improve.
  3. Prepare prompts or prompt-response examples for the target use case.
  4. Generate or use teacher responses.
  5. Fine-tune the student model on the generated or logged examples.
  6. Evaluate quality, latency, and cost against the teacher and base student model.

When Distillation Fits

Use distillation when:

  • A larger model performs well but is too costly or slow.
  • You need lower latency for production.
  • The use case is stable enough to train a student model.
  • You can accept some possible quality tradeoff for better price/performance.

Do not choose distillation just to add changing company knowledge. Use RAG for changing knowledge.

7. RLHF and Reinforcement Fine-Tuning

RLHF stands for reinforcement learning from human feedback. At exam level, it means using human feedback to align model behavior with human preferences. Amazon Bedrock documentation also describes reinforcement fine-tuning as a reward-driven customization method where reward functions score responses and the model learns to maximize those rewards.

RLHF / Reinforcement Fine-Tuning Flow

  1. Collect prompts or use invocation logs.
  2. Generate multiple responses.
  3. Have humans, reward functions, or model-as-a-judge graders score or rank the responses.
  4. Train the model using those feedback signals.
  5. Monitor reward metrics and validation quality.
  6. Check that the model is not overfitting to the reward signal.

When It Fits

Scenario Why RLHF / reinforcement tuning fits
Multiple valid answers exist, but some are better Rewards can rank quality
Human preference matters Feedback aligns outputs with user expectations
Objective scoring is possible A reward function can grade correctness or structure
Labels are expensive to produce Reward-based tuning can use feedback instead of full input-output labels

Exam Trap

RLHF is not the same as ordinary supervised fine-tuning. Supervised fine-tuning learns from labeled correct examples. RLHF or reinforcement fine-tuning learns from feedback, preferences, or reward scores.

8. Data Preparation for Fine-Tuning

Fine-tuning quality depends heavily on data. The exam expects you to recognize what "good training data" means before you tune a model.

Data Preparation Checklist

Step What to do Why it matters
Curate Select relevant, high-quality examples Poor data teaches poor behavior
Clean Remove duplicates, noise, invalid records, irrelevant content Reduces confusion and overfitting
Govern Check permissions, privacy, lineage, retention, and compliance Prevents legal and security risk
Label Create correct input-output pairs where supervised tuning is used Fine-tuning needs target outputs
Balance Include representative classes, user groups, edge cases, and domains Reduces bias and brittle behavior
Size Use enough examples for the task and model requirements Too little data may underfit; too much poor data hurts
Format Match the model's required JSONL, conversation, image, or modality format Invalid format blocks training
Split Create training, validation, and test sets Enables tuning, early checks, and final evaluation
Secure Store data in approved locations with access controls and encryption Protects proprietary data
Evaluate Test baseline and tuned model on holdout data Proves the customization helped

Train / Validation / Test Split

Split Used for Exam clue
Training set Updates model weights or adapter parameters "Model learns from these examples"
Validation set Checks performance during tuning and helps detect overfitting "Holdout validation during training"
Test set Final unbiased evaluation after tuning "Final performance evaluation"

Loss, Accuracy, and Overfitting

During supervised fine-tuning, the model generates completions for training prompts. The training process compares those completions to labels, calculates loss, and updates weights or adapters to reduce that loss.

Know this:

  • Lower training loss alone is not enough.
  • Validation quality should improve with training.
  • If training improves but validation gets worse, suspect overfitting.
  • A holdout test set gives the final quality signal after tuning.

9. AWS Services to Recognize

Need AWS service or feature Exam role
Customize FMs with managed workflow Amazon Bedrock custom models Fine-tuning, reinforcement fine-tuning, distillation, custom model workflow
Prepare Bedrock training data Amazon Bedrock model customization data preparation JSONL datasets, model-specific formats, training and validation data
Fine-tune open or hosted FMs Amazon SageMaker JumpStart Fine-tune selected foundation models, including domain adaptation and instruction tuning
Low-code data preparation Amazon SageMaker Canvas Create data flows for ML preprocessing with little or no code
Scalable Spark-based data preparation AWS Glue interactive sessions Serverless Spark data transformation
Big data preparation in notebooks Amazon EMR integration with SageMaker Studio Classic Use Spark, Hive, Presto, and related frameworks
Feature discovery and storage Amazon SageMaker Feature Store Store and retrieve standardized features for model training
Bias detection in data Amazon SageMaker Clarify Detect potential bias and imbalanced representations
Label training data Amazon SageMaker Ground Truth Manage data labeling workflows
Store training data and logs Amazon S3 Common storage location for training datasets, validation datasets, and invocation logs
Reward functions for reinforcement fine-tuning AWS Lambda Score model outputs with custom code in Bedrock reinforcement fine-tuning

10. Hyperparameters and Training Controls

The exam is not asking you to tune hyperparameters deeply, but you should recognize the concepts.

Hyperparameter Meaning Exam impact
Epoch count Number of times the model processes the full training dataset More epochs process more tokens, can increase cost and overfitting risk
Batch size Number of examples processed before updating model parameters Affects training dynamics and resource use
Learning rate How much parameters are updated after each batch Too high can destabilize training; too low can train slowly
Steps Number of update iterations or exposures to batches More steps can increase cost and overfitting risk
Early stopping Stop training when validation improvement stalls Helps avoid unnecessary training and overfitting

Cost Pattern

Amazon Bedrock custom model training cost is tied to tokens processed during training, commonly thought of as training data tokens multiplied by epochs, plus model storage. Distillation can also incur teacher model inference charges when generating synthetic data.

Exam shortcut: More data, more epochs, larger models, and more generated teacher responses usually mean higher cost.

11. Catastrophic Forgetting

Catastrophic forgetting happens when fine-tuning on a narrow task improves that task but degrades the model's performance on other tasks. Skill Builder calls this out as a limitation of fine-tuning, especially when the model is tuned heavily for a single task.

How to Think About It

If the application needs... Concern level
Excellent performance on one narrow task only Lower concern
Broad general-purpose behavior after tuning Higher concern
Many task types Use diverse or multitask data; evaluate broad capabilities

Mitigations to Recognize

  • Use representative and diverse training data.
  • Include multiple tasks if broad behavior matters.
  • Evaluate against both task-specific and general benchmarks.
  • Consider parameter-efficient fine-tuning when appropriate.
  • Use RAG or prompting if changing weights is unnecessary.

12. Full Fine-Tuning vs PEFT / LoRA / ReFT

Skill Builder introduces parameter-efficient methods so you can recognize them conceptually.

Approach What changes Why it matters
Full fine-tuning All or most model parameters can be updated More compute and memory; stronger specialization risk
PEFT Most base parameters are frozen; small task-specific adapters are trained Lower compute and memory than full fine-tuning
LoRA A PEFT technique that adds trainable low-rank matrices while preserving base weights Popular efficient adaptation method
ReFT Freezes base model and learns task-specific interventions on hidden representations Adapts representations rather than all base weights

Exam shortcut: If the question mentions reducing memory and compute for fine-tuning, look for parameter-efficient fine-tuning or LoRA.

13. Choose the Right Customization Method

Scenario Best answer pattern
Need current answers from company policies with citations RAG, not fine-tuning
Need model to follow a repeated response format from examples Supervised fine-tuning or instruction tuning
Need model to understand industry jargon Domain adaptation fine-tuning or continued pre-training
Have a large unlabeled corpus of domain documents Continued pre-training
Have labeled prompt-response examples Supervised fine-tuning
Have human preference rankings RLHF
Have a reward function that can score responses Reinforcement fine-tuning
Need lower cost and latency than a large model Distillation
Need to train a brand-new FM from scratch Pre-training
Need better prompt clarity only Prompt engineering
Need private knowledge without changing model weights RAG
Need to label examples for training SageMaker Ground Truth
Need to detect bias in training data SageMaker Clarify
Need low-code preprocessing SageMaker Canvas
Need scalable Spark preprocessing AWS Glue interactive sessions or EMR with SageMaker Studio Classic

14. Common Wrong Answers

  • Use fine-tuning to keep fast-changing documents up to date.
  • Use pre-training from scratch for a narrow business task.
  • Assume a larger dataset is automatically better than a curated representative dataset.
  • Fine-tune without labeled data for a supervised task.
  • Skip validation and test sets because training loss improved.
  • Ignore governance, privacy, licensing, or data lineage before training.
  • Tune on a single narrow task and assume broad model performance will remain unchanged.
  • Use distillation when the real need is source-grounded answers.
  • Use RLHF when simple labeled examples are available and sufficient.
  • Treat prompt engineering, RAG, and fine-tuning as interchangeable.

15. Mini Flashcards

Q: What is pre-training? A: The initial large-scale training process that teaches a foundation model broad capabilities from massive data.

Q: What is fine-tuning? A: Additional training of a pre-trained model on task-specific examples to improve performance for a use case.

Q: What is continued pre-training? A: Further training a pre-trained model on additional usually unlabeled domain data.

Q: What is instruction tuning? A: Fine-tuning with instruction and response examples so the model follows instructions better.

Q: What is domain adaptation fine-tuning? A: Adapting a pre-trained model to domain-specific language, terminology, and patterns.

Q: What is transfer learning? A: Reusing a pre-trained model and adapting it to a new task instead of training from scratch.

Q: What is RLHF? A: Reinforcement learning from human feedback, used to align model outputs with human preferences.

Q: What is model distillation? A: Training a smaller student model to approximate a larger teacher model for lower cost or latency.

Q: What is catastrophic forgetting? A: A tuned model improves on the fine-tuned task but loses performance on other tasks.

Q: What are the three key data splits? A: Training, validation, and test.

Q: What does SageMaker Ground Truth help with? A: Data labeling workflows for training datasets.

Q: What does SageMaker Clarify help detect? A: Bias and imbalanced representations in data or model behavior.

Q: What is an epoch? A: One pass through the full training dataset.

Q: Why can more epochs increase cost? A: Each epoch processes the training data again, increasing tokens processed.

16. Practice Questions

  1. A company has thousands of labeled customer emails and desired response categories. It wants better classification accuracy for this stable task. What should it consider?

    • Answer: Supervised fine-tuning with labeled examples, after testing prompt engineering and baseline performance.

  2. A company needs a model to answer from policies that change weekly and include citations. What should it use?

    • Answer: RAG with a knowledge base, not fine-tuning.

  3. A legal team has a large unlabeled corpus of legal documents and wants the model to better understand legal terminology. Which method fits best?

    • Answer: Continued pre-training or domain adaptation, depending on the available training setup and data format.

  4. A large teacher model gives excellent answers, but production latency and cost are too high. Which method should the team consider?

    • Answer: Model distillation to a smaller student model.

  5. A model is fine-tuned on one narrow task and then performs worse on other general tasks. What is this called?

    • Answer: Catastrophic forgetting.

  6. A team has human reviewers rank multiple model responses by usefulness and safety. Which tuning concept does this support?

    • Answer: RLHF or reinforcement fine-tuning.

  7. Which dataset split is used for final unbiased performance measurement after training is complete?

    • Answer: Test set.

  8. Which AWS service helps manage data labeling workflows?

    • Answer: Amazon SageMaker Ground Truth.

  9. Which AWS service helps detect imbalanced representations or potential bias in training data?

    • Answer: Amazon SageMaker Clarify.

  10. What happens to cost when the number of epochs increases?

    • Answer: Training processes the dataset more times, so tokens processed and training cost can increase.

17. Last-Day Review Checklist

  • I can distinguish pre-training, continued pre-training, fine-tuning, RLHF, and distillation.
  • I know that fine-tuning changes model behavior or parameters, while prompting and RAG do not change base model weights.
  • I can explain when RAG is better than fine-tuning.
  • I can identify supervised fine-tuning from labeled input-output examples.
  • I can identify continued pre-training from unlabeled domain data.
  • I can identify distillation from teacher and student models.
  • I can identify RLHF from human feedback, rankings, preferences, or reward signals.
  • I know why data curation, governance, labeling, size, and representativeness matter.
  • I know the purpose of training, validation, and test splits.
  • I can explain catastrophic forgetting.
  • I know PEFT and LoRA reduce fine-tuning memory and compute by training fewer parameters.
  • I can map SageMaker Ground Truth to labeling and SageMaker Clarify to bias detection.
  • I can explain how epochs, batch size, learning rate, and steps affect training at a high level.

Official Sources