GenAI and LLMs Summary · Andrew Nedilko

Study Guide

Part 01

LLM Training: Pretraining, SFT & RLHF

Unsupervised pretraining, supervised fine-tuning, RLHF, rejection sampling, PPO, and the advantage function.

Part 02

Policy Optimization: GRPO, DPO & Reasoning Models

GRPO for reasoning LLMs, DPO/IPO/KTO, PPO vs. DPO, ORPO, and SimPO.

Part 03

Advanced Training: Efficiency & Parallelism

Imitation and proxy learning, continual pretraining, RLAIF, FlashAttention, gradient checkpointing, mixed precision, distributed parallelism, and ZeRO.

Part 04

Mixture of Experts & LLM Use Cases

MoE and Mixture of Agents, plus what trained LLMs are actually used for in the enterprise.

Part 05

PEFT, Quantization & Pruning

LoRA, QLoRA, DoRA, GaLore, quantization schemes, and pruning.

Part 06

RAG & Agents: Quick Overview

Condensed cross-reference notes on RAG and agents from the LLM summary.

Part 07

Prompt Engineering

Prompting techniques, Google's prompt-engineering guidance, verbalized sampling, RAT, and RankPrompt.

Part 08

Temperature, Function Calling & LSTMs

Sampling temperature, function calling, and LSTMs vs. transformers.

Part 09

Embeddings

Embeddings in deep learning, embedding quantization and truncation, Word2Vec and GloVe.

Part 10

Sampling & Decoding Strategies

Deterministic vs. stochastic decoding, concrete examples, library integration, and when to use each.

Part 11

Transformer Architecture

The transformer stack, input representation, subword tokenization, positional embeddings (absolute, relative, RoPE), residual connections, and Transformer-XL.

Part 12

GPT vs. BERT & Encoder Models

GPT vs. BERT, base vs. large BERT, fine-tuning BERT, inference in masked language models, symbolic logic, and BERT varieties.

Part 13

Attention Mechanisms

Attention explained end-to-end: self-attention, cross-attention, visual walkthroughs, parallelism, and long-range dependencies.

Part 14

Vision & Multimodal Models

Image and video models, multimodal architectures, and VL-JEPA.

Part 15

Responsible AI & Guardrails

Responsible AI principles and practical guardrails for LLM and agent systems.

Full Topic Map

The original quick-navigation index — every link below jumps straight to the relevant topic.

LLM TRAINING	Unsupervised learning, SFT, RLFH, Rejection sampling, RFT (rejection sampling fine-tuning), PPO (Proximal Policy Optimization), 2 PPO methods, ADVANTAGE
Policy Optimization	GRPO (Reasoning LLMs), DPO, IPO, KTO, PPO vs. DPO, ORPO, SimPO
LLM TRAINING 2	Reasoning models, Imitation Learning, Proxy Learning, Continual Pretraining, RLAIF, OAIF (Online AI Feedback), ZeRO, Flash attention, Gradient Checkpointing, Mixed Precision Training, Distributed Parallelism, Trained LLM Uses, LLM Enterprise Use Cases
MOE	Mixture of Experts, Mixture of Agents
PEFT	PEFT, QLORA, DORA, Galore, Quantization, Pruning
PROMPTING	Prompt Engineering, Google Prompt Engineering, Prompt Guidelines, Verbalized sampling, RAT (Retrieval Augmented Thoughts) & RankPrompt
MISC	Temperature, Func Calling
LSTMs	LSTM vs. Transformers
EMBEDDINGS	Embeddings in DL, Embedding Quantization & Truncation, Word2Vec, Glove
SAMPLING	Sampling - Deterministic, Stochastic, Examples, How Integrated in Libraries When use stochastic, When use deterministic
TRANSFORMERS	Transformers stack, Input Representation, Subword Tokenization, Positional Embeddings (Absolute (Fixed & Learned) and Relative), RoPE Positional Embeddings, Residual Connections, Transformer-XL, Brief Vision Transformer GTP vs. BERT, Base vs. Large BERT, Fine-Tune BERT, Inference in MLM, Symbolic Logic (Wolfram), BERT Varieties
ATTENTION	Attention Explained, Self-Attention, Cross-Attention, Attention Figures, Parallel Processing & Long-Range Dependencies
VISION	Image and Vision Models, Multimodal models, VL-JEPA
RESPONSIBLE AI	Responsible AI, Guardrails