AI/ML Interview Preparation Notes

Deep Learning and Machine Learning

From learning paradigms and backpropagation to loss functions, regularization, optimizers, classic ML algorithms, metrics, and ML at production scale.

Study Guide

Learning Paradigms: Active, Weak, Semi- & Self-Supervised

Reducing labeling cost with active learning, weak supervision, semi-supervised and self-supervised learning.

How Neural Networks Learn & the DL Training Process

Forward and backward propagation, the training loop, activation functions, monotonic and convex functions.

Normalization & Regression Summary

Batch/layer normalization and a compact summary of regression models.

Loss Functions

Loss-function theory plus the common losses used across regression and classification.

Regularization: L1, L2 & Beyond

L1/L2 regularization, regression variants with regularization, dropout and other techniques.

Optimization & Learning-Rate Schedules

Optimizers from SGD to Adam, and learning-rate schedules.

MLE vs. Bayesian Estimation

Maximum-likelihood vs. Bayesian views of parameter estimation.

DL Basics & CNN Essentials

Deep-learning theory fundamentals and convolutional networks.

DNN Vectorization

Vectorizing one and multiple training examples in a deep network.

RNN, LSTM, GRU & Beam Search

Recurrent architectures, gated units, and beam-search decoding.

Bias–Variance, Ensembles & Model Drift

The bias–variance trade-off, boosting and bagging refreshers, and detecting/handling model drift.

Metrics

Classification and regression metrics, ROC/AUC, and when to use which.

Miscellaneous: Determinism & Distance Measures

Deterministic vs. stochastic processes and Euclidean vs. cosine distance.

Classic ML: Naïve Bayes

The Naïve Bayes family, assumptions, and use cases.

Linear & Logistic Regression, OLS & Gradient Descent

Linear regression, ordinary least squares, gradient-descent variants, and logistic regression.

KNN, SVM, Random Forest & KL Divergence

Distance-based and margin-based classifiers, tree ensembles, and KL divergence.

Clustering

K-means and friends: algorithms, initialization, and evaluation.

Dimensionality Reduction: PCA, LDA & SVD

PCA, LDA, SVD, and how PCA relates to SVD.

A/B Testing

Experiment design, hypothesis testing, and pitfalls.

DL Math: Gradients & Derivatives

The calculus behind backprop: gradients, Jacobians, and chain rule.

Model Interpretability

Interpreting models: feature importance, SHAP-style reasoning, and trade-offs.

Machine Learning at Facebook

How ML systems are organized and shipped at Facebook scale.

UNIX, ML at Scale & Causal Analysis

Practical UNIX, distributed ML systems design, and causal analysis.

Full Topic Map

The original quick-navigation index — every link below jumps straight to the relevant topic.

TOC

Active Learning, Weak supervision, Semi-Supervised Learning, Self-Supervised Learning
How do neural networks learn (forward, backward propagation)
DL Training Process, Activation Functions, Monotonic Func, Convex & Concave Func
Normalization, LR Schedules
Loss Functions (Theory), Common Loss Functions
L1 and L2 Regularization, Regressions summary w/L1 and L2 regularization
Other Forms of Regularization
Optimization (Optimizers)
MLE vs. Bayesian Estimation
DL Basics (Theory), CNN Essentials
DNN Vectorizing One and Multiple Examples
RNN, LSTM, GRU
Beam search
Bias Variance Trade Off
ML Aspects To Refresh (Boosting, Bagging)
Model drift
Metrics
Miscellaneous (deterministic, stochastic)
Euclidean vs. Cosine Distance
Classic ML Algorithms - Naïve Bayes
Linear Regression, Ordinary Least Square Method (OLS), Types of Gradient Descent
Logistic regression
KNN, SVM, Random Forest, KL Divergence
Clustering
DIMENSIONALITY REDUCTION (PCA, LDA), SVD, PCA vs. SVD
A/B Testing
DL Math (gradients, derivatives)
Model Interpretability
Machine Learning At Facebook
UNIX, ML At Scale, Causal Analysis

Python notebooks covering the entire text of “Understanding Deep Learning” - multiple practical examples in code! (book downloaded in Recent ML Downloads on my laptop). Another link here.