Principal Component Analysis
ELI5 - The Vibe Check
PCA is a way to take data with a hundred confusing, overlapping columns and boil it down to just a handful of new super-columns that capture most of what actually matters, throwing away the redundant noise. It's like discovering that height and shoe size are basically measuring the same underlying thing, so instead of tracking both, you track one combined size score that loses barely any information.
Real Talk
Principal Component Analysis is a dimensionality reduction technique that transforms a set of possibly correlated features into a smaller set of uncorrelated variables, called principal components, ordered by how much variance in the original data each one explains. It's commonly used to compress high-dimensional data before modeling, speed up training, visualize data in two or three dimensions, and reduce noise from correlated features.
Show Me The Code
from sklearn.decomposition import PCA
pca = PCA(n_components=2)
X_reduced = pca.fit_transform(X)
print(pca.explained_variance_ratio_) # how much info each component keeps
When You'll Hear This
"We ran PCA down to 20 components and kept 95% of the variance, training got way faster." / "Plot the first two principal components, it's the fastest way to eyeball whether the classes even separate."
Related Terms
Embedding
An embedding is turning words, sentences, or entire documents into lists of numbers (vectors) that capture their meaning.
Feature Engineering
Feature engineering is the unglamorous, hugely important work of turning raw, messy real-world data into the specific columns a model can actually learn fr...
K-Means
K-Means is a clustering algorithm that groups your data into a set number of buckets without ever being told what the buckets should mean.
Machine Learning (ML)
Machine Learning is teaching a computer by showing it thousands of examples instead of writing out every rule.