Correlation basis or covariance basis
The two bases answer different questions. See both bases on one dataset.
PCA on the correlation basis is the eigendecomposition of the matrix that the correlation coefficient calculator computes one pair at a time. For two columns, use that calculator, and for one column at a time use descriptive statistics. PCA only says something once the columns are read together.
Vertical distance or perpendicular distance
Regression asks a directional question and minimizes vertical distance to the line. PCA asks a symmetric one and minimizes perpendicular distance. The two lines differ even on the same two columns.
Eigenvectors and loadings are two different columns
An eigenvector is a unit direction. A loading multiplies that direction by the square root of its eigenvalue.
The sign of a component is arbitrary
Flipping one component changes neither its eigenvalue nor the geometry.
How many components to keep
Parallel analysis compares each observed eigenvalue with a same-shape random-data band. Kaiser, cumulative variance and broken stick are shown as alternative rules.
Reading the biplot
The equal-aspect frame preserves the angles between arrows. Under correlation scaling, arrow inner products approximate correlations.
Fewer observations than variables
Centering leaves at most min(n - 1, p) components.
Reproduce this in R and in Python
fit <- prcomp(X, center = TRUE, scale. = TRUE)
fit = PCA().fit(Z)
Assumptions, limits, and privacy
PCA describes variance structure. It does not identify causes. Recalculation happens in your browser and requires JavaScript.