How to reduce dimensions with PCA in scikit-learn

High-dimensional feature matrices can be hard to inspect, plot, or pass into later learning steps without compressing the columns first. PCA in scikit-learn projects numeric features onto a smaller set of principal components, keeping the directions that explain the most variance in the fitted data.

The smoke test uses the built-in digits dataset because it starts with 64 numeric pixel features and makes the shape change easy to verify. StandardScaler runs before PCA so each feature has comparable scale; PCA centers input before singular value decomposition, but it does not scale each feature for you.

A two-component projection is useful for quick visualization and dimensionality checks, not as proof that two columns keep enough signal for every model. When reduced features feed supervised evaluation, fit scaling and PCA only on training rows, or keep both transformers inside a Pipeline so validation or test rows are transformed without being used to learn the projection.

Related: How to standardize features with scikit-learn
Related: How to create a scikit-learn pipeline

Steps to reduce dimensions with scikit-learn PCA:

  1. Save the PCA smoke script as reduce_dimensions_pca.py.
    reduce_dimensions_pca.py
    from sklearn.datasets import load_digits
from sklearn.decomposition import PCA
from sklearn.preprocessing import StandardScaler
 
X, _ = load_digits(return_X_y=True)
 
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)
 
pca = PCA(n_components=2)
X_reduced = pca.fit_transform(X_scaled)
 
print(f"original shape: {X.shape}")
print(f"scaled shape: {X_scaled.shape}")
print(f"reduced shape: {X_reduced.shape}")
print(f"components kept: {pca.n_components_}")
print(f"explained variance ratio: {pca.explained_variance_ratio_.round(4).tolist()}")
print(f"total explained variance: {pca.explained_variance_ratio_.sum():.4f}")
print(f"first reduced row: {X_reduced[0].round(3).tolist()}")

    load_digits() returns 1,797 samples with 64 numeric features, so the two-column PCA output is easy to check.

  2. Run the script from a Python environment that has scikit-learn installed. 
    $ python3 reduce_dimensions_pca.py
original shape: (1797, 64)
scaled shape: (1797, 64)
reduced shape: (1797, 2)
components kept: 2
explained variance ratio: [0.1203, 0.0956]
total explained variance: 0.2159
first reduced row: [-1.914, -0.955]
  3. Check the shape lines before using the reduced matrix.

    original shape and scaled shape should both show 64 columns. reduced shape should show 2 columns because n_components=2 was used.

  4. Check the explained variance lines.

    The two ratio values show how much variance each retained component explains in the fitted, scaled data. The total is low in this smoke test because two components compress 64 digit-pixel features aggressively.

  5. Replace the built-in dataset with your own numeric feature matrix after the smoke test works. 
    X, _ = load_digits(return_X_y=True)

    Use PCA with dense numeric features. For sparse text or count matrices, consider TruncatedSVD because PCA centers data before decomposition.

  6. Keep scaling and PCA together when the reduced features feed a model. 
    from sklearn.pipeline import make_pipeline
 
projector = make_pipeline(
    StandardScaler(),
    PCA(n_components=2),
)

    Fit the pipeline on training rows, then call transform() on validation or test rows so preprocessing statistics do not leak across the split.

  7. Remove the sample script if it was created only for the smoke test. 
    $ rm reduce_dimensions_pca.py