How to reduce dimensions with PCA in scikit-learn

Principal component analysis trades exact reconstruction for a smaller set of orthogonal features. scikit-learn can select the minimum number of components that reaches a retained-variance target, so the reduced width follows an explicit information rule instead of an arbitrary column count.

The digits dataset contains 1,797 samples with 64 pixel features whose active standard deviations span different scales. PCA centers input values but does not scale each feature, so StandardScaler gives active features unit variance before the decomposition.

A 90% retained-variance rule selects 31 components for the scaled digits matrix and retains about 90.05% of its variance. Applying inverse_transform() exposes the tradeoff as reconstruction error in standardized feature units; a downstream supervised model still needs scaling and PCA fitted only on training rows.

1. Build the raw active-feature profile at the start of reduce_dimensions_pca.py.

   reduce_dimensions_pca.py

   import numpy as np
   from sklearn.datasets import load_digits
   from sklearn.decomposition import PCA
   from sklearn.preprocessing import StandardScaler
    
   X, _ = load_digits(return_X_y=True)
   active_std = X.std(axis=0)
   active_std = active_std[active_std > 0]
    
   print(f"original shape: {X.shape}")
   print(
       "raw active-feature std range: "
       f"{active_std.min():.4f} to {active_std.max():.4f}"
   )

   Constant pixel columns are excluded only from the spread calculation; they remain in X so the PCA input keeps its original 64-column shape.

2. Execute the profiling block to inspect the input shape and active-feature scale range.

   $ python3 reduce_dimensions_pca.py
   original shape: (1797, 64)
   raw active-feature std range: 0.0236 to 6.5361

3. Standardize the feature matrix immediately after the profiling output.

   scaler = StandardScaler()
   X_scaled = scaler.fit_transform(X)

4. Profile the standardized active features immediately after the scaling block.

   scaled_active_std = X_scaled.std(axis=0)
   scaled_active_std = scaled_active_std[scaled_active_std > 0]

5. Set a 90% retained-variance rule below the scaling block.

   target_variance = 0.90
   pca = PCA(n_components=target_variance, svd_solver="full")

   With svd_solver=“full”, a float between 0 and 1 makes PCA select the smallest component count whose cumulative explained variance exceeds that value.

6. Apply fit_transform() to the standardized values below the component rule.

   X_reduced = pca.fit_transform(X_scaled)

7. Reconstruct the standardized matrix from the retained components below the transformation.

   X_reconstructed = pca.inverse_transform(X_reduced)

8. Calculate the retained variance below the reconstruction.

   retained_variance = pca.explained_variance_ratio_.sum()

9. Calculate the reconstruction RMSE below the retained variance.

   reconstruction_rmse = np.sqrt(np.mean((X_scaled - X_reconstructed) ** 2))

10. Add fail-capable guards for PCA row preservation, width reduction, variance retention, and finite reconstruction error.

    if X_reduced.shape[0] != X.shape[0]:
        raise RuntimeError("PCA changed the sample count")
    if X_reduced.shape[1] >= X.shape[1]:
        raise RuntimeError("PCA did not reduce the feature count")
    if retained_variance < target_variance:
        raise RuntimeError("PCA missed the retained-variance target")
    if not np.isfinite(reconstruction_rmse):
        raise RuntimeError("PCA reconstruction error is not finite")

11. Add the final PCA report below the validation guards.

    print(
        "scaled active-feature std range: "
        f"{scaled_active_std.min():.4f} to {scaled_active_std.max():.4f}"
    )
    print(f"variance target: {target_variance:.4f}")
    print(f"selected components: {pca.n_components_}")
    print(f"retained variance: {retained_variance:.4f}")
    print(f"reduced shape: {X_reduced.shape}")
    print(f"reconstruction RMSE: {reconstruction_rmse:.4f}")

12. Run the completed PCA analysis from the file directory.

    $ python3 reduce_dimensions_pca.py
    original shape: (1797, 64)
    raw active-feature std range: 0.0236 to 6.5361
    scaled active-feature std range: 1.0000 to 1.0000
    variance target: 0.9000
    selected components: 31
    retained variance: 0.9005
    reduced shape: (1797, 31)
    reconstruction RMSE: 0.3080

13. Compare the reported retained variance with the 0.9000 target.

    The fitted model retained 0.9005, so 31 components satisfy the selected rule while reducing the original 64-feature width.

14. Confirm the reduced shape preserves all 1,797 samples in 31 component columns.
15. Confirm the reconstruction RMSE is a finite 0.3080 in standardized feature units.

    A smaller reconstruction error means the retained components reproduce the scaled matrix more closely. Reconstruction error does not measure classification accuracy; a downstream estimator needs its own evaluation.