How to run cross validation with scikit-learn

Cross validation estimates how a model performs on data it did not train on by rotating the train/test split across several folds. In scikit-learn, it is a model evaluation check to run before trusting one lucky split or comparing candidate estimators.

The cross_val_score() helper fits a fresh estimator clone for each fold and returns one test score per split. Pairing it with a Pipeline keeps preprocessing inside each training fold, which avoids fitting the scaler on rows that later become test data.

A compact iris classification run uses the built-in dataset, a linear SVC classifier, and an explicit StratifiedKFold splitter. The printed mean summarizes the folds, while the standard deviation shows how much the score changes across held-out partitions.

1. Create run_cross_validation.py with a pipeline, splitter, and scoring call.

   run_cross_validation.py

   import numpy as np
   import sklearn
   from sklearn.datasets import load_iris
   from sklearn.model_selection import StratifiedKFold, cross_val_score
   from sklearn.pipeline import make_pipeline
   from sklearn.preprocessing import StandardScaler
   from sklearn.svm import SVC
    
    
   X, y = load_iris(return_X_y=True)
    
   classifier = make_pipeline(
       StandardScaler(),
       SVC(kernel="linear", C=1.0, random_state=42),
   )
    
   cv = StratifiedKFold(
       n_splits=5,
       shuffle=True,
       random_state=42,
   )
    
   metric = "accuracy"
    
   scores = cross_val_score(
       classifier,
       X,
       y,
       cv=cv,
       scoring=metric,
   )
    
   print(f"scikit-learn {sklearn.__version__}")
   print(f"Metric: {metric}")
   print(f"Folds: {len(scores)}")
   print(f"Fold {metric} scores:", np.array2string(scores, precision=3, floatmode="fixed"))
   print(f"Mean {metric}: {scores.mean():.3f}")
   print(f"Standard deviation: {scores.std():.3f}")

   StratifiedKFold keeps each class represented in every test fold for classification data. The explicit shuffle=True and random_state make the fold assignment repeatable for the same input rows.

2. Run the script.

   $ python run_cross_validation.py
   scikit-learn 1.9.0
   Metric: accuracy
   Folds: 5
   Fold accuracy scores: [1.000 1.000 0.867 1.000 0.967]
   Mean accuracy: 0.967
   Standard deviation: 0.052

3. Confirm that the fold count matches the splitter.

   The output shows Folds: 5 because n_splits=5 created five train/test rotations. A nan score or fewer values in the score array means at least one fold failed or the script did not run the configured splitter.

4. Confirm that the reported metric matches the comparison being made.

   The script sets metric = “accuracy” and prints Metric: accuracy before the fold scores. Change that single value to another supported scorer, such as f1_macro, before rerunning when accuracy is not the model objective.

5. Read the mean and standard deviation together before comparing model runs.

   In this run, mean accuracy is 0.967 and the standard deviation is 0.052. A larger standard deviation means the estimate depends more on which rows land in each held-out fold.