How to run randomized search with scikit-learn

Randomized search samples a fixed number of hyperparameter settings instead of evaluating every combination in a large grid. In scikit-learn, RandomizedSearchCV is a focused way to tune a model when continuous ranges such as C or gamma would make exhaustive search too expensive.

An SVC classifier can sit inside a Pipeline so scaling is fitted separately within each cross-validation fold. RandomizedSearchCV receives a parameter distribution, a fold splitter, a scoring metric, and a sample budget through n_iter.

A small Iris train/test split keeps the run fast while still proving that the search object can sample candidates, choose the best cross-validation score, refit the selected pipeline, and score held-out rows. Increase n_iter only after the candidate ranges and scoring metric match the project model.

1. Save the randomized search script as run_randomized_search.py.

   run_randomized_search.py

   from scipy.stats import loguniform
   import sklearn
   from sklearn.datasets import load_iris
   from sklearn.metrics import accuracy_score
   from sklearn.model_selection import RandomizedSearchCV, StratifiedKFold, train_test_split
   from sklearn.pipeline import make_pipeline
   from sklearn.preprocessing import StandardScaler
   from sklearn.svm import SVC
    
    
   iris = load_iris()
   X_train, X_test, y_train, y_test = train_test_split(
       iris.data,
       iris.target,
       test_size=0.25,
       stratify=iris.target,
       random_state=42,
   )
    
   model = make_pipeline(
       StandardScaler(),
       SVC(),
   )
    
   param_distributions = {
       "svc__C": loguniform(1e-2, 1e2),
       "svc__gamma": loguniform(1e-4, 1e0),
       "svc__kernel": ["rbf"],
   }
    
   cv = StratifiedKFold(
       n_splits=5,
       shuffle=True,
       random_state=42,
   )
    
   search = RandomizedSearchCV(
       estimator=model,
       param_distributions=param_distributions,
       n_iter=8,
       scoring="accuracy",
       cv=cv,
       random_state=42,
       refit=True,
   )
   search.fit(X_train, y_train)
    
   best_params = search.best_params_
   best_params_display = {
       "svc__C": round(float(best_params["svc__C"]), 4),
       "svc__gamma": round(float(best_params["svc__gamma"]), 4),
       "svc__kernel": best_params["svc__kernel"],
   }
    
   test_predictions = search.predict(X_test)
   test_accuracy = accuracy_score(y_test, test_predictions)
    
   print(f"scikit-learn {sklearn.__version__}")
   print(f"candidate settings tried: {len(search.cv_results_['params'])}")
   print(f"cross-validation folds: {search.n_splits_}")
   print(f"best params: {best_params_display}")
   print(f"best CV accuracy: {search.best_score_:.3f}")
   print(f"held-out accuracy: {test_accuracy:.3f}")

   loguniform() samples C and gamma across orders of magnitude. The svc__ prefix targets the automatically named SVC step inside make_pipeline().

2. Run the script from a Python environment that has scikit-learn installed.

   $ python3 run_randomized_search.py
   scikit-learn 1.9.0
   candidate settings tried: 8
   cross-validation folds: 5
   best params: {'svc__C': 0.3149, 'svc__gamma': 0.6351, 'svc__kernel': 'rbf'}
   best CV accuracy: 0.965
   held-out accuracy: 0.921

3. Confirm that the sampled candidate count matches n_iter.

   candidate settings tried: 8 matches n_iter=8. cross-validation folds: 5 matches StratifiedKFold(n_splits=5).

4. Read the selected parameter values and cross-validation score.

   best_params_ identifies the sampled setting selected by cross-validation. best_score_ is the mean cross-validation score for that setting, using the configured accuracy scorer.

5. Check the held-out score before reusing the refit pipeline.

   refit=True fits the selected pipeline on the full training split after the search. The held-out accuracy: 0.921 line comes from search.predict(X_test), so it checks rows outside the randomized search fit.

6. Replace the dataset, estimator, and parameter distributions after the smoke run works.

   Use pipeline parameter names such as step__parameter for nested estimators, keep random_state while comparing candidate ranges, and raise n_iter when a larger search budget is worth the extra cross-validation fits.

7. Remove the smoke-test script if it was created only for the check.

   $ rm run_randomized_search.py