Scikit-learn's model-selection tools clone and score estimators through a fixed contract, and a compiled Keras model does not follow it. The SKLearnClassifier wrapper in current Keras 3 releases adds that contract, so a Keras classifier can return cross_val_score fold scores and let GridSearchCV choose the parameters for a fitted estimator that predicts held-out samples.
Searching layer sizes or adapting preprocessing inside each fold needs a function that builds and compiles the model rather than a finished network. Each fit call passes that function the current training features and one-hot encoded labels, so every fold starts from new weights and adapts its preprocessing to its own training rows.
Search parameters reach Keras through two dictionaries, model_kwargs for the builder and fit_kwargs for model.fit(), and a grid replaces each dictionary whole. Training settings need particular care because scikit-learn clones the estimator before every fold, and the Keras wrapper does not carry a constructor fit_kwargs value into that clone.
Steps to use a Keras model with scikit-learn:
- Install Keras, JAX, and scikit-learn in your project's virtual environment.
$ python -m pip install keras jax scikit-learn
Tested with Keras 3.15.1, JAX 0.12.0.1, and scikit-learn 1.9.1 on Python 3.12 on Linux. JAX 0.12.0.1 requires Python 3.12 or newer. It runs on the CPU here and keeps the backend install small.
Related: How to install Keras with pip - Create a file named
wine_classifier.pyin your project directory using your text editor. - Enter the following imports, backend selection, and data split.
- wine_classifier.py
import os os.environ["KERAS_BACKEND"] = "jax" import keras from keras.wrappers import SKLearnClassifier from sklearn.datasets import load_wine from sklearn.model_selection import ( GridSearchCV, cross_val_score, train_test_split ) keras.utils.set_random_seed(17) X, y = load_wine(return_X_y=True) X_train, X_heldout, y_train, y_heldout = train_test_split( X, y, test_size=0.2, stratify=y, random_state=17 )
The backend setting must come before the first
kerasimport.load_wineships with scikit-learn and holds 178 samples of 13 numeric features across three classes. Cross-validation and grid search both use onlyX_train, which keepsX_heldoutunseen until the final prediction.
Related: How to set the Keras backend - Add the model-building function at the end of
wine_classifier.py.def build_model(X, y, units=16): normalizer = keras.layers.Normalization() normalizer.adapt(X) model = keras.Sequential([ keras.Input(shape=(X.shape[1],)), normalizer, keras.layers.Dense(units, activation="relu"), keras.layers.Dense(y.shape[1], activation="softmax"), ]) model.compile( optimizer="adam", loss="categorical_crossentropy", ) return model
The wrapper calls this function with
Xandyas keyword arguments, so a builder without those parameters raisesTypeError.yarrives one-hot encoded, which makesy.shape[1]the class count and requires a categorical loss; a sparse loss raisesValueErrorhere. AdaptingNormalizationinside the builder computes feature scaling from each fold's training rows only. - Add the wrapper and the cross-validation call at the end of
wine_classifier.py.classifier = SKLearnClassifier( model=build_model, model_kwargs={"units": 16}, ) fit_settings = {"epochs": 30, "batch_size": 16, "verbose": 0} scores = cross_val_score( classifier, X_train, y_train, cv=5, params=fit_settings ) print("fold scores:", scores.round(3)) print(f"mean accuracy: {scores.mean():.3f}")
Do not rely on a constructor
fit_kwargsvalue here. In Keras 3.15.1 the clone made for each fold drops it, so every fold trains for a single epoch without an error and scores far lower. Theparamsargument passes the settings to each fold'sfit()call instead.A compiled
Modelinstance is also accepted, but the wrapper rebuilds it withclone_modelbefore fitting. Trained weights and adaptedNormalizationvalues are lost, andmodel_kwargshas no effect.
Related: Keras scikit-learn wrappers - Save
wine_classifier.py. - Run the script to collect the fold scores.
$ python wine_classifier.py 1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 85ms/step 1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 195ms/step 1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 70ms/step 1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 225ms/step 1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 201ms/step fold scores: [1. 0.862 0.964 1. 0.893] mean accuracy: 0.944
Each progress line is one fold's
predict()call, and the wrapper'spredict()takes noverboseargument. Five folds give five scores between 0 and 1, and they are accuracies becauseSKLearnClassifierinherits itsscore()method from scikit-learn. The fixed seed repeated these scores between runs here, but exact values can differ on another machine or package version. - Open
wine_classifier.pyin your text editor. - Add the parameter grid and grid search at the end of
wine_classifier.py.param_grid = { "model_kwargs": [{"units": 4}, {"units": 32}], "fit_kwargs": [ {"epochs": 5, "batch_size": 16, "verbose": 0}, {"epochs": 30, "batch_size": 16, "verbose": 0}, ], } search = GridSearchCV(classifier, param_grid, cv=5) search.fit(X_train, y_train)
Each grid value is a whole dictionary because
model_kwargsandfit_kwargsare plain dictionaries. A nested key such asmodel_kwargs__unitsraisesAttributeError. Search values are applied after cloning, sofit_kwargssurvives here. Four combinations across five folds plus one refit trains 21 models.
Related: scikit-learn grid search - Add the search report and held-out prediction at the end of
wine_classifier.py.best = search.best_estimator_ best_params = search.best_params_ print("best units:", best_params["model_kwargs"]["units"]) print("best epochs:", best_params["fit_kwargs"]["epochs"]) print(f"best cv accuracy: {search.best_score_:.3f}") print("refit epochs:", len(best.history_.history["loss"])) predictions = search.predict(X_heldout) print("predicted:", predictions[:10]) print("actual: ", y_heldout[:10]) accuracy = search.score(X_heldout, y_heldout) print(f"held-out accuracy: {accuracy:.3f}")
GridSearchCVrefits the best combination on all ofX_train, andsearch.predict()uses that refit estimator.best.model_holds the fitted Keras model, andbest.history_holds its training history. - Save
wine_classifier.py. - Run the script to select the parameters and predict the held-out samples.
$ python wine_classifier.py 1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 58ms/step ##### snipped ##### fold scores: [1. 0.862 0.964 1. 0.893] mean accuracy: 0.944 1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 83ms/step ##### snipped ##### best units: 32 best epochs: 30 best cv accuracy: 0.958 refit epochs: 30 2/2 ━━━━━━━━━━━━━━━━━━━━ 1s 273ms/step predicted: [1 0 0 1 1 0 1 0 0 1] actual: [1 0 0 1 1 0 1 0 0 1] 2/2 ━━━━━━━━━━━━━━━━━━━━ 0s 443us/step held-out accuracy: 1.000
refit epochsmatchesbest epochs, which shows that the search setting reachedmodel.fit().best cv accuracyis the winning combination's mean validation-fold score, which the search also used to pick it. Onlyheld-out accuracycomes from rows that neither the search nor the refit saw. The complete run took less than a minute on two CPU cores. - Confirm that the
predictedlabels match theactuallabels for the first ten held-out samples.predictedholds class labels from the originalyrather than one-hot vectors, so the two rows compare directly. Aheld-out accuracynear 0.33 means the model is guessing among the three classes; training on shuffled labels scored 0.222 here.