How to calculate regression metrics with scikit-learn

Regression evaluation compares continuous predictions by measuring both error size and goodness of fit. In scikit-learn, regression metric functions take held-out target values and predicted values directly, so estimators from different training scripts can be checked with the same calls.

The metric inputs must describe the same samples in the same order. mean_absolute_error() and root_mean_squared_error() stay in the target unit, mean_squared_error() squares larger misses more heavily, and r2_score() compares the predictions with a baseline that always predicts the target mean.

Use test or validation rows that were not used for fitting. A built-in diabetes dataset and a small Ridge model are enough to produce reproducible predictions for a smoke check; replace that setup with the project's existing y_test and prediction array when a regression model is already trained.

1. Save the regression metric smoke script as regression_metrics_demo.py.

   regression_metrics_demo.py

   import sklearn
   from sklearn.datasets import load_diabetes
   from sklearn.linear_model import Ridge
   from sklearn.metrics import (
       mean_absolute_error,
       mean_squared_error,
       r2_score,
       root_mean_squared_error,
   )
   from sklearn.model_selection import train_test_split
    
    
   X, y = load_diabetes(return_X_y=True)
    
   X_train, X_test, y_train, y_test = train_test_split(
       X,
       y,
       test_size=0.2,
       random_state=42,
   )
    
   model = Ridge(alpha=1.0)
   model.fit(X_train, y_train)
   predictions = model.predict(X_test)
    
   mae = mean_absolute_error(y_test, predictions)
   mse = mean_squared_error(y_test, predictions)
   rmse = root_mean_squared_error(y_test, predictions)
   r2 = r2_score(y_test, predictions)
    
   print(f"scikit-learn {sklearn.__version__}")
   print(f"test rows: {len(y_test)}")
   print(f"mean absolute error: {mae:.2f}")
   print(f"mean squared error: {mse:.2f}")
   print(f"root mean squared error: {rmse:.2f}")
   print(f"r2 score: {r2:.3f}")
   print(f"first actual/predicted: {y_test[0]:.1f} / {predictions[0]:.1f}")

   The script uses load_diabetes() and Ridge only to create a reproducible test split and prediction array.
   Related: How to split data into train and test sets with scikit-learn

2. Run the script from a Python environment with scikit-learn installed.

   $ python3 regression_metrics_demo.py
   scikit-learn 1.9.0
   test rows: 89
   mean absolute error: 46.14
   mean squared error: 3077.42
   root mean squared error: 55.47
   r2 score: 0.419
   first actual/predicted: 219.0 / 157.3

3. Check that the row count and sample pair come from the held-out split.

   test rows: 89 confirms the metric calls used the test split. The first actual/predicted pair confirms that y_test and predictions contain comparable regression values.

4. Read the absolute and squared error metrics together.

   mean_absolute_error() and root_mean_squared_error() are easier to compare with the target unit. mean_squared_error() is larger because the residuals are squared before averaging.

5. Read the R2 score as the held-out goodness-of-fit value.

   r2_score() is best at 1.0, is 0.0 for a model no better than predicting the target mean, and can be negative when the predictions are worse than that baseline.

6. Reuse the metric calls with the project's held-out targets and predictions.

   mae = mean_absolute_error(y_test, y_pred)
   mse = mean_squared_error(y_test, y_pred)
   rmse = root_mean_squared_error(y_test, y_pred)
   r2 = r2_score(y_test, y_pred)

   Keep y_test and y_pred from the same split and row order. Current scikit-learn releases include root_mean_squared_error(); upgrade the environment if that import is missing.

7. Remove the smoke-test script if it was created only for validation.

   $ rm regression_metrics_demo.py