How to calculate regression metrics with scikit-learn

Model predictions become actionable only when their misses are measured on values the model did not fit. scikit-learn exposes regression losses and the R2 score as functions that compare paired ground-truth and predicted arrays, so the same evaluation works regardless of which regressor produced the predictions.

Mean absolute error (MAE) and root mean squared error (RMSE) retain the target unit, while mean squared error (MSE) squares each residual and therefore gives larger errors more influence. R2 measures improvement over always predicting the mean target value; 1.0 is perfect, 0.0 matches that mean baseline, and a negative value is worse.

The ground-truth and prediction arrays must contain the same samples in the same order. Use validation or test targets that were excluded from fitting, because metrics calculated on training predictions usually overstate how well the model handles unseen data.

Related: How to split data into train and test sets with scikit-learn
Related: How to train linear regression with scikit-learn

Steps to calculate scikit-learn regression metrics:

  1. Create regression_metrics.py with the metric imports and paired held-out values.
    regression_metrics.py
    from sklearn.metrics import (
    mean_absolute_error,
    mean_squared_error,
    r2_score,
    root_mean_squared_error,
)
 
y_true = [100, 120, 140, 160, 180]
y_pred = [110, 115, 145, 150, 190]

    The sample y_true and y_pred arrays represent project-held-out targets and predictions in matching sample order.

  2. Add the four regression metric calculations below the prediction array. 
    mae = mean_absolute_error(y_true, y_pred)
mse = mean_squared_error(y_true, y_pred)
rmse = root_mean_squared_error(y_true, y_pred)
r2 = r2_score(y_true, y_pred)
  3. Append labeled output formatting below the metric calculations. 
    print(f"MAE: {mae:.2f}")
print(f"MSE: {mse:.2f}")
print(f"RMSE: {rmse:.2f}")
print(f"R2: {r2:.4f}")
  4. Run the completed regression metric script. 
    $ python3 regression_metrics.py
MAE: 8.00
MSE: 70.00
RMSE: 8.37
R2: 0.9125

    For one target definition and data split, lower MAE, MSE, and RMSE values indicate smaller errors, while a higher R2 indicates more variance explained.