How to plot an ROC curve with scikit-learn

ROC curves show how a binary classifier trades true positive rate against false positive rate across score thresholds. In scikit-learn, RocCurveDisplay can draw that curve from a fitted estimator and held-out labels, while also exposing the area under the curve as ROC AUC.

RocCurveDisplay.from_estimator() uses the estimator's predict_proba() output when it is available and falls back to decision_function() through the default response_method=“auto” behavior. The positive class controls which score column becomes the curve, so the smoke script converts malignant cancer rows to label 1 before plotting.

Use ROC and AUC on held-out data instead of rows used for fitting. The curve is threshold-independent, so it complements thresholded checks such as a confusion matrix, a classification report, or a tuned decision threshold.

1. Install scikit-learn and matplotlib in the active Python environment.

   $ python3 -m pip install --upgrade scikit-learn matplotlib

   matplotlib is required because RocCurveDisplay renders the curve through a plotting axis.

2. Save the ROC plotting script as plot_roc_curve.py.

   plot_roc_curve.py

   import matplotlib
    
   matplotlib.use("Agg")
    
   import matplotlib.pyplot as plt
   from sklearn.datasets import load_breast_cancer
   from sklearn.linear_model import LogisticRegression
   from sklearn.metrics import RocCurveDisplay
   from sklearn.model_selection import train_test_split
   from sklearn.pipeline import make_pipeline
   from sklearn.preprocessing import StandardScaler
    
    
   cancer = load_breast_cancer()
   X = cancer.data
   y = (cancer.target_names[cancer.target] == "malignant").astype(int)
    
   X_train, X_test, y_train, y_test = train_test_split(
       X,
       y,
       test_size=0.25,
       stratify=y,
       random_state=42,
   )
    
   model = make_pipeline(
       StandardScaler(),
       LogisticRegression(max_iter=1000),
   )
   model.fit(X_train, y_train)
    
   display = RocCurveDisplay.from_estimator(
       model,
       X_test,
       y_test,
       name="Logistic regression",
       pos_label=1,
       plot_chance_level=True,
       despine=True,
   )
   display.ax_.set_title("ROC curve for malignant cancer class")
   display.figure_.set_size_inches(7, 5)
   display.figure_.tight_layout()
   display.figure_.savefig("roc-curve-plot.png", dpi=160)
   plt.close(display.figure_)
    
   print(f"train rows: {X_train.shape[0]}")
   print(f"test rows: {X_test.shape[0]}")
   print("positive class: malignant")
   print(f"roc auc: {display.roc_auc:.3f}")
   print(f"curve points: {len(display.fpr)}")
   print("plot saved: roc-curve-plot.png")

   matplotlib.use(“Agg”) lets the script save a PNG in shells, containers, and scheduled jobs that do not have an interactive display.

3. Run the script.

   $ python3 plot_roc_curve.py
   train rows: 426
   test rows: 143
   positive class: malignant
   roc auc: 0.996
   curve points: 10
   plot saved: roc-curve-plot.png

4. Confirm that the positive class matches the label being evaluated.

   The script prints positive class: malignant because label 1 was assigned to malignant rows before fitting. Set pos_label to the class that should count as positive in the project data.

5. Check the AUC and saved plot output.

   roc auc: 0.996 shows strong ranking on the held-out split, and plot saved: roc-curve-plot.png confirms that the figure was written in the current directory.

6. Reuse the display call with the project's fitted classifier and validation data.

   display = RocCurveDisplay.from_estimator(
       fitted_classifier,
       X_validation,
       y_validation,
       pos_label=positive_label,
       plot_chance_level=True,
   )

   Keep X_validation and y_validation from the same held-out split, and choose pos_label before comparing AUC values across classifiers.

7. Remove the smoke script if it was created only to check the plotting workflow.

   $ rm plot_roc_curve.py