Lesson 6 of 13 · 11 min

The scikit-learn Workflow: Classification and Regression

Every model in scikit-learn obeys one contract, and once you internalize it the library stops being a catalog of algorithms and becomes a single composable interface. An estimator learns state from data with fit, a predictor adds predict, a classifier usually adds predict_proba, and a transformer adds transform. This lesson runs that contract end to end on two real biomedical datasets, one classification task and one regression task.

The Estimator Contract

fit(X, y) mutates the estimator in place and returns self, storing learned attributes under trailing-underscore names such as coef_ and classes_. predict(X) returns hard labels or point estimates, while predict_proba(X) returns an array of shape (n_samples, n_classes) whose columns follow the order in classes_. A Pipeline chains transformers and a final estimator so that one fit call refits every step on the training data only, which is exactly what stops preprocessing from leaking test information.

Classification: Tumor Diagnosis

python

from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
from sklearn.dummy import DummyClassifier
from sklearn.metrics import roc_auc_score, accuracy_score

X, y = load_breast_cancer(return_X_y=True)  # 569 samples, 30 features; classes_ = [0=malignant, 1=benign]
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.25, stratify=y, random_state=42
)

clf = Pipeline([
    ("scale", StandardScaler()),
    ("model", LogisticRegression(max_iter=10000, random_state=42)),
])
clf.fit(X_train, y_train)

proba = clf.predict_proba(X_test)[:, 1]   # P(class == clf.classes_[1])
pred = clf.predict(X_test)
print(f"ROC AUC:  {roc_auc_score(y_test, proba):.3f}")
print(f"Accuracy: {accuracy_score(y_test, pred):.3f}")

dummy = DummyClassifier(strategy="most_frequent").fit(X_train, y_train)
print(f"Baseline accuracy: {accuracy_score(y_test, dummy.predict(X_test)):.3f}")

ROC AUC:  0.998
Accuracy: 0.986
Baseline accuracy: 0.629

Tip: The scaler lives inside the Pipeline, so StandardScaler learns its mean and variance from X_train alone and reapplies them to X_test. Fitting a scaler on the full matrix before splitting leaks test statistics into training and silently inflates every score. Note that stratify=y preserves the malignant/benign ratio across the split, and that accuracy of 0.986 is only impressive relative to the 0.629 baseline that a DummyClassifier reaches by always guessing the majority class.

Regression: Disease Progression

python

from sklearn.datasets import load_diabetes
from sklearn.model_selection import train_test_split
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import Ridge
from sklearn.dummy import DummyRegressor
from sklearn.metrics import mean_absolute_error, r2_score

X, y = load_diabetes(return_X_y=True)  # 442 patients, 10 features; y = 1-year progression
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.25, random_state=42
)

reg = Pipeline([
    ("scale", StandardScaler()),
    ("model", Ridge(alpha=1.0)),
])
reg.fit(X_train, y_train)
pred = reg.predict(X_test)
print(f"MAE: {mean_absolute_error(y_test, pred):.1f}")
print(f"R2:  {r2_score(y_test, pred):.3f}")

base = DummyRegressor(strategy="mean").fit(X_train, y_train)
print(f"Baseline MAE: {mean_absolute_error(y_test, base.predict(X_test)):.1f}")
print(f"Baseline R2:  {r2_score(y_test, base.predict(X_test)):.3f}")

MAE: 41.5
R2:  0.486
Baseline MAE: 65.5
Baseline R2:  -0.014

The Ridge model explains roughly 49 percent of the variance in one-year disease progression and cuts mean absolute error by about 24 units versus the mean-predicting baseline, whose R2 sits near zero by construction. Fixing random_state=42 on train_test_split and on any stochastic estimator makes every number here reproducible run to run, which is the precondition for trusting a later comparison between models.

Try it yourself: Swap LogisticRegression for RandomForestClassifier(random_state=42) inside the same Pipeline and compare ROC AUC and accuracy against both the linear model and the DummyClassifier. Then change the baseline to DummyClassifier(strategy='stratified') and explain why its accuracy falls below the most_frequent baseline while still being an honest reference point.

Train/Val/Test Splits and Sequence-Similarity Leakage Logistic Regression and SVMs