Framing a Biological Problem as ML

A biological question becomes a machine learning task only after you fix four things: the unit each prediction is made about, the target label attached to that unit, the task type, and the metric that decides success. Skip any one and you get a model that trains cleanly yet answers the wrong question. This lesson works a concrete variant-effect example and marks where the biology, not implementation convenience, must drive the choice.

Pin the prediction unit

The unit of prediction is the row of your design matrix, and choosing it wrong is the most expensive early mistake: a pathogenicity model can predict per variant, per gene, or per patient, and a splicing model per read, per junction, or per transcript. The unit dictates what a train and test split must separate, because if the unit is the variant while two variants share a gene, a random split leaks gene-level signal and inflates every downstream number. Group splits by the biological entity that would be unseen at deployment, for example GroupKFold on gene or a held-out chromosome.

import pandas as pd

# One row per variant: the variant is the unit of prediction.
variants = pd.DataFrame({
    "variant_id": ["chr1:12345:A>G", "chr7:55019:T>C", "chr17:7579:G>A"],
    "cadd": [3.1, 24.5, 32.0],
    "gnomad_af": [0.12, 3e-4, 1e-5],
    "clinvar_sig": ["Benign", "Pathogenic", "Pathogenic"],
    "review_status": ["single_submitter", "reviewed_by_expert_panel", "single_submitter"],
})

# Target label: binary pathogenicity derived from the ClinVar assertion.
variants["y"] = (variants["clinvar_sig"] == "Pathogenic").astype(int)

# A confidence weight the biology gives us for free: not all labels are equal.
weight = {"reviewed_by_expert_panel": 1.0, "single_submitter": 0.3}
variants["label_weight"] = variants["review_status"].map(weight)

print(variants[["variant_id", "y", "label_weight"]].to_string(index=False))

    variant_id  y  label_weight
chr1:12345:A>G  0           0.3
chr7:55019:T>C  1           1.0
chr17:7579:G>A  1           0.3

Task type and metric

The same labels support different task types, and the biology decides which one is well posed: a yes or no clinical call is binary classification, a continuous MAVE functional score or expression level is regression, and shortlisting candidates for the wet lab is ranking. Ranking does not need a calibrated 0.5 threshold; its operating point is the assay budget, so it is judged by precision at k or PR-AUC rather than accuracy. Fix the task type before modeling, because both the loss function and the reported metric change with it.

import numpy as np
from sklearn.metrics import roc_auc_score, average_precision_score

rng = np.random.default_rng(0)

# Realistic clinical imbalance: ~2% of screened variants are truly pathogenic.
n = 20000
y = rng.binomial(1, 0.02, size=n)

# A mediocre model score: weakly shifted for true positives, heavy overlap.
score = rng.normal(loc=0.6 * y, scale=1.0)

# Precision@k: the metric a fixed wet-lab follow-up budget actually cares about.
k = 100
top_k = np.argsort(-score)[:k]
precision_at_k = y[top_k].mean()

print(f"prevalence base : {y.mean():.3f}  ({y.sum()} / {n} positive)")
print(f"ROC-AUC         : {roc_auc_score(y, score):.3f}")
print(f"PR-AUC (AP)     : {average_precision_score(y, score):.3f}")
print(f"precision@{k}    : {precision_at_k:.3f}")

prevalence base : 0.021  (412 / 20000 positive)
ROC-AUC         : 0.689
PR-AUC (AP)     : 0.045
precision@100    : 0.090