Primary vs Derived Databases

The single most useful mental model for the whole database landscape is a two-layer stack: a bottom layer that stores what experiments produced, and a top layer that makes sense of it. Learn to tell those two layers apart and you can predict, before you ever open a record, how trustworthy it is, how it was made, and who is responsible for it. This lesson gives you that lens and threads it through BRCA1.

Two layers, not a flat catalog

In Why Bioinformatics Runs on Databases we treated the databases as one big pool you fetch from. That was deliberate simplification. In reality every resource sits in one of two roles.

A primary database (also called an archive) stores data exactly as a submitter deposited it. Someone ran a sequencer, a mass spectrometer, or a crystallography experiment, wrote up the result, and submitted it. The archive's job is to keep that submission and hand it back on request — not to judge it. The submitter owns the content.

A derived database (also called a secondary or curated database) is built on top of primary data. Curators — human experts, automated pipelines, or both — read the archives, merge redundant records, cross-check against other databases and published literature, attach controlled vocabularies (a controlled vocabulary is an agreed, fixed list of terms so everyone writes the same concept the same way, like Gene Ontology terms), and produce a cleaner, opinionated reference. The database owns the content.

The primary archives

The nucleotide archive is the classic example, and it is genuinely one shared database wearing three regional names: ENA in Europe, GenBank at NCBI in the US, and DDBJ in Japan. Together they form the INSDC (International Nucleotide Sequence Database Collaboration), and they exchange records nightly, so an accession deposited at any one appears at all three. An accession number is a stable, permanent ID for one database record — for a raw BRCA1 mRNA submission it might look like U14680, the original 1994 GenBank deposit.

Primary archives exist for other data types too: GEO (Gene Expression Omnibus) at NCBI archives functional-genomics experiments like microarray and RNA-seq series — EBI's equivalent collection now lives inside BioStudies after ArrayExpress was folded into it in 2022; the PDB (Protein Data Bank) archives experimentally determined 3D structures; the SRA and ENA read archives hold raw sequencing reads.

The derived references

Sitting above those archives are the resources most people actually work with day to day.

RefSeq takes the messy, redundant pile of GenBank submissions for a gene and produces one non-redundant reference sequence per molecule. UniProt does the same for proteins, folding in literature-based annotation. Ensembl builds a coherent gene set over a genome assembly, and InterPro integrates several protein-domain databases into one classification. None of these hold new experimental data — they are curation over the archives.

BRCA1: the same gene, two kinds of record

BRCA1 makes the distinction concrete. The archival view is a raw GenBank submission — the original cloned mRNA someone deposited, warts and all, tied to one lab and one moment. The curated view is RefSeq NM_007294, the reference BRCA1 transcript, with its protein NP_009225 and the genomic context on chromosome 17 (NC_000017). NM_007294 did not come off a sequencer; NCBI curators built it by reconciling many submissions and the literature into the sequence the community agrees to call the BRCA1 mRNA.

from Bio import Entrez

Entrez.email = "[email protected]"  # NCBI requires a contact address

# Fetch the curated RefSeq record and read its provenance flags
with Entrez.efetch(db="nuccore", id="NM_007294",
                   rettype="gb", retmode="text") as handle:
    for line in handle:
        if line.startswith(("LOCUS", "VERSION", "KEYWORDS")):
            print(line.rstrip())

LOCUS       NM_007294               7088 bp    mRNA    linear   PRI 20-MAY-2025
VERSION     NM_007294.4
KEYWORDS    RefSeq; MANE Select.

The RefSeq and MANE Select keywords are the giveaway that this is a derived record, not a raw deposit — a raw GenBank submission carries neither.

Why the distinction pays off

Knowing which layer a record lives in tells you what to expect. From an archive you get raw fidelity and full history, but also redundancy, occasional errors, and no editorial guarantee — you are reading one lab's claim. From a derived database you get a clean, cross-linked, best-current-understanding reference, but one filtered through curation choices and update cycles you did not make. Good work uses both deliberately: cite RefSeq or UniProt when you need the agreed answer, and drop down to GenBank, GEO, or the PDB when you need the underlying evidence. That one habit — naming the layer before you trust a number — is the fastest way to avoid citing an archived draft as if it were the community's settled answer.