Subsetting and Filtering Data

A data frame is R's version of a spreadsheet: each row is one observation (here, a gene) and each column is a variable (like its chromosome or its expression level). Real datasets have thousands of rows, so almost every analysis begins by trimming the table down to just the rows and columns you care about. Choosing part of a table is called subsetting, and picking rows by a rule is called filtering.

A small dataset

# data.frame() builds a table from equal-length columns
genes <- data.frame(
  gene = c("TP53", "BRCA1", "EGFR", "MYC", "PTEN"),
  chromosome = c("17", "17", "7", "8", "10"),
  expression = c(12.4, 8.1, 25.7, 30.2, 5.6)
)

# typing the name on its own line prints the whole table
genes

   gene chromosome expression
1  TP53         17       12.4
2 BRCA1         17        8.1
3  EGFR          7       25.7
4   MYC          8       30.2
5  PTEN         10        5.6

The main tool for picking parts of a table is square brackets, written as df[rows, columns]. The comma inside the brackets is the key: whatever you put before it selects rows, whatever you put after it selects columns, and leaving a side blank means keep all of them. You can name columns as text in quotes, and choose rows by their number, where 1:3 is shorthand for the numbers 1, 2, 3. One quirk to notice: pulling out a single column this way, as in genes[, "gene"], hands back a plain vector of values rather than a one-column table, which is why that result prints with a [1] position label instead of as a grid with row numbers.

# row 1, all columns (note the blank space after the comma)
genes[1, ]

# all rows, only the "gene" column
genes[, "gene"]

# rows 1 to 3, and only two named columns
genes[1:3, c("gene", "expression")]

  gene chromosome expression
1 TP53         17       12.4
[1] "TP53"  "BRCA1" "EGFR"  "MYC"   "PTEN" 
   gene expression
1  TP53       12.4
2 BRCA1        8.1
3  EGFR       25.7

Filter by a condition

To pick rows by a rule instead of by number, you first build a condition. Writing genes$gene reaches into the data frame and pulls out just the gene column, and == asks which of its values equal something, giving back a TRUE or FALSE for every row. Put that condition before the comma to keep only the TRUE rows, join several rules with & when both must hold or | when either is enough, use which() to see the matching row numbers, and reach for subset() to do the same filtering with less typing.

# a condition produces one TRUE/FALSE value per row
genes$expression > 10

# keep the rows where gene equals "TP53"
genes[genes$gene == "TP53", ]

# & means BOTH conditions must be TRUE
genes[genes$expression > 10 & genes$chromosome == "17", ]

# | means EITHER condition can be TRUE
genes[genes$chromosome == "7" | genes$chromosome == "8", ]

# which() returns the positions that are TRUE
which(genes$expression > 10)

# subset() filters rows with less typing (no genes$ needed)
subset(genes, expression > 10)

[1]  TRUE FALSE  TRUE  TRUE FALSE
  gene chromosome expression
1 TP53         17       12.4
  gene chromosome expression
1 TP53         17       12.4
  gene chromosome expression
3 EGFR          7       25.7
4  MYC          8       30.2
[1] 1 3 4
  gene chromosome expression
1 TP53         17       12.4
3 EGFR          7       25.7
4  MYC          8       30.2