Transforming Data with dplyr

Meet the Tidyverse

The tidyverse is a collection of R packages that share a common style and are built for working with data. Its main package for transforming tables is called dplyr, and we make it available with the library() function, which loads an installed package into your current R session. Throughout this lesson our table is a small set of genes, where log2FC is the log2 fold change (how strongly a gene's activity goes up or down between two conditions) and padj is an adjusted p value (smaller values mean we are more confident the change is real).

install.packages("tidyverse")  # run this only once, to download the packages
library(tidyverse)             # load dplyr and its companions for this session

# tibble() builds a tidyverse table; each argument becomes one column
genes <- tibble(
  gene   = c("TP53", "BRCA1", "EGFR", "MYC", "GAPDH"),  # c() combines values into one column
  log2FC = c(2.1, -1.8, 3.4, 0.2, 0.05),
  padj   = c(0.001, 0.02, 0.0001, 0.4, 0.9)
)

genes  # type a table's name to print it to the console

# A tibble: 5 × 3
  gene  log2FC   padj
  <chr>  <dbl>  <dbl>
1 TP53    2.1  0.001 
2 BRCA1  -1.8  0.02  
3 EGFR    3.4  0.0001
4 MYC     0.2  0.4   
5 GAPDH   0.05 0.9   

Chaining with the Pipe

The pipe operator, written as |>, takes the result on its left and feeds it as the first input to the function on its right. You can read it out loud as the word then, so a stack of pipes becomes a step by step recipe. This lets us start with a table and apply one change per line without saving messy in-between copies.

genes |>                                                 # start with the genes table, then
  filter(padj < 0.05) |>                                 # keep only rows where padj is below 0.05, then
  select(gene, log2FC) |>                                # keep only the gene and log2FC columns, then
  arrange(desc(log2FC)) |>                               # sort rows by log2FC, largest first, then
  mutate(direction = if_else(log2FC > 0, "up", "down"))  # add a new column labelling the direction

# A tibble: 3 × 3
  gene  log2FC direction
  <chr>  <dbl> <chr>    
1 EGFR     3.4 up       
2 TP53     2.1 up       
3 BRCA1   -1.8 down     

Each line is one dplyr verb doing a single job. filter() keeps or drops rows based on a condition, select() chooses which columns to keep, and arrange() reorders the rows, where desc() means descending so the largest value comes first. mutate() creates or changes a column, and here if_else() returns the text up when log2FC is positive and down otherwise.

Before dplyr, the same filtering was done with square-bracket indexing in base R, the plain R that comes with no extra packages installed. To reach a column you repeat the table's name followed by a dollar sign, and a trailing comma after the condition means keep every column. Compare the two styles below.

# Base R: genes$padj reaches the padj column, and the comma keeps all columns
genes[genes$padj < 0.05, ]

# dplyr: name the column directly, which reads almost like a sentence
genes |> filter(padj < 0.05)