Lesson 9 of 14 · 11 min

Summarizing Tables: cut, sort, uniq, wc

Most bioinformatics tools report their results as tables: plain-text files where each line is one record and the columns within a line are separated by a tab character. You will meet many such formats, including TSV (tab-separated values), BED, and GFF. To peek at a file we use cat, a command that prints a file's contents to the screen.

A BED file of peaks

bash

cat peaks.bed

chr1	1000	1500	peak1
chr2	2200	2600	peak2
chr1	5000	5400	peak3
chrX	9000	9600	peak4
chr1	12000	12500	peak5
chr2	15000	15300	peak6
chrX	22000	22800	peak7
chr1	30000	30500	peak8
chrX	41000	41600	peak9

In a BED file each row describes a region of a genome. The first column is the chromosome, the second is the start position, the third is the end position, and any later columns hold extra details such as a feature name. Those values are separated by tabs, not spaces, which is what lets the next tools split each line apart cleanly.

The cut command pulls out specific columns. The -f flag chooses which fields (columns) to keep by number, and cut assumes the columns are tab-separated by default. If a file uses a different separator, such as the commas in a CSV file, you say so with -d, for example cut -d, -f2 file.csv.

bash

cut -f1 peaks.bed

chr1
chr2
chr1
chrX
chr1
chr2
chrX
chr1
chrX

The pipe symbol, written as a vertical bar, sends the output of one command straight into the next so you can chain small tools together. sort reorders the lines it receives (alphabetically by default), and uniq -c then collapses runs of identical neighbouring lines into one line prefixed by how many times it occurred. Because uniq only compares adjacent lines you must sort first, and a final tool, wc -l, simply counts how many lines it is handed.

bash

# tally how many peaks fall on each chromosome
cut -f1 peaks.bed | sort | uniq -c

# rank the chromosomes from most peaks to fewest (-n numeric, -r reverse)
cut -f1 peaks.bed | sort | uniq -c | sort -nr

# count how many DIFFERENT chromosomes appear (-u keeps one of each)
cut -f1 peaks.bed | sort -u | wc -l

      4 chr1
      2 chr2
      3 chrX

      4 chr1
      3 chrX
      2 chr2

3

Try it yourself: In the same peaks.bed, count the total number of peaks with wc -l peaks.bed, then read the ranked tally to see which single chromosome carries the most. For a challenge, order the rows by position with sort -k1,1 -k2,2n peaks.bed, where -k1,1 sorts on the first column and -k2,2n sorts the second column numerically.

Searching Text with grep A Gentle Intro to awk and sed