# Linux commands: uniq

> Learn how the Linux uniq command finds and removes duplicate lines in text, why you pair it with sort, and how -d, -u, and -c change what it reports.

Author: [Flavio Copes](https://flaviocopes.com/about/) | Published: 2020-10-04 | Topics: [CLI](https://flaviocopes.com/tags/cli/) | Canonical: https://flaviocopes.com/linux-command-uniq/

`uniq` is a command useful to sort lines of text.

You can get those lines from a file, or using pipes from the output of another command:

```bash
uniq dogs.txt

ls | uniq
```

You need to consider this key thing: `uniq` will only detect adjacent duplicate lines.

This implies that you will most likely use it along with `sort`:

```bash
sort dogs.txt | uniq
```

The `sort` command has its own way to remove duplicates with the `-u` (_unique_) option. But `uniq` has more power.

By default it removes duplicate lines:

![Terminal showing cat dogs.txt with duplicate names, then sort dogs.txt | uniq removing duplicates to show each name once](https://flaviocopes.com/images/linux-command-uniq/Screen_Shot_2020-09-07_at_08.39.35.png)

You can tell it to only display duplicate lines, for example, with the `-d` option:

```bash
sort dogs.txt | uniq -d
```

![Terminal showing cat dogs.txt with duplicate names, then uniq -d displaying only the duplicate lines Roger and Syd](https://flaviocopes.com/images/linux-command-uniq/Screen_Shot_2020-09-07_at_08.36.50.png)

You can use the `-u` option to only display non-duplicate lines:

![Terminal showing cat dogs.txt with duplicate names, then uniq -u displaying only non-duplicate lines Ivica Luna Tina Vanille](https://flaviocopes.com/images/linux-command-uniq/Screen_Shot_2020-09-07_at_08.38.50.png)

You can count the occurrences of each line with the `-c` option:

![Terminal showing cat dogs.txt with duplicates, then uniq -c displaying count numbers before each name 1 Ivica 1 Luna 2 Roger 2 Syd](https://flaviocopes.com/images/linux-command-uniq/Screen_Shot_2020-09-07_at_08.37.15.png)

Use the special combination:

```bash
sort dogs.txt | uniq -c | sort -nr
```

to then sort those lines by most frequent:

![Terminal showing sort uniq -c sort -nr output with names sorted by frequency 2 Syd 2 Roger 1 Vanille 1 Tina 1 Luna 1 Ivica](https://flaviocopes.com/images/linux-command-uniq/Screen_Shot_2020-09-07_at_08.37.49.png)

The `uniq` command works on Linux, macOS, WSL, and anywhere you have a UNIX environment

For a quick paste-and-clean pass over a block of text, I built a free [line sorter](https://flaviocopes.com/tools/line-sorter/) that sorts lines and removes duplicates.