Remove duplicate lines from a list

Paste your list and get one copy of every line. You can also show only the repeated lines and see which ones appear most often.

  • Free
  • No sign-up
  • Runs in your browser
remove-duplicate-lines

100% private — your list is processed in your browser and never sent to any server.

How it works

Paste your list

One item per line: emails, keywords, URLs, names, log lines or anything else.

Set how to compare

Ignore case or stray spaces if you want "Apple" and "apple " treated as the same line.

Copy the result

Get one copy of each line, only the repeated ones, or only those that appear once.

Cleaning a list without opening a spreadsheet

Repeated lines creep into every kind of list: a mailing list built from several exports, a set of keywords collected over months, a column of URLs, a log file, or a list of names typed by different people. Spreadsheets can remove duplicates, but only after you import the data, pick the column and check what the tool did to the rest. Pasting the list here and copying the result back is faster.

Two ways to define "the same line"

By default two lines are duplicates only if they match exactly. Real lists are rarely that tidy, which is why there are two comparison options. Ignore upper/lower case treats Apple and apple as one item. Ignore spaces at line edges treats apple and apple as one item, which matters because a trailing space is invisible and is the most common reason duplicates survive a clean-up.

Keep the first copy, or the last

The result keeps the order in which lines first appear, so your list stays recognizable. If you paste an ordered log where the newest entry is the one you care about, switch to keeping the last occurrence instead. You can also sort the result alphabetically, with numbers ordered naturally (item 2 before item 10), or by how many times each line repeated.

Find the duplicates instead of removing them

Sometimes the useful answer is which lines repeat. Choose "Only lines that repeat" to list them once each, or "Only lines that appear once" to get the entries that have no duplicate at all. Below the result you also see a ranking of the most repeated lines with their counts.

What it does not do

The tool compares whole lines. It will not spot two lines that are similar but not equal (for example a misspelling), and it does not look inside a line, so for a spreadsheet with several columns, copy the single column you want to check.

In Excel and Google Sheets

With a list in A2:A7 (apple, banana, apple, cherry, banana, apple):

Spreadsheet formulas for this calculation
To getFormulaResult
The list without repeats, in order of first appearance=UNIQUE(A2:A7)apple, banana, cherry
How many different values=COUNTA(UNIQUE(A2:A7))3
Is this row a repeat of an earlier one?=COUNTIF($A$2:A2,A2)>1FALSE, FALSE, TRUE, FALSE, TRUE, TRUE

The formulas use English function names and commas. If your Excel or Google Sheets is set to another language, the function names change, and in many countries the separator between arguments is a semicolon.

  • Without formulas: in Excel select the list and use Data, Remove Duplicates; in Google Sheets use Data, Data cleanup, Remove duplicates.
  • UNIQUE exists in Excel for Microsoft 365, Excel 2021 and later, and in Google Sheets, but not in Excel 2019 or earlier.
  • In Google Sheets, UNIQUE sees apple and Apple as two different values. The tool lets you choose whether case matters and whether to keep the first or the last occurrence.

Frequently asked questions

Does it keep my original order?

Yes, unless you choose to sort. Each line appears at the position where it first showed up in your list (or last, if you pick that option).

Why does a duplicate sometimes survive?

Usually because of a trailing space, a different capital letter or an invisible character. Turn on "Ignore spaces at line edges" and "Ignore upper/lower case". If it still survives, run the list through the text cleaner first to remove hidden characters.

Can I remove duplicates from a column in Excel or Google Sheets?

Yes: copy the column, paste it here, copy the result and paste it back. It is quicker than the spreadsheet's own feature when you also want to see how many were repeated.

How many lines can it handle?

Lists of several hundred thousand lines work well on a computer. The work happens in your browser, so very large lists depend on your device's memory.

Can I get only the lines that were repeated?

Yes. Choose "Only lines that repeat" under Show. You will get each repeated line once, and the ranking below tells you how many times each appeared.

Is my list uploaded anywhere?

No. Everything is processed in your browser and nothing is sent to a server, so it is safe for customer lists or internal data.

How do I remove duplicates in Excel?

Select the list and use Data, Remove Duplicates. With a formula, =UNIQUE(A2:A7) returns the list without repeats, in Excel 2021, Microsoft 365 and Google Sheets.