text tool
Remove duplicate lines
Paste a list with one item per line and every repeated line is removed, keeping the first copy in its original order. Choose whether capitals and extra spaces make two lines different, and see a tally of which lines repeated and how many times.
Loading the tool…
- Runs on your device, nothing is uploaded
- Free, no sign in
- No watermark, no file limit
How it works
- 1
Paste the list
One item per line, from a spreadsheet column, an email or a list of options. The result updates as you type.
- 2
Choose what counts as the same
Ignore capitals, Ignore extra spaces and Remove empty lines are ticked to start. Untick one to make that difference count.
- 3
Choose which copy stays
The first one keeps the order of first appearances. The last one keeps each line where it last appeared. None drops every line that repeated and leaves only the lines that appeared once.
- 4
Copy or download the result
Open What repeated to see each duplicate and its count. Copy the clean list or download it as a .txt file. Nothing is uploaded and nothing is stored, ever.
How lines are compared
Each line is turned into a comparison key. With Ignore extra spaces ticked, spaces at either end are dropped and runs of spaces and tabs inside are read as one; with Ignore capitals ticked, the key is lower cased using your browser's language rules. Two lines with the same key are duplicates. The first time a key appears its line is remembered, and later copies are counted and left out, so the result keeps your order rather than an alphabetical one.
First, last or none
Keeping the first copy is right for most lists, because the order you pasted is the order you get back. Keeping the last copy suits a log or an export in which a later line is the newer version of the same record. None is a different tool in disguise: it drops every line that appeared more than once and leaves only the unique ones, which is how you find the entries that are in one list but not another after pasting both together.
Cleaning an option list for a form
A dropdown, radio or checkbox question is only as good as its options. Lists copied from a spreadsheet often carry the same value typed twice, once with a capital or once with a trailing space that is invisible on screen. Remove those here before building the question, then sort the list with the sort lines tool if alphabetical order suits it. The form generator takes a prompt of up to 10,000 characters, enough to paste a long, clean option list into the description of the question you want.
What counts as a line
A line ends wherever the text has a line break, whether it was written on Windows, macOS or Linux, so a list copied from any of them splits the same way. An empty line is a line too. With Remove empty lines ticked they are all dropped and counted in the summary; with it unticked, the first empty line is kept like any other line and the rest are removed as its duplicates. A line holding only spaces counts as empty. Spaces at the start of a line matter only while Ignore extra spaces is unticked, which suits outlines and code where the indent carries meaning.
Limits worth knowing
Only exact matches are removed, after the space and capital rules you chose. Sydney and Sydney NSW stay as two lines, as do colour and color, St Kilda and Saint Kilda, or a name with and without a middle initial. Punctuation always counts, so a line ending in a full stop is different from one without. With Ignore capitals on, the copy you keep is spelled the way it appeared, so check that the copy kept is the spelling you want.
Limits
- Near duplicates are not merged: Sydney and Sydney NSW, or colour and color, stay as two lines.
- With Ignore capitals on, the copy kept is spelled the way it appeared in the list.
- Punctuation always counts, so a line ending in a full stop is different from the same line without one.
What it does not do
- It does not sort the list; the sort lines tool does that.
- It does not merge spelling variants or abbreviations.
- It does not compare one column inside each line, such as one field of a CSV row.
Questions people ask
Can I remove duplicates from a spreadsheet column?
Yes. Copy the column, paste it here, and copy the result back into an empty column. Each cell becomes one line, so a cell that holds a line break splits into two lines; check those first. To remove duplicate rows based on one column while keeping whole rows, use the spreadsheet itself, since this tool only sees the text you paste.
Does it change the order of my list?
No. The lines that stay keep the order they had, so a list of options or steps comes back in the same sequence with the repeats taken out. With The last one chosen, each line sits where its final copy was. To put the list in alphabetical or numeric order as well, run the result through the sort lines tool.
How do I find the lines that appear only once?
Choose None under Which copy stays. Every line that appeared more than once is dropped entirely, and only the lines that appeared exactly once are left. Paste two lists one after the other first, and the result is the entries that are in one list but not the other, as long as neither list repeats lines within itself.
Is there a limit on how long the list can be?
There is no fixed limit. The comparison runs in this tab, and a list of a hundred thousand lines is handled in a moment on a typical laptop. Very large pastes are slowed more by the text boxes drawing the text than by the comparison itself, so for millions of lines a spreadsheet or a command line tool is the better choice.
Is my list sent anywhere?
No. Everything happens in this page on your device, and no network request carries the text, which you can check in the network tab of your browser while you paste. Nothing is stored either, so a customer list, a set of email addresses or a staff roster leaves no copy behind when you close the tab.
Build a form with this option list
Take what you made here into forms.
Create a form with OneCraftMore tools like this
- Sort lines alphabeticallySort lines alphabetically A to Z or Z to A, by the number in each line, by length, or shuffle them. Language aware, with natural number order. No upload.
- Remove line breaksRemove line breaks from pasted text: hard breaks joined into paragraphs, blank lines kept as paragraph breaks, split words rejoined. Nothing is uploaded.
- Text diffCompare two versions line by line and word by word, with side by side or one column view. Runs in your browser, so a draft contract never leaves your machine.
- Case converterSee sentence case, title case, camelCase, snake_case and eleven more rendered from your own text, then press the one you want to copy it. Runs in your browser.
- CSV to tablePaste CSV and read it as a sortable grid. Every column header shows its type, blanks and range, and any row with the wrong field count is named by line.
Background reading: the question page on this.
Written and checked by the OneCraft team. Last checked .