Sort, Dedupe and Clean Lines
Sort lines, remove duplicates and empty lines, trim spaces.
About this tool
Paste a list and clean it up: trim spaces, drop empty lines, remove duplicates (optionally ignoring case), and sort alphabetically, naturally (so item2 comes before item10), by length, in reverse, or shuffle it.
How to use it
- Paste a list with one item per line.
- Choose what to clean: trim spaces, remove empty lines, remove duplicates (optionally ignoring case).
- Pick a sort order: natural, A to Z, Z to A, by length, reversed or shuffled.
- Copy the result. The status line shows how many lines and duplicates were removed.
Questions
- What is natural sort?
- Natural sort compares numbers inside text by their value, so item2 comes before item10. A plain alphabetical sort would put item10 first because 1 comes before 2.
- Which duplicate is kept?
- The first occurrence of each line is kept and later copies are removed, so the original order is preserved unless you also sort.
- Is the shuffle random enough for a draw?
- It uses the browser’s cryptographically secure random generator with an unbiased Fisher–Yates shuffle, so every order is equally likely.
- Why are some duplicate lines not removed?
- The lines look the same but aren’t identical. Common causes are invisible characters: a non-breaking space instead of a normal space inside the line, a zero-width space copied from a web page, or two Unicode forms of the same accented letter. Trim spaces only cleans the ends of a line, and with Ignore case off, Apple and apple stay separate.
- Can I count how many times each line appears?
- Not here. The status line only shows how many duplicates were removed in total. To count occurrences, use the shell: sort list.txt | uniq -c | sort -rn prints every distinct line with its count, the most frequent first.
- How many lines can it handle?
- There is no fixed limit. Lists with tens of thousands of lines are processed as you type, and Open file accepts text files up to 50 MB. Everything runs in your browser, so very large files depend on your device’s memory, and nothing is uploaded.
How to remove duplicate lines from a list
Paste the list, keep Remove duplicates on, and every line that already appeared earlier is dropped. With Ignore case on, Apple and apple count as the same line, and the spelling of the first one is kept. Trim spaces runs before deduplication, so “apple ” with a trailing space matches “apple”, which is the most common reason a dedupe seems to miss obvious copies. The status line shows the line count before and after.
Typical uses for removing duplicate lines include cleaning a mailing list before an import, merging two lists of hostnames or package names, and listing the unique values of a spreadsheet column, which you can copy from Excel and paste in directly. For email addresses, Ignore case is usually right, since domains are case-insensitive and most providers ignore case in the name part too. For file paths and case-sensitive IDs, turn it off.
Sorting lines online vs the Unix sort command
A to Z and Z to A sort lines with your browser’s locale rules while ignoring case and accents, so apple, Banana and Éclair come out in dictionary order. Plain byte-order sorting, as done by sort with LC_ALL=C or by Array.prototype.sort() without a comparator, puts every uppercase letter before every lowercase one and accented letters after z. By length puts the shortest lines first and breaks ties in natural order.
For reference, the Unix equivalents are sort -u to sort and remove duplicates, sort -f to ignore case, sort -V for natural or version order, and uniq, which only removes adjacent duplicates and is therefore usually run after sort. Unlike uniq, this tool removes duplicates anywhere in the list and can keep the original order when Sort is set to Keep order. Shuffle randomises the list, and Shuffle again gives a fresh order.
How to remove empty lines and trim whitespace
Remove empty lines drops lines that are empty or contain only spaces and tabs, so the gaps left over from copying a table or a log don’t survive. Trim spaces removes leading and trailing whitespace from every line, including tabs and the non-breaking spaces that often come from web pages and PDFs. Turn trimming off when indentation matters, for example in a YAML block or a Python snippet, and only remove the empty lines.
Line endings are normalised along the way. Windows CRLF and Unix LF input both split into lines correctly, and the result always uses LF line breaks. That makes the line tools a quick way to clean a list copied from Excel or a Windows text file before pasting it into a shell script, a .gitignore file or a config. For long lists, Download saves the result as lines.txt.
Something not working, or a result looks wrong? Report a problem with this tool