← Back to Text & Writing Tools
Remove Duplicate Lines
Delete repeated lines from a list and keep one copy of each, in the original order.
Checking tool availability…
Remove Duplicate Lines works on your text locally in your browser.
Clean up lists with repeated entries
Merged mailing lists, copied spreadsheet columns, log extracts, and keyword lists often end up with the same line several times. Paste the list and every repeated line is removed as you type, keeping the first copy of each in the order it first appeared — so the list is shorter but otherwise unchanged. Options let you ignore capitalization or spaces at the ends of lines when deciding what counts as a duplicate.
How to remove duplicate lines
- Type or paste your text, or choose Upload Document to open a .txt, .md, .docx, or .pdf file.
- The result appears immediately below, with counts of lines kept and removed; turn on the matching options if you need them.
- Choose Copy Result to use it elsewhere, or Replace Input with Result to keep working here — Undo Replace brings the original back.
How duplicates are matched
- By default, two lines are duplicates only if they match exactly — every letter, capital, and space.
- “Ignore case when comparing” treats “Apple” and “apple” as the same line.
- “Ignore spaces at the start and end of lines” treats “ apple” and “apple” as the same line.
- The options only change the comparison: the first copy of each line is always returned exactly as it was typed.
- Blank lines are compared like any other line, so only the first blank line is kept.
| You enter | You get |
|---|---|
| apple orange apple banana orange | apple orange banana |
| Red red Red | Red red |
Your text stays on your device
- Text is processed by JavaScript running in your browser.
- Your text is never uploaded to Looty Tools or any other server.
- Uploaded documents are opened and read in your browser; the file itself is never sent to Looty Tools.
- Nothing is stored — refreshing or closing the page clears both boxes.
Limitations
- Uploads accept .txt and .md files up to 5 MB, Word .docx files up to 15 MB, and PDFs up to 25 MB and 50 pages. Older .doc files are not supported, and scanned or image-only PDFs have no text to use.
- Lines that look identical but contain different invisible characters, such as a non-breaking space or a zero-width space, are treated as different.
- “Ignore case” uses standard lowercasing, so special cases such as the German ß are not matched with “SS”.
- The list is not sorted; use Sort Lines Alphabetically afterwards if you need it in order.
Related text tools
- Remove Empty LinesDelete blank lines from a list or block of text.
- Remove Extra SpacesCollapse repeated spaces and trim each line.
- Sort Lines AlphabeticallySort each line from A to Z.
Guides
- Why Duplicate Lines AppearHow merged lists, repeated exports, logs, and double submissions create duplicate lines — and when a repeated line is meaningful and should stay.
- Exact vs. Near-Duplicate LinesWhy lines that look identical may not match: capitalization, trailing spaces, non-breaking and zero-width characters, and Unicode normalization.
- Duplicate Rows vs. LinesThe difference between removing duplicate lines from text and duplicate rows from a spreadsheet, why CSV files need care, and which approach to use.
Remove Duplicate Lines FAQ
- Which copy of a duplicate line is kept?
- The first one. Every later line that matches an earlier line is removed, and the lines that remain stay in their original order.
- Are “Apple” and “apple” duplicates?
- Not by default — matching is exact, so capitalization counts. Turn on “Ignore case when comparing” to treat them as the same line.
- What about lines that differ only by spaces?
- By default they are different lines. “Ignore spaces at the start and end of lines” compares lines without their leading and trailing spaces; the lines that are kept are still returned exactly as they were.
- What happens to blank lines?
- Blank lines are compared like any other line, so only the first blank line is kept. To remove every blank line, use Remove Empty Lines.
- Does it sort my list?
- No. The order of the lines is preserved. Use Sort Lines Alphabetically if you also want the list in order.
- Can I upload a Word document or PDF instead of pasting?
- Yes. Upload Document accepts .txt, .md, Word .docx, and text-based .pdf files, or you can drop a file onto the text box. The file is read in your browser — it is not sent to Looty Tools — and its text replaces whatever is in the box. Scanned PDFs without a text layer and older .doc files are not supported.
- Is my list sent anywhere?
- No. Duplicates are removed in your browser, and the text never leaves your device or gets saved.
