Remove duplicate lines and keep list order

Paste a list and keep the first or last matching line in its original position. Choose whether to ignore case, Vietnamese accents and empty lines before copying the result.

Paste a list and choose which line to keep

Processed in your browser · saves drafts when allowed Ready
Deduplicated list

Deduplicate a pasted list in three steps

01

Paste a list

Paste the list into the input field, one item per line. You can use the sample first to try the matching rules.

Which lines count as duplicates?

The tool compares complete lines and ignores leading or trailing whitespace for matching, while keeping the selected line’s original spelling and spacing in the result.

RuleExampleMatching result
Ignore case (default)Hà Nội / hà nộiOne match; keep the first or last line you select.
Ignore accents (optional)Đà Nẵng / Da NangOne match when this option and Ignore case are enabled.
Leading or trailing spaces" Huế " / "Huế"One match; spaces on the retained line remain unchanged.
Spaces inside a lineHà Nội / Hà NộiStill two different lines.

Keep the first or last occurrence?

Keep first or last
Keep first uses the first line’s spelling in each duplicate group; Keep last uses the final occurrence. The survivors remain in their original relative order and are not sorted A–Z.
Removed line counts
Duplicates removed counts repeated lines among the lines checked. Removed blank lines have their own count; if you turn off Remove empty lines, blank lines are also deduplicated and at most one is retained.
Meaning is not matching
The tool does not find repeated words within a line, typos, equivalent URLs or sentences with the same meaning. Review the result when case or accents matter for codes, names and identifiers.

Where are the text and draft stored?

Line matching runs in your browser and does not send the text to an KitoTool deduplication API. When local storage is available, the page saves your input and options for a later visit; Clear removes that draft. On a shared device, select Clear when finished. The downloaded TXT contains the result and a UTF-8 marker for accented text.

Comments & feedback

Rate this tool

Loading ratings…

Write a commentNo account needed
Up to 4,000 characters0 / 4.000

Do not share passwords, sensitive information or the contents of files you are processing.

Reviewed before posting. Email stays private.

Embed this tool on your websiteFree · Copy once, receive tool updates automatically

Paste the iframe into HTML, WordPress or any platform that supports embeds. The tool runs in the visitor’s browser; adjust its height and theme as needed.

View the embed guide and supported platforms →

<div style="max-width:900px">
  <iframe data-kitotool-embed src="https://kitotool.com/embed/en/duplicate-line-cleaner/" title="Duplicate Line Remover — KitoTool" width="100%" height="920" style="display:block;width:100%;border:1px solid #dbe3ee;border-radius:12px" loading="lazy" referrerpolicy="strict-origin-when-cross-origin"></iframe>
  <p style="margin:6px 2px 0;text-align:right;font:12px/1.5 system-ui,sans-serif;color:#64748b">Free tool by <a href="https://kitotool.com/en/duplicate-line-cleaner/" target="_blank" rel="nofollow noopener noreferrer" style="color:#2359d4;text-decoration:underline">KitoTool</a></p>
</div>

The code includes a visible credit below the iframe; its link uses nofollow for a widely distributed widget.

Optional: automatically resize the iframe

Add this script once to the page containing the iframe.

<script>
(function(){var origin="https://kitotool.com";function frames(){return document.querySelectorAll("iframe[data-kitotool-embed]")}function measure(f){if(f.contentWindow&&new URL(f.src).origin===origin)f.contentWindow.postMessage({type:"kitotool:measure"},origin)}window.addEventListener("message",function(e){if(e.origin!==origin||!e.data||e.data.type!=="kitotool:resize"||!Number.isFinite(e.data.height))return;frames().forEach(function(f){if(f.contentWindow===e.source&&new URL(f.src).origin===origin)f.style.height=Math.max(320,Math.min(12000,Math.ceil(e.data.height)))+"px";});});function connect(){frames().forEach(function(f){f.addEventListener("load",function(){measure(f)});measure(f)})}if(document.readyState==="loading")document.addEventListener("DOMContentLoaded",connect,{once:true});else connect()})();
</script>