Duplicate File Finder - Revision history

Casey at 16:15, 22 April 2025

2025-04-22T16:15:55Z

Casey at 16:14, 22 April 2025

2025-04-22T16:14:01Z

Casey at 23:15, 7 March 2025

2025-03-07T23:15:30Z

Casey at 22:28, 7 March 2025

2025-03-07T22:28:48Z

Casey: Created page with "Originally from [Reddit https://stackoverflow.com/questions/76242708/find-duplicate-files-with-hash-and-length-but-use-other-algorithm] but I added a .bat wrapper"

2025-03-07T22:28:04Z

Created page with "Originally from [Reddit https://stackoverflow.com/questions/76242708/find-duplicate-files-with-hash-and-length-but-use-other-algorithm] but I added a .bat wrapper"

New page

Originally from [Reddit https://stackoverflow.com/questions/76242708/find-duplicate-files-with-hash-and-length-but-use-other-algorithm] but I added a .bat wrapper

@@ Line 8: / Line 8: @@
      certutil -hashfile "%%f" MD5 | findstr /V ":"
+)
 </PRE>

@@ Line 1: / Line 1: @@
-Originally from [https://stackoverflow.com/questions/76242708/find-duplicate-files-with-hash-and-length-but-use-other-algorithm Reddit] but I added a .bat wrapper
+Originally from [https://superuser.com/questions/1315365/how-can-i-generate-an-md5-sum-for-a-folder-on-windows Superuser.com]
-==duplicate.ps1==
+==getsums.bat==
 <PRE>
-$srcDir = 'C:\test'
+@echo off
-$maxThreads = 6 # Tweak this value for more or less threads
+for /R . %%f in (*.*) do (
-$rs = [runspacefactory]::CreateRunspacePool(1, $maxThreads)
+     echo | set/p="%%f ~~~ "
-$rs.Open()
+    certutil -hashfile "%%f" MD5 | findstr /V ":"
-−
+)
 </PRE>

← Older revision		Revision as of 23:15, 7 March 2025
Line 1:		Line 1:
	Originally from [https://stackoverflow.com/questions/76242708/find-duplicate-files-with-hash-and-length-but-use-other-algorithm Reddit] but I added a .bat wrapper		Originally from [https://stackoverflow.com/questions/76242708/find-duplicate-files-with-hash-and-length-but-use-other-algorithm Reddit] but I added a .bat wrapper
		+
		+	==duplicate.ps1==
		+	<PRE>
		+	$srcDir = 'C:\test'
		+	$maxThreads = 6 # Tweak this value for more or less threads
		+	$rs = [runspacefactory]::CreateRunspacePool(1, $maxThreads)
		+	$rs.Open()
		+
		+	$tasks = Get-ChildItem -Path $srcDir -File -Recurse \| Group-Object Length \|
		+	Where-Object Count -GT 1 \| ForEach-Object {
		+	$ps = [powershell]::Create().AddScript({
		+	$args[0] \| Get-FileHash -Algorithm MD5 \|
		+	Group-Object Hash \|
		+	Where-Object Count -GT 1
		+	}).AddArgument($_.Group)
		+
		+	$ps.RunspacePool = $rs
		+
		+	@{ ps = $ps; iasync = $ps.BeginInvoke() }
		+	}
		+
		+	$tasks \| ForEach-Object {
		+	try {
		+	$_.ps.EndInvoke($_.iasync)
		+	}
		+	finally {
		+	if($_.ps) {
		+	$_.ps.Dispose()
		+	}
		+	}
		+	}
		+
		+	if($rs) {
		+	$rs.Dispose()
		+	}
		+	</PRE>
		+
		+	==run.bat==
		+	<PRE>
		+	powershell -ExecutionPolicy Bypass -File duplicate.ps1 > out.txt
		+	</PRE>

← Older revision		Revision as of 22:28, 7 March 2025
Line 1:		Line 1:
−	Originally from [~~Reddit~~ https://stackoverflow.com/questions/76242708/find-duplicate-files-with-hash-and-length-but-use-other-algorithm] but I added a .bat wrapper	+	Originally from [https://stackoverflow.com/questions/76242708/find-duplicate-files-with-hash-and-length-but-use-other-algorithm Reddit] but I added a .bat wrapper