• 0

Software to detect duplicates, with a difference?


Question

So to cut out a load of boring waffle for you, I need software to detect duplicate files.

 

I've tried Auslogics (as it was free) and it seemed to work well - for 100% identical files.

 

However, I have a ton of files that are 99% identical. By that I mean I have files that are basically the same but they have a -1 at the end of the file name. The rest of the file name is the same, the file size is the same, if they're a video then the duration is the same - but one will have a -1.

 

So we're talking about a file like: mycat.jpg and then another file called mycat-1.jpg ... where they're the exact same picture.

 

Is there such a software for Windows that can detect duplicates in this manner? I swear I must have many GB I could free up. I've already done 10GB off of 1 small subfolder.

8 answers to this question

Recommended Posts

  • 0

I used HashMyFiles to scan MD5 hashes (other hash types selectable) in various directories to scan for duplicate photos. Worked great. Name of file was irrelevant. You can open/scan multiple directories simultaneously. Duplicate hashes are handily color coded. Just picked which one to delete and done.

 

https://www.nirsoft.net/utils/hash_my_files.html

 

 

HashMyFiles.png

Edited by JustGeorge
  • Like 3
  • 0

If this is specific to digital camera photo's, check out https://www.advancedrenamer.com/

 

(Free for personal use)

 

It plugins to exif command line, and you can rename files to be Camera Model, Shutter Count (provided the camera embeds the the info).

 

I do like JustGeorge suggestion and taking a look at that myself for my various back up archive files.

  • 0

Thanks for the suggestions guys.

 

Turns out the software I was using, Auslogics, which is also free, does exactly what I need.

 

Issue was - I just didn't know how to use it properly at the point of asking.

 

Got it sorted now. Scanned 2 folders already & 10GB of duplicates in each for a 20GB free up.

  • 0

I used AllDup portable for video and images, but it isn't great, needs multiple passes to be able to pick out all dupes.

 

I will have to give a few of the suggestions in this topic a go.

  • 0
  On 23/09/2022 at 16:37, Technique said:

So to cut out a load of boring waffle for you, I need software to detect duplicate files.

 

I've tried Auslogics (as it was free) and it seemed to work well - for 100% identical files.

 

However, I have a ton of files that are 99% identical. By that I mean I have files that are basically the same but they have a -1 at the end of the file name. The rest of the file name is the same, the file size is the same, if they're a video then the duration is the same - but one will have a -1.

 

So we're talking about a file like: mycat.jpg and then another file called mycat-1.jpg ... where they're the exact same picture.

 

Is there such a software for Windows that can detect duplicates in this manner? I swear I must have many GB I could free up. I've already done 10GB off of 1 small subfolder.

Expand  

If the files are the same, the filename shouldn't matter because filenames don't affect checksums, so you would think Auslogics would detect it as being a duplicate anyway.

 

Just saw your follow-up comment, glad you got it sorted, 🙂

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Recently Browsing   0 members

    • No registered users viewing this page.
  • Posts

    • Microsoft PC Manager 3.17.4.0 (Offline Installer) by Razvan Serea With Microsoft PC Manager, users can easily perform basic computer maintenance and enhance the speed of their devices with just one click. This app offers a range of features, including disk cleanup, startup app management, virus scanning, Windows Update checks, process monitoring, and storage management. Microsoft PC Manager key features: Storage Manager- easily uninstall infrequently used apps, manage large files, perform a cleanup, and set up Storage Sense to automatically clear temporary files. Health Checkup feature -scans for potential problems, viruses, and startup programs to turn off. It helps you identify unnecessary items to remove, optimizing your system's performance. Pop-up Management - block pop-up windows from appearing in apps. Windows Update - scans your system for any pending updates. Startup Apps - enable or disable startup apps on your PC, allowing you to optimize your system's startup performance. Browser Protection - rest assured that harmful programs cannot alter your default browser. Also enables you to change your default browser. Process Management - allows you to conveniently terminate any active process, ensuring optimal system performance and resource utilization. Anti-virus protection - Fully integrated with Windows Security. Safeguard your PC anytime. Quick Steps: Download Microsoft PC Manager Offline Installer (APPX/MSIX) with Adguard Adguard serves as a third-party online service, offering a user-friendly method for directly downloading appx, appxbundle, and msixbundle files from the Microsoft Store. Official download links will be generated for both the app's various versions and its dependency packages. How to download Microsoft PC Manager Offline Installer (APPX/MSIX) 1. Initially, you must find the app URL within the Microsoft Store. Access the Microsoft Store via your browser and search for "Microsoft PC Manager". Once located, copy the app URL, which includes the product ID, either from the address bar or from the provided link below. https://apps.microsoft.com/detail/9PM860492SZD 2. Now paste the app URL into the designated area, then click the check mark button to produce a direct download link. 3. To download, right-click the relevant link and select “Save link as…” from your browser's menu. Occasionally, Microsoft Edge may flag the download as insecure. In such cases, consider utilizing alternative browsers such as Google Chrome or Firefox to successfully complete the download. Microsoft PC Manager is a completely free tool optimized exclusively for use on Windows 10 (version 1809 or newer) and Windows 11. Download: Microsoft PC Manager 3.17.4.0 | from Microsoft Store View: Microsoft PC Manager Home Page Get alerted to all of our Software updates on Twitter at @NeowinSoftware
    • You might want to update to 1TB drive.  They are not very expensive these days, there are plenty of options - simple look I would think you could find a 1TB ssd or even nvme somewhere between 50 and 100 bucks. when I have moved to larger ssds in laptop in the past, you can just connect the drive via usb - image.  Then take the old drive out, put in the new drive and there you go..  If your concerned about how much space you have left this prob your best bet..   
    • I see! Wasn’t aware of that. Even though they started off with desktop app for Mac before Windows. Should be on it’s way soon! 🤞
    • support.microsoft.com/en-us/topic/microsoft-defender-update-for-windows-operating-system-installation-images-1c89630b-61ff-00a1-04e2-2d1f3865450d
    • It's a Developer Beta 1 not even a Public or User Beta, I doubt any of the design is 100% finalized yet.
  • Recent Achievements

    • First Post
      Johnny Mrkvička earned a badge
      First Post
    • Week One Done
      viraltui earned a badge
      Week One Done
    • One Month Later
      serfegyed earned a badge
      One Month Later
    • Dedicated
      firey earned a badge
      Dedicated
    • Dedicated
      fettermanj earned a badge
      Dedicated
  • Popular Contributors

    1. 1
      +primortal
      627
    2. 2
      ATLien_0
      223
    3. 3
      Michael Scrip
      217
    4. 4
      Xenon
      149
    5. 5
      Steven P.
      139
  • Tell a friend

    Love Neowin? Tell a friend!