• 0

Software to detect duplicates, with a difference?


Question

So to cut out a load of boring waffle for you, I need software to detect duplicate files.

 

I've tried Auslogics (as it was free) and it seemed to work well - for 100% identical files.

 

However, I have a ton of files that are 99% identical. By that I mean I have files that are basically the same but they have a -1 at the end of the file name. The rest of the file name is the same, the file size is the same, if they're a video then the duration is the same - but one will have a -1.

 

So we're talking about a file like: mycat.jpg and then another file called mycat-1.jpg ... where they're the exact same picture.

 

Is there such a software for Windows that can detect duplicates in this manner? I swear I must have many GB I could free up. I've already done 10GB off of 1 small subfolder.

8 answers to this question

Recommended Posts

  • 0

I used HashMyFiles to scan MD5 hashes (other hash types selectable) in various directories to scan for duplicate photos. Worked great. Name of file was irrelevant. You can open/scan multiple directories simultaneously. Duplicate hashes are handily color coded. Just picked which one to delete and done.

 

https://www.nirsoft.net/utils/hash_my_files.html

 

 

HashMyFiles.png

Edited by JustGeorge
  • Like 3
  • 0

If this is specific to digital camera photo's, check out https://www.advancedrenamer.com/

 

(Free for personal use)

 

It plugins to exif command line, and you can rename files to be Camera Model, Shutter Count (provided the camera embeds the the info).

 

I do like JustGeorge suggestion and taking a look at that myself for my various back up archive files.

  • 0

Thanks for the suggestions guys.

 

Turns out the software I was using, Auslogics, which is also free, does exactly what I need.

 

Issue was - I just didn't know how to use it properly at the point of asking.

 

Got it sorted now. Scanned 2 folders already & 10GB of duplicates in each for a 20GB free up.

  • 0

I used AllDup portable for video and images, but it isn't great, needs multiple passes to be able to pick out all dupes.

 

I will have to give a few of the suggestions in this topic a go.

  • 0
  On 23/09/2022 at 16:37, Technique said:

So to cut out a load of boring waffle for you, I need software to detect duplicate files.

 

I've tried Auslogics (as it was free) and it seemed to work well - for 100% identical files.

 

However, I have a ton of files that are 99% identical. By that I mean I have files that are basically the same but they have a -1 at the end of the file name. The rest of the file name is the same, the file size is the same, if they're a video then the duration is the same - but one will have a -1.

 

So we're talking about a file like: mycat.jpg and then another file called mycat-1.jpg ... where they're the exact same picture.

 

Is there such a software for Windows that can detect duplicates in this manner? I swear I must have many GB I could free up. I've already done 10GB off of 1 small subfolder.

Expand  

If the files are the same, the filename shouldn't matter because filenames don't affect checksums, so you would think Auslogics would detect it as being a duplicate anyway.

 

Just saw your follow-up comment, glad you got it sorted, 🙂

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Recently Browsing   0 members

    • No registered users viewing this page.
  • Posts

    • Wikipedia has become hot molasses with mostly filth masquerading as truth. Most of it is the imagination of few writers who think of themselves as above god. The narrative setup is mind boggling.
    • he donates it all to his foundation to launder, trying to avoid inheritance tax while buying up africa
    • No. 1/10th the cost of a Mac? The Mini starts at 599. What mini-pc costs 1/10th of that? The OS is nothing like macOS. Sure you can make it look like it, but it won’t work anything like it. Applications won’t install, or remove the same. Application compatibility will be different. Managing the OS is completely dissimilar, the biggest common ground will be a few CLI utilities. You can definitely get yourself an inexpensive little setup that has a good Linux distro and be happy. It won’t be 1/10th the cost or essentially a Mac though.
    • Vivaldi 7.4.3684.50 by Razvan Serea Vivaldi is a cross-platform web browser built for – and with – the web. A browser based on the Blink engine (same in Chrome and Chromium) that is fast, but also a browser that is rich in functionality, highly flexible and puts the user first. A browser that is made for you. Vivaldi is produced with love by a founding team of browser pioneers, including former CEO Jon Stephenson von Tetzchner, who co-founded and led Opera Software. Vivaldi’s interface is very customizable. Vivaldi combines simplicity and fashion to create a basic, highly customizable interface that provides everything a internet user could need. The browser allows users to customize the appearance of UI elements such as background color, overall theme, address bar and tab positioning, and start pages. Vivaldi features the ability to "stack" and "tile" tabs, annotate web pages, add notes to bookmarks and much more. Vivaldi 7.4.3684.50 changelog: [Chromium] Update to 136.0.7103.170 [Crash][Mail][JSError] RangeError: Maximum call stack size exceeded (VB-115288) Download: Vivaldi 64-bit | 125.0 MB (Freeware) Download: Vivaldi 32-bit | ARM64 View: Vivaldi Home Page | Screenshot Get alerted to all of our Software updates on Twitter at @NeowinSoftware
    • I must be doing something right because I feel like I don’t really come across that many cheaters. I’m not saying you don’t, I just must be really lucky.
  • Recent Achievements

    • Apprentice
      Adrian Williams went up a rank
      Apprentice
    • Reacting Well
      BashOrgRu earned a badge
      Reacting Well
    • Collaborator
      CHUNWEI earned a badge
      Collaborator
    • Apprentice
      Cole Multipass went up a rank
      Apprentice
    • Posting Machine
      David Uzondu earned a badge
      Posting Machine
  • Popular Contributors

    1. 1
      +primortal
      519
    2. 2
      ATLien_0
      260
    3. 3
      +Edouard
      192
    4. 4
      +FloatingFatMan
      178
    5. 5
      snowy owl
      135
  • Tell a friend

    Love Neowin? Tell a friend!