• 0

Software to detect duplicates, with a difference?


Question

So to cut out a load of boring waffle for you, I need software to detect duplicate files.

 

I've tried Auslogics (as it was free) and it seemed to work well - for 100% identical files.

 

However, I have a ton of files that are 99% identical. By that I mean I have files that are basically the same but they have a -1 at the end of the file name. The rest of the file name is the same, the file size is the same, if they're a video then the duration is the same - but one will have a -1.

 

So we're talking about a file like: mycat.jpg and then another file called mycat-1.jpg ... where they're the exact same picture.

 

Is there such a software for Windows that can detect duplicates in this manner? I swear I must have many GB I could free up. I've already done 10GB off of 1 small subfolder.

8 answers to this question

Recommended Posts

  • 0

I used HashMyFiles to scan MD5 hashes (other hash types selectable) in various directories to scan for duplicate photos. Worked great. Name of file was irrelevant. You can open/scan multiple directories simultaneously. Duplicate hashes are handily color coded. Just picked which one to delete and done.

 

https://www.nirsoft.net/utils/hash_my_files.html

 

 

HashMyFiles.png

Edited by JustGeorge
  • Like 3
  • 0

If this is specific to digital camera photo's, check out https://www.advancedrenamer.com/

 

(Free for personal use)

 

It plugins to exif command line, and you can rename files to be Camera Model, Shutter Count (provided the camera embeds the the info).

 

I do like JustGeorge suggestion and taking a look at that myself for my various back up archive files.

  • 0

Thanks for the suggestions guys.

 

Turns out the software I was using, Auslogics, which is also free, does exactly what I need.

 

Issue was - I just didn't know how to use it properly at the point of asking.

 

Got it sorted now. Scanned 2 folders already & 10GB of duplicates in each for a 20GB free up.

  • 0

I used AllDup portable for video and images, but it isn't great, needs multiple passes to be able to pick out all dupes.

 

I will have to give a few of the suggestions in this topic a go.

  • 0
  On 23/09/2022 at 16:37, Technique said:

So to cut out a load of boring waffle for you, I need software to detect duplicate files.

 

I've tried Auslogics (as it was free) and it seemed to work well - for 100% identical files.

 

However, I have a ton of files that are 99% identical. By that I mean I have files that are basically the same but they have a -1 at the end of the file name. The rest of the file name is the same, the file size is the same, if they're a video then the duration is the same - but one will have a -1.

 

So we're talking about a file like: mycat.jpg and then another file called mycat-1.jpg ... where they're the exact same picture.

 

Is there such a software for Windows that can detect duplicates in this manner? I swear I must have many GB I could free up. I've already done 10GB off of 1 small subfolder.

Expand  

If the files are the same, the filename shouldn't matter because filenames don't affect checksums, so you would think Auslogics would detect it as being a duplicate anyway.

 

Just saw your follow-up comment, glad you got it sorted, 🙂

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Recently Browsing   0 members

    • No registered users viewing this page.
  • Posts

    • That will generate <1min shorts videos for me, based on text input? (and maybe a stock image upload, if not, text input only is fine) I understand that there may not be any that DON'T require a login or signup, which is fine, I'll create a login if needed - but are there any free services where i don't need a sub or to refill credits or whatnot to continue using? Thanks!
    • I'll be getting one (upgrading from AW6). Hoping it's got 3 day battery life. I'll take 2, but prefer 3.
    • I would suggest Ubiquiti Cloud Gateway Fiber + U7 Pro XGS   https://store.ui.com/us/en/category/all-cloud-gateways/collections/cloud-gateway-fiber/products/ucg-fiber https://store.ui.com/us/en/category/wifi-flagship/products/u7-pro-xgs?variant=u7-pro-xgs-us I have been using this combo for 3 months now.
    • YouTube app is no longer compatible with several iPhones by Taras Buria Google is rolling out a new update for the YouTube app on iOS, and it is bad news for those still using an old iPhone. Starting with version 20.22.1, the latest release in the App Store, YouTube for iPhone now requires iOS 16 and newer. Previously, the app required a device with iOS 15 and newer. The new hardware requirement makes several iPhones incompatible with the latest version of the YouTube app. Those include the iPhone 6S / 6S Plus, iPhone 7 / 7 Plus, and the original iPhone SE. The seventh-generation iPod touch is also no longer compatible. On the iPad side, the update kicks out the second-generation iPad Air and the fourth-generation iPad mini. Although old iPhones can no longer install YouTube version 20.22.1 and newer, old versions continue working just fine. However, going forward, users will not receive new features and improvements. Also, at some point in the future, the app will stop working altogether. Another workaround is to use the web version of the app, but that also provides a limited experience compared to the up-to-date client. Google is not the only service that kicks out old and now unsupported Apple devices. Recently, WhatsApp developers dropped iPhones that cannot run iOS 15.1 or newer. The news is clearly a bummer for those who are still using their old iPhones, which, to Apple's credit, have a solid lifespan (as long as you are happy with the battery life). However, it is also worth keeping in mind that the iPhone 6S series was released almost a decade ago. Having ten years of support is impressive, but it is clearly time to move on. Perhaps my friend with his day-one iPhone 7 Black can finally consider buying something less ancient.
  • Recent Achievements

    • One Month Later
      Parotel earned a badge
      One Month Later
    • Reacting Well
      Cryptecks earned a badge
      Reacting Well
    • Week One Done
      primussolicitors798 earned a badge
      Week One Done
    • First Post
      nothin earned a badge
      First Post
    • Enthusiast
      Epaminombas went up a rank
      Enthusiast
  • Popular Contributors

    1. 1
      +primortal
      188
    2. 2
      snowy owl
      134
    3. 3
      ATLien_0
      130
    4. 4
      Xenon
      119
    5. 5
      +FloatingFatMan
      97
  • Tell a friend

    Love Neowin? Tell a friend!