• 0

Software to detect duplicates, with a difference?


Question

So to cut out a load of boring waffle for you, I need software to detect duplicate files.

 

I've tried Auslogics (as it was free) and it seemed to work well - for 100% identical files.

 

However, I have a ton of files that are 99% identical. By that I mean I have files that are basically the same but they have a -1 at the end of the file name. The rest of the file name is the same, the file size is the same, if they're a video then the duration is the same - but one will have a -1.

 

So we're talking about a file like: mycat.jpg and then another file called mycat-1.jpg ... where they're the exact same picture.

 

Is there such a software for Windows that can detect duplicates in this manner? I swear I must have many GB I could free up. I've already done 10GB off of 1 small subfolder.

8 answers to this question

Recommended Posts

  • 0

I used HashMyFiles to scan MD5 hashes (other hash types selectable) in various directories to scan for duplicate photos. Worked great. Name of file was irrelevant. You can open/scan multiple directories simultaneously. Duplicate hashes are handily color coded. Just picked which one to delete and done.

 

https://www.nirsoft.net/utils/hash_my_files.html

 

 

HashMyFiles.png

Edited by JustGeorge
  • Like 3
  • 0

If this is specific to digital camera photo's, check out https://www.advancedrenamer.com/

 

(Free for personal use)

 

It plugins to exif command line, and you can rename files to be Camera Model, Shutter Count (provided the camera embeds the the info).

 

I do like JustGeorge suggestion and taking a look at that myself for my various back up archive files.

  • 0

Thanks for the suggestions guys.

 

Turns out the software I was using, Auslogics, which is also free, does exactly what I need.

 

Issue was - I just didn't know how to use it properly at the point of asking.

 

Got it sorted now. Scanned 2 folders already & 10GB of duplicates in each for a 20GB free up.

  • 0

I used AllDup portable for video and images, but it isn't great, needs multiple passes to be able to pick out all dupes.

 

I will have to give a few of the suggestions in this topic a go.

  • 0
  On 23/09/2022 at 16:37, Technique said:

So to cut out a load of boring waffle for you, I need software to detect duplicate files.

 

I've tried Auslogics (as it was free) and it seemed to work well - for 100% identical files.

 

However, I have a ton of files that are 99% identical. By that I mean I have files that are basically the same but they have a -1 at the end of the file name. The rest of the file name is the same, the file size is the same, if they're a video then the duration is the same - but one will have a -1.

 

So we're talking about a file like: mycat.jpg and then another file called mycat-1.jpg ... where they're the exact same picture.

 

Is there such a software for Windows that can detect duplicates in this manner? I swear I must have many GB I could free up. I've already done 10GB off of 1 small subfolder.

Expand  

If the files are the same, the filename shouldn't matter because filenames don't affect checksums, so you would think Auslogics would detect it as being a duplicate anyway.

 

Just saw your follow-up comment, glad you got it sorted, 🙂

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Recently Browsing   0 members

    • No registered users viewing this page.
  • Posts

    • Apple: lets take our inspiration from Windows 7/Vista and call it innovative
    • Nice! My $100 refurbished iPad 8 that I bought last year for a small project (and still runs like new cuz it's been mostly in storage the past several months) will get this v26 upgrade. I'm definitely gonna scope out OS26 on this device before upgrading my also-supported refurbished iPhone SE 3rd Gen (only got it for the 5G UW support).
    • Hello, The separate discussion about VLC has been moved to its own thread at  Regards, Aryeh Goretsky  
    • Hello, I am guessing this is the result of using a third-party tweaking tool instead of changing things within Windows settings. Windows 11 has a decent set of configuration options for the display, but they tend to be scattered throughout the operating system due to fragmentation of product ownership/inconsistent standards being applied by Microsoft. There are a few places in Windows 11 where you can modify the various display settings in order to get them to your liking: Under Settings → System → Display you can stroll down to the Scale & layout section and change the Scale and Display resolution settings to whatever you prefer.  For the Scale option, you can click on the text box to choose between the various pre-populated settings.  If none of those are exactly what you are looking for, click on the caret ("") caret to open the Custom scaling setting (also available under Settings → Display → Custom scaling).  I would recommend this as an option of last resort as this can cause display issues in various programs. I strongly recommend using the screen's native Display resolution here; it should show up as the "(Recommended)" resolution.  If it does not, you may need to install your monitor's profile information file (basically, a device driver for your monitor) to get the correct setting. Another place to look at making changes to fonts is under Accessiblity → Text size.  There is a slider bar for changing the text size from 100% to 225% there. You can try making changes to the ClearType settings.  It's a little bit finer-grained than the others, but it may make the text easier to view on the display.  Run the ClearType Text Tuner (filename: CTTUNE.EXE) and go through the wizard. You may want to do this several times under different lighting conditions or times of the day to figure out what works best for you overall. Lastly, you may want to look into using a dedicated assistive screen technology program from a third-party.  Microsoft provides a Screen Magnifier and a Narrator for text-to-speech, but they provide only basic functionality, and you may find that a third-party program works a lot better.  The pros of this is that third-party assistive technology programs tend to work quite well under Windows; Microsoft has historically worked closely with third-party developers of assistive technologies.  The cons of this are that such tools tend to be on the expensive side, since they are often sold to schools, businesses, or paid for by insurance companies, not individuals. Regards, Aryeh Goretsky  
    • Totally agree with you. I think people, especially on tech sites, need to realize that they’re the minority. Microsoft isn’t going to waste time on things that, in the big picture, don’t matter to the majority.
  • Recent Achievements

    • Dedicated
      MadMung0 earned a badge
      Dedicated
    • Rookie
      CHUNWEI went up a rank
      Rookie
    • Enthusiast
      the420kid went up a rank
      Enthusiast
    • Conversation Starter
      NeoToad777 earned a badge
      Conversation Starter
    • Week One Done
      VicByrd earned a badge
      Week One Done
  • Popular Contributors

    1. 1
      +primortal
      506
    2. 2
      ATLien_0
      267
    3. 3
      +FloatingFatMan
      257
    4. 4
      Edouard
      203
    5. 5
      snowy owl
      178
  • Tell a friend

    Love Neowin? Tell a friend!