• 0

Any FileSync software that does portable delta synchs?


Question

So I?m familiar with a lot of file sync tools, but I've never encountered one that can work this way, and am hoping someone out there might know of something.

Basically, I have a lot of data that moves with me daily. I would like to leave my portable drive permanently at work, and perform a synch where a third "cache" drive holds the knowledge base of both drives as well holds the files as they move between both locations. Cloud storage is impractical.

In the end I want to use a 2TB drive to synch 8TB datasets. A file sync program that is capable of detecting bitrot via an internal hashing system is preferable, but at this point anything that can do a delta sync is worth inspecting. No file in the data set is larger than 200gigs, and most of it doesn't change frequently, but I make changes on both sides of the data.

Link to comment
Share on other sites

4 answers to this question

Recommended Posts

  • 0

So you want to use 3rd drive as transport method of the diffs.

That would be rdiff, create signatures of your files, then use that signature to compare files in location 2 to create the diffs(delta), then apply those diffs(patch) to location 1. Rinse and Repeat ;)

http://linux.die.net/man/1/rdiff

Sure it could be scripted, there might be some gui type application/frontend?

What OS(es) are you working with? If you can get network connectivity between the 2 locations I would just use rsync.. No 3rd party cloud anything.. Only amount of data moved would be the diffs created every day.

Do you think you actually created 2TB of diffs? More than likely even if you go the rdiffs route you could just use a simple flash drive to move the diffs back and forth. If your not talking LARGE diffs - I would look to just using rsync over network connection. Doesn't even have to be all that fast depending on how much diff and how long your willing to wait for the changes. For example if you run the sync when your done at work, I would hope by time you got home the changes would be done. Then say you do some work at home, kick off a sync and when you go to work the next morning you again should be in sync.

Link to comment
Share on other sites

  • 0

Presently I use Unison for sync, Robocopy for mirror, and fciv.exe for hashing. So long as a tool can run in Windows Server 2012 and supports Unicode characters I can consider it. Rdiff is probably more headache than it?s worth unless someone?s made a native Windows version.

2TB of diffs really are unlikely unless something prevents me from synching for a month+. Really the maximum daily would probably cap out around 300 gigs on a bad day, and 25-50gigs on an average day.

I have bandwidth caps from my ISP, and I already skim real close to my monthly max. I either need a solution that uses the portable eSATA drive I have now, or just replace it with semi-portable raid array.

Link to comment
Share on other sites

  • 0

This is port of rdiff for windows

http://personal.hlfs...aszu/rdiff.html

so if I have file1.doc at both work and home

Create sig of the file1.doc

rdiff signature file1.doc file1.sig

then make my changes to file1, then create the delta

rdiff delta file1.sig file1.doc file1.delta

take the file1.delta home

rdiff patch file1.doc file1.delta file1.tmp

then just move file1.tmp to file1.zip and your good to go.

rinse and repeat as needed, sure you could right a simple script to run through all your files and create deltas of files that have changed and then copy the delta's to your transport disk. Then repeat that in your remote location to move your remote location changes back, etc.

This is was just quick off the top of my head how you could do it with rdiff, you could prob write a simple gui around it with autoIT or something as well if you really wanted.

Link to comment
Share on other sites

This topic is now closed to further replies.
  • Recently Browsing   0 members

    • No registered users viewing this page.