• 0

[Bash] Shell script that does backups


Question

Ive been asked (Im thinking about doing this to my system too) to make a shell script that backups all files in x location to another location in y. The thing is that the backup must only be made if something has been changed. Example:

/x contains:

a (1 byte)

b (1 byte)

c (1 byte)

/y contains:

nothing

Ill put this script into crontab and everytime (say 3) it will execute. 3 arrives and there is nothing in /y and it will copy all files in /x to /y. To compensate space I think a good idea (this I will do to my script for my system) is gzip it all up. Anyways the current state (at 3:01) right now is.

/x contains:

a (1 byte)

b (1 byte)

c (1 byte)

/y contains:

a (1 byte)

b (1 byte)

c (1 byte)

Again 3 arrives and /y is the same as /x so nothing changes and nothing happens. But now, I open c in /x , type something inside and save it now it is:

/x contains:

a (1 byte)

b (1 byte)

c (2 bytes)

This changes it so now when the script excutes at 3 it has to delete everything in /y and rewrite it all over again with everything that is in /x (regardless that the other files havent changed).

How do I do this? I imagine something like doing a ls -a on /x and using cut to cut certain columns and seeing if it equals /y's files/file sizes but im not sure how to make a filename and file size relation or if it is even correct and a bit humiliating but I dont know how to use cut either (I know something but not enough to do this)

If someone could help thank you very much.

Link to comment
Share on other sites

8 answers to this question

Recommended Posts

  • 0

What you're lookingfor is rsync.

For the simple task you're looking for, just schedule "rsync -a source/ destination/" to run regularly and you're done.

If you'd like your backup to delete files that are removed from your source directory then add the --delete option:

/source
-- a
-- b
-- c

/destination
-- a
-- b
-- c

If you delete B, and add D from source and then run "rsync -a --delete source/ destination/" then you'll have:

/source
-- a
-- c
-- d
/destination
-- a
-- c
-- d

If you leave out the delete option then you'll end up with destination containing "a, b, c, and d" while source only contains "a,c, and d".

No, if you want to get really tricky you can use a combination of hard links and rsync to create backups spanning several days using only the disk space required for changes. You can google for that one: it's not hard to do and there are dozens of examples to be found.

Using file size alone is not a good way to determine if a file has been edited. For a trivial example consider the following:

int main(int argc, char** argv) {
  printf("Die in a fire!");
  return 1;
}

vs:

int main(int argc, char** argv) {
  printf("I love my cats");
  return 0;
}

Clearly very different files, but a simple file-size comparison won't pick that up, you'd need to compare hashes in order to be sure that your files are being updated correctly.

If you're doing this for source code then you should probably look into a proper versioning system like git, svn, hg, etc. They work just fine pulling from local drives as well as remote servers and will allow you to mix & match to get a stable edition from multiple earlier versions.

Link to comment
Share on other sites

  • 0

And if you want to maintain a file how it was during a previous backup, you might look at rdiff-backup. It's based on rsync but you can get a file back the way it was at a certain date, assuming a backup was done then.

Link to comment
Share on other sites

  • 0
Thanks but I rather use/make a shell script

If you've got a lust for using BASH's scripting syntax exclusively then you're not going to get very far. It can do simple tests ("file exists", "x > y", etc) but it lacks the equivalent of say Python's DirHandle/FileHandle modules, nor can it do detailed file comparison (ie: using cryptographic hashes). BASH also lacks facilities for copying or moving files itself, it farms those requests out to other utilities: typically cp/mv/mkdir/etc from fileutils.

Once you start using BASH to call external applications you're in the strange realm of "why not just use rsync?" It's about as common on modern *nix systems as the BASH shell and you don't gain anything by ignoring it.

If you just want a shell script but don't care what you write it in then Python, Ruby, and PERL are all good choices, BASH: not so much.

Link to comment
Share on other sites

  • 0

I understand :) and again thank you veru much for all of your help. But Id still rather use a shell script. If some things cant be done, so be it. Ill communicate this small problem. But please stop suggesting rsync. Thank you :)

And if someone can give me tips on a shell script for this, thanks again.

Link to comment
Share on other sites

  • 0
I understand :) and again thank you veru much for all of your help. But Id still rather use a shell script. If some things cant be done, so be it. Ill communicate this small problem. But please stop suggesting rsync. Thank you :)

And if someone can give me tips on a shell script for this, thanks again.

This is pretty basic but should do what you need:

cd x
for i in `find . -mtime 0`; do install -D $i ../y/$i; done
cd ../y
for i in `find .`; do if [ ! -e "../$i" ]; then rm $i; fi; done

It uses modified times instead of file sizes which should be more accurate.

Link to comment
Share on other sites

This topic is now closed to further replies.
  • Recently Browsing   0 members

    • No registered users viewing this page.