need help with rsync

trexos

Hi,

I want to sync my local music files with the older backup on my vps. As my upload speed is quite low, I don't want to just upload my whole music library (~10GB), just the new files and maybe if I deleted a file locally, so it should be deleted on my vps. I read that rsync is perfect for that, but I'm not sure which command I have to use for it. Is this the right one?

rsync -av --delete /Directory1/ /Directory2/


What I need:

*sync files between two directorys

*if there is a new file on my local pc upload it to the server

*if a file is deleted locally, delete it on my server

*if a file is changed locally, change it on my server

So I need my music locally backuped 1:1 to my VPS. Can you please help me with the rsync command?

Thank you

rajprakash

http://www.thegeekstuff.com/2010/09/rsync-command-examples/

trexos

@rajprakash said:
http://www.thegeekstuff.com/2010/09/rsync-command-examples/

Thanks. According to this website this is my command:

rsync -avz /root/temp/ [email protected]:/home/thegeekstuff/temp/

Does this change anything on my local directory? Does it delete files on remote if they are deleted locally?

hdpixel

This is what I use to back up a VPS folder to another VPS server.

one line below

/usr/bin/rsync -avzHx --delete --stats --progress --exclude-from '/home/localfolder/rsync-exclude.txt' -e "ssh -2 -p 22" /home/localfolder/ [email protected]:/home/remoteuser/remote-backup-folder/

The content of rsync-exclude.txt is as follows:

logs/*

public_html/tmp/*

MassNodes

@trexos said:
Does this change anything on my local directory? Does it delete files on remote if they are deleted locally?

It wont delete locally unless you add the --delete flag.

trexos

@MassNodes

and will it delete remotely? without --delte

sleddog

trexos said: Does this change anything on my local directory? Does it delete files on remote if they are deleted locally?

No, it doesn't change anything locally. It mirrors the local /root/temp dir to the remote machine.

If you want to delete file on the remote that have been removed on the local, add in the --delete switch you had in your first post.

Also, you need to be careful with trailing slashes:

rsync -avz --delete /root/temp/ [email protected]:/home/thegeekstuff/temp

Note the absence of a trailing slash on the destination directory (the remote end).

You can use --dry-run to test without transferring anything:

rsync -avz --delete --dry-run /root/temp/ [email protected]:/home/thegeekstuff/temp

And you probably aren't running an rsync server on the remote, so you can transport it over ssh.

rsync -avz --delete --dry-run -e 'ssh' /root/temp/ [email protected]:/home/thegeekstuff/temp

It'll ask you for your ssh password. If you want to automate the syncing, setup a keypair between the two boxes, then:

rsync -avz --delete --dry-run -e 'ssh -i /path/to/key' /root/temp/ [email protected]:/home/thegeekstuff/temp

And when it's all working as you want, change the v (verbose) to q (quiet) and remove the dry-run:

rsync -aqz --delete -e 'ssh -i /path/to/key' /root/temp/ [email protected]:/home/thegeekstuff/temp

trexos

@sleddog

Thank you very much for this great explanation!
Just a little questions, what would happen if I would use the command you posted with the trailing slash on the remote end:

rsync -avz --delete --dry-run /root/temp/ [email protected]:/home/thegeekstuff/temp/

sleddog

I think you'd end up with everything on the remote end stored in /home/thegeekstuff/temp/temp/

Do some small transfers to test....

trexos

error found^^

trexos

So now I want it the other way round. I already checked the man page, so I just have to change the order from:

rsync -avz --delete --dry-run /root/temp/ [email protected]:/home/thegeekstuff/temp/

to:

rsync -avz --delete --dry-run [email protected]:/home/thegeekstuff/temp/ /root/temp/

Am I right?

mikho

Should work

JustAMacUser

Just a thought: I usually use --delete-after instead of --delete in case I (or an application) accidentally move files. This way I'll notice rsync uploading several gigabytes before deleting them. It gives me a chance to abort and move files on the server to save time and bandwidth.

trexos

@MikHo said:
Should work

Thank you. Will try it later

@JustAMacUser said:
Just a thought: I usually use --delete-after instead of --delete in case I (or an application) accidentally move files. This way I'll notice rsync uploading several gigabytes before deleting them. It gives me a chance to abort and move files on the server to save time and bandwidth.

Thank you for the suggestions, so with the --delete-after flag it uploads the new files and then in the end it deletes the old files?

---

Is there a way to exclude system folders starting with . like .ssh from syncing? I didn't find anything.

mikho

theres the --exclude parameter .
Note: the directory path is relative to the folder you are backing up. Best to use full path.

sleddog

trexos said: Is there a way to exclude system folders starting with . like .ssh from syncing? I didn't find anything.

Use --exclude or --exclude-from.

The path you specify is relative to your source base dir, i.e., if your syncing /home/thegeekstuff and want to exclude /home/thegeekstuff/.ssh you'd specify:

rsync --exclude=.ssh/ ...

If you have mutiple dirs/files to exclude you can put them in a text file and reference it with exclude-from:

rsync --exclude-from=/path/to/exclude-file ...

The file /path/to/exclude-file contains a list of relative paths and filenames, one per line. e.g.:

.ssh/
some-dir/subdir/
somefile.png

trexos

@MikHo and @sleddog

Thank you. I will do this later

---

But somehow my new command doesn't work as expected. I use this command on my local machine.

rsync -avz --delete -c --dry-run -e 'ssh -p 1234' [email protected]:/home/USER/bilder /home/USER/Pictures/bilder/

I already have a few files on my local machine under

/home/user/Pictures/bilder

but it looks like it will sync a lot of files (I only tried it with --dry-run). I added -c so that it uses md5sums for file comparison, but it still wants to sync way too much files.

Edit: if it helps, I copied the files which are already on my local machine with FileZilla.

JustAMacUser

@trexos said:
Thank you for the suggestions, so with the --delete-after flag it uploads the new files and then in the end it deletes the old files?

Yes. It will delete files on the remote side that have been deleted from the local side. (Excluded files are ignored.)

sleddog

From the man page...

            --delete                delete extraneous files from dest dirs
            --delete-before         receiver deletes before xfer, not during
            --delete-during         receiver deletes during the transfer
            --delete-delay          find deletions during, delete after
            --delete-after          receiver deletes after transfer, not during
            --delete-excluded       also delete excluded files from dest dirs

trexos

@sleddog said:
From the man page...

            --delete                delete extraneous files from dest dirs
>             --delete-before         receiver deletes before xfer, not during
>             --delete-during         receiver deletes during the transfer
>             --delete-delay          find deletions during, delete after
>             --delete-after          receiver deletes after transfer, not during
>             --delete-excluded       also delete excluded files from dest dirs

I already read that.