Tuesday, May 20, 2014

How to: CentOS Virtualmin Automated Remote Back-ups with rdiff-backup – PULL

1. Lets start with installing rdiff-backup on the backup server and then host-server (server to be backed up to the backup server) – make sure you get the same version installed on both servers:
1yum install rdiff-backup
2. On the backup server, create a new account (Using Webmin) which will be used to perform the backup:
Username:
1rdiff-backup
Real name:
1rdiff-backup
Shell:
1/etc/false
Password:
1No Login Allowed

3. Create a passphrase-free ssh key on backup server that will be placed on the  host server so that thebackup server can log in to the host server and access the files automatically:
So on the backup server:
1su -m rdiff-backup
1ssh-keygen -t rsa
1Generating public/private rsa key pair.
2Enter file in which to save the key (/rdiff-backup/.ssh/id_rsa): /backup/.ssh/id_rsa
3Enter passphrase (empty for no passphrase):
4Enter same passphrase again:
5Your identification has been saved in /backup/.ssh/id_rsa.
6Your public key has been saved in /backup/.ssh/id_rsa.pub.
7The key fingerprint is: e0:fc:4a:8a:51:a8:c7:3a:e4:3a:3c:22:f9:4e:35:ca rdiff-backup@example
 Your key fingerprint will almost certainly differ from the example here.
Create an ssh config alias which defines how to contact the host server with the backup key. Place the following into /home/rdiff-backup/.ssh/config:
1vim /home/rdiff-backup/.ssh/config
Paste this into the file and insert your settings where needed:
1host example_backup
2hostname xxx.xxx.xxx.xxx
3port xx
4user root
5identityfile /home/rdiff-backup/.ssh/id_rsa
6compression yes
7protocol 2
## host = your name for the host server
## hostname = the hostname or IP of the host server
## port = your SSH port
 Note that “compression yes” is optional, and you may wish to omit it the servers are connected over high-speed nets. 
This config entry enables the backup server to use the “hostname” example-backup wherever ssh expects a real hostname. ssh will use the information specified in the config file, which will result in a  automatic connection to the host server, using the specified key
You may need to make some file permission adjustments, it depends on your system:
1chmod -R go-rwx /home/rdiff-backup/.ssh

4. Give permission for the backup server to access the host server and run rdiff-backup.
You need the public portion of the key you just generated on the backup server:
1cat /home/rdiff-backup/.ssh/id_rsa.pub
You should see a really long key that starts with something like this – with example.com being your backup servershostname.
1ssh-rsa AAAAB3NzaC1yc2EAAAAB[...] rdiff-backup@example.com<script type="text/javascript">
2/* <![CDATA[ */
3(function(){try{var s,a,i,j,r,c,l,b=document.getElementsByTagName("script");l=b[b.length-1].previousSibling;a=l.getAttribute('data-cfemail');if(a){s='';r=parseInt(a.substr(0,2),16);for(j=2;a.length-j;j+=2){c=parseInt(a.substr(j,2),16)^r;s+=String.fromCharCode(c);}s=document.createTextNode(s);l.parentNode.replaceChild(s,l);}}catch(e){}})();
4/* ]]> */
5</script>

Assuming that root@host server`s home directory is /root, we will construct a terribly long line in the file /root/.ssh/authorized_keys2 (on the host server). The line is so long that I’m going to break it in two here for demonstration purposes only, you must join this first line and the public key from above on one line, with only a space between them:

Now on the host server:
1vim /root/.ssh/authorized_keys2
We will no paste the really long line in there – but we are splitting in up into two to make it easier to do-
so the firs part is the command that is allowed – with xxx.xxx.xxx.xxx being ether your backup servers hostname or IP
1rdiff-backup --server --restrict-read-only /",from="xxx.xxx.xxx.xxx",no-port-forwarding,no-X11-forwarding,no-pty
now place a space after the no-pty
then the second part is the long ssh-rsa we got from - cat /home/rdiff-backup/.ssh/id_rsa.pub in the previous step
1ssh-rsa AAAAB3NzaC1yc2EAAAAB[...] rdiff-backup@example.com<script type="text/javascript">
2/* <![CDATA[ */
3(function(){try{var s,a,i,j,r,c,l,b=document.getElementsByTagName("script");l=b[b.length-1].previousSibling;a=l.getAttribute('data-cfemail');if(a){s='';r=parseInt(a.substr(0,2),16);for(j=2;a.length-j;j+=2){c=parseInt(a.substr(j,2),16)^r;s+=String.fromCharCode(c);}s=document.createTextNode(s);l.parentNode.replaceChild(s,l);}}catch(e){}})();
4/* ]]> */
5</script>
so your final line should look something like this:
1command="rdiff-backup --server --restrict-read-only /",from="xxx.xxx.xxx.xxx",no-port-forwarding,no-X11-forwarding,no-pty ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQBFRCWgRVs0BhZ2FPxUgRsuWgSSLV7dD+7Db8uV5PRZxAUWWmO8Wbqh0C1daMT86D3BbBwtbC9Clxr7XgalHoL8v65qgl0TZBecGdFWC4Zt+s2YmkUmVcPv/idfnRCHgIhoUKsadsadzzHWef6p7MMCfCp/z7/34357fiwbQ0RCcWnQMORO/dgMJ5LSsXKCOPzsbTt1/i8NulTUNCjlVb4LsCGFX1wscRbXNT4Y6fKMURK0y3kvZ7s888+6utjPnLjVeR7FNL3NtAZyhvbZe6mDFf9G6Usef+4bCgpNBG2Ff2JPbpv0KW+S/Q2jIPhmhJFXs7IYwhPRSoJNUefjdBpvF7xRs/w== rdiff-backup@example.com<script type="text/javascript">
2/* <![CDATA[ */
3(function(){try{var s,a,i,j,r,c,l,b=document.getElementsByTagName("script");l=b[b.length-1].previousSibling;a=l.getAttribute('data-cfemail');if(a){s='';r=parseInt(a.substr(0,2),16);for(j=2;a.length-j;j+=2){c=parseInt(a.substr(j,2),16)^r;s+=String.fromCharCode(c);}s=document.createTextNode(s);l.parentNode.replaceChild(s,l);}}catch(e){}})();
4/* ]]> */
5</script>
This entry in /root/.ssh/authorized_keys2 permits anyone with the specified key (i.e. rdiff-backup@example.com(example.com = backup servers hostname)) to connect with ssh from the host named xxx.xxx.xxx.xxx (backup servers hostname or IP)  and issue the forced rdiff-backup command. It further restricts the ssh connection to eliminate port forwarding, X11 forwarding and a pty. The rdiff-backup invocation is also restricted to read-only operations starting from the root of the file system.
If you have any troubles, this step is the one which has most likely caused you problems. Here are some troubleshooting guidelines:
  • Make sure there are no line breaks in the authorized_keys2 entry.
  • Use the reverse DNS response for kitty’s IP address in from="backup server".
  • Make sure you copied the public key properly.
  • Ensure file permissions are set properly:
    To set the permissions run this command on the host server
    1chmod -R go-rwx /root/.ssh
  • Make sure rdiff-backup is in root’s PATH, or add a full path to command="/path/to/rdiff-backup...".
  • Look in /var/log for help.

5. Perform a test backup and populate known_hosts.
You should now be able to perform a test backup. During this test ssh will probably ask you to accept the host serverhost key — you will need to complete this step before you can begin an unattended backup.
On the backup server:
1cd /home/rdiff-backup
1rdiff-backup example_backup::/tmp test-backup
If you are asked for a password or passphrase then something is wrong. Other than asking you to verify the host key it should succeed in performing a backup of example_backup::/tmp in  test-backup.
Assuming the first attempt asked you to verify the host key, run the test a second time to verify that it asks you nothing.

6. To include and exclude files and folders in the backup process.
We will create a file list that can both include and exclude files and folders on the host server to be backed up to thebackup server. Its best to place this file in a folder in the rdiff-backup users home directory.
On the backup server:
Create a file called filelist.txt
1vim /home/rdiff-backup/filelist.txt
##To exclude files/folders we have to place a “-” in front of the file/folder.
##To include files/folders we simply place the file/folders name in the list.
So lets place something like this into to file:
1- /tmp
2- /mnt
3- /proc
4- /var/tmp
5- /var/cache
6- /home/rdiff-backup
7- /home/vdomains
## I exclude some of the normal directories but also the directory where other server backups are stored on the host server (/home/rdiff-backup) as as well as the home of my Virtualmin Domains, as I back this up using the Virtualmin backup feature as to also backup the Databases!
7. Create a cron job on the backup server  to initiate your backup – use webmin.
Execute as: 
1rdiff-backup
Command:
1rdiff-backup --include-globbing-filelist /home/rdiff-backup/config/filelist.txt example_backup::/ /home/rdiff-backup/example.com
Time: