|
|
You've got the theory of backups under your belt, and you know some of the tools. Now let's get down to it and start giving some examples of useful methods in which to perform various types of backups.
You may have seen this done, but here is how to tar up all the user's home directories in the /home directory:
# tar -czvf home-dir-userdirs.tgz /home
The flags (czvf) stand for create, z compress, verbose, and output file. Then the two (or more) options the command takes are the target tgz file (including the file extension) and then the director(ies) to be tar-balled into the target file.
Want something more creative? How about combining the find tool with tar. Use them together to find all new files on the hard drive that belong to the user tweeks and are less than 30 days old, and put them in backup archive:
# find / -user tweeks -ctime -30 -exec tar Azvf /home/BACKUPS/
tweeks-new- files.tgz {}\ ;
Or use tar to back up all the standard OS and user files and directories on a system to tape locally:
# tar czvf /dev/tape /bin /boot /dev /etc /home /lib /lost+found
/opt /root /sbin /tmp /usr /var
That will back up everything to the tape. One nice command, no unneeded fancy backup software to get in the way.
| Caution |
This last tar command is not guaranteed to backup active and changing files. You still must shut off important services and stop writing to key files that you want backed up. The undefined system runlevel of 4 is perfect for shutting down active disk writing services and doing backups. Look back to the Shutting Off Services" section to see how this is accomplished. |
Using the customized runlevel 4 Stop and Back up method (not a Freeze and Copy; that was for the database), we can now apply a simple backup script such as this one to go to runlevel 4, run a set of full tar backups, and e-mail us the results.
#!/bin/bash ## backup-rl -4.sh ## A simple script that switches to runlevel 4, tars up the dirs ## to tape, emails the results, and changes back to the previous runlevel. ## Use the COMPRESS and BACKUPDIRS variables below to change how it runs ## Tape device or symlink expected is /dev/tape. ######## User Variables ########### RLBAK="4" ##COMPRESS="" # compression off="" COMPRESS="z" # compression on="z" BACKUPDIRS="/bin /boot /dev /etc /home /lib /lost+found /opt /root /sbin /tmp /usr /var" ## All dirs "/bin /boot /dev /etc /home /lib /lost+found /opt /root /sbin /tmp /usr /var" MYEMAIL="admin@example.com" TAPE=/dev/tape ########### Do not change ########### RL="$(runlevel | cut -f2 -d" ")" RLORIG="$RL" CURDIR=$(pwd) ## Do a MySQL-HOT-COPY to /root/mysql-bak ####################################### echo -Performing a MySQL Hot Copy to /root/mysql-backup mysqlhotcopy --user=root --allowold --regexp=".*" /root/mysql-backup echo -MySQL hotcopy complete... ## Change Run Levels and Log Activity ###################################### echo echo -Current runlevel is $RLORIG logger -t BACKUP "Runlevel 4 tar backup script was run..." echo -Changing run levels to $RLBAK for backups logger -t BACKUP "Changing run levels to $RLBAK for backups" init "$RLBAK" sleep 10 echo -Now in runlevel $(runlevel | cut -f2 -d" "), starting backups logger -t BACKUP "Now in runlel $(runlevel | cut -f2 -d" "), starting backups" echo " -preparing to backup directories $BACKUPDIRS to tape" logger it BACKUP " -preparing to backup directories $BACKUPDIRS to tape" ## Do the needful ####################################### echo " -rewinding tape" mt rewind echo -tarring content to tape tar -c$COMPRESS -f $TAPE $BACKUPDIRS > /dev/null 2>&l ERROR=$? if [ $ERROR -ne 0 ] ; then #logger -t BACKUP "Failure in backup script couldn't run tar" $ERROR echo "tar errored out this " $ERROR | mail -s "BACKUP FAIL: tar could not run" $MYEMAIL exit " -FAILURE doing tar" $ERROR fi ## Cleanup... ################################### echo -Backups complete... logger -t BACKUP "Backups complete..." echo $BACKUPDIRS | mail -s "BACKUP GOOD: directories listed below" $MYEMAIL echo -Ejecting tape... mt eject echo echo -Returning to runlevel $RLORIG logger -t BACKUP "Returning to runlevel $RLORIG" init $RLORIG echo sleep 5 echo Now back in runlevel $Rl echo -Done
| Note |
If you are going to use this type of script, be sure that you are not shutting off syslog if you use the logger logging function to record backup status to the system log. |
This is a good example of a backup script that works for performing simple, full system backups. It will get mysql using mysqlhotcopy and everything else (assuming that you have shut such services off in runlevel 4). There are tons of good backup scripts out there for you to choose from-most of them in shell or perl scripts for free. You can find many good backup scripts at http://linux-backup.net.
Tar can handle incremental backups. However, if you want to do level-based incremental/ differential or TOH backups using this script, then you need to do this with dump (see the Using Dump section later in this chapter).
| Note |
For more information on tar, see the man tar page, or check out http://linux-backup.net/full_inc.gwif.html. |
A very nice tar trick I like to show is how to use tar (and other such tools) remotely. Most sysadmins are very familiar with ssh (or its older insecure cousin, rsh), but not all have ever used it to transport standard output (or stdout) from a Linux server, over the network to a backup server. This is particularly useful for systems without tape drives.
# tar cvf - /home /var /etc | ssh root@10.1.1.10 "cat >/mnt/backup/remoteserver1_$(date +%Y-%m-%d).tar"
Dump is probably one of the more fun backup packages. However, you need to be mindful that it is not backing up the file system through the file system as you and other programs see it, but at the block device level. But what makes dump even more sensitive than tar and other such tools to disk content changes is that dump looks at the block device before the backup begins and at all the inodes that it needs to back up. If many of those inodes change before it gets them to tape, these excessive errors can cause dump to abort its backup all together.
This means that before you actually perform a backup, you should definitely use runlevel 4 or a similar method to shut down disk-chatty services, lock and copy your databases (or just shut them off completely), and quiet anything else on the system before using dump to back up to tape. If you do all of this (and are running a supported file system (such as ext2/ext3), then you shouldn't have any problems with dump.
Here's the basic usage for a local dump session doing a level-0 (full) backup to tape by the hard drive device name /dev/hda1:
# dump -Ouf /dev/tape /dev/hda1
This will back up the first file system (/ boot) on the first hard drive to tape (on this particular system). It's fairly fast and will create few (if any) errors if you've sufficiently quieted your system.
| Note |
Notice the u option. This is important as it allows you to keep track of the backup dates and levels automatically in the /etc/dumpdates file. |
If you have three file systems on your server, /boot, /, and /home (for example), then you do not want to rewind the tape after each backup. This would be bad in that it would only yield the last partition you backed up on the tape. So if you have three partitions on a given disk, you need to use the /dev/nst0 device for at least the first two partitions and then the /dev/st0 or /dev/tape for the last backup on that given tape. These are the three commands that you might want to execute for the first level-0 backup:
dump -Ouf /dev/nst0 /dev/dha1 dump -Ouf /dev/nst0 /dev/dha2 dump -Ouf /dev/st0 /dev/dha3
You might call this your Saturday backup. If you are using the TOH backup plan, what do you think your Monday backup would look like? Probably something like this:
dump -3uf /dev/nst0 /dev/dha1 dump -3uf /dev/nst0 /dev/dha2 dump -3uf /dev/st0 /dev/dha3
If you want to get fancy and also give each day's tape its own dump label, this might help things a bit if tapes get mixed up:
Based on this general usage, we can create a total system backup script. Using some fancy shell script conditionals and the date command, you can autodetect the correct date in our 3-week TOH backup schedule, use the correct related runlevel, and with tools like mysqlhotcopy can build a simple backup script that can pretty much take care of your whole system. Here's a simple example script that employs everything that we've discussed thus far:
#!/bin/bash # Name: dump-disks.sh # Version: v2004-04-26 by tweeks # Takes: /dev/dev1 [/dev/dev2] [/dev/dev3] # Does: Does modified Tower of Hanoi tape backup on three week rotation # Details: Looks at day of the week and day of the month and implements a # modified TOH dump of /dev/dev1 [/dev/dev2] [/dev/dev3] as per: # S S M T W T F # 0 - 3 2 5 4 7 # 1 - 3 2 5 4 7 # 1 - 3 2 5 4 7 # Output: log to /tmp/backuplog.txt emailed to $MYEMAIL # Returns: "1" on Sunday error ######## User Variables ############ RLBAK="4" MYEMAIL="backups@example.com" TAPEDEV=/dev/nst0 # use the "n" device for no rewind (e.g. /dev/nst0) MYSQL=/etc/init.d/mysqld ########### Do not change ########### RL="$(runlevel | cut -f2 -d" ")" RLORIG="$RL" CURDIR="$(pwd)" LEVEL="" DEV1="$1" DEV2="$2" DEV3="$3" ## Detect Day of Week/Month #################################### # Thanks to Henry Pugsley/Rackspace dow='date "+%a"' # Day of Week dom='date "+%d"' # Day of Month #disks="/dev/hda1 /dev/hda2 /dev/hda3" if [ $dow == "Sat" ] && [ $dom -lt 7 ] ; then LEVEL=0 elif [ $dow == "Sun" ] ; then echo "Not on Sunday" && exit 1 elif [ $dow == "Sat" ] ; then LEVEL=1 elif [ $dow == "Mon" ] ; then LEVEL=3 elif [ $dow == "Tue" ] ; then LEVEL=2 elif [ $dow == "Wed" ] ; then LEVEL=5 elif [ $dow == "Thu" ] ; then LEVEL=4 elif [ $dow == "Fri" ] ; then LEVEL=7 fl ## Command Line Error Check/Usage ################################## if [ "$DEV1" = "" ] ; then echo echo ERROR echo "Usage: dump-disks /dev/dev1 [/dev/dev2] [/dev/dev3]" echo "Modified TOH backups run Mon-Sat. Will abort on Sundays..." exit fi #### sanity check #### echo -----IS THIS RIGHT?------ echo dump-disks.sh about to run like this: echo Backup Level=$LEVEL, Backup Devices= $DEV1 , $DEV2 , $DEV3 echo echo Backup will be: echo dump -$LEVEL -auL "L$LEVEL_$(date +%Y-%m-%d)" -f $DEV1 $DEV2 $DEV3 echo -----CTRL-C TO STOP------ sleep 10 echo ## Set up logfile: /tmp/backuplog.txt ####################################### echo dump-disk.sh run on $(date +%Y-%m-%d_%H:%M) by $USERNAME >/tmp/backuplog.txt echo" o--------------------------------o" >>/tmp/backuplog.txt ## Do a MySQL-HOT-COPY to /root/mysql-bak ####################################### echo -Performing a MySQL Hot Copy to /root/mysql-backup echo -Performing a MySQL Hot Copy to /root/mysql-backup>>/tmp/backuplog.txt mkdir -p /root/mysql-backup # ensure mysql is actually running... $MYSQL start mysqlhotcopy --user=root --allowold --regexp=".*" /root/mysql-backup >>/tmp/backuplog.txt 2>&1 $MYSQL stop echo -MySQL hotcopy complete...>>/ tmp/backuplog.txt echo " o-------------------------------o" >>/tmp/backuplog.txt ## Change Run Levels and Log Activity ####################################### echo echo -Current runlevel is $RLORIG echo -Changing run levels to $RLBAK for backups init "$RLBAK" sleep 10 echo -Now in runlel $(runlevel | cut -f2 -d" "), starting backups echo "-preparing to L-$LEVEL dump Devices=$DEV1, $DEV2, $DEV3 to tape" echo "-preparing to L-$LEVEL dump Devices=$DEV1, $DEV2, $DEV3 to tape" >>/tmp/backuplog.txt ## Do the needful #################################### echo " -rewinding tape" mt -f $TAPEDEV rewind # ---------------- if [ "$DEV1" != "" ] ; then echo -dump of $DEV1... dump -$LEVEL -auL "L$LEVEL_$(date +%Y-%m-%d)" -f $TAPEDEV $DEV1 >>/tmp/backuplog.txt 2>&1 echo " o--------------------------------o" >>/tmp/backuplog.txt fi # ---------------- if [ "$DEV?" != ""] ; then echo -dump of $DEV2... dump -$LEVEL -auL "L$LEVEL_$(date +%Y-%m-%d)" -f $TAPEDEV $DEV2 >>/tmp/backuplog.txt 2>&1 echo " o------------------------------o" >>/tmp/backuplog.txt fi # ---------------- if [ "$DEV3" != ""] ; then echo -dump of $DEV3... dump -$LEVEL -auL "L$LEVEL_$(date +%Y-%m-%d)" -f $TAPEDEV $DEV3 >>/tmp/backuplog.txt 2>&1 echo " o--------------------------------o" >>/tmp/backuplog.txt fi ## Cleanup... ################################### echo -Backups complete... echo -Ejecting tape... mt -f $TAPEDEV rewind echo echo -Returning to runlevel $RLORIG init $RLORIG echo sleep 5 echo Now back in runlevel $RL echo -Done >>/tmp/backuplog.txt echo -Done if [ "$LEVEL" = "0" ] ; then echo | mail -s"Following Backup was Level=0 dump. Make a copy." $MYEMAIL fi cat /tmp/backuplog.txt | mail -s "$(date +%a) Dump status" $MYEMAIL
The command line usage for backing up to three partitions with this script looks like this:
And this will give us a modified TOH backup based on the day of the week and the calendar date. Additionally, the first Saturday of the month will always get a level-0 dump and the script will even send us an e-mail reminder to tell us to clone our level-0 tape! This is a configuration that we can fire and forget, and it will be completely automated (except for the insertion of the tapes).
The nice thing about this date smart script configuration is that now we can easily automate this into a daily cronjob and it will know what to do. This is all you need in root's cronjob (by editing with contab -e as root):
########### Smart Three Week TOH Backup Script ############ 34 1 * * * /root/bin/dump-disks.sh /dev/hda1 /dev/hda2 /dev/hda3
And that's it. The smart TOH dump script will now run every morning at 1:34 A.M., and will even autoabort on Sundays (see the script for how this works).
As you can see, with the backup tools provided in most Linux distributions, it doesn't take that much to get an intelligent, hands off, backup configuration set up that makes good use of backup levels, tape usage, and easy recovery strategies. You just need to sit down and plan these things out a bit, and automate things with scripts whenever possible.
If the aforementioned backup script is still a little to "in the rough" for your tastes, you definitely owe it to yourself to check out Amanda at http://backupcentral.com/amanda.html and www.amanda.org. Newer backup suites such as Amanda even go beyond backup levels and the TOH as automate things by simply asking you questions about number of tapes, retention times, and data sets. Very nice indeed!
Although it is not often done, you can run dump remotely over ssh. You do this with a special usage of dd on the machine with the tape drive on it. The stream is pushed over ssh, into dd on the tape machine, and get put directly to tape. The usage looks like this:
# dump -f - /dev/hda2 | ssh root@backupserver.example.com dd \ obs=10k of=/dev/st0
Pretty neat! You can also use a pull method from the tape/backup server, like this:
# ssh -n root@desktop.example.com dump f - /dev/hda2 | dd \ obs=10k of=/dev/nst0
These can be very convenient if you can't be in more than one place at a time.
|
|