FatMin

Rsync How to

Rsync is a very very useful command with a whole bunch of options. However most of the
time if you run rsync on the command line you only need to know a few. 


 rsync -avz foo:src/bar /data/tmp

This would recursively transfer all files from the directory src/bar 
on the machine foo into the /data/tmp/bar directory on the local machine. 
The files are transferred in "archive" mode, which ensures that symbolic links,
devices, attributes, permissions, ownerships etc are preserved in the transfer. 
Additionally, compression will be used to reduce the size of data portions of the transfer.

 rsync -avz foo:src/bar/ /data/tmp

A trailing slash on the source changes this behavior to transfer all files
from the directory src/bar on the machine foo into the /data/tmp/. 
A trailing / on a source name means "copy the contents of this directory". 
Without a trailing slash it means "copy the directory". 
This difference becomes particularly important when using the --delete option.

Below are a few of the options that I find useful.


What to Copy:
-r	recursive

How to Copy:
-n	dry run
-g 	preserve group

Destination Options:
-a	archive
-b	backup

Syntax Examples:
rsync [OPTION]... SRC [SRC]... [USER@]HOST:DEST
rsync [OPTION]... [USER@]HOST:SRC DEST
rsync [OPTION]... SRC [SRC]... DEST
rsync [OPTION]... [USER@]HOST::SRC [DEST]
rsync [OPTION]... SRC [SRC]... [USER@]HOST::DEST
rsync [OPTION]... rsync://[USER@]HOST[:PORT]/SRC [DEST]
rsync [OPTION]... SRC [SRC]... rsync://[USER@]HOST[:PORT]/DEST

mkisofs - Make an iso from a cd or directory

This morning while working on getting a CentOS Jumpstart Server setup with a Knoppix rescue boot image I ran across the following simple utility mkisofs. It was not installed on my box so I had to get it via yum.

In the example below I created an iso from a directory knoppix51
mkisofs -o knoppix51.iso knoppix51

Below are the common usage options:

mkisofs [-a] [-b boot_image] [-c boot_catalog]
        [-A application_id] [-f] [-d] [-D] [-i include-list]
        [-l] [-L] [-p preparer] [-P publisher] [-r] [-R]
        [-T] [-v] [-V] [-V volid] [-x path] [-z] [-m glob]
        -o filename path

Here is how to use mkisofs to make an iso from a cd/dvd

  dd if=/dev/dvd of=dvd.iso    # for dvd  
dd if=/dev/cdrom of=cd.iso   # for cdrom  
dd if=/dev/scd0 of=cd.iso    # if cdrom is scsi

You can also download it from http://freshmeat.net/projects/mkisofs/

RotateLog

More often than not I run into applications that do not know how to clean up after themselves or their log files. On a couple of my boxes, there are applications that have two years of logs squirreled away, just taking up space. Since the application owners do not seem to care about these logs, and do not seem to have any intention of removing any of these logs, I decided to try rotatelog which I used at my old job.

One of the nice things about rotatelog is its config file. A sample file is shown below. After running "make install" all I needed to do was edit a few paths in the rotatelog script.

 FILES:

  # File  Trigger  Owner:Group  Mode  Compress  Archive_Limit

 # =========================================================

  /var/log/wtmp      1M   root:wheel  664  none  4

 /var/log/wtmpx     4M   root:wheel  644  none  4
 /var/log/messages  74K  root:wheel  664  Z     5
 /var/log/maillog   60K  root:wheel  664  gz    7

 

 ACTIONS:

 

 # Shell Command :file1,file2,file3 ... fileN
 # ==========================================

  rotate1                             : /var/log/wtmp,/var/log/wtmpx

 kill -HUP `cat /var/run/syslog.pid` : /var/log/messages,/var/log/maillog

 

 NOTIFY:

 

 # Person to email rotation notification
 # =====================================

Rotatelog installed itself to /usr/local/sbin/rotatelog, and created its config file to /usr/local/etc.

After giving it a test, I then I added it to crontab, and let it rip.
Learn more about rotatelog here

Using LFTP to Mirror a Web Repo

While in the process of researching the best way to create my own local repository of Solaris packages, I ran across LFTP which is a ftp/http client with I built-in mirror command.

For more information on LFTP, look here or  here

The repo that I wanted to Mirror was Blastwave.org, so the first thing that I did was mirror thir directory structure on my local server. I created a 100 gb mount on /v1, and then created the following directories (these are the only ones that I need). They pretty much match the structure that blastwave is using.

/v1/repo/stable/sparc/5.8
/v1/repo/stable/sparc/5.10
/v1/repo/stable/i386/5.10

Here is an example of what I ran to create the initial x86 Solaris 10 repo.

lftp -c 'open -e "mirror /csw/stable/i386/5.10/ /v1/repo/stable/5.10/" http://blastwave.network.com'

Which is based on the basic structure outlined below

lftp -c 'open -e "mirror <remote_dir_path> <local_dir>/" <remote_site>'

Below are a couple of the options that you can use with LFTP

In the example below, LFTP removes any local packages that no longer exist on the source, and only downloads newer package versions from the host mirror.

lftp -e "mirror --delete --only-newer

You can also use the -f option to read your settings from a file.

lftp -f /scriptfile

Or you can log to a file.
--log=FILE      write lftp commands being executed to FILE