miscellaneous: 9月 2012

前一篇有提到我還需要備份字典的 Content , 如法泡製 , 相關內容如下:

先用 crontab 增加一項目:

crontab -l 的內容如下: (擷取相關的 )

0 23 * * 5 /home/jeff/bin/backup-content.sh > /home/jeff/backup-content.txt 2>&1

相對的 ,寫一個 shell file . 如下:
動作和之前 SVN 備份差不多 , 只是這次直接對 NAS 上進行備份 , 並且使用 7z 壓縮程式 !! (tar 好像超過 4G 會有問題 ) .

#!/bin/sh
#
# vi:set ts=4:

## ===== defind echo color
COLOR_ECHO="echo -e"        ## in Fedora the color echo need have -e argument.
COLOR_NORMAL="\033[0m"
COLOR_BOLD="\033[1m"
COLOR_RED="\033[31m"
COLOR_GREEN="\033[32m"
COLOR_YELLOW="\033[33m"

## ===== define source and distnation dir .
BACKUP_SRC="/home/${USER}/NAS/Project/Content"
BACKUP_DEST="/home/${USER}/NAS/Project/backup"
NAS_KEEP_ITEM=12

################################################
#### check log file size.
################################################
LOG_MAX_SZ=`echo "1048576 * 100" | bc -l    `

LOG_FILE=/home/${USER}/backup-content.log
TXT_FILE=/home/${USER}/backup-content.txt

if [ -e ${LOG_FILE} ];then
    _file_sz=`ls -l ${LOG_FILE} | awk '{print $5}'`
    if [ ${_file_sz} -gt ${LOG_MAX_SZ} ];then
        rm ${LOG_FILE}
    fi
fi

################################################
#### chekc NAS direct
################################################
if [ ! -e /home/${USER}/NAS ];then
    ${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_RED}""#### NAS file system not exist !! ""${COLOR_NORMAL}"
    exit 1
elif [ ! -e ${BACKUP_DEST} ];then
    ${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_RED}""#### backup dir not exist !! ""${COLOR_NORMAL}"
    exit 1
fi

#### Check NAS content source dir

if [ ! -e ${BACKUP_SRC} ];then
    ${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_RED}""#### Content source dir not exist !! ""${COLOR_NORMAL}"
    exit 1
fi

################################################
#### Check 7z tools
################################################
COMPRESS_7Z="/usr/local/bin/7z"
VER_7Z=`${COMPRESS_7Z} | grep Copyright | sed 's/Copyright.*//g' | awk '{print $2}'`
## echo "VER_7Z:${VER_7Z}"
## echo "COMPRESS_7Z:${COMPRESS_7Z}"

if [ -z "${VER_7Z}" ];then
    ${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_RED}""#### Compress Tool 7Z not found !! ""${COLOR_NORMAL}"
    exit 1
else
    ${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_GREEN}""#### 7Z Version is ${VER_7Z} . ""${COLOR_NORMAL}"
fi

################################################
#### Start back up
################################################

_cur_date=`date +%Y_%m_%d`
_bak_name=${BACKUP_DEST}/Content_${_cur_date}.7z

${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_GREEN}""#### Strating Compress ${BACKUP_SRC} ......   ""${COLOR_NORMAL}"

## echo "***************"" 7z a ${_bak_name} ${BACKUP_SRC} "
${COMPRESS_7Z} a ${_bak_name} ${BACKUP_SRC}

################################################
#### Remove oldset back up files .
################################################

## rm oldest back up file
_lists=`find ${BACKUP_DEST}/* | xargs ls -t | grep -n ".7z"`

## echo "_lists:${_lists}"

for _del_item in ${_lists} ; do
    _list_id=`echo ${_del_item} | sed 's/:.*//g' `
    _item_name=`echo ${_del_item} | sed 's/.*://g' `
    if [ ${_list_id} -gt ${NAS_KEEP_ITEM} ];then
##       echo "***************"" rm -vf ${_item_name} "
       rm -vf ${_item_name}
    fi
done

################################################

if [ -e ${TXT_FILE} ];then
    cat ${TXT_FILE} >> ${LOG_FILE}
fi

################################################

因為工作需要 , 每星期我要負責 backup SVN datebase 和字典的 Content 資料 .
每次都需要在 console 下執行一堆 command , 並且還讓自己的 PC 處於忙碌 , 修改 code 後要重新 build 都會很慢 , 想說 SVN 那台 server PC (後面簡稱 SVNPC) 都沒有事情作 , 好吧 , 寫一個 shell scrip 讓 SVNPC 定時自動幫我備份資料 .

linux 要自動執行一個 shell 可以用 crontal -e 指令來填寫啟動時間 , 詳細使用方式請參考"鳥哥" 網頁 !!

http://linux.vbird.org/linux_basic/0430cron.php

crontab -l 的內容如下: (擷取相關的 )

0 2 * * 6 /home/jeff/bin/backup-svn.sh ec510 auto > /home/jeff/backup-svn-ec510.txt 2>&1

接著 , 寫一個備份的 shell scrip 吧 !! 首先 , 備份 SVN 資料 , 完整 shell 如下:
這個 shell 會將 /home/svn/repos/<project> 用 tar 的方式壓縮起來 , 並且壓縮的檔名為svn-db-<project>-<date>.tar.bz2 .
執行過程中可以將 message 會放在svnbackup-svn-<project>.txt 內 .
執行結束後會將 .txt append 到 .log 內 , 如果 .log file 過大 , 會自動清除.
壓縮完畢後本地會留下最近的 4 個壓縮檔 .
並將最新的壓縮檔 copy 到 NAS 上.
檢查 NAS上的備份資料數 , 如果超過 20 個就會刪除最舊的.

大功告成 , 每星期六早上2:00 開始備份 !!

#!/bin/sh
#
# vi:set ts=4:

NAS_KEEP_ITEM=20
LOCAL_KEEP_ITEM=4

## ===== defind echo color
COLOR_ECHO="echo -e"        ## in Fedora the color echo need have -e argument.
COLOR_NORMAL="\033[0m"
COLOR_BOLD="\033[1m"
COLOR_RED="\033[31m"
COLOR_GREEN="\033[32m"
COLOR_YELLOW="\033[33m"

################################################
## check log file size and remove it if too large.
################################################
LOG_MAX_SZ=`echo "1048576 * 100" | bc -l    `

if [ "$1" == "" ];then
    echo " FIXME ****************** not finish "
    exit
else
    LOG_FILE=/home/${USER}/backup-svn-$1.log
    TXT_FILE=/home/${USER}/backup-svn-$1.txt
fi

if [ -e ${LOG_FILE} ];then
    _file_sz=`ls -l ${LOG_FILE} | awk '{print $5}'`
    if [ ${_file_sz} -gt ${LOG_MAX_SZ} ];then
        rm ${LOG_FILE}
    fi
fi

################################################
## show some message .
################################################

if [ "$2" == "" ];then
    ${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_GREEN}"\
                "## Back up $1 ...."\
                "${COLOR_NORMAL}"

elif [ "$2" == "auto" ];then
    _start_time=`date +%Y/%m/%d\ \ %k:%M:%S`
    ${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_GREEN}"\
                "## Auto back up $1 at \"${_start_time}\" ...."\
                "${COLOR_NORMAL}"

else
    ${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_RED}"\
                "## Bad command $2 ...."\
                "${COLOR_NORMAL}"

    exit 1
fi

################################################
## start back up svn .
################################################

_work_dir=`pwd`
_cur_date=`date +%Y%m%d`

    if [ $1 == "" ];then
        _svn_dir="/home/svn"
        _bak_name="$_work_dir/svn-db-$_cur_date.tar.bz2"
        ${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_GREEN}" \
                        "Back up whole svn dir.. "\
                        "${COLOR_NORMAL}"
    else
        _svn_dir="/home/svn/repos/$1"
        _bak_name="$_work_dir/svn-db-$1-$_cur_date.tar.bz2"
        ${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_GREEN}" \
                        "Back up $_svn_dir dir..." \
                        "${COLOR_NORMAL}"
    fi

    if [ "$_svn_dir" == ""   ] || [ ! -e $_svn_dir ]; then
        ${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_RED}" \
                        "## The SVN_DIR is not correct ..... "
                        "${COLOR_NORMAL}"
        exit
    else
        if [ ! -e "$_bak_name" ];then
            tar -jcf $_bak_name $_svn_dir
        else
            ${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_RED}" \
                            "## The file $_bak_name already exist , Please check it .. " \
                            "${COLOR_NORMAL}"
            exit
        fi
    fi

################################################
## Remove local oldest back up file.
################################################
${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_GREEN}" \
                 "Remove local oldest backup file..." \
                "${COLOR_NORMAL}"

if [ $1 == "" ];then
    echo " FIXME ****************** not finish "
    exit
else
    _lo_lists=`find /home/${USER}/* -maxdepth 1 -type f | grep "$1" | sort -r | grep -n "tar.bz2" `
fi

##    echo "_lo_lists:${_lo_lists}"

for _lo_del_item in ${_lo_lists} ; do
    _lo_list_id=`echo "${_lo_del_item}" | sed 's/:.*//g' `
    _lo_item_name=`echo ${_lo_del_item} | sed 's/.*://g' `
    if [ ${_lo_list_id} -gt ${LOCAL_KEEP_ITEM} ] && [ ${_lo_item_name} != ${_bak_name} ] ;then \
        rm -vf ${_lo_item_name}
    fi
done

################################################
## Copy local backup file to NAS, back up.
################################################

## ===== define source and distnation dir .

if [ $1 == "" ];then
    echo " FIXME ****************** not finish "
    exit
else
    BACKUP_DEST="/home/${USER}/NAS/Project/$1/backup/SVN_Backup"
fi

#### chekc NAS direct
if [ ! -e /home/${USER}/NAS ];then
    ${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_RED}""#### NAS file system not exist !! ""${COLOR_NORMAL}"
    exit 1
elif [ ! -e ${BACKUP_DEST} ];then
    ${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_RED}""#### backup dir not exist !! ""${COLOR_NORMAL}"
    exit 1
fi

#### Copy to tar file to NAS .
${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_GREEN}""#### Copy tar file to NAS ... !! ""${COLOR_NORMAL}"
cp -v ${_bak_name} ${BACKUP_DEST}/

################################################
#### Remove oldest backup file in NAS .
################################################
${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_GREEN}" \
                "Remove NAS oldest backup file..." \
                "${COLOR_NORMAL}"

if [ $1 == "" ];then
    echo " FIXME ****************** not finish "
    exit
else
    _lists=`find ${BACKUP_DEST}/* | sort -r | grep -n ".tar.bz2"`
fi

## echo "_lists:${_lists}"

for _del_item in ${_lists} ; do
    _list_id=`echo ${_del_item} | sed 's/:.*//g' `
    _item_name=`echo ${_del_item} | sed 's/.*://g' `

    if [ ${_list_id} -gt ${NAS_KEEP_ITEM} ];then \
        rm -vf ${_item_name}
    fi
done

################################################
## Show dome message.
################################################
if [ "$2" == "" ];then
    ${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_GREEN}"\
                "## Back up $1 done ...."\
                "${COLOR_NORMAL}"

elif [ "$2" == "auto" ];then
    _end_time=`date +%Y/%m/%d\ \ %k:%M:%S`
    ${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_GREEN}"\
                "## Auto back up $1 done at \"${_end_time}\" ...."\
                "${COLOR_NORMAL}"
fi

################################################

if [ -e ${TXT_FILE} ];then
    cat ${TXT_FILE} >> ${LOG_FILE}
fi

################################################

miscellaneous

2012年9月26日星期三

自動執行備份 part 2

自動執行.... 備份

2012年9月26日 星期三

自動執行 備份 part 2

自動執行.... 備份

2012年9月26日星期三

自動執行備份 part 2