這篇文章我不知道要如何命名比較恰當 , 主要會使用 mkfs.ext4 和 stat 這兩個utility 所以就簡單命名了 !!
在 embedded system 中 nand flash space 是很寶貴的 , 不能太浪費 !!
需求來了, 我需要計算要放入的資料大小(包含一堆目錄)然後建立一個 剛剛好大小的 image file , 這個 image file 需要包含 ext4 filesystem ,量產時只要將這個 image file 直接燒錄到eMMC 對應的partition 中即可.
問題是.... 現在我每次都計算部準確 .... 常常 空間不足夠 , 又不能浪費一堆空間 (這個是純資料 , Device 會使用 mount -o r ..... read-only type , 所以浪費的也用不到 !! )
先說 stat 吧 !!
stat 可以將檔案 , 資料夾所需要的 block & size都列印出來 , 指令如下：
STAT="/bin/stat"
size=`${STAT} -c%s $1`
block=`${STAT} -c%b $1`
也可以利用 stat 來知道是檔案還是資料夾
${STAT} ${dir_list} | grep -q "directory"
if [ $? != 0 ];then
## echo "###### FileStatus ${dir_list}"
FileStatus ${dir_list}
else
## echo "###### DirectStatus ${dir_list}"
DirectStatus ${dir_list}
if [ DIRECT_COUNT != 0 ];then
let DIRECT_COUNT=DIRECT_COUNT-1
fi
fi
然後用 shell file 計算所有資料夾需要的 size 和 block count .
這邊要注意 , ext2/ext4 的 block size 有 1024/2048 & 4096 區分 .
一樣會有內部斷裂問題 (小於 block size會使用一個 block 去儲存) .
stat -c%b 回應的值是 disk 的最小block 單位 目前都是 512 bytes.
或者可以用 stat -c%B 去讀取 確定的大小值 .
我寫了一個 xstat.sh shell scrip 去計算所有資料的大小 , 如下：
example :
[jeff@instant ec910-sdk]$ ./build/xstat.sh src/
file count:145
folder count:7
file size:2055196
folder size:94208
total need block:603
[jeff@instant ec910-sdk]$
這個 shell 統計出 需要 603 個 block 也就是需要 603 * 4096 bytes.
ext4 filesystem structure:
2013年4月16日 星期二
shell script 中的 getopt & getopts
在 linux shell 中我們常常用到 傳入一些參數 . 之前我都使用 getopts 的方式, 使用上都好好的 , example file 如下(片段)：
while getopts ":O:h" opt; do
echo "opt:${opt}"
case ${opt} in
## ---- output size setting.
O )
OUT_BLOCK_SZ=${OPTARG}
;;
## ---- help menu.
\? | h )
${MKMATE} color -fred "## Bad command ${OPTARG}...."
Usage
return 1
;;
esac
done
今天想要輸入 一堆其它的值 例如: <$> foo.sh -O 1234 dir_1 dir_2 dir_3. or
<$> foo.sh dir_1 dir_2 dir_3 ..... -O 1234
使用之前就的方式卻出問題, 這些 dir 訊息沒有辦法取出(maybe I got some mistake in getopts ),所以就換成使用 getopt 了, example 如下：
args=`getopt O:h $@ `
if [ $? != 0 ];then
${MKMATE} color -fred "## Bad command ${OPTARG}...."
return 1
fi
set -- ${args}
while [ -n "$1" ]
do
case $1 in
## ---- output size setting.
-O )
shift
OUT_BLOCK_SZ=$1
;;
-h | - )
Usage
return 1
;;
\? )
${MKMATE} color -fred "## Bad command ${OPTARG}...."
Usage
return 1
;;
-- )
DIR_LIST=
;;
* )
if [ -z ${DIR_LIST} ];then
DIR_LIST="$1"
else
DIR_LIST=${DIR_LIST}:$1
fi
;;
esac
shift
done
不管輸入方式是 <$>foo.sh -O 1234 dir_1 ..... or <$>foo.sh dir_1 ..... -O 1234
最後 srgs 都會 -O 1234 -- dir_1 dir_2 .....
接著用 set -- ${args} 就可以重新設定 ＄＠ 參數 .
在利用 while 和 shift 就可以逐一取出想要的參數 , 利用 -- 當區隔就可以將 dir_1 dir_2 取入 .
我利用 : 將這些 dir 資料並起來, 有利於我的處理需求 ！！
Reference :
http://changyy.pixnet.net/blog/post/25660234-%5Bbash%5D-%E5%B0%87%E6%AA%94%E6%A1%88%E5%85%A7%E7%9A%84%E5%AD%97%E4%B8%B2%E9%80%B2%E8%A1%8C%E5%8F%96%E4%BB%A3
while getopts ":O:h" opt; do
echo "opt:${opt}"
case ${opt} in
## ---- output size setting.
O )
OUT_BLOCK_SZ=${OPTARG}
;;
## ---- help menu.
\? | h )
${MKMATE} color -fred "## Bad command ${OPTARG}...."
Usage
return 1
;;
esac
done
今天想要輸入 一堆其它的值 例如: <$> foo.sh -O 1234 dir_1 dir_2 dir_3. or
<$> foo.sh dir_1 dir_2 dir_3 ..... -O 1234
使用之前就的方式卻出問題, 這些 dir 訊息沒有辦法取出(maybe I got some mistake in getopts ),所以就換成使用 getopt 了, example 如下：
args=`getopt O:h $@ `
if [ $? != 0 ];then
${MKMATE} color -fred "## Bad command ${OPTARG}...."
return 1
fi
set -- ${args}
while [ -n "$1" ]
do
case $1 in
## ---- output size setting.
-O )
shift
OUT_BLOCK_SZ=$1
;;
-h | - )
Usage
return 1
;;
\? )
${MKMATE} color -fred "## Bad command ${OPTARG}...."
Usage
return 1
;;
-- )
DIR_LIST=
;;
* )
if [ -z ${DIR_LIST} ];then
DIR_LIST="$1"
else
DIR_LIST=${DIR_LIST}:$1
fi
;;
esac
shift
done
不管輸入方式是 <$>foo.sh -O 1234 dir_1 ..... or <$>foo.sh dir_1 ..... -O 1234
最後 srgs 都會 -O 1234 -- dir_1 dir_2 .....
接著用 set -- ${args} 就可以重新設定 ＄＠ 參數 .
在利用 while 和 shift 就可以逐一取出想要的參數 , 利用 -- 當區隔就可以將 dir_1 dir_2 取入 .
我利用 : 將這些 dir 資料並起來, 有利於我的處理需求 ！！
Reference :
http://changyy.pixnet.net/blog/post/25660234-%5Bbash%5D-%E5%B0%87%E6%AA%94%E6%A1%88%E5%85%A7%E7%9A%84%E5%AD%97%E4%B8%B2%E9%80%B2%E8%A1%8C%E5%8F%96%E4%BB%A3
2013年2月20日 星期三
auto mount NAS direct on /home/
最近因為需要處理NAS 上的資料 , 每天早上開機都要 mount 一次 , 麻煩 , 寫一個 shell 來自動 mount 吧 !!
開機完畢後只要執行這個 shell 就會將需要的 mount 好. 預設值是 Public & Software 資料夾 , mount 位置在 /home/${USER}/NAS/ , 並且會自動建立對應位置 , 如果需要額外資料夾 , 可以 使用 nas_mount.sh abc 的方式.
這個 shell 會檢查 NAS 是否在線上 (ping IP) .
也會自動檢查是否已經 mount 好了.
內容如下:
#!/bin/sh
#
# vi:set ts=4:
## ===== defind echo color
COLOR_ECHO="echo -e" ## in Fedora the color echo need have -e argument.
COLOR_NORMAL="\033[0m"
COLOR_BOLD="\033[1m"
COLOR_RED="\033[31m"
COLOR_GREEN="\033[32m"
COLOR_YELLOW="\033[33m"
COLOR_BLUE="\033[34m"
## =====================================
## NAS Mount function.
## =====================================
function NAS_mount()
{
nas_dir="$1"
mnt_dir="$2"
## ---- echo some message .
${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_BLUE}" \
"## Mount NAS \"${nas_dir}\" director on \"${mnt_dir}\" ........" \
"${COLOR_NORMAL}"
## ---- check mnt_dir exist
if [ ! -e "${mnt_dir}" ];then
mkdir -p ${mnt_dir}
fi
## ---- check already mounted
_mounted=`cat /proc/mounts | grep NAS/${nas_dir} | awk '{print $2}'`
_mount_src=`cat /proc/mounts | grep NAS/${nas_dir} | awk '{print $1}'`
if [ -n "${_mounted}" ];then
${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_GREEN}" \
" # NAS \"${_mount_src}\" Already mount on \"${mnt_dir}\" ......" \
"${COLOR_NORMAL}"
else
_nas_aliv=`ping ${NAS_IP} -c 1 | grep errors | sed 's/.*errors/errors/g' | sed 's/errors.*/errors/g' `
if [ -z "${_nas_aliv}" ];then
echo xxxxxxx | su -c " mount.cifs //192.168.2.6/${nas_dir} ${mnt_dir} -o user=jeff -o pass=xxxxxxxxx" > /dev/null 2>&1
else
${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_RED}" \
"NAS Server not ready \"${NAS_IP}\" ........" \
"${COLOR_NORMAL}"
fi
fi
}
## ===== Mount NAS on System.
NAS_IP="192.168.2.6"
if [ -z "$1" ];then
NAS_mount "Software" "/home/${USER}/NAS/Software"
NAS_mount "Public" "/home/${USER}/NAS/Public"
else
NAS_mount "${1}" "/home/${USER}/NAS/${1}"
fi
## =================
開機完畢後只要執行這個 shell 就會將需要的 mount 好. 預設值是 Public & Software 資料夾 , mount 位置在 /home/${USER}/NAS/ , 並且會自動建立對應位置 , 如果需要額外資料夾 , 可以 使用 nas_mount.sh abc 的方式.
這個 shell 會檢查 NAS 是否在線上 (ping IP) .
也會自動檢查是否已經 mount 好了.
內容如下:
#!/bin/sh
#
# vi:set ts=4:
## ===== defind echo color
COLOR_ECHO="echo -e" ## in Fedora the color echo need have -e argument.
COLOR_NORMAL="\033[0m"
COLOR_BOLD="\033[1m"
COLOR_RED="\033[31m"
COLOR_GREEN="\033[32m"
COLOR_YELLOW="\033[33m"
COLOR_BLUE="\033[34m"
## =====================================
## NAS Mount function.
## =====================================
function NAS_mount()
{
nas_dir="$1"
mnt_dir="$2"
## ---- echo some message .
${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_BLUE}" \
"## Mount NAS \"${nas_dir}\" director on \"${mnt_dir}\" ........" \
"${COLOR_NORMAL}"
## ---- check mnt_dir exist
if [ ! -e "${mnt_dir}" ];then
mkdir -p ${mnt_dir}
fi
## ---- check already mounted
_mounted=`cat /proc/mounts | grep NAS/${nas_dir} | awk '{print $2}'`
_mount_src=`cat /proc/mounts | grep NAS/${nas_dir} | awk '{print $1}'`
if [ -n "${_mounted}" ];then
${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_GREEN}" \
" # NAS \"${_mount_src}\" Already mount on \"${mnt_dir}\" ......" \
"${COLOR_NORMAL}"
else
_nas_aliv=`ping ${NAS_IP} -c 1 | grep errors | sed 's/.*errors/errors/g' | sed 's/errors.*/errors/g' `
if [ -z "${_nas_aliv}" ];then
echo xxxxxxx | su -c " mount.cifs //192.168.2.6/${nas_dir} ${mnt_dir} -o user=jeff -o pass=xxxxxxxxx" > /dev/null 2>&1
else
${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_RED}" \
"NAS Server not ready \"${NAS_IP}\" ........" \
"${COLOR_NORMAL}"
fi
fi
}
## ===== Mount NAS on System.
NAS_IP="192.168.2.6"
if [ -z "$1" ];then
NAS_mount "Software" "/home/${USER}/NAS/Software"
NAS_mount "Public" "/home/${USER}/NAS/Public"
else
NAS_mount "${1}" "/home/${USER}/NAS/${1}"
fi
## =================
2013年2月18日 星期一
用 shell + crontab 固定發出 openkm report Mail.
最近因為 上級要知道 OpenKM(簡稱 OKM) 內容的更動狀況 , 需要每週 report 一次 !!
主要是看看這星期 OKM 內容有哪些人更動!!
Google 一下 有關 OKM 的 auto report , 發現需要會 java script , 我不會 @@ !!
想說 , OKM 的 dashboard 可以看到 log 內容 , 應該 找一下 log file , 也會有吧 !!
找了半天 , 發現 /openkm-6.2.2-community/tomcat/repository/okmdb.log 內有我要的訊息 , 好吧 , 山不轉 人轉 , 寫一個 shell script + crontab 來執行 auto report 吧 !!
首先寫一個 shell , 由 okmdb.log 內容取出 document create/delete & check in 的項目
到一個 temp file , shell 如下:
#!/bin/sh
#
# vi:set ts=4:
#############################
## ===== defind echo color
#############################
COLOR_ECHO="echo -e" ## in Fedora the color echo need have -e argument.
COLOR_NORMAL="\033[0m"
COLOR_BOLD="\033[1m"
COLOR_RED="\033[31m"
COLOR_GREEN="\033[32m"
COLOR_YELLOW="\033[33m"
COLOR_BLUE="\033[34m"
#############################
## ==== check log file exist.
#############################
OKMDB_LOG="/home/jeff/openkm-6.2.2-community/tomcat/repository/okmdb.log"
TEMP_FILE="/home/jeff/okm_report.txt"
if [ ! -e ${OKMDB_LOG} ];then
${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_RED}" \
"The log file \"${OKMDB_LOG}\" not exist !! " \
"${COLOR_NORMAL}"
exit
fi
#############################
## ==== Crate title
#############################
echo -e "ITEM\t\t\tDATE\t\t\tFILES\t\t\tUSER" > ${TEMP_FILE}
echo >> ${TEMP_FILE}
#############################
## ==== get document create event log.
#############################
DOC_CREATE=`cat ${OKMDB_LOG} | grep "CREATE_DOCUMENT" | sed 's/\ /:/g'`
IFS="
"
for list in ${DOC_CREATE} ; do
IFS=','
_date=`echo ${list} | awk '{print $3}' | sed 's/\..*//g' | sed 's/'\''//g' `
_file=`echo ${list} | awk '{print $7}' | sed 's/'\''//g' | sed 's/\/okm\:root\///g' `
_user=`echo ${list} | awk '{print $8}' | sed 's/'\''//g' | sed 's/)//g'`
echo -e "CREATE\t\t${_date}\t\t${_file}\t\t${_user}" >> ${TEMP_FILE}
done
echo >> ${TEMP_FILE}
#############################
## ==== get document delete event log.
#############################
DOC_DELETE=`cat ${OKMDB_LOG} | grep "DELETE_DOCUMENT" | sed 's/\ /:/g'`
IFS="
"
for list in ${DOC_DELETE} ; do
IFS=','
_date=`echo ${list} | awk '{print $3}' | sed 's/\..*//g' | sed 's/'\''//g' `
_file=`echo ${list} | awk '{print $6}' | sed 's/'\''//g' | sed 's/\/okm\:root\///g' `
_user=`echo ${list} | awk '{print $7}' | sed 's/'\''//g' | sed 's/)//g'`
echo -e "DELETE\t\t${_date}\t\t${_file}\t\t${_user}" >> ${TEMP_FILE}
done
echo >> ${TEMP_FILE}
#############################
## ==== get document modify event log.
#############################
DOC_CHECKIN=`cat ${OKMDB_LOG} | grep "CHECKIN_DOCUMENT" | sed 's/\ /:/g'`
IFS="
"
for list in ${DOC_CHECKIN} ; do
IFS=','
_date=`echo ${list} | awk '{print $3}' | sed 's/\..*//g' | sed 's/'\''//g' `
_file=`echo ${list} | awk '{print $7}' | sed 's/'\''//g' | sed 's/\/okm\:root\///g' `
_user=`echo ${list} | awk '{print $8}' | sed 's/'\''//g' | sed 's/)//g'`
echo -e "CHECKIN\t\t${_date}\t\t${_file}\t\t${_user}" >> ${TEMP_FILE}
done
echo >> ${TEMP_FILE}
##############################
## ==== send notify mail
#############################
NOTIFY_MAIL="jeff@xxxxxxx.com.tw,ch@xxxxxxx.com.tw,emiliehuang@xxxxxxx.com.tw,alvin@xxxxxxx.com.tw,pp@xxxxxxx.com.tw"
#NOTIFY_MAIL="jeff@xxxxxxx.com.tw"
mail -s "Content report" ${NOTIFY_MAIL} < ${TEMP_FILE}
rm -rf ${TEMP_FILE}
## ==== remove log file ??
if [ ! -z "$1" ] && [ "$1" == "-d" ];then
echo > ${OKMDB_LOG}
fi
#############################
接著 , 我不想每天(星期)都去檢查 report file , 就將這個 file 內容直接mail 給上級吧^^ !!
要讓 fedora 發 mail 其實蠻簡單的 用 "mail -s "Content report" ${NOTIFY_MAIL} < ${TEMP_FILE}"
就可以了!!
不過要先設定 smtp 的內容 , google 一下 其實也不難 , 只要建立 /home/jeff/.mailrc 檔案, 且.mailrc 內容填入下列即可:
set from=Server_AUTO<jeff@xxxxxx.com.tw> smtp=xxxx.com.tw
set smtp-auth-user=jeff@xxxx.com.tw smtp-auth-password=*******
set smtp-auth=login
上面有些個人訊息 , 就用 xxxx 和 **** 代替 !!
收到 mail 的內容就會如下:
ITEM DATE FILES USER
CREATE 2013-02-06:10:28:40 share/data_sip_image_ec510/sip/Image/English/Lower/btn_backspace_press.png alvin
CREATE 2013-02-06:10:29:18 share/data_sip_image_ec510/sip/Image/English/Upper/btn_backspace_press.png alvin..........
主要是看看這星期 OKM 內容有哪些人更動!!
Google 一下 有關 OKM 的 auto report , 發現需要會 java script , 我不會 @@ !!
想說 , OKM 的 dashboard 可以看到 log 內容 , 應該 找一下 log file , 也會有吧 !!
找了半天 , 發現 /openkm-6.2.2-community/tomcat/repository/okmdb.log 內有我要的訊息 , 好吧 , 山不轉 人轉 , 寫一個 shell script + crontab 來執行 auto report 吧 !!
首先寫一個 shell , 由 okmdb.log 內容取出 document create/delete & check in 的項目
到一個 temp file , shell 如下:
#!/bin/sh
#
# vi:set ts=4:
#############################
## ===== defind echo color
#############################
COLOR_ECHO="echo -e" ## in Fedora the color echo need have -e argument.
COLOR_NORMAL="\033[0m"
COLOR_BOLD="\033[1m"
COLOR_RED="\033[31m"
COLOR_GREEN="\033[32m"
COLOR_YELLOW="\033[33m"
COLOR_BLUE="\033[34m"
#############################
## ==== check log file exist.
#############################
OKMDB_LOG="/home/jeff/openkm-6.2.2-community/tomcat/repository/okmdb.log"
TEMP_FILE="/home/jeff/okm_report.txt"
if [ ! -e ${OKMDB_LOG} ];then
${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_RED}" \
"The log file \"${OKMDB_LOG}\" not exist !! " \
"${COLOR_NORMAL}"
exit
fi
#############################
## ==== Crate title
#############################
echo -e "ITEM\t\t\tDATE\t\t\tFILES\t\t\tUSER" > ${TEMP_FILE}
echo >> ${TEMP_FILE}
#############################
## ==== get document create event log.
#############################
DOC_CREATE=`cat ${OKMDB_LOG} | grep "CREATE_DOCUMENT" | sed 's/\ /:/g'`
IFS="
"
for list in ${DOC_CREATE} ; do
IFS=','
_date=`echo ${list} | awk '{print $3}' | sed 's/\..*//g' | sed 's/'\''//g' `
_file=`echo ${list} | awk '{print $7}' | sed 's/'\''//g' | sed 's/\/okm\:root\///g' `
_user=`echo ${list} | awk '{print $8}' | sed 's/'\''//g' | sed 's/)//g'`
echo -e "CREATE\t\t${_date}\t\t${_file}\t\t${_user}" >> ${TEMP_FILE}
done
echo >> ${TEMP_FILE}
#############################
## ==== get document delete event log.
#############################
DOC_DELETE=`cat ${OKMDB_LOG} | grep "DELETE_DOCUMENT" | sed 's/\ /:/g'`
IFS="
"
for list in ${DOC_DELETE} ; do
IFS=','
_date=`echo ${list} | awk '{print $3}' | sed 's/\..*//g' | sed 's/'\''//g' `
_file=`echo ${list} | awk '{print $6}' | sed 's/'\''//g' | sed 's/\/okm\:root\///g' `
_user=`echo ${list} | awk '{print $7}' | sed 's/'\''//g' | sed 's/)//g'`
echo -e "DELETE\t\t${_date}\t\t${_file}\t\t${_user}" >> ${TEMP_FILE}
done
echo >> ${TEMP_FILE}
#############################
## ==== get document modify event log.
#############################
DOC_CHECKIN=`cat ${OKMDB_LOG} | grep "CHECKIN_DOCUMENT" | sed 's/\ /:/g'`
IFS="
"
for list in ${DOC_CHECKIN} ; do
IFS=','
_date=`echo ${list} | awk '{print $3}' | sed 's/\..*//g' | sed 's/'\''//g' `
_file=`echo ${list} | awk '{print $7}' | sed 's/'\''//g' | sed 's/\/okm\:root\///g' `
_user=`echo ${list} | awk '{print $8}' | sed 's/'\''//g' | sed 's/)//g'`
echo -e "CHECKIN\t\t${_date}\t\t${_file}\t\t${_user}" >> ${TEMP_FILE}
done
echo >> ${TEMP_FILE}
##############################
## ==== send notify mail
#############################
NOTIFY_MAIL="jeff@xxxxxxx.com.tw,ch@xxxxxxx.com.tw,emiliehuang@xxxxxxx.com.tw,alvin@xxxxxxx.com.tw,pp@xxxxxxx.com.tw"
#NOTIFY_MAIL="jeff@xxxxxxx.com.tw"
mail -s "Content report" ${NOTIFY_MAIL} < ${TEMP_FILE}
rm -rf ${TEMP_FILE}
## ==== remove log file ??
if [ ! -z "$1" ] && [ "$1" == "-d" ];then
echo > ${OKMDB_LOG}
fi
#############################
接著 , 我不想每天(星期)都去檢查 report file , 就將這個 file 內容直接mail 給上級吧^^ !!
要讓 fedora 發 mail 其實蠻簡單的 用 "mail -s "Content report" ${NOTIFY_MAIL} < ${TEMP_FILE}"
就可以了!!
不過要先設定 smtp 的內容 , google 一下 其實也不難 , 只要建立 /home/jeff/.mailrc 檔案, 且.mailrc 內容填入下列即可:
set from=Server_AUTO<jeff@xxxxxx.com.tw> smtp=xxxx.com.tw
set smtp-auth-user=jeff@xxxx.com.tw smtp-auth-password=*******
set smtp-auth=login
上面有些個人訊息 , 就用 xxxx 和 **** 代替 !!
收到 mail 的內容就會如下:
ITEM DATE FILES USER
CREATE 2013-02-06:10:28:40 share/data_sip_image_ec510/sip/Image/English/Lower/btn_backspace_press.png alvin
CREATE 2013-02-06:10:29:18 share/data_sip_image_ec510/sip/Image/English/Upper/btn_backspace_press.png alvin..........
2012年9月26日 星期三
自動執行 備份 part 2
前一篇有提到我還需要備份 字典的 Content , 如法泡製 , 相關內容如下:
先用 crontab 增加一項目:
crontab -l 的內容如下: (擷取 相關的 )
0 23 * * 5 /home/jeff/bin/backup-content.sh > /home/jeff/backup-content.txt 2>&1
相對的 ,寫一個 shell file . 如下:
動作和 之前 SVN 備份差不多 , 只是這次直接對 NAS 上 進行備份 , 並且使用 7z 壓縮程式 !! (tar 好像超過 4G 會有問題 ) .
#!/bin/sh
#
# vi:set ts=4:
## ===== defind echo color
COLOR_ECHO="echo -e" ## in Fedora the color echo need have -e argument.
COLOR_NORMAL="\033[0m"
COLOR_BOLD="\033[1m"
COLOR_RED="\033[31m"
COLOR_GREEN="\033[32m"
COLOR_YELLOW="\033[33m"
## ===== define source and distnation dir .
BACKUP_SRC="/home/${USER}/NAS/Project/Content"
BACKUP_DEST="/home/${USER}/NAS/Project/backup"
NAS_KEEP_ITEM=12
################################################
#### check log file size.
################################################
LOG_MAX_SZ=`echo "1048576 * 100" | bc -l `
LOG_FILE=/home/${USER}/backup-content.log
TXT_FILE=/home/${USER}/backup-content.txt
if [ -e ${LOG_FILE} ];then
_file_sz=`ls -l ${LOG_FILE} | awk '{print $5}'`
if [ ${_file_sz} -gt ${LOG_MAX_SZ} ];then
rm ${LOG_FILE}
fi
fi
################################################
#### chekc NAS direct
################################################
if [ ! -e /home/${USER}/NAS ];then
${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_RED}""#### NAS file system not exist !! ""${COLOR_NORMAL}"
exit 1
elif [ ! -e ${BACKUP_DEST} ];then
${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_RED}""#### backup dir not exist !! ""${COLOR_NORMAL}"
exit 1
fi
#### Check NAS content source dir
if [ ! -e ${BACKUP_SRC} ];then
${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_RED}""#### Content source dir not exist !! ""${COLOR_NORMAL}"
exit 1
fi
################################################
#### Check 7z tools
################################################
COMPRESS_7Z="/usr/local/bin/7z"
VER_7Z=`${COMPRESS_7Z} | grep Copyright | sed 's/Copyright.*//g' | awk '{print $2}'`
## echo "VER_7Z:${VER_7Z}"
## echo "COMPRESS_7Z:${COMPRESS_7Z}"
if [ -z "${VER_7Z}" ];then
${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_RED}""#### Compress Tool 7Z not found !! ""${COLOR_NORMAL}"
exit 1
else
${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_GREEN}""#### 7Z Version is ${VER_7Z} . ""${COLOR_NORMAL}"
fi
################################################
#### Start back up
################################################
_cur_date=`date +%Y_%m_%d`
_bak_name=${BACKUP_DEST}/Content_${_cur_date}.7z
${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_GREEN}""#### Strating Compress ${BACKUP_SRC} ...... ""${COLOR_NORMAL}"
## echo "***************"" 7z a ${_bak_name} ${BACKUP_SRC} "
${COMPRESS_7Z} a ${_bak_name} ${BACKUP_SRC}
################################################
#### Remove oldset back up files .
################################################
## rm oldest back up file
_lists=`find ${BACKUP_DEST}/* | xargs ls -t | grep -n ".7z"`
## echo "_lists:${_lists}"
for _del_item in ${_lists} ; do
_list_id=`echo ${_del_item} | sed 's/:.*//g' `
_item_name=`echo ${_del_item} | sed 's/.*://g' `
if [ ${_list_id} -gt ${NAS_KEEP_ITEM} ];then
## echo "***************"" rm -vf ${_item_name} "
rm -vf ${_item_name}
fi
done
################################################
if [ -e ${TXT_FILE} ];then
cat ${TXT_FILE} >> ${LOG_FILE}
fi
################################################
先用 crontab 增加一項目:
crontab -l 的內容如下: (擷取 相關的 )
0 23 * * 5 /home/jeff/bin/backup-content.sh > /home/jeff/backup-content.txt 2>&1
相對的 ,寫一個 shell file . 如下:
動作和 之前 SVN 備份差不多 , 只是這次直接對 NAS 上 進行備份 , 並且使用 7z 壓縮程式 !! (tar 好像超過 4G 會有問題 ) .
#!/bin/sh
#
# vi:set ts=4:
## ===== defind echo color
COLOR_ECHO="echo -e" ## in Fedora the color echo need have -e argument.
COLOR_NORMAL="\033[0m"
COLOR_BOLD="\033[1m"
COLOR_RED="\033[31m"
COLOR_GREEN="\033[32m"
COLOR_YELLOW="\033[33m"
## ===== define source and distnation dir .
BACKUP_SRC="/home/${USER}/NAS/Project/Content"
BACKUP_DEST="/home/${USER}/NAS/Project/backup"
NAS_KEEP_ITEM=12
################################################
#### check log file size.
################################################
LOG_MAX_SZ=`echo "1048576 * 100" | bc -l `
LOG_FILE=/home/${USER}/backup-content.log
TXT_FILE=/home/${USER}/backup-content.txt
if [ -e ${LOG_FILE} ];then
_file_sz=`ls -l ${LOG_FILE} | awk '{print $5}'`
if [ ${_file_sz} -gt ${LOG_MAX_SZ} ];then
rm ${LOG_FILE}
fi
fi
################################################
#### chekc NAS direct
################################################
if [ ! -e /home/${USER}/NAS ];then
${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_RED}""#### NAS file system not exist !! ""${COLOR_NORMAL}"
exit 1
elif [ ! -e ${BACKUP_DEST} ];then
${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_RED}""#### backup dir not exist !! ""${COLOR_NORMAL}"
exit 1
fi
#### Check NAS content source dir
if [ ! -e ${BACKUP_SRC} ];then
${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_RED}""#### Content source dir not exist !! ""${COLOR_NORMAL}"
exit 1
fi
################################################
#### Check 7z tools
################################################
COMPRESS_7Z="/usr/local/bin/7z"
VER_7Z=`${COMPRESS_7Z} | grep Copyright | sed 's/Copyright.*//g' | awk '{print $2}'`
## echo "VER_7Z:${VER_7Z}"
## echo "COMPRESS_7Z:${COMPRESS_7Z}"
if [ -z "${VER_7Z}" ];then
${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_RED}""#### Compress Tool 7Z not found !! ""${COLOR_NORMAL}"
exit 1
else
${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_GREEN}""#### 7Z Version is ${VER_7Z} . ""${COLOR_NORMAL}"
fi
################################################
#### Start back up
################################################
_cur_date=`date +%Y_%m_%d`
_bak_name=${BACKUP_DEST}/Content_${_cur_date}.7z
${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_GREEN}""#### Strating Compress ${BACKUP_SRC} ...... ""${COLOR_NORMAL}"
## echo "***************"" 7z a ${_bak_name} ${BACKUP_SRC} "
${COMPRESS_7Z} a ${_bak_name} ${BACKUP_SRC}
################################################
#### Remove oldset back up files .
################################################
## rm oldest back up file
_lists=`find ${BACKUP_DEST}/* | xargs ls -t | grep -n ".7z"`
## echo "_lists:${_lists}"
for _del_item in ${_lists} ; do
_list_id=`echo ${_del_item} | sed 's/:.*//g' `
_item_name=`echo ${_del_item} | sed 's/.*://g' `
if [ ${_list_id} -gt ${NAS_KEEP_ITEM} ];then
## echo "***************"" rm -vf ${_item_name} "
rm -vf ${_item_name}
fi
done
################################################
if [ -e ${TXT_FILE} ];then
cat ${TXT_FILE} >> ${LOG_FILE}
fi
################################################
自動執行.... 備份
因為工作需要 , 每星期我要負責 backup SVN datebase 和 字典的 Content 資料 .
每次都需要 在 console 下執行一堆 command , 並且還讓自己的 PC 處於忙碌 , 修改 code 後要重新 build 都會很慢 , 想說 SVN 那台 server PC (後面簡稱 SVNPC) 都沒有事情作 , 好吧 , 寫一個 shell scrip 讓 SVNPC 定時自動幫我備份資料 .
linux 要自動執行 一個 shell 可以用 crontal -e 指令來填寫啟動時間 , 詳細使用方式請參考"鳥哥" 網頁 !!
http://linux.vbird.org/linux_basic/0430cron.php
crontab -l 的內容如下: (擷取 相關的 )
0 2 * * 6 /home/jeff/bin/backup-svn.sh ec510 auto > /home/jeff/backup-svn-ec510.txt 2>&1
接著 , 寫一個 備份的 shell scrip 吧 !! 首先 , 備份 SVN 資料 , 完整 shell 如下:
這個 shell 會將 /home/svn/repos/<project> 用 tar 的方式壓縮起來 , 並且壓縮的檔名為svn-db-<project>-<date>.tar.bz2 .
執行過程中可以將 message 會放在svnbackup-svn-<project>.txt 內 .
執行結束後 會將 .txt append 到 .log 內 , 如果 .log file 過大 , 會自動清除.
壓縮完畢後本地會留下 最近的 4 個壓縮檔 .
並將最新的 壓縮檔 copy 到 NAS 上.
檢查 NAS上的備份資料數 , 如果超過 20 個就會刪除 最舊的.
大功告成 , 每星期六早上2:00 開始備份 !!
#!/bin/sh
#
# vi:set ts=4:
NAS_KEEP_ITEM=20
LOCAL_KEEP_ITEM=4
## ===== defind echo color
COLOR_ECHO="echo -e" ## in Fedora the color echo need have -e argument.
COLOR_NORMAL="\033[0m"
COLOR_BOLD="\033[1m"
COLOR_RED="\033[31m"
COLOR_GREEN="\033[32m"
COLOR_YELLOW="\033[33m"
################################################
## check log file size and remove it if too large.
################################################
LOG_MAX_SZ=`echo "1048576 * 100" | bc -l `
if [ "$1" == "" ];then
echo " FIXME ****************** not finish "
exit
else
LOG_FILE=/home/${USER}/backup-svn-$1.log
TXT_FILE=/home/${USER}/backup-svn-$1.txt
fi
if [ -e ${LOG_FILE} ];then
_file_sz=`ls -l ${LOG_FILE} | awk '{print $5}'`
if [ ${_file_sz} -gt ${LOG_MAX_SZ} ];then
rm ${LOG_FILE}
fi
fi
################################################
## show some message .
################################################
if [ "$2" == "" ];then
${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_GREEN}"\
"## Back up $1 ...."\
"${COLOR_NORMAL}"
elif [ "$2" == "auto" ];then
_start_time=`date +%Y/%m/%d\ \ %k:%M:%S`
${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_GREEN}"\
"## Auto back up $1 at \"${_start_time}\" ...."\
"${COLOR_NORMAL}"
else
${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_RED}"\
"## Bad command $2 ...."\
"${COLOR_NORMAL}"
exit 1
fi
################################################
## start back up svn .
################################################
_work_dir=`pwd`
_cur_date=`date +%Y%m%d`
if [ $1 == "" ];then
_svn_dir="/home/svn"
_bak_name="$_work_dir/svn-db-$_cur_date.tar.bz2"
${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_GREEN}" \
"Back up whole svn dir.. "\
"${COLOR_NORMAL}"
else
_svn_dir="/home/svn/repos/$1"
_bak_name="$_work_dir/svn-db-$1-$_cur_date.tar.bz2"
${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_GREEN}" \
"Back up $_svn_dir dir..." \
"${COLOR_NORMAL}"
fi
if [ "$_svn_dir" == "" ] || [ ! -e $_svn_dir ]; then
${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_RED}" \
"## The SVN_DIR is not correct ..... "
"${COLOR_NORMAL}"
exit
else
if [ ! -e "$_bak_name" ];then
tar -jcf $_bak_name $_svn_dir
else
${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_RED}" \
"## The file $_bak_name already exist , Please check it .. " \
"${COLOR_NORMAL}"
exit
fi
fi
################################################
## Remove local oldest back up file.
################################################
${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_GREEN}" \
"Remove local oldest backup file..." \
"${COLOR_NORMAL}"
if [ $1 == "" ];then
echo " FIXME ****************** not finish "
exit
else
_lo_lists=`find /home/${USER}/* -maxdepth 1 -type f | grep "$1" | sort -r | grep -n "tar.bz2" `
fi
## echo "_lo_lists:${_lo_lists}"
for _lo_del_item in ${_lo_lists} ; do
_lo_list_id=`echo "${_lo_del_item}" | sed 's/:.*//g' `
_lo_item_name=`echo ${_lo_del_item} | sed 's/.*://g' `
if [ ${_lo_list_id} -gt ${LOCAL_KEEP_ITEM} ] && [ ${_lo_item_name} != ${_bak_name} ] ;then \
rm -vf ${_lo_item_name}
fi
done
################################################
## Copy local backup file to NAS, back up.
################################################
## ===== define source and distnation dir .
if [ $1 == "" ];then
echo " FIXME ****************** not finish "
exit
else
BACKUP_DEST="/home/${USER}/NAS/Project/$1/backup/SVN_Backup"
fi
#### chekc NAS direct
if [ ! -e /home/${USER}/NAS ];then
${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_RED}""#### NAS file system not exist !! ""${COLOR_NORMAL}"
exit 1
elif [ ! -e ${BACKUP_DEST} ];then
${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_RED}""#### backup dir not exist !! ""${COLOR_NORMAL}"
exit 1
fi
#### Copy to tar file to NAS .
${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_GREEN}""#### Copy tar file to NAS ... !! ""${COLOR_NORMAL}"
cp -v ${_bak_name} ${BACKUP_DEST}/
################################################
#### Remove oldest backup file in NAS .
################################################
${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_GREEN}" \
"Remove NAS oldest backup file..." \
"${COLOR_NORMAL}"
if [ $1 == "" ];then
echo " FIXME ****************** not finish "
exit
else
_lists=`find ${BACKUP_DEST}/* | sort -r | grep -n ".tar.bz2"`
fi
## echo "_lists:${_lists}"
for _del_item in ${_lists} ; do
_list_id=`echo ${_del_item} | sed 's/:.*//g' `
_item_name=`echo ${_del_item} | sed 's/.*://g' `
if [ ${_list_id} -gt ${NAS_KEEP_ITEM} ];then \
rm -vf ${_item_name}
fi
done
################################################
## Show dome message.
################################################
if [ "$2" == "" ];then
${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_GREEN}"\
"## Back up $1 done ...."\
"${COLOR_NORMAL}"
elif [ "$2" == "auto" ];then
_end_time=`date +%Y/%m/%d\ \ %k:%M:%S`
${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_GREEN}"\
"## Auto back up $1 done at \"${_end_time}\" ...."\
"${COLOR_NORMAL}"
fi
################################################
if [ -e ${TXT_FILE} ];then
cat ${TXT_FILE} >> ${LOG_FILE}
fi
################################################
每次都需要 在 console 下執行一堆 command , 並且還讓自己的 PC 處於忙碌 , 修改 code 後要重新 build 都會很慢 , 想說 SVN 那台 server PC (後面簡稱 SVNPC) 都沒有事情作 , 好吧 , 寫一個 shell scrip 讓 SVNPC 定時自動幫我備份資料 .
linux 要自動執行 一個 shell 可以用 crontal -e 指令來填寫啟動時間 , 詳細使用方式請參考"鳥哥" 網頁 !!
http://linux.vbird.org/linux_basic/0430cron.php
crontab -l 的內容如下: (擷取 相關的 )
0 2 * * 6 /home/jeff/bin/backup-svn.sh ec510 auto > /home/jeff/backup-svn-ec510.txt 2>&1
接著 , 寫一個 備份的 shell scrip 吧 !! 首先 , 備份 SVN 資料 , 完整 shell 如下:
這個 shell 會將 /home/svn/repos/<project> 用 tar 的方式壓縮起來 , 並且壓縮的檔名為svn-db-<project>-<date>.tar.bz2 .
執行過程中可以將 message 會放在svnbackup-svn-<project>.txt 內 .
執行結束後 會將 .txt append 到 .log 內 , 如果 .log file 過大 , 會自動清除.
壓縮完畢後本地會留下 最近的 4 個壓縮檔 .
並將最新的 壓縮檔 copy 到 NAS 上.
檢查 NAS上的備份資料數 , 如果超過 20 個就會刪除 最舊的.
大功告成 , 每星期六早上2:00 開始備份 !!
#!/bin/sh
#
# vi:set ts=4:
NAS_KEEP_ITEM=20
LOCAL_KEEP_ITEM=4
## ===== defind echo color
COLOR_ECHO="echo -e" ## in Fedora the color echo need have -e argument.
COLOR_NORMAL="\033[0m"
COLOR_BOLD="\033[1m"
COLOR_RED="\033[31m"
COLOR_GREEN="\033[32m"
COLOR_YELLOW="\033[33m"
################################################
## check log file size and remove it if too large.
################################################
LOG_MAX_SZ=`echo "1048576 * 100" | bc -l `
if [ "$1" == "" ];then
echo " FIXME ****************** not finish "
exit
else
LOG_FILE=/home/${USER}/backup-svn-$1.log
TXT_FILE=/home/${USER}/backup-svn-$1.txt
fi
if [ -e ${LOG_FILE} ];then
_file_sz=`ls -l ${LOG_FILE} | awk '{print $5}'`
if [ ${_file_sz} -gt ${LOG_MAX_SZ} ];then
rm ${LOG_FILE}
fi
fi
################################################
## show some message .
################################################
if [ "$2" == "" ];then
${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_GREEN}"\
"## Back up $1 ...."\
"${COLOR_NORMAL}"
elif [ "$2" == "auto" ];then
_start_time=`date +%Y/%m/%d\ \ %k:%M:%S`
${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_GREEN}"\
"## Auto back up $1 at \"${_start_time}\" ...."\
"${COLOR_NORMAL}"
else
${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_RED}"\
"## Bad command $2 ...."\
"${COLOR_NORMAL}"
exit 1
fi
################################################
## start back up svn .
################################################
_work_dir=`pwd`
_cur_date=`date +%Y%m%d`
if [ $1 == "" ];then
_svn_dir="/home/svn"
_bak_name="$_work_dir/svn-db-$_cur_date.tar.bz2"
${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_GREEN}" \
"Back up whole svn dir.. "\
"${COLOR_NORMAL}"
else
_svn_dir="/home/svn/repos/$1"
_bak_name="$_work_dir/svn-db-$1-$_cur_date.tar.bz2"
${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_GREEN}" \
"Back up $_svn_dir dir..." \
"${COLOR_NORMAL}"
fi
if [ "$_svn_dir" == "" ] || [ ! -e $_svn_dir ]; then
${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_RED}" \
"## The SVN_DIR is not correct ..... "
"${COLOR_NORMAL}"
exit
else
if [ ! -e "$_bak_name" ];then
tar -jcf $_bak_name $_svn_dir
else
${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_RED}" \
"## The file $_bak_name already exist , Please check it .. " \
"${COLOR_NORMAL}"
exit
fi
fi
################################################
## Remove local oldest back up file.
################################################
${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_GREEN}" \
"Remove local oldest backup file..." \
"${COLOR_NORMAL}"
if [ $1 == "" ];then
echo " FIXME ****************** not finish "
exit
else
_lo_lists=`find /home/${USER}/* -maxdepth 1 -type f | grep "$1" | sort -r | grep -n "tar.bz2" `
fi
## echo "_lo_lists:${_lo_lists}"
for _lo_del_item in ${_lo_lists} ; do
_lo_list_id=`echo "${_lo_del_item}" | sed 's/:.*//g' `
_lo_item_name=`echo ${_lo_del_item} | sed 's/.*://g' `
if [ ${_lo_list_id} -gt ${LOCAL_KEEP_ITEM} ] && [ ${_lo_item_name} != ${_bak_name} ] ;then \
rm -vf ${_lo_item_name}
fi
done
################################################
## Copy local backup file to NAS, back up.
################################################
## ===== define source and distnation dir .
if [ $1 == "" ];then
echo " FIXME ****************** not finish "
exit
else
BACKUP_DEST="/home/${USER}/NAS/Project/$1/backup/SVN_Backup"
fi
#### chekc NAS direct
if [ ! -e /home/${USER}/NAS ];then
${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_RED}""#### NAS file system not exist !! ""${COLOR_NORMAL}"
exit 1
elif [ ! -e ${BACKUP_DEST} ];then
${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_RED}""#### backup dir not exist !! ""${COLOR_NORMAL}"
exit 1
fi
#### Copy to tar file to NAS .
${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_GREEN}""#### Copy tar file to NAS ... !! ""${COLOR_NORMAL}"
cp -v ${_bak_name} ${BACKUP_DEST}/
################################################
#### Remove oldest backup file in NAS .
################################################
${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_GREEN}" \
"Remove NAS oldest backup file..." \
"${COLOR_NORMAL}"
if [ $1 == "" ];then
echo " FIXME ****************** not finish "
exit
else
_lists=`find ${BACKUP_DEST}/* | sort -r | grep -n ".tar.bz2"`
fi
## echo "_lists:${_lists}"
for _del_item in ${_lists} ; do
_list_id=`echo ${_del_item} | sed 's/:.*//g' `
_item_name=`echo ${_del_item} | sed 's/.*://g' `
if [ ${_list_id} -gt ${NAS_KEEP_ITEM} ];then \
rm -vf ${_item_name}
fi
done
################################################
## Show dome message.
################################################
if [ "$2" == "" ];then
${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_GREEN}"\
"## Back up $1 done ...."\
"${COLOR_NORMAL}"
elif [ "$2" == "auto" ];then
_end_time=`date +%Y/%m/%d\ \ %k:%M:%S`
${COLOR_ECHO} "${COLOR_BOLD}""${COLOR_GREEN}"\
"## Auto back up $1 done at \"${_end_time}\" ...."\
"${COLOR_NORMAL}"
fi
################################################
if [ -e ${TXT_FILE} ];then
cat ${TXT_FILE} >> ${LOG_FILE}
fi
################################################
訂閱：
文章 (Atom)