HACMP安装手册
说明:在实际安装过程中,本文中的主机名和IP地址需要根据实际情况进行调整;
1. 安装前准备
1.1. 硬件设备
1.1.1. 主机
主机两台,每台主机至少需要两块网卡,一个空闲的串口。
如果主机中配了千兆以太网卡,建议使用该千兆网卡,不要使用内置网卡。(因为内置网卡是集成在主板上的,如果一旦出现故障,不容易更换,这样HACMP中的配置就需要修改,增加了工作量)
1.1.2. 心跳线
心跳线一根。
如果心跳线的口和主机的串口不能对上,需要两根转接线,具体的长度由两台主机的相对位置决定。
1.2. 操作系统环境
主机 |
内容 |
文件集 |
备注 |
主机1 |
OS:5200-02
Kerl:64
文件系统:JFS2 |
bos.adt.*
bos.net.*
bos.rte.*
rsct.basic.*
rsct.compat.*
rsct.core.* |
oslevel
–r
prtconf
lslpp
-l |
主机1 |
与主机1一致 |
1.3. 软件介质
HACMP
4.5(CRM)
1.4. 磁盘划分
建立需要切换的VG等。
如果是对ORACLE9i RAC数据库做双机系统,对存放数据库数据的VG(oradatavg)建立时不需要建立属性为concurrent的卷组。(在配置HACMP时设定为concurrent属性即可)
1.5. 操作系统参数配置
1.5.1. /sbin/rc.boot
修改/sbin/rc.boot文件中:
##Modified
By Sinosoft For Hacmp Software Start
#nohup
/usr/sbin/syncd 60
> /dev/null 2>&1 &
nohup
/usr/sbin/syncd 10
> /dev/null 2>&1 &
##Modified
By Sinosoft For Hacmp Software End |
1.5.2. water
mark
smit system -->
Change
/ Show Characteristics of Operating System
HIGH
water mark for pending write I/Os per file
[32] +
LOW
water mark for pending write I/Os per file
[24] +
或者直接用命令:
chdev
-l sys0 -a maxpout='32'
-a minpout='24' |
1.6. 网络地址配置
1.6.1. 主机名及IP地址
主机 |
内容 |
备注 |
主机1 |
boot1 |
10.48.178.201 |
|
serv1 |
10.48.178.202 |
|
stby1 |
10.48.179.3 |
|
主机2 |
boot2 |
10.48.178.203 |
|
serv2 |
10.48.178.204 |
|
stby2 |
10.48.179.4 |
|
具体的hosts文件参考附录中的“主机hosts文件”。
1.6.2. 主机信任关系
在两台主机的/目录下,建立.rhosts文件,
注意改变文件权限:chmod 600 /.rhosts
文件内容如下:
CZXCDB1
boot1
serv1
stby1
CZXCDB2
boot2
serv2
stby2 |
1.6.3. 测试信任关系
在两台主机上做以下操作,看结果是否正常:
ping
CZXCDB1
ping
CZXCDB2
ping
boot1
ping
boot2
ping
stdby1
ping
stdby2
rlogin
CZXCDB1
rlogin CZXCDB2 |
1.7. 心跳线配置及测试
在选择哪个串口作为心跳线的端口时,原则是这样:
在串口富裕的情况下,尽量不要选择第1个和第2个。(因为第一个串口一般做console,第二个常用作远程维护程序使用)
如果不这样,虽然同步和简单的测试没有问题,但在实际应用中回出现HACMP经常自动切换,所以应该必须不使用第1和第2个串口(P630用最后一个,P650可以用最后两个)
1.7.1. 心跳线配置
1.7.1.1. 主机1
1.7.1.1.1. 第一步
登录root用户
smitty
tty
1.7.1.1.2. 第二步
选择”Add a TTY”
TTY
Move
cursor to desired item and press Enter.
List
All Defined TTYs
Add
a TTY
Move
a TTY to Another Port
Change
/ Show Characteristics of a TTY
Remove
a TTY
Configure
a Defined TTY
Generate
Error Report
Trace
a TTY |
1.7.1.1.3. 第三步
Add
a TTY
Type
or select values in entry fields.
Press
Enter AFTER making all desired changes.
[TOP] [Entry
Fields]
TTY
type
tty
TTY
interface
rs232
Description
Asynchronous Terminal
Parent
adapter
sa2
*
PORT number
[0] +
Enable
LOGIN
disable +
BAUD
rate
[9600]
PARITY [none] +
BITS
per character
[8] +
Number
of STOP BITS
[1] +
TIME
before advancing to next port setting
[0]
+
TERMINAL
type
[dumb]
FLOW
CONTROL to be used
[xon] |
1.7.1.2. 主机2
步骤与主机1雷同。
1.7.2. 心跳线测试
在两台主机上,通过lsdev –Cc tty查看新增的串行口,会有类似下面的内容:
tty0
Available 00-00-S3-00 Asynchronous
Terminal |
序号 |
主机 |
内容 |
1. |
主机1 |
stty
</dev/tty0 |
2. |
主机2 |
stty
</dev/tty0
这时在两台主机的命令行下会有内容的显示,否则,tty配置失败。示例如下:
speed
9600 baud; -parity hupcl
eol2
= ^?
brkint
-inpck -istrip icrnl -ixany ixoff onlcr tab3
echo
echoe echok |
3. |
主机1 |
cat
/etc/hosts > /dev/tty0 |
4. |
主机2 |
cat
< /dev/tty0
这时在主机2的命令行下有主机1的/etc/hosts文件的内容,否则,tty配置失败。 |
2. 安装软件
2.1. 主机1
将HACMP V4.5 CRM 软件CD放入CD-ROM,在主控台上执行以下操作:
smitty
installp
->Install
Software
->INPUT
device / directory for software [/dev/cd0]
SOFTWARE
to install
[不选择cluster.haview,
cluster.hativoli,
cluster.man.en_US.haview]
ACCEPT
new license agreements?
yes
安装软件 |
2.2. 主机2
主机2上安装步骤和主机1一致。
3. 配置HACMP
3.1. 配置Topology
3.1.1. hacmp
cluster
#smit
hacmp
Cluster
Configuration
Cluster
Topology
Configure
Cluster
Add
a Cluster Definition
*Cluster
ID [1 ]
*Cluster
Name [cluster
] |
3.1.2. hacmp
node
#smitty
hacmp
Cluster
Configuration
Cluster
Topology
Configure
Node
Add
Cluster Nodes
Node
Names [ ]
加入两个Node:CZXCDB1 CZXCDB2 |
3.1.3. hacmp
network
#smit
hacmp
Cluster
Configuration
Cluster
Topology
Configure
Networks
Configure
IP-based Networks
Configure
Non IP-based Networks
分别增加cz_ether和cz_serial
cz_ether的subnets为10.48.178.0
10.48.179.0 |
3.1.4. hacmp
adapter
适配器有两种类型(网卡、串口),每台节点4个,共8个。
3.1.4.1. 主机1
3.1.4.1.1. service适配器
IP Label serv1
New
IP Label
[]
*
Network Type
[ether]
*
Network Name
[cz_ether]
*
Network Attribute
[public]
*
Interface / IP-Label Function
[service]
IP
Address
[10.48.178.202]
Hardware
Address
[]
Node
Name
[CZXCDB1]
Netmask [255.255.255.0]
Interface
Name |
3.1.4.1.2. boot适配器
IP Label boot1
New
IP Label
[]
*
Network Type
[ether]
*
Network Name
[cz_ether]
*
Network Attribute
[public]
*
Interface / IP-Label Function
[boot]
IP
Address
[10.48.178.201]
Hardware
Address
[]
Node
Name
[CZXCDB1]
Netmask [255.255.255.0]
Interface
Name
en1 |
3.1.4.1.3. standby适配器
IP Label stby1
New
IP Label
[]
*
Network Type [ether]
*
Network Name
[cz_ether]
*
Network Attribute
[public]
*
Interface / IP-Label Function
[standby]
IP
Address
[10.48.179.3]
Hardware
Address []
Node
Name
[CZXZDB1]
Netmask [255.255.255.0]
Interface
Name
en2 |
3.1.4.1.4. rs232适配器
Adapter
Label tty1
New
Adapter Label
[]
Network
Type [rs232]
*
Network Name
[cz_serial]
*
Device Name
[/dev/tty0]
*
Node Name
[CZXCDB1] |
3.1.4.2. 主机2
3.1.4.2.1. service适配器
IP Label serv2
New
IP Label
[]
*
Network Type
[ether]
*
Network Name
[cz_ether]
*
Network Attribute [public]
*
Interface / IP-Label Function
[service]
IP
Address
[10.48.178.204]
Hardware
Address
[]
Node
Name
[CZXCDB2]
Netmask [255.255.255.0]
Interface
Name |
3.1.4.2.2. boot适配器
IP Label boot2
New
IP Label
[]
*
Network Type
[ether]
*
Network Name
[cz_ether]
*
Network Attribute
[public]
*
Interface / IP-Label Function
[boot]
IP
Address
[10.48.178.204]
Hardware
Address
[]
Node
Name
[CZXCDB2]
Netmask [255.255.255.0]
Interface
Name
en1 |
3.1.4.2.3. standby适配器
IP Label stby2
New
IP Label []
*
Network Type
[ether]
*
Network Name
[cz_ether]
*
Network Attribute
[public]
*
Interface / IP-Label Function
[standby]
IP
Address [10.48.179.4]
Hardware
Address
[]
Node
Name
[CZXZDB2]
Netmask [255.255.255.0]
Interface
Name
en2 |
3.1.4.2.4. rs232适配器
Adapter
Label tty2
New
Adapter Label
[]
Network
Type [rs232]
*
Network Name
[cz_serial]
*
Device Name
[/dev/tty0]
*
Node Name [CZXCDB2] |
3.2. 配置Resource Group
3.2.1. hacmp
resource group
增加三个资源组:
3.2.1.1. 1.orares
*Resource
Group Name [orares]
*Node
Relationship concurant
*Site
Relationship
ignore
*Participating
Node Names [CZXCDB1 CZXCDB2 ]
*Service
IP lable []
*Volume Groups
[oradatavg] |
3.2.1.2. 2.serv1net
*Resource
Group Name [serv1net]
*Node
Relationship cascading
*Site
Relationship
ignore
*Participating
Node Names [CZXCDB1 CZXCDB2 ]
*Service
IP lable
[] |
3.2.1.3. 3.serv2net
*Resource
Group Name [serv2net]
*Node
Relationship cascading
*Site
Relationship
ignore
*Participating
Node Names [CZXCDB2 CZXCDB1]
*Service
IP lable
[] |
3.2.2. hacmp
app server
增加一个app server
*Server
Name [app1]
*start
script [/usr/es/script/start.sh]
*stop
script [/usr/es/script/stop.sh] | | | 在两台主机创建目录:
mkdir
–p /usr/es/script
touch
/usr/es/script/start.sh
touch
/usr/es/script/stop.sh
touch
/usr/es/script/oracle_start
touch
/usr/es/script/oracle_stop
chown
–R 777 /usr/es/script/*.sh
脚本的内容可以参考如下:
3.2.2.1.1. start.sh
查看附录中的Application的启动和关闭脚本/start.sh脚本。
3.2.2.1.2. stop.sh
查看附录中的Application的启动和关闭脚本/stop.sh脚本。
3.2.2.1.3. oracle_start
查看附录中的Application的启动和关闭脚本/oracle_start.sh脚本。
3.2.2.1.4. oracle_stop
查看附录中的Application的启动和关闭脚本/oracle_stop脚本。
3.2.3. hacmp
resource group attribute
3.2.3.1. orares
Resource
Group Name
orares
Node
Relationship
concurant
Site
Relationship ignore
Participating
Node Names / Default Node Priority
CZXCDB1
CZXCDB2
Dynamic
Node Priority
[]
Service
IP label
[]
Filesystems
(default is All)
[]
Filesystems
Consistency Check
fsck
Filesystems
Recovery Method
sequential
Filesystems/Directories
to Export
[]
Filesystems/Directories
to NFS mount
[]
Network
For NFS Mount []
Volume
Groups
[]
Concurrent
Volume groups
[oradatavg]
Raw
Disk PVIDs
[]
Connections
Services
[]
Fast
Connect Services []
Tape
Resources
[]
Application
Servers
[app1]
Communication
Links
[]
Primary
Workload Manager Class
[]
Secondary
Workload Manager Class
[]
Miscellaneous
Data
[]
Automatically
Import Volume Groups
false |
3.2.3.2. serv1net
Resource
Group Name
serv1net
Node
Relationship cascading
Site
Relationship
ignore
Participating
Node Names / Default Node Priority
CZXCDB1
CZXCDB2
Dynamic
Node Priority
[]
Service
IP label
[serv1]
Filesystems
(default is All)
[]
Filesystems
Consistency Check
fsck
Filesystems
Recovery Method
sequential
Filesystems/Directories
to Export
[]
Filesystems/Directories
to NFS mount
[]
Network
For NFS Mount
[]
Volume
Groups
[]
Concurrent
Volume groups
[]
Raw
Disk PVIDs
[]
Connections
Services []
Fast
Connect Services
[]
Tape
Resources
[]
Application
Servers
[]
Communication
Links
[]
Primary
Workload Manager Class []
Secondary
Workload Manager Class
[]
Miscellaneous
Data
[]
Automatically
Import Volume Groups
false |
3.2.3.3. serv2net
Resource
Group Name
serv2net
Node
Relationship
cascading
Site
Relationship
ignore
Participating
Node Names / Default Node Priority
CZXCDB2
CZXCDB1
Dynamic
Node Priority
[]
Service
IP label
[serv2]
Filesystems
(default is All)
[]
Filesystems
Consistency Check
fsck
Filesystems
Recovery Method
sequential
Filesystems/Directories
to Export
[]
Filesystems/Directories
to NFS mount
[]
Network
For NFS Mount
[]
Volume
Groups
[]
Concurrent
Volume groups
[]
Raw
Disk PVIDs []
Connections
Services
[]
Fast
Connect Services
[]
Tape
Resources
[]
Application
Servers
[]
Communication
Links []
Primary
Workload Manager Class
[]
Secondary
Workload Manager Class
[]
Miscellaneous
Data
[]
Automatically
Import Volume Groups
false |
4. 同步HACMP
4.1. 同步Topology
#smit
hacmp
Cluster
Configuration
Cluster
Topology
Synchronize
Cluster Topology |
4.2. 同步Resource Group
#smit
hacmp
Cluster
Configuration
Cluster
Resources
Synchronize Cluster Resource | 5.2. 测试
装ORACLE9i RAC之前需要测试,在ORACLE9i
RAC安装完后还需要测试,现列出在ORACLE9i RAC安装完后的测试内容:
序号 |
主机1 |
主机2 |
备注 |
1. |
smitty
clstart |
smitty
clstart |
看VG是否正常挂上,数据库实例和监听进程能否正常启动。 |
2. |
smitty
clstop
takeover |
(主机2正常运行) |
看主机2数据库是否正常,主机1的service
ip能否漂移到主机2的standby网卡上。 |
3. |
(主机1正常运行) |
smitty
clstop
takeover |
看主机1数据库是否正常,主机2的service
ip能否漂移到主机1的standby网卡上。 |
4. |
拔掉boot网卡的网线 |
(主机2正常运行) |
看主机1的service ip能否切换到本机的standby网卡上,standby ip切换到bootip网卡上。 |
5. |
接通原boot ip网卡的网线 |
(主机2正常运行) |
看主机1的service ip不会切回到本机的boot网卡上,standby ip仍然在原boot ip的网卡上。
如果接着用graceful关掉hacmp,他们的IP也不会回到原来的位置。 |
6. |
(主机1正常运行) |
拔掉boot网卡的网线 |
看主机2的service ip能否切换到本机的standby网卡上,standby ip切换到bootip网卡上。 |
7. |
(主机1正常运行) |
接通原boot ip网卡的网线 |
看主机2的service ip不会切回到本机的boot网卡上,standby ip仍然在原boot ip的网卡上。
如果接着用graceful关掉hacmp,他们的IP也不会回到原来的位置。 |
具体的IP切换的情况,参考《hacmp_status.ppt》。
6. 维护和管理
7. 问题集锦
7.1. .rhosts文件的属性
chmod
600 .rhosts
7.2. Application脚本的属性
chmod
777 脚本
7.3. 配置Resource
Group的属性
注意看3.2.3中对资源组的配置信息。
8. 附录
8.1. 主机hosts文件
8.1.1. 主机1
127.0.0.1 loopback localhost CZXCDB1 # loopback
(lo0) name/address
######################################
#Hosts
For CZXCDB1
10.48.178.201 boot1
10.48.178.202 serv1
10.48.179.3 stby1
CZXCDB1
######################################
######################################
#Hosts
For CZXCDB2
10.48.178.203 boot2
10.48.178.204 serv2
10.48.179.4 stby2
CZXCDB2
###################################### |
8.1.2. 主机2
127.0.0.1 loopback localhost CZXCDB2# loopback
(lo0) name/address
######################################
#Hosts
For CZXCDB1
10.48.178.201 boot1
10.48.178.202 serv1
10.48.179.3 stby1
CZXCDB1
######################################
######################################
#Hosts
For CZXCDB2
10.48.178.203 boot2
10.48.178.204 serv2
10.48.179.4 stby2
CZXCDB2
###################################### |
8.2. Application的启动和关闭脚本
8.3. start.sh
##############################################################
##
start oracle server
echo
"`hostname`:The ORACLE Server is starting,Please Waiting."
sleep
5
su
- oracle -c ". /usr/es/script/oracle_start"
>>/usr/es/script/oracle_start.log
sleep
5
echo
"`hostname`:The ORACLE Server is started."
############################################################## |
8.4. stop.sh
##############################################################
##
stop oracle server
echo
"`hostname`:The ORACLE Server is stopping,Please Waiting."
su
- oracle -c ". /usr/es/script/oracle_stop"
>>/usr/es/script/oracle_stop.log
sleep
5
echo
"`hostname`:The ORACLE Server is stoped."
############################################################## |
8.5. oracle_start
echo
"Start Oracle DataBase Begin"
sqlplus
/nolog <<EOF
connect
sys/change_on_install as sysdba;
startup;
show
sga;
exit;
EOF
sleep
5
lsnrctl
start
echo
"Start Oracle DataBase End" |
8.6. oracle_stop
echo
"Stop Oracle DataBase Begin"
lsnrctl
stop
sleep
5
sqlplus
/nolog <<EOF
connect
sys/change_on_install as sysdba
show
sga;
shutdown
immediate;
exit;
EOF
echo
"Stop Oracle DataBase End" |
|