E10K Admin

`
E10K Administration

(cavi99@chollian.net)
E10K System Administration
1. Installing SSP 3.5.......................................................................................................... 7

1.1. SSP Release Note........................................................................................... 7
1.2. Configuring SSP Network files ...................................................................... 8
1.2.1. 2 Sub net Configuration .......................................................................... 8
1.2.2. 3 subnet Configuration............................................................................ 9
1.2.3. 3 sbunet (spare SSP) Configuration ..................................................... 10
1.2.4. Configuring Network Files.................................................................... 10
1.2.5. Reboot SSP............................................................................................ 12
1.3. Install SSP software packages..................................................................... 12
1.3.1. Web Start ...................................................................................... 12
1.3.2. ssp_install ..................................................................................... 12
1.4. Configuring SSP ........................................................................................... 12
1.5. Reboot SSP ................................................................................................... 13
2. Preparing the domain ................................................................................................. 13
2.1. Creating an eeprom.image file..................................................................... 13
2.2. Creating the domain resource ..................................................................... 14
2.3. Bringing up the domain ................................................................................ 14
3. Installing the OS on the domain ................................................................................. 14
3.1. Configuring the SSP as boot server ............................................................ 14
3.1.1. Configuring OS Image (Solaris 8) .................................................. 14
3.1.2. Configuring Sysidcfg file ...................................................................... 15
3.1.3. Configuring Rules and Profile............................................................... 15
3.1.4. Editing /etc/vfstab................................................................................. 16
3.1.5. Adding Client......................................................................................... 16
3.1.6. E10K ssphostname Configuration......................................................... 16
3.1.7. Booting Client using Network............................................................... 16
4. Using netcon ............................................................................................................... 17
5. Booting the domain for the first time......................................................................... 17
5.1. Configuring boot devices ............................................................................. 17
5.1.1. Disk Alias ...................................................................................... 17
5.1.2. Network Alias ............................................................................... 18
6. Finishing the installation............................................................................................. 18
6.1. SSP hostname ...................................................................................... 18
6.2. NTP ...................................................................................................... 18
2
Written by (cavi99@chollian.net)
http://cavi99.mchol.com
6.2.1. Main SSP ............................................................................................... 18

6.2.2. Spare SSP.............................................................................................. 18
6.2.3. Domain................................................................................................... 19
6.2.4. Starting NTP daemon............................................................................ 19
7. Multipathing ................................................................................................................ 19
7.1. Alternate Pathing (AP) ................................................................................. 19
7.2. AP Initial Configuration ................................................................................ 19
7.2.1. Package ........................................................................................ 19
7.2.2. Creating AP database ........................................................................... 19
7.2.3. To view AP database ............................................................................ 20
7.3. Disk Path Group ........................................................................................... 20
7.3.1. Disk Path group ............................................................................ 20
7.3.2. AP and Boot disk................................................................................... 20
7.3.3. To remove AP support from the boot disk .......................................... 20
7.3.4. AP and mirrored boot disk.................................................................... 20
7.3.5. To remove a mirror from AP................................................................ 21
7.3.6. Manually switching the Active PATH................................................... 21
7.4. Network Pathgroup ...................................................................................... 21
7.4.1. Creating Nework Pathgroup ................................................................. 21
7.4.2. Deleting Network Pathgroup ................................................................ 22
7.4.3. Manually switching the Active Path ..................................................... 22
7.5. MpxIO (Sun StorEdge Traffic Manager)...................................................... 22
7.5.1. MPxIO Installation (STMS 4.2 on Solaris 8) ........................................ 22
7.5.2. T3 Restriction ....................................................................................... 23
8. IPMP............................................................................................................................ 23
8.1. IPMP Overview............................................................................................. 23
8.2. IPMP Requirement........................................................................................ 23
8.3. IPMP Setup ................................................................................................... 23
8.3.1. Multiple Active Paths without Standby ................................................ 23
8.3.2. Single Active Path with Standby .......................................................... 24
8.4. Automating IPMP.......................................................................................... 24
8.4.1. Multiple Active Paths without Standby ................................................ 24
8.4.2. Single Active Path with Standby .......................................................... 24
9. Dynamic Reconfiguration (DR) ................................................................................... 24
9.1. DR Model ...................................................................................................... 24
9.2. DR 2.0 ........................................................................................................... 25
3
9.2.1. Checking DR model............................................................................... 25

9.2.2. View DR status...................................................................................... 25
9.2.3. Attaching a Board ................................................................................. 25
9.2.4. Detaching a Board................................................................................. 25
9.2.5. Using Automatic Dynamic Reconfiguration (ADR)Commands ............. 26
9.3. DR 3.0 ........................................................................................................... 26
9.3.1. DR 3.0 Overview ................................................................................... 26
9.3.2. DR 3.0 Commands ................................................................................. 27
10. Inter-Domain Network ........................................................................................ 27
10.1. IDN Overview ............................................................................................... 27
10.2. Setting up IDN between 2 domains.............................................................. 28
10.2.1. Monitor IDN........................................................................................... 28
10.2.2. Remove IDN .......................................................................................... 28
11. Troubleshooting .................................................................................................. 29
11.1. redlist and blacklist file................................................................................ 29
11.2. Recovery from domain hang ........................................................................ 29
11.2.1. Domain soft hang .................................................................................. 29
11.2.2. Force a panic from the SSP .................................................................. 29
11.2.3. Force and XIR from the SSP................................................................. 29
11.2.4. Force bringup from the SSP (last resort)............................................. 29
12. Useful Administration on E10K........................................................................... 29
12.1. SSP Failover ................................................................................................. 29
12.1.1. Command Syntax .................................................................................. 29
12.1.2. SSP failover Example ........................................................................... 30
12.2. SSP Backup and Restore.............................................................................. 31
12.3. Control Board Failover................................................................................. 31
12.3.1. setfailover .............................................................................. 31
12.3.2. ssp_config .............................................................................. 31
12.4. Control Board Firmware Upgrade ............................................................... 31
12.4.1. Control Board .............................................................. 31
12.4.2. Control Board Firmware Upgrade ........................................................ 31
12.5. Platform Management Command ................................................................. 31
12.5.1. hostinfo.................................................................................................. 31
12.5.2. power..................................................................................................... 32
12.5.3. fan.......................................................................................................... 32
12.5.4. hostview ................................................................................................ 33
4
12.6. Upgrading to SSP 3.5 ................................................................................... 33

12.6.1. SSP 3.4 to SSP 3.5 ................................................................................ 33
12.6.2. SSP 3.2,3.3 to SSP 3.5.......................................................................... 33
12.7. Disable DMP with AP ................................................................................... 34
13. E10K vs E15K/12K.............................................................................................. 34
13.1. Daemon......................................................................................................... 34
13.2. Commands .................................................................................................... 35
13.3. Configuration Files ....................................................................................... 36
13.4. Miscellaneous ............................................................................................... 37
5
Document History
Version
1.0 2003. 8. 1 Initial
1.1 2003.9.29 15/12K reference
6
SSP 3.5, SSP Solaris 8, Domain Solaris 8

.
E10K
Install SSP Operating System
Install SSP Package
Configuring the SSP as boot server
Preparing the domain
Installing the OS on the domain
Booting the domain for the first time
1. Installing SSP 3.5

1.1. SSP Release Note
SSP Verson SSP OS Domain OS DR Model IDN COD
2.5.1, 2.6 2.0

3.5 7,8,9 Supported 1.0
7,8,9 3.0
2.6 2.5.1, 2.6

3.4 2.0 Supported 1.0
7,8,9 7,8
2.6 2.5.1, 2.6

7,8 7,8
2.6 2.5.1, 2.6

7,8 7,8
2.5.1, 2.6 2.5.1, 2.6 Not

3.1.1 1.0 1.0
7 7 Supported
2.5.1, 2.6 Not

3.1 2.5.1 1.0 1.0
7 Supported
Not
3.0 2.5.1 2.5.1 1.0 1.0
Supported
7
1.2. Configuring SSP Network files

Sub Net Equipments Failover
2 1 SSP, 1 Hub, 1 CB SSP, CB failover disabled
3 1 SSP, 2 Hub, 2 CB only CB failover enabled
3 (spare SSP) 2 SSP, 2 Hun, 2 CB SSP, CB failover enabled
1.2.1. 2 Sub net Configuration
8
1.2.2. 3 subnet Configuration
9
1.2.3. 3 sbunet (spare SSP) Configuration
1.2.4. Configuring Network Files

.
/etc/hosts
# cat /etc/hosts (Main SSP)
127.0.0.1 localhost
#############################
# Entry for SSP
#############################
192.168.0.1 ses-main-ssp # floating IP
192.168.0.2 ssp0 loghost # main SSP
192.168.0.3 ssp1 # spare SSP
#############################
# Entry for Domain_Network
#############################
203.234.247.100 domain1
10
203.234.247.101 domain2
#############################
# Entry for cb0_Network
#############################
192.168.1.1 ssp0-hme0
192.168.1.2 ssp1-hme0
192.168.1.3 ses-cb0
#############################
# Entry for cb1_Network
#############################
192.168.2.1 ssp0-qfe1
192.168.2.2 ssp1-qfe1
192.168.2.3 ses-cb1
Spare SSP loghost

spare SSP .
/etc/hostname.*
.
/etc/hostname.hme0
/etc/hostname.qfe0
/etc/hostname.qfe1 (Spare SSP )
/etc/ethers
Domain, SSP, Control Board Ethernet Address .
# cat /etc/ethers
0:0:be:1:2:3 domain1
0:0:be:4:5:6 domain2
0:0:be:7:8:9 ses-cb0
0:0:be:a:b:c ses-cb1
8:0:20:1:2:3 ssp0
8:0:20:4:5:6 ssp1
/etc/defaultrouter /etc/notrouter
# touch /etc/notrouter
NIC router
/etc/notrouter default router .
11
/etc/netmasks
# cat /etc/netmasks
203.234.247.0 255.255.255.0
192.168.0.0 255.255.255.0
SSP network netmask .
/etc/nsswitch.conf
Name Service .
1.2.5. Reboot SSP

# reboot
1.3. Install SSP software packages

SSP .
# pkginfo l SUNWsspr
SSP package 2 .
1.3.1. Web Start

# cd /cdrom_base_directory
# ./installer
1.3.2. ssp_install
# cd /cdrom_base_directory/
System_Service_Processor_3.5/Tools
# ./ssp_install ../Product
1.4. Configuring SSP

# /opt/SUNWssp/bin/ssp_config
You must specify what type of processor modules you have installed in your
Enterprise-10000 platform. If you have a mixture of processors, select the
option corresponding to the lowest processor speed in your configuration.
Please
select one of the options below.
1) 250 MHz processors
5) Unlisted (manually enter clock values)
12
What speed of processors do you have installed? 3

Your selections are apparently for a 336 MHz processor.
Is this correct? (y/n) y

The platform name identifies the entire host machine to the SSP software.
The
platform name occupies a different name space than domain names (hostnames
of
bootable systems).
What is the name of the platform this SSP will service? ses
Please enter the floating host name of the main SSP [ses-ssp-main]:
ses-ssp-main
main SSP name = ses-ssp-main
main SSP address = 192.168.95.109
Is this correct? (y/n) y
Do you have a control board 0? (y/n)y
Please enter the host name of the control board 0 [allxf4cb0]: ses-cb0
Do you have a control board 1? (y/n)y
Please enter the host name of the control board 1 [allxf4cb1]: ses-cb1
Please identify the primary control board.
Is Control Board 0 [ses-cb0] the primary? (y/n)y

Platform name = ses
Control board 0 = ses-cb0 => 192.168.151.123
Control board 1 = ses-cb1 => 192.168.152.123
Primary Control Board = 0
Is this correct? (y/n)y

SSP configuration completed.
1.5. Reboot SSP

# reboot
2. Preparing the domain

2.1. Creating an eeprom.image file
ssp% cd $SSPVAR/.ssp_private/eeprom_save
ssp% sys_id -f eeprom.image.domain_name -k key -h hostid
13
eeprom
ssp% sys_id -d -f eeprom.image.domain_name
2.2. Creating the domain resource

ssp% domain_status
ssp% domain_create -d new_domain_name -b 0 3 4 o 5.8 -p platform_name
ssp% domain_status

ssp% domain_remove -d domain_name
ssp% domain_rename -d old_name -n new_name -o OS_Version
2.3. Bringing up the domain

ssp% echo $SUNW_HOSTNAME
domain1
ssp% domain_switch domain2 ( domain .)
ssp% power -on
ssp% bringup -A off
ssp% netcon
ok
3. Installing the OS on the domain

3.1. Configuring the SSP as boot server
3.1.1. Configuring OS Image (Solaris 8)
# mkdir /export/install
Solaris 1/2 CD
# cd /cdrom/cdrom0/s0/Solaris_8/Tools
# ./setup_install_server /export/install
# cd /
# eject cdrom
Solaris 2/2 CD
# cd /cdrom/cdrom0/Solaris_8/Tools
# ./add_to_install_server /export/install
# cd /
# eject cdrom
Language Pack CD
14
# cd /cdrom/cdrom0/Solaris_8/Tools
# ./add_to_install_server /export/install
3.1.2. Configuring Sysidcfg file

# mkdir /export/config
# cd /export/install/Solaris_8/Misc/jumpstart_sample
# cp r * /export/config
# cd /export/config
sysidcfg .
# cat sysidcfg
system_locale=ko
timezone=ROK
timeserver=localhost
network_interface=primary {netmask=255.255.255.0 protocol_ipv6=no}
terminal=dtterm
name_service=NONE
security_policy=NONE
root_password=jVcibwphl0aRs (/etc/shadow )
3.1.3. Configuring Rules and Profile

# rm rules
rules .
# cat rules
any - - class end_script
class profile . ( partition .)
# cat class
install_type initial_install
system_type standalone
partitioning explicit
filesys c0t0d0s0 free /
filesys c0t0d0s1 1024 swap
filesys c0t0d0s4 5 (AP Database )
filesys c0t0d0s5 5 (AP Database )
cluster SUNWCXall
locale ko
# vi end_script
touch /a/noautoshutdown
15
# chmod +x end_script
# ./check
# ls class rules.ok
( rules.ok .)
3.1.4. Editing /etc/vfstab

# vi /etc/vfstab
share -F nfs o ro /export/config
# shareall
# dfshares ( .)
3.1.5. Adding Client

# cd /export/install/Solaris_8/Tools
/etc/hosts, /etc/ethers
.
# ./add_install_client c ssp_name:/export/config p ssp_name:/export/config
client_name sun4u
( -p: sysidcfg c: profile, rules.ok )
3.1.6. E10K ssphostname Configuration

E10000 Server Domain ssp
script .
# cd /export/install/Solaris_8/Tools/Boot/usr/sbin/install.d/install_config
# vi ssp_config
#!/sbin/sh
echo float_ssp_hostname > /tmp/ssphostname
echo float_ssp_IP_address > /tmp/ssphostaddr
3.1.7. Booting Client using Network

ssp% netcon
ok show-nets
ok devalias
ok nvunalias net (alias alias )
ok nvalias net ^Y
ok nvstore
ok boot net install
SSP SSP IP address .
16
(ssp_begin script .)
4. Using netcon
Command option Description
~. exit from netcon
~# send BREAK signal (stop-a)
~= toggle JTAG betwwen cvcd mode
~? show status of all netcon session
~^ -r read only session (release write lock)
~@ -g unlocked-write session
~& -l locked-write session
~* -f exclusive session
ssp% domain_switch domain1

ssp% netcon
5. Booting the domain for the first time

5.1. Configuring boot devices
5.1.1. Disk Alias
alias
ok nvunalias disk
ok reset
ok show-disks ( )
ok nvalias disk ^Y
( @ Fiber Disk WWN, SCSI Disk
target,LUN:slice .
.
ok nvalias disk /sbus@4d,0/SUNW.socal@0,0/
sf@0,0/ssd@200000008a1085
ok nvalias disk /sbus@4c,0/QLGC,isp@1,10000
/sd@3,0:a)
ok nvstore
17
ok devalias
ok boot
5.1.2. Network Alias

ok nvunalias net
ok show-nets
ok nvalias net ^Y
ok nvstore
ok devalias
6. Finishing the installation
6.1. SSP hostname

domain /etc/ssphostname /etc/hosts floating SSP
IP .
6.2. NTP
6.2.1. Main SSP
# cd /etc/inet
# cp ntp.server ntp.conf
# vi ntp.conf
peer spare_ssp_name
server 127.127.1.0
fudge 127.127.1.0 stratum 8
driftfile /var/ntp/ntp.drift
peer machine1 (machine is a IP address of external NTP Server if necessary)
peer machine2
# touch /var/ntp/ntp.drift
6.2.2. Spare SSP

# cd /etc/inet
# cp ntp.server ntp.conf
# vi ntp.conf
peer main_ssp_name
server 127.127.1.0
fudge 127.127.1.0 stratum 8
18
driftfile /var/ntp/ntp.drift
peer machine1 (machine is a IP address of external NTP Server if necessary)
peer machine2
# touch /var/ntp/ntp.drift
6.2.3. Domain
# cd /etc/inet
# vi ntp.conf
server main_ssp_name prefer
server spare_ssp_name
6.2.4. Starting NTP daemon

on SSP and All domains
# /etc/init.d/xntpd start
7. Multipathing
7.1. Alternate Pathing (AP)
Sbus based Multipathing Solution
automatic failover capabilities for disks
manual failover capabilities for network interfaces
Only one path is active ( does not increase IO performace)
VxVM DMP is not installed if AP metadevices are found in the system
AP needs small partition for configuration database
7.2. AP Initial Configuration
7.2.1. Package
# pkgadd d . SUNWapr SUNWapu SUNapdev SUNWapdoc
7.2.2. Creating AP database

# apdb -c /dev/rdsk/c0t0d0s4 f
(-c create, AP database f (forcely) )
# apdb -c /dev/rdsk/c0t0d0s5
) Deleting AP database
# apdb -d /dev/rdsk/c0t0d0s4 -f
19
7.2.3. To view AP database

# apconfig D
7.3. Disk Path Group

7.3.1. Disk Path group
# apinst
(view the metadisk configuration)
# apdisk -c -p sf0 -a sf2

(-c creat, -p pathgroup -a alternate path)
# apconfig S u
(view uncommitted disk entries)
# apdb C
(commit database)
# devfsadm
# vi /etc/vfstab
Edit /etc/vfstab entry except boot disk file system

(boot disk file system should be edited using apboot command)
# ls l /dev/ap/dsk
# apconfig S
(View committed Disk entries)
7.3.2. AP and Boot disk

if you want to placing the boot disk under AP control,
first, create AP pathgroup for a physical path that include the boot disk and
then
# apboot mc0t0d0
(apboot command replaces /etc/system, /etc/vfstab entry automatically)
7.3.3. To remove AP support from the boot disk

# apboot c0t0d0
( this command also cleans up unnecessary entry in /etc/vfstab, /etc/system )
7.3.4. AP and mirrored boot disk

# apboot mc0t0d0
20
# apboot m mc1t0d0
and then create mirror using disk management software
Do not put either of mirrors into OBP boot-device parameter
7.3.5. To remove a mirror from AP

# apboot -u mc1t0d0
7.3.6. Manually switching the Active PATH

# apconfig S
(view current configuration)
# apconfig P sf0 a sf2
(-P pathgroup -a the path to become active)
if you want to switch back

# apconfig P sf0 a sf0
AP for disk can be switched automatically, but when a path is marked T

(apconfig S), AP does not automatically switch. You can reset the tried flag by
one of these 3 methods
- # apdisk w sf2
(sf2 is the path that is marked T)
- rebooting the domain
- DR detach and DR attach
7.4. Network Pathgroup

7.4.1. Creating Nework Pathgroup
AP database database
# apdb c /dev/rdsk/c0t0d0s7 f
# apnet c p qfe0 a qfe4
( -c create p pathgroup a alternate path)
# apconfig N u
(view uncommitted network entries)
# apdb C
(commit database)
# apconfig N
(view Network Entries)
# ifconfig qfe0 down unplumb
21
# mv /etc/hostname.qfe0 /etc/hostname.mether0
# ifconfig mether0 plumb
# ifconfig mether0 inet 192.168.10.1 netmask + broadcast + up
# ifconfig a
7.4.2. Deleting Network Pathgroup

# ifconfig mether0 unplumb
# apnet d mether0
# apdb C
# apconfig N
7.4.3. Manually switching the Active Path

# apconfig P mether0 a qfe4
# apconfig -N
7.5. MpxIO (Sun StorEdge Traffic Manager)

PCI based Multipathing Solution
A5200, T3 array Fiber Channel array
automatic failover capabilities for disks
does not support network interface
increases IO performace by load balancing across multiple IO channels
7.5.1. MPxIO Installation (STMS 4.2 on Solaris 8)
MPxIO Packages Install
# pkgadd d . SUNWsan SUNWcfpl SUNWcflx
Pathch Installation
# patchadd M /patch_dir 111412-xx 11095-xx 111096-xx 111097-xx
111413-xx
Enable STMS
# vi /kernel/drv/scsi_vhci.conf
...
mpxio-disable=no (yes no )
....
scsi_vhci device
# grep scsi_vhci /etc/name_to_major
/etc/system
22
set ssd:ssd_io_time = 180

set ssd:ssd_max_throttle = 8
fp_offline_ticker=15
fp_retry_count=2 Failover Time 34
Reconfiguration boot
# reboot -- -r
7.5.2. T3 Restriction
T3 array STMS
# telnet t3
Login: root
Password:
T3> sys mp_support mpxio
T3> sys list
8. IPMP
8.1. IPMP Overview
Failure detection (auto failover)
Repair detection (auto failback)
Outbound load spreading (not inbound)
8.2. IPMP Requirement

the same type of network interface
OBP local-mac-address? true
IPMP is supported on Solaris 8 update 2 (SUNWcsr)
8.3. IPMP Setup

8.3.1. Multiple Active Paths without Standby
# eeprom local-mac-address?=true
# ifconfig hme0 plumb 192.168.0.231 group group1 deprecated failover
netmask + broadcast + up
# ifconfig hme0 addif 192.168.0.232 netmask + broadcast + up
# ifconfig qfe1 plumb 192.168.0.233 group group1 deprecated failover
23
# ifconfig qfe1 addif 192.168.0.234 netmask + broadcast + up

# ifconfig -a
8.3.2. Single Active Path with Standby

# eeprom local-mac-address?=true
# ifconfig hme0 plumb 192.168.0.231 group group1 deprecated failover
# ifconfig hme0 addif 192.168.0.232 netmask + broadcast + up
# ifconfig qfe1 plumb 192.168.0.233 group group1 netmask + broadcast +
deprecated -failover standby up
# ifconfig a
8.4. Automating IPMP

8.4.1. Multiple Active Paths without Standby
# cat /etc/hostname.hme0
192.168.0.231 group group1 -failover deprecated up
addif 192.168.0.232 up
# cat /etc/hostname.qfe1
addif 192.168.0.234 up
8.4.2. Single Active Path with Standby

# cat /etc/hostname.hme0
addif 192.168.0.232 up
# cat /etc/hostname.qfe1
192.168.0.233 group group1 deprecated -failover standby up
9. Dynamic Reconfiguration (DR)

9.1. DR Model
DR SSP Solaris
Domain Solaris Version SSP Version
Model Version
2.0 Solaris 7,8 Solaris 2.5.1, 6,7,8 3.3, 3.4, 3,5
24
3.0 Solaris 8 10/01, 02/02, Solaris 9 3.5 only
9.2. DR 2.0
9.2.1. Checking DR model
ssp% domain_status m
only one of DR model can be used at a time
9.2.2. View DR status

ssp% dr
dr> drshow 0 CPU
dr> drshow 0 MEM
dr> drshow 0 IO
dr> drshow 0 OBP
dr> drshow 0 DRAIN
dr> drshow 0 UNSAFE
dr> drshow ALL
dr> exit
9.2.3. Attaching a Board

ssp% dr
dr> init_attach 6
dr> complete_attach 6
dr> exit
if you want to abort attach process, you can use

dr> abort_attach 6
before you have executed complete_attach command
9.2.4. Detaching a Board

ssp% dr
dr> drain 6
dr> drshow 6 drain
dr> complete_detach 6
dr> exit
also you can abort detach operation using abort_detach command
25
9.2.5. Using Automatic Dynamic Reconfiguration (ADR)Commands

beginning in SSP 3.3, ADR commands can be used
# showusages
# addboard -b 2 d domain1
or
# addboard d domain1 SB1
# deleteboard b 2
or
# deleteboard SB1
# moveboard b 5 d domain2
or
# moveboard d domain2 SB5
9.3. DR 3.0
9.3.1. DR 3.0 Overview
SSP 3.5 Domain OS Solaris 10/01 DR 3.0 .
E10K DR 2.0 .
DR Version
/platform/SUNW,Ultra-Enterprise-10000/kernel/drv/nfdr.conf
DR 2.0
legacy-dr-model=0
DR 3.0
legacy-dr-model=1
rebooting.
DR Version
# domain_status m
-m SSP 3.5 .
DR 2.0 DR 3.0 .
platform DR 2.0 DR 3.0 .
DR 3.0 RCM(Reconfiguration Coordination Manager) systevent(System

Event Facility) .
26
9.3.2. DR 3.0 Commands

rcfgadm
ssp
Attaching a Board
ssp% rcfgadm -d domain1 a
cfgadm
Attaching a Board
domain1# cfgadm -d domain1 c configure Ap_ID
Detaching a Board
domain1# cfgadm -d domain1 -a
domain1# cfgadm -d domain1 -c disconnect Ap_ID
Moving boards between domains

domain# cfgadm -a
domain1# cfgadm -d domain1
domain1# cfgadm -d domain1 -c disconnect Ap_ID
domain1# cfgadm -d domain1 x unassign Ap_ID
domain2# cfgadm -d domain2 x assign
domain2# cfgadm -d domain2 c configure Ap_ID
domain2# cfgadm -d domain2 -a
ADR commands
addboard, deleteboard, moveboard DR 2.0 DR 3.0
.
10. Inter-Domain Network

10.1. IDN Overview
High-speed batch data transfer
External network load reduction
Inter-Domain communication
The Domain must have no early than the Solaris 7 Hardware 11/99 (SUNWidn.u
SUNWidnx.u)
The SSP version should be no early than SSP 3.2
27
10.2. Setting up IDN between 2 domains

ssp% netcon
domain1# vi /etc/hostname.idn0
domain1-idn0
domain1# vi /etc/hosts
192.200.2.1 domain1-idn0
192.200.2.2 domain2-idn0
domain1# init 0
domain1-ok setenv idn-smr-size 32
domain1-ok boot
domain1# ifconfig -a

ssp:domain2% netcon
domain2# vi /etc/hostname.idn0
domain2-idn0
domain2 # vi /etc/hosts
192.200.2.1 domain1-idn0
192.200.2.2 domain2-idn0
domain2# init 0
domain2-ok setenv idn-smr-size 32
domain2-ok boot
domain2# ifconfig a
ssp:domain2% domain_link domain1 domain2

ssp:domain2% domain_status
10.2.1. Monitor IDN

domain1# netstat -k idn
domain1# netstat -k idn0
domain1# snoop -d idn0
10.2.2. Remove IDN

ssp:domain2% domain_unlink domain1
28
11. Troubleshooting
11.1. redlist and blacklist file
To enable blacklist, you can edit blacklist file or use Hostview
Default location and file name

$SSPVAR/etc/platform_name/redlist
$SSPVAR/etc/platform_name/blacklist
location and file name can be overridden by directives in .postrc
Changes take effect the next time a domain runs bringup
11.2. Recovery from domain hang

11.2.1. Domain soft hang
ssp:domain1% netcon
~#
ok sync
A panic dump is created
11.2.2. Force a panic from the SSP

ssp:domain1% hostint
A panic dump is created
11.2.3. Force and XIR from the SSP

ssp:domain1% hostreset
A resetinfo file is created
11.2.4. Force bringup from the SSP (last resort)

ssp:domain1% bringup f
No failure information is recorded at all
12. Useful Administration on E10K

12.1. SSP Failover
12.1.1. Command Syntax
ssp% showfailover [ -r ]
ssp% setfailover [ on | off | force ]
29
12.1.2. SSP failover Example

ssp% showfailover
Failover State:
SSP Failover: Active
CB Failover: Active
Failover Connection Map:
Main SSP to Spare SSP thru Main Hub: GOOD
Main SSP to Spare SSP thru Spare Hub: GOOD
Main SSP to Primary Control Board: GOOD
Main SSP to Spare Control Board: GOOD
Spare SSP to Main SSP thru Main Hub: GOOD
Spare SSP to Main SSP thru Spare Hub: GOOD
Spare SSP to Primary Control Board: GOOD
Spare SSP to Spare Control Board: GOOD
SSP/CB Host Information:
Main SSP: ssp1
Spare SSP: ssp0
Primary Control Board (JTAG source): ses-cb1
Spare Control Board: ses-cb0
System Clock source: ses-cb1
ssp% setfailover force
ssp% showfailover
Failover State:
SSP Failover: Failed
CB Failover: Active
Failover Connection Map:
Main SSP to Spare SSP thru Main Hub: GOOD
Main SSP to Spare SSP thru Spare Hub: GOOD
Main SSP to Primary Control Board: GOOD
Main SSP to Spare Control Board: GOOD
Spare SSP to Main SSP thru Main Hub: GOOD
Spare SSP to Main SSP thru Spare Hub: GOOD
Spare SSP to Primary Control Board: GOOD
Spare SSP to Spare Control Board: GOOD
30
SSP/CB Host Information:

Main SSP: ssp0
Spare SSP: ssp1
Primary Control Board (JTAG source): ses-cb1
Spare Control Board: ses-cb0
System Clock source: ses-cb1
ssp% setfailover on
12.2. SSP Backup and Restore

ssp-main% ssp_backup /backup_dir
ssp-main% ls /backup_dir
ssp_backup.cpio
ssp_backup% cpio it I /backup_dir/ssp_backup.cpio
ssp-main% ssp_restore /backup_dir/ssp_backup.cpio
12.3. Control Board Failover

12.3.1. setfailover
ssp% setfailover t cb [ on | off | force ]
12.3.2. ssp_config
# /opt/SUNWssp/bin/ssp_config cb
# reboot
12.4. Control Board Firmware Upgrade

12.4.1. Control Board
ssp% cb_prom -r -h cb_name
12.4.2. Control Board Firmware Upgrade

ssp% cd /opt/SUNWssp/cbobjs
ssp% cb_prom -p flash_boot.ima -h cb_name
ssp% cb_reset cb_name
12.5. Platform Management Command

12.5.1. hostinfo
ssp% hostinfo h ( CPU status )
ssp% hostinfo F ( fan status )
ssp% hostinfo t ( Temperature information )
ssp% hostinfo S ( CPU signature )
ssp% hostinfo p ( Power status )
31
12.5.2. power
power
ssp% power
E10K power
ssp% power on all
ssp% power off all
power-off Circuit Breaker off

ssp% power off -B
System Board Power command

ssp% power off -sb 12
ssp% power on sb 0
Control Board Power command

ssp% power off -cb 0
(0 rear, 1 front Control Board Control Board power
on . Control Board Centerplane power on
.)
Centerplane Support Board Power command

ssp% power on -csb 1
ssp% power off csb 0
Power Supply Power command

ssp% power -off ps 0 1
12.5.3. fan
10 fan tray power-off ( -t fan tray
.)
ssp% fan t 10 p off
fan tray power-on

ssp% fan p on
fan speed fast (fan speed SSP

nominal, fast 2 speed .)
ssp% fan s fast
32
12.5.4. hostview
E10K GUI Tool
ssp% setenv DISPLAY workstation_name:0.0
ssp% hostview &
12.6. Upgrading to SSP 3.5

12.6.1. SSP 3.4 to SSP 3.5
if there is a spare SSP, perform data synchronization , backup, and disable
failover
ssp-main% showdatasync
ssp-main% setdatasync backup
ssp-main% setfailover off
ssp-spare% /etc/init.d/ssp stop
ssp-main% cd /cdrom/cdrom0/System_Service_Processor_3.5/Tools
ssp-main% ./ssp_backup /backup_dir
ssp-main% ./ssp_upgrade ../Products
ssp-main% su -
ssp-main% reboot
ssp-spare% cd /cdrom/cdrom0/ System_Service_Processor_3.5/Tools

ssp-spare% ./ssp_upgrade ../Products
ssp-spare% su -
ssp-spare% reboot
12.6.2. SSP 3.2,3.3 to SSP 3.5

ssp-main% cd /cdrom/cdrom0/System_Service_Provider_3.5/Tools
ssp-main% ./ssp_backup /backup_dir
ssp-main% ./ssp_upgrade ../Products
ssp-main% /opt/SUNWssp/bin/ssp_restore /backup_dir/ssp_backup.cpio
ssp-main% su -
ssp-main# reboot
On each domain, Verify /etc/ssphostname and /etc/hosts file that the floating
host name is specified
Redirect console communication to the new floating host name
33
each-domain# ps ef | grep cvcd

each-domain# kill 9 cvcd_pid
each-domain# /platform/SUNW,Ultra-Enterprise-10000/lib/cvcd
on each domain, update the /etc/syslog.conf file to replace the host name of
the former main SSP with the floating host name
ssp-spare% cd /cdrom/cdrom0/System_Service_Provider_3.5/Tools
ssp-spare% ./ssp_upgrade ../Products
ssp-main% su -
ssp-main# reboot
12.7. Disable DMP with AP

DMP is incompatible with AP. DMP must be disabled to allow AP to function
correctly
How to disabled DMP, if VxVM (earlier than VxVM 3.1) is already installed
# rm r /dev/vx/dmp /dev/vx/rdmp
# ln -s /dev/vx/dsk /dev/vx/dmp
# ln -s /dev/vx/rdsk /dev/vx/rdmp
# rm /kernel/drv/vxdmp
/etc/system forceload drv/vxdmp
13. E10K vs E15K/12K

13.1. Daemon
E10K 15K/12K Function
CBE & CBS hwad Hardware Access daemon
snmpd pcd Platform configuration daemon
obp_helper osd OpenBoot PROM server daemon
machine_server mld Machine logger daemon
cbs tmd Task management daemon
netcon_server dxs Domain X server daemon
34
- dca Domain configuration agent daemon
ssp_startup ssd SMS startup daemon
edd dsmd, esmd Domain status monitoring daemon and its

client, environmental status monitoring
daemon
fod fomd Failover management daemon
codd codd COD daemon
13.2. Commands
/etc/init.d/ssp /etc/init.d/sms Start and stop SMS daemon
addboard, addboard, Add, remove, or reconfigure a system

deleteboard, deleteboard, board
moveboard moveboard
bringup setkeyswitch Power on and power off a domain, load

OpenBoot PROM
bringup -st setdate Set a different data for a domain
cb_prom flashupdate Display and update board flash PROM

code
codlit addcodlicense Add a COD license
domain_create addtag Associate a host name with a domain
domain_link, linkdomain, Link and unlink IDN

domain_unlink unlinkdomain
domain_remove deletetag Remove a host name
domain_status showplatform Display board and domain status
hostinfo showenvironmsent Display environment data (temperature,

voltage fans)
35
voltage, fans)
hostint reset Interrupt the CPU
hostview Sun Management Display and manage the SunFire 15K/12K

Center server using GUI
hpost hpost POST utility
initcmdsync, initcmdsync, Failover cndsync synchronization

savecmdsync, savecmdsync, command
cancelcmdsync, cancelcmdsync,
runcmdsync, runcmdsync,
showcmdsync showcmdsync
netcon console Domain console window
power poweron, Turn on and turn off supplies

poweroff
redx redx Low-level access I2 C, JTAG, Console bus
showfailover, showfailover, Display and set the failover status

setfailover setfailover
sigbcmd reset -X Send and XIR to all CPU ports of a domain
thermcal_config thermcal Read and write thermistor calibration data
13.3. Configuration Files

/opt/SUNWssp/bin /opt/SUNWSMS/bin/ Configure the system controller as main

/ssp_config sms_config or spare
/export/home/ssp/ /export/home/sms- POST configuration file

.postrc svc/.postrc
/var/opt/SUNWssp /var/opt/SUNWSMS Domain configuration file

/.ssp_private/doma /.pcd/domain_info
in config
36
in_config
/var/opt/SUNWssp /var/opt/SUNWSMS System board configuration file

/.ssp_private/doma /.pcd/sysboard_info
in_config
/var/opt/SUNWssp /var/opt/SUNWSMS NVRAM/IDPROM information

/etc/platform/dom /data/domain/idpro
ain/eeprom.image m.image
/var/opt/SUNWssp /var/opt/SUNWSMS Logging level control

/adm/.logger /adm/.logger
13.4. Miscellaneous
User ssp(id=12) User sms-svc(id=6) administrator user ID
Control Board(CB) and System Controller Monitors and controls

System Service system boards
Provider(SSP)
Centerplane Support Centerplane Support Support for the centerplane

Board(CSB) Board(CSB) board
System boards(SB) Domain Configuration System board

Unit(DCU)
$SSPETC $SMSETC /etc/opt/SUNWSMS
$SSPVAR $SMSVAR /var/opt/SUNWSMS
$SSPLOGGER $SMSLOGGER /var/opt/SUNWSMS/adm
BBSRAM IOSRAM I/O SRAM used by the PC

ASIC accessed by JTAG or
the processor
37

E10K Admin

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

E10K Admin

Uploaded by

Copyright:

Available Formats

`

1. Installing SSP 3.5.......................................................................................................... 7

6.2.1. Main SSP ............................................................................................... 18

9.2.1. Checking DR model............................................................................... 25

12.6. Upgrading to SSP 3.5 ................................................................................... 33

SSP 3.5, SSP Solaris 8, Domain Solaris 8

1. Installing SSP 3.5

2.5.1, 2.6 2.0

2.6 2.5.1, 2.6

2.6 2.5.1, 2.6

2.6 2.5.1, 2.6

2.5.1, 2.6 2.5.1, 2.6 Not

2.5.1, 2.6 Not

1.2. Configuring SSP Network files

2 1 SSP, 1 Hub, 1 CB SSP, CB failover disabled

3 1 SSP, 2 Hub, 2 CB only CB failover enabled

3 (spare SSP) 2 SSP, 2 Hun, 2 CB SSP, CB failover enabled

1.2.1. 2 Sub net Configuration

1.2.2. 3 subnet Configuration

1.2.3. 3 sbunet (spare SSP) Configuration

1.2.4. Configuring Network Files

Spare SSP loghost

1.2.5. Reboot SSP

1.3. Install SSP software packages

1.3.1. Web Start

1.4. Configuring SSP

Enterprise-10000 platform. If you have a mixture of processors, select the

option corresponding to the lowest processor speed in your configuration.

select one of the options below.

1) 250 MHz processors

2) 336 MHz processors

3) 400 MHz processors

4) 466 MHz processors

5) Unlisted (manually enter clock values)

What speed of processors do you have installed? 3

Is this correct? (y/n) y

main SSP address = 192.168.95.109

Is this correct? (y/n) y

Do you have a control board 0? (y/n)y

Do you have a control board 1? (y/n)y

Is Control Board 0 [ses-cb0] the primary? (y/n)y

Control board 0 = ses-cb0 => 192.168.151.123

Control board 1 = ses-cb1 => 192.168.152.123

Primary Control Board = 0

Is this correct? (y/n)y

1.5. Reboot SSP

2. Preparing the domain

2.2. Creating the domain resource

2.3. Bringing up the domain

3. Installing the OS on the domain

3.1.2. Configuring Sysidcfg file

3.1.3. Configuring Rules and Profile

3.1.4. Editing /etc/vfstab

3.1.5. Adding Client

3.1.6. E10K ssphostname Configuration

3.1.7. Booting Client using Network

ssp% domain_switch domain1

5. Booting the domain for the first time

5.1.2. Network Alias

6. Finishing the installation

6.1. SSP hostname

6.2.2. Spare SSP

6.2.4. Starting NTP daemon

7.2.2. Creating AP database