Professional Documents
Culture Documents
SM-340
Copyright 2004 Sun Microsystems, Inc. 4150 Network Circle, Santa Clara, California 95054, U.S.A. All rights reserved.
This product or document is protected by copyright and distributed under licenses restricting its use, copying, distribution, and decompilation. No part of this product or document may
be reproduced in any form by any means without prior written authorization of Sun and its licensors, if any.
Third-party software, including font technology, is copyrighted and licensed from Sun suppliers.
Sun, Sun Microsystems, the Sun logo, Java, Netra, OpenBoot, Solaris, Sun Enterprise, Sun Fire, Sun HPC Cluster Tools, Sun Java, and Sun StorEdge are trademarks or registered
trademarks of Sun Microsystems, Inc. in the U.S. and other countries.
All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International, Inc. in the U.S. and other countries. Products bearing SPARC
trademarks are based upon an architecture developed by Sun Microsystems, Inc.
UNIX is a registered trademark in the U.S. and other countries, exclusively licensed through X/Open Company, Ltd.
RESTRICTED RIGHTS: Use, duplication, or disclosure by the U.S. Government is subject to restrictions of FAR 52.227-14(g)(2)(6/87) and FAR 52.227-19(6/87), or DFAR 252.227-7015
(b)(6/95) and DFAR 227.7202-3(a).
DOCUMENTATION IS PROVIDED “AS IS” AND ALL EXPRESS OR IMPLIED CONDITIONS, REPRESENTATIONS, AND WARRANTIES, INCLUDING ANY IMPLIED WARRANTY
OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE OR NON-INFRINGEMENT, ARE DISCLAIMED, EXCEPT TO THE EXTENT THAT SUCH DISCLAIMERS ARE
HELD TO BE LEGALLY INVALID.
THIS MANUAL IS DESIGNED TO SUPPORT AN INSTRUCTOR-LED TRAINING (ILT) COURSE AND IS INTENDED TO BE USED FOR REFERENCE PURPOSES IN CONJUNCTION
WITH THE ILT COURSE. THE MANUAL IS NOT A STANDALONE TRAINING TOOL. USE OF THE MANUAL FOR SELF-STUDY WITHOUT CLASS ATTENDANCE IS NOT
RECOMMENDED.
Copyright 2004 Sun Microsystems Inc. 4150 Network Circle, Santa Clara, California 95054, Etats-Unis. Tous droits réservés.
Ce produit ou document est protégé par un copyright et distribué avec des licences qui en restreignent l’utilisation, la copie, la distribution, et la décompilation. Aucune partie de ce
produit ou document ne peut être reproduite sous aucune forme, par quelque moyen que ce soit, sans l’autorisation préalable et écrite de Sun et de ses bailleurs de licence, s’il y en a.
Le logiciel détenu par des tiers, et qui comprend la technologie relative aux polices de caractères, est protégé par un copyright et licencié par des fournisseurs de Sun.
Sun, Sun Microsystems, le logo Sun, Java, Netra, OpenBoot, Solaris, Sun Enterprise, Sun Fire, Sun HPC Cluster Tools, Sun Java, et Sun StorEdge sont des marques de fabrique ou des
marques déposées de Sun Microsystems, Inc. aux Etats-Unis et dans d’autres pays.
Toutes les marques SPARC sont utilisées sous licence sont des marques de fabrique ou des marques déposées de SPARC International, Inc. aux Etats-Unis et dans d’autres pays. Les
produits portant les marques SPARC sont basés sur une architecture développée par Sun Microsystems, Inc.
UNIX est une marques déposée aux Etats-Unis et dans d’autres pays et licenciée exclusivement par X/Open Company, Ltd.
LA DOCUMENTATION EST FOURNIE “EN L’ETAT” ET TOUTES AUTRES CONDITIONS, DECLARATIONS ET GARANTIES EXPRESSES OU TACITES SONT FORMELLEMENT
EXCLUES, DANS LA MESURE AUTORISEE PAR LA LOI APPLICABLE, Y COMPRIS NOTAMMENT TOUTE GARANTIE IMPLICITE RELATIVE A LA QUALITE MARCHANDE, A
L’APTITUDE A UNE UTILISATION PARTICULIERE OU A L’ABSENCE DE CONTREFAÇON
Ce manuel de référence doit être utilisé dans le cadre d’un cours de formation dirigé par un instructeur (ILT). Il ne s’agit pas d’un outil de formation indépendant. Nous vous
déconseillons de l’utiliser dans le cadre d’une auto-formation.
Course Contents
Sun Fire 4800/E4900, 4810, and 6800/E6900 Server Eight-Slot PCI I/O Assembly ................... 2-40
Sun Fire V1280/E2900 Server I/O Assembly Location ................................................................... 2-41
Sun Fire 3800 Server I/O Assembly Locations ................................................................................. 2-42
Sun Fire 4800/E4900 Server I/O Assembly Locations .................................................................... 2-43
Sun Fire 4810 Server I/O Assembly Locations ................................................................................ 2-44
Sun Fire 6800/E6900 Server I/O Assembly Locations ................................................................... 2-45
Eight-Slot PCI I/O Assembly Slot Locations and LEDs .................................................................. 2-46
Eight-Slot Assembly Electrical Characteristics ................................................................................ 2-47
Six-Slot cPCI I/O Assembly Slot Locations and LEDs .................................................................... 2-48
Six-Slot cPCI I/O Slot Electrical Characteristics ............................................................................... 2-49
Four-Slot cPCI I/O Assembly Slot Locations and LEDs ................................................................. 2-50
Four-Slot cPCI I/O Slot Electrical Characteristics ............................................................................ 2-51
PCI and cPCI I/O Adapters ................................................................................................................ 2-52
Sun Fire V1280/E2900 Server Sun Fireplane Switchboard ............................................................. 2-54
Sun Fire 4800/E4900, 4810, and 6800/E6900 Server Sun Fireplane Switchboard ....................... 2-55
Sun Fireplane Switchboard Physical Locations ................................................................................ 2-56
Sun Fireplane Switchboard LEDs ....................................................................................................... 2-58
Sun Fire V1280/E2900 Server Baseplane .......................................................................................... 2-59
System Configuration Card Reader (SCCR) .................................................................................... 2-60
Sun Fire 6800/E6900 Server Centerplane and ID Board (Front View) ......................................... 2-61
Sun Fire 6800/E6900 Server Centerplane and ID Board (Rear View) ........................................... 2-62
ID Board ................................................................................................................................................. 2-63
Replacing a Centerplane or ID Board ................................................................................................ 2-65
ID Board MAC Addresses ................................................................................................................... 2-66
AC Power Distribution ......................................................................................................................... 2-67
Sun Fire 4810 and 4800/E4900 Server AC Component Locations ................................................. 2-69
Sun Fire 6800/E6900 Server AC Component Locations .................................................................. 2-70
RTU and RTS ......................................................................................................................................... 2-71
Redundant Transfer Unit Panel .......................................................................................................... 2-72
Redundant Transfer Unit LED Functions ......................................................................................... 2-73
AC Input Box ......................................................................................................................................... 2-74
Sun Services
Preface
Course Goals
Upon completion of this course, you should be able to:
• Locate online resources for the Sun Fire™ midrange
server product line, which includes the following
servers:
• Sun Fire V1280 server and Sun Fire E2900 server
• Sun Fire 3800 server
• Sun Fire 4800 server and Sun Fire E4900 server
• Sun Fire 4810 server
• Sun Fire 6800 server and Sun Fire E6900 server
• Describe the server configuration and key features of
each model in the Sun Fire midrange server line
Course Goals
• Configure the Sun Fire midrange server platforms
• Perform system maintenance-related activities on the
Sun Fire midrange servers
Course Map
Sun Fire™ Midrange Introduction
Introducing the
Sun Fire™ Midrange Field-Replaceable Units
Servers
Platform Assessment
and Management
Troubleshooting the
Sun Fire Midrange
Servers
Introductions
• Name
• Company affiliation
• Title, function, and job responsibility
• Experience related to topics presented in this course
• Reasons for enrolling in this course
• Expectations for this course
Icons
Additional resources
! Discussion
?
Note
Caution
Typographical Conventions
• Courier is used for the names of commands, files,
directories, programming code, programming
constructs, and on-screen computer output.
• Courier bold is used for characters and numbers
that you type, and for each line of programming code
that is referenced in a textual description.
• Courier italic is used for variables and
command-line placeholders that are replaced with a
real name or value.
Typographical Conventions
• Courier italic bold is used to represent variables
whose values are to be entered by the student as part of
an activity.
• Palatino italic is used for book titles, new words or
terms, or words that are emphasized.
Additional Conventions
Java™ programming language examples use the following
additional conventions:
• Courier is used for the class names, methods, and
keywords.
• Methods are not followed by parentheses unless a
formal or actual parameter list is shown.
• Line breaks occur where there are separations,
conjunctions, or white space in the code.
• If a command on the Solaris OS is different from the
Microsoft Windows platform, both commands are
shown.
Sun Services
Module 1
Objectives
• List the functional goals of the Sun Fire™ midrange
server product line
• Locate Sun Microsystems web sites containing
important Sun Fire midrange server information
• List the server models that comprise the Sun Fire
midrange server product line
• Identify the input and output (I/O) components that
the Sun Fire midrange servers support
• Describe the key features of each Sun Fire midrange
server model
• Power on and off each Sun Fire midrange server
Relevance
• Which Sun Fire midrange server models are available?
• How is each Sun Fire midrange server model used?
• What are the key features of each Sun Fire midrange
server model?
System Controllers
• Sets up the system and coordinates the boot process
• Generates system clocks
• Monitors the environmental sensors
• Analyzes errors and takes corrective action
• Sets up the system partitions and domains
• Provides the system console capabilities
L2 Switches L2 Switches
Segment 0 Segment 1
Multipathed I/O
• Sun StorEdge™ Traffic Manager software – Provides a
high level of disk availability and performance using
multipath access to I/O devices. This was formerly
known as Multiplexed IO or MPxIO.
• Internet Protocol Multipathing – Provides a high level
of network availability and performance using
automatic failover and load balancing on existing
Internet Protocol-based networking products.
Sun Services
Module 2
Field-Replaceable Units
Objectives
• Describe the various administrative and service layers
on the Sun Fire midrange server products
• Describe the Sun Fire midrange server FRU strategy
• Locate and describe the function of the Sun Fire
midrange server system controller boards
• Locate and describe the function of the Sun Fire
midrange server system boards
• Locate and describe the function of the Sun Fire
midrange server I/O boards
• Locate and describe the function of the Sun Fire
midrange server Sun Fireplane switchboards
Objectives
• Locate and describe the function of the Sun Fire
midrange server baseplane and centerplanes
• Locate and describe the function of the Sun Fire
midrange server AC and DC power distribution FRUs
• Locate and describe the function of the Sun Fire
midrange server fan tray assemblies
• Locate and describe the function of the Sun Fire
midrange server FrameManager
• Locate and describe the function of the Sun StorEdge™
D240 media tray
• Install and administer the console
• Install the server in a rack configuration
Sun Fire™ Midrange Server Maintenance Module 2, slide 3 of 112
Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision C
Sun Services
Relevance
• What URLs are needed to install and configure the Sun
Fire midrange server models?
• What is the difference between hot-plug and hot-swap?
• Which components make up the Sun Fire midrange
server models?
• Where are the components located?
• Which status indicators are associated with each
component?
Applications
Solaris
Operating
System
OpenBoot PROM 5
LOM Shell
Platform
Hardware
Open Boot PROM Open Boot PROM Open Boot PROM Open Boot PROM
Platform Hardware
FRU Administration
Non-hot-pluggable FRUs:
• PCI cards
• Centerplane
• Sun Fireplane switchboard
FRU Administration
Hot-pluggable FRUs:
• System boards
• I/O boards
• System controller (only with failover enabled)
FRU Administration
Hot-swappable FRUs:
• DC power supplies
• Fan trays
• cPCI cards
IB_SSC FRU
SC1
SC0
SC1
SC0
SC0
SC1
SC0
SSC1 B
TTYB
(reserved)
SSC1 A
LOM serial
port
J 14600
J 14601
J14500 CPU 1
J 14501
J 14400 (P1)
J 14401
J 14300
J 14301
J 13600
J 13601
J 13500 CPU 0
J 13501 (P0)
J13400
J 13401
J13300
J 13301
J16600
J 16601 CPU 3
J16500 (P3)
J 16501
J 16400
J 16401
J 16300
J 16301
J 15600 CPU 2
J 15601 (P2)
J 15500
J 15501
J 15400
J 15401
J 15300
J 15301
DIMM 3, bank 0
DIMM 3, bank 1
DIMM 2, bank 0
DIMM 2, bank 1
DIMM 1, bank 0
DIMM 1, bank 1
DIMM 0, bank 0
DIMM 0, bank 1
SB0
SB2
SB4
SB2
SB0
SB0
SB2
SB4
SB0
SB2
SB4
SB4 SB1
SB2 SB3
SB0 SB5
IB6
IB8
IB8
IB6
IB8
IB9 IB8
IB7 IB6
Removal OK LED
(Amber)
Activated LED
(Green)
cPCI
5 Slot Fault LED
4 LEDs
3 (Amber)
2
1 Removal OK LED
cPCI (Amber)
0 Board
Slots
LEDs
Activated LED
(Green)
cPCI
3 Slot Fault LED
LEDs (Amber)
2
1
0 Removal OK LED
cPCI (Amber)
Slots Board
LEDs
Ejector handle
AR
D110 DC-DC I2C Aux
SEPROM
SDC
DX0 LEDs
On
Fault
Ok-to-Remove
DX1
Echip
RP0
RP2
RP0
RP2
RP0
RP2
System Board 0
System Board 2
System Board 4
530-3073
1B6_SSC1
J9001
System Board 0
System Board 2
System Board 4
Sun Fireplane Switchboard 2
Sun Fireplane Switchboard 0
1B6_SSC1
J9301
to DC Distrubution Board
System configuration
card reader (SCCR)
System Board 2
System Board 3
System Board 0
System Board 4
System Board 1
System Board 5
System Board 2
System Board 3
System Controller 0
System Board 0
System Board 4
System Board 1
System Board 5
Sun Fire™ Midrange Server Maintenance Module 2, slide 61 of 112
Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision C
Sun Services
I/O Board 8
Sun Fireplane Switchboard 2
I/O Board 8
I/O Board 9
ID Board
ID Board
The ID board contains a serial electrically erasable
programmable read-only memory (SEEPROM)
application-specific integrated circuit (ASIC) with the
following information:
• It has the server chassis ID.
• It has the server serial number/host ID.
• It incorporates six media access control (MAC)
addresses for the Sun Fire 6800/E6900 server and four
MAC addresses for the Sun Fire 3800, 4800/E4900, and
4810 servers. This includes one per possible domain
and one each for the system controllers.
• It has the server and component power-on hours.
AC Power Distribution
All Sun Fire midrange servers installed in a data center
cabinet are equipped with the following AC components:
• The redundant transfer unit (RTU)
• The redundant transfer switch (RTS)
• An AC input box
AC Power Distribution
Rack
Fan
Trays
(2)
220VAC
Primary
200–240VAC To AC input box for all servers
RTS
Secondary 220VAC except the Sun Fire 3800 servers.
RTU
200–240VAC To power supplies for the
RTS Sun Fire 3800 servers only.
AC
input
box
AC
input
boxes
RTU
RTS
Front Rear
J3 J5 J7 J9 J4 J6 J8 J10
On-Off breakers On-Off breakers
J13 J15
J14 J16
To Frame
Not Used
Manager
RTUR RTS1 J3 J7 J8 J4
RTUF RTS3 J5 J9 J10 J6
RTUR RTS0
RTUF RTS2 J13 J14
J15 J16
J124A J224A
AC Input Box
• For the Sun Fire 6800/E6900 systems, the AC input box
receives power from the RTU through two power
cables, each with a corresponding power switch.
• For the Sun Fire 4810 and 4800/E4900 systems, the AC
input box receives power from the RTU through three
power cables, each with a corresponding power switch.
• Sun Fire 3800 systems do not use AC input boxes.
DC Power Distribution
Sun Fire midrange DC power distribution systems include
comprised of the following major components:
• System centerplane
• Power centerplane
• Fan centerplane
• DC power supplies
Auxiliary
3V3
Primary Main Standby
200–240VAC Feed A (2) 48VDC 12VDC
RTS AC
Secondary RTU Input 220VAC AC-DC Converters (PS0-PS3)
200–240VAC Box
RTS
Feed B (2)
Rack
Fan
Trays DC-DC Board Converters DC-DC Board Converters
(2) (system, I/O, and Sun (system controller and
Fireplane switch boards) ID boards)
220VAC
Primary Main Auxiliary
200–240VAC 56VDC 56VDC
RTS AC 3
Secondary 220VAC To fans (3)
RTU Input System Centerplane
200–240VAC Main 56VDC
RTS 220VAC Box
Main Auxiliary
220VAC
56VDC 56VDC
AC-DC Converters (PS0-PS2)
Primary
RTS Main
200-240VAC 56V F
1 A
RTUF AC0 220VAC AC-DC
CONV N
Secondary Power Auxiliary
RTS Centerplane PS0-2 56V C
200-240VAC
E
AC SC0 N
Input SC1 T
Box Auxiliary DC-DC E
56V R
Primary Power P
RTS Centerplane 220VAC AC-DC
200-240VAC Main L
1 CONV A
RTUR AC1 PS3-5 56V N
Secondary E
RTS
200-240VAC
DC-DC
SB1, 3, 5
ib7, 9
RP2, 3
DC Power Supplies
Power Supply Specifications
PS3
PS2
PS1
PS0
PS2
PS1
PS0
PS1
PS0
PS2
PS3
PS4
PS5
} Grid 1
Grid 0
{ PS1
PS0
PS2
Fan tray
FT0 FT1
FT2 FT3
FT1
FT0 FT2
FT0
FT2
FT1
FT1 FT0
FT3 FT2
Front Rear
Active LED
(green)
Fault LED
(amber)
Removal OK LED
(amber)
FrameManager
Front RTU
SC1
Sun Fire
Serial
Midrange
Port B
Server
SC0
Rear RTU
RTS RTS
Full bus
Configuration switch
DVD ROM (ID6) Disk (ID1) Disk (ID0) DAT tape (ID4)
Split bus
Configuration switch
To domain To domain
DVD ROM (ID6) Disk (ID0) Disk (ID0) DAT tape (ID6)
Power LED
(green)
DC output
Fault
Remove OK
AC input
Sun Services
Module 3
Objectives
• Describe an overview of the Sun Fire V1280/E2900
server platform administrative functions
• Perform user maintenance and administrative
functions using the LOM shell
• Display and change system controller parameters by
using LOM shell commands
• Monitor the Sun Fire V1280/E2900 server platform by
using LOM shell commands
• Power cycle the Sun Fire V1280/E2900 server with
LOM shell commands and with the front panel power
rocker switch
Objectives
• Identify the administrative and service tasks that you
can perform with the platform shell
• Identify the platform shell commands you can use to
display system information, set up system parameters,
and test system hardware
• Describe three methods you can use to connect to the
system controller shells
• Describe how to navigate between shells on the Sun
Fire 3800, 4800/E4900, 4810, and 6900/E6900 servers
• Configure the platform and system controller by using
the platform shell
Objectives
• Display and change system controller parameters by
using the platform shell
• Describe how to manage the system controller for
reboot and failover operations
• Describe how to use system controller commands to
monitor platforms and domains
• Describe how to power on and off the system
components
• Describe how to update the platform firmware
• Describe the capability and effects of splitting the Sun
Fire 3800, 4800/E4900, 4810, and 6800/E6900 servers
into segments and domains
Objectives
• Describe how to start, stop, and power cycle domains
• Describe how to perform device configuration with
OpenBoot™ programmable read-only memory
(PROM) to add or remove devices from a Sun Fire
midrange server
• Describe OpenBoot PROM capabilities
• Describe how to use OpenBoot PROM to represent
interconnected busses and their devices in a device tree
• Describe how to map Sun Fire midrange server
physical devices
Relevance
• Which functions does the system controller perform?
• How do the system controller maintenance buses
communicate with the platform?
• Which role does the platform shell play in configuring
the Sun Fire midrange servers?
• Which commands are available in the platform shell?
• How is each platform shell command used to configure
the Sun Fire midrange server platform?
Applications
Solaris
Operating
System
OpenBoot PROM 5
LOM Shell
Platform
Hardware
LOM Shell
The LOM shell that runs on the system controller has been
significantly modified from versions on earlier Netra server
platforms. Commands that were originally developed for the
Sun Fire 3800, 4800, 4810, and 6800 server domain shell have
been adopted.
Default #.
Escape
Sequence
lom> showdate
Mon Feb 06 12:31:40 EDT 2004
CPUs
----
/partition0/domain0/SB0/bbcGroup0/cpuAB/cpusafariagent0:
part_no=0x919c version=2 manufacturer=0x07d
devid=0x102234058a9 (lot=0x022340 bin=1 wafer=22 col=5
row=9)
speed=750
e$ (8MB):
J4400: 4MB, part# 0, serial# 090AXB
J4300: 4MB, part# 0, serial# 090AVU
DRAM DIMM Group 0 (1024MB):
J13300: 256MB, part# 0, serial# 005898
J13400: 256MB, part# 0, serial# 005870
J13500: 256MB, part# 0, serial# 005878
J13600: 256MB, part# 0, serial# 005881
DRAM DIMM Group 1 (Empty):
Component Description
--------- -----------
/N0/SB0/P2 UltraSPARC-III, 0MHz, 8M ECache
/N0/SB0/P3 UltraSPARC-III, 0MHz, 8M ECache
/N0/SB2/P2 UltraSPARC-III, 0MHz, 8M ECache
/N0/SB2/P3 UltraSPARC-III, 0MHz, 8M ECache
Power-Cycle Operations
The following power-cycle operations are available when you
enable the system indicator board rocker switch.
• If the system is in standby mode, pressing the switch
powers on the system. This action is equivalent to
executing the LOM poweron command.
• If the system running the Solaris OS, pressing the
switch for four seconds executes an orderly shutdown.
This action is the equivalent of executing the LOM
shutdown command.
• If the system is powered on, pressing this switch for
more than four seconds executes a system power down
to standby mode. This action is equivalent to the LOM
poweroff command.
Sun Fire™ Midrange Server Maintenance Module 3, slide 45 of 132
Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision C
Sun Services
Telnet Connection
Press: CTRL ]
at the telnet> prompt
type: send break
SSH Connection
Solaris #.
Operating
System Tip Connection
~.
Telnet Connection
Press: CTRL ]
at the telnet> prompt
type: send break
SSH Connection
#.
OpenBoot PROM
Tip Connection
~.
Domain Shell
Type:
disconnect
Type: console
Platform Shell -d domain ID
SC: SSC0
Main System Controller
SC Failover: disabled
Clock failover enabled.
Software Reset...
.
.
.
Failover
The system controller failover event is logged in the platform
message log file, which is viewed on the console of the new
main system controller or through the showlogs command
on the system controller.
Platform Shell - Spare System Controller
sp4-sc0:sc> Nov 12 01:15:42 sp4-sc0 Platform.SC: SC Failover: enabled and
active.
Nov 12 01:16:42 sp4-sc0 Platform.SC: SC Failover: no heartbeat detected
from the Main SC
Nov 12 01:16:42 sp4-sc0 Platform.SC: SC Failover: becoming main SC
Nov 12 01:16:49 sp4-sc0 Platform.SC: Chassis is in single partition mode.
Nov 12 01:17:04 sp4-sc0 Platform.SC: Main System Controller
Nov 12 01:17:04 sp4-sc0 Platform.SC: SC Failover: disabled
sp4-sc1:SC>
SC: SSC0
Main System Controller
SC Failover: disabled
Clock failover enabled
Platform Assessment
You can use system controller commands to monitor the
platform and domains. These commands include:
• showplatform
• showboards
• showcomponent
• showsc
• showenvironment
• history
• connections
• showlogs
• showfru
Sun Fire™ Midrange Server Maintenance Module 3, slide 77 of 132
Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision C
Sun Services
Network
-------
The system controller is configured to be on a network.
Network settings: static
Hostname: marlins0
IP Address: 10.6.11.10
Netmask: 255.0.0.0
Gateway:
DNS Domain:
Primary DNS Server:
Secondary DNS Server:
Loghosts
--------
Loghost for Platform: 10.6.5.120
Log Facility for Platform: local0
SNMP
----
SNMP Agent: enabled
Chassis Description: Sun Fire 3800
Chassis Contact: Bob,Jackson,
Dave , John , Sam ,
Matt , Ted
Chassis Location: back_of_room
Trap Hosts: 10.6.5.120
Public Community String:
Private Community String:
SNMP packets received: 0
SNMP packets sent: 263
SNMP traps sent: 263
SC
--
SC POST diag Level: min
SC Failover: disabled
Logical Hostname:
Security Options
----------------
Telnet servers: Enabled
Idle connection timeout : No timeout
CPUs
----
/partition0/domain0/SB0/bbcGroup0/cpuAB/cpusafariagent0:
part_no=0x919c version=2 manufacturer=0x07d
devid=0x102234058a9 (lot=0x022340 bin=1 wafer=22 col=5
row=9)
speed=750
e$ (8MB):
J4400: 4MB, part# 0, serial# 090AXB
J4300: 4MB, part# 0, serial# 090AVU
DRAM DIMM Group 0 (1024MB):
J13300: 256MB, part# 0, serial# 005898
J13400: 256MB, part# 0, serial# 005870
J13500: 256MB, part# 0, serial# 005878
J13600: 256MB, part# 0, serial# 005881
DRAM DIMM Group 1 (Empty):
Component Description
--------- -----------
/N0/SB0/P2 UltraSPARC-III, 0MHz, 8M ECache
/N0/SB0/P3 UltraSPARC-III, 0MHz, 8M ECache
/N0/SB2/P2 UltraSPARC-III, 0MHz, 8M ECache
/N0/SB2/P3 UltraSPARC-III, 0MHz, 8M ECache
Open Boot PROM Open Boot PROM Open Boot PROM Open Boot PROM
Platform Hardware
Segments
• A segment refers to all or part of the Sun Fireplane
interconnect.
• Dual-partition (segment) mode splits the Sun Fireplane
interconnect into two independent snoopy coherent
systems.
• The Sun Fireplane switch boards are divided
between the two segments.
• All connections between segments are disabled.
• To enable dual-segment mode, run:
schostname:SC> setupplatform -p partition
Configure chassis for single or dual-partition mode? [single]:
dual
Domains
• A domain is a logical division of a segment.
• Each domain has an independent instance of the
Solaris OS.
• Each segment can have a maximum of two domains.
• Domains are useful for testing new applications or
operating system updates.
• Temporary resources can be borrowed from existing
domains.
• Upon completion, resources can be returned.
• System reboot is not required.
RP0/RP1 RP2/RP3
SB0 SB2 IB6 SB4 IB8 SB1 SB3 IB7 SB5 IB9
Segment 0 Segment 1
Configuring ACLs
Configure ACLs by using the showplatform -p acls
command. Type:
schostname:SC> setupplatform -p acls
ACL for domain A [SB0 SB1 SB2 SB3 SB4 SB5 IB6 IB7 IB8 IB9]:-r
SB1 SB3 SB5 IB7 IB9
ACL for domain B [SB0 SB1 SB2 SB3 SB4 SB5 IB6 IB7 IB8 IB9]:-
ACL for domain C [SB0 SB1 SB2 SB3 SB4 SB5 IB6 IB7 IB8 IB9]:-r
SB0 SB2 SB4 SB5 IB6 IB8 IB9
ACL for domain D [SB0 SB1 SB2 SB3 SB4 SB5 IB6 IB7 IB8 IB9]:-r
SB0 SB1 SB2 SB4 IB6 IB7 IB8
Viewing ACLs
Display the current ACLs by using the
showplatform -p acls command. Type:
schostname:SC> showplatform -p acls
ACL for Domain A: SB0 SB2 IB4 IB6 IB8
ACL for Domain B:
ACL for Domain C: SB1 SB3 IB7
ACL for Domain D: SB3 SB5 IB9
Solaris Solaris
Operating Operating
System System
Sun Fire
Midrange Server
Device Tree
Each device node can have the following components:
• Properties – Data structures describing the node and its
associated device.
• Methods – The software procedures used to access the
device.
• Data – The initial values of the private data used by the
methods.
• Children – Other device nodes attached to a given node
and that lie directly below it in the device tree.
• Parent – The node that lies directly above a given node
in the device tree.
IOC 0 IOC 1
Pci@18 Pci@19
Hard Drive
Target 1 PCI Slot 1 PCI Slot 2
Tape Drive
Target 5 PCI Slot 3
Gigabit
PCI Slot 5 DVD Drive PCI Slot 4
Ethernet 1
SunFire3800,4800/E4900,4810,and6800/E6900
Server Device Tree Components
ssm@0,0
IOC
8 pci@ 18,600000 pci@ 18,700000
Device Device
/ssm@0,0
Node ID 0
/ssm@0,0/SUNW,UltraSPARC-III@b,0
CPU offset
CPU AID
/ssm@0,0/SUNW,memory-controller@b,400000
Memory offset
Memory AID
/ssm@0,0/pci@19,700000/pci@3/SUNW,isptwo@4/sd@5,0
Node ID IOC AID Bus offset Device # PCI controller Device instance
/ssm@0,0/pci@19,700000.....
Bus offset
Device Number
The PCI controller slots, located in the PCI (cPCI) chassis, are
referenced by the device number.
Device number:
/ssm@0,0/pci@19,700000/pci@3.......
Device #
PCI Slot 0
pci@1 PCI Slot 2
PCI Slot 5
network@1
pci@1 pci@1
PCI Slot 1
pci@2 PCI Slot 3
scsi@2
pci@2 network@2
Hard Drive
ide@3 Target 0 PCI Slot 4
disk@0,0 pci@3
Tape Drive
Target 5
st@5,0
pci@1 (Slot 0)
pci@2 (Slot 1)
pci@3 (Slot 2)
pci@1 (Slot 3)
pci@1 (Slot 4)
pci@2 (Slot 5)
pci@3 (Slot 6)
pci@1 (Slot 7)
pci@1 (Slot 0)
pci@1 (Slot 1)
pci@1 (Slot 2)
pci@2 (Slot 3)
pci@1 (Slot 4)
pci@2 (Slot 5)
IOC 0 IOC 1
pci@18,700000 A A pci@19,700000
B pci@18,600000 pci@19,600000 B
pci@1 (Slot 0)
pci@1 (Slot 1)
pci@1 (Slot 2)
pci@1 (Slot 3)
Sun Services
Module 4
Objectives
• Describe the basic architecture of the Sun Fire server
system
• Describe the two levels of Sun Fireplane interconnect
switches
• Describe how the system boards provide CPU and
memory resources to the operating system in Sun Fire
midrange servers
• Describe how Sun Fire midrange servers use PCI and
cPCI I/O assemblies
• Describe how the Sun Fireplane interconnect plane is
the main system bus of the Sun Fire family of servers
Objectives
• Describe the different integrated service processors
supported by the Sun Fire midrange server architecture
• Describe the troubleshooting methodology for fault
analysis and diagnosis of failed components
• Describe the system tools available for gathering
background information on Sun Fire midrange server
problems
• Describe the testing tools available for isolating faults
in the Sun Fire midrange servers
• Describe how Sun Fire midrange servers use parity to
detect system interconnect errors
Objectives
• Describe how Sun Fire midrange server subsystems
can use error correcting code (ECC) to recover from
errors
• Describe how console port error messages are reported
to help isolate faulty components in the console bus
hub (CBH)
• Describe how Sun Fire midrange server environmental
faults are reported
• Describe the enhanced availability features
implemented in the new firmware update 5.15.3
• Describe how blacklisting is used to reconfigure Sun
Fire midrange server hardware to avoid parts with
errors
Objectives
• Describe how domain shell operating messages
provide an aid to troubleshooting a system error in Sun
Fire midrange servers.
• Recover from a hung domain
Relevance
• Which diagnostic tools are available to test the Sun Fire
servers?
• How do you free a hung domain?
• How do you create an action plan to replace failed
FRUs?
Additional Resources
• Sun Microsystems, Inc. Sun Fire™ 6800/4810/4800/3800
Systems Platform Administration Manual, part number
817-0999.
• Sun Microsystems, Inc. Sun Fire™ 6800/4810/4800/3800
System Controller Command Reference Manual, part
number 817-1000.
• Sun Microsystems, Inc. Sun Fire™ V1280/Netra™ 1280
System Administration Guide, 817-0509.
• Sun Microsystems, Inc. Sun Fire™ V1280/Netra™ 1280
Systems Service Manual, part number 817-0510.
• Sun Microsystems, Inc. Sun Fire™ V1280/Netra™ 1280
System Controller Command Reference Manual, part
number 817-0511.
Additional Resources
• Sun Microsystems, Inc. Sun Fire™ Midrange Systems
Hardware Reference Manual, part number 805-7363.
• Sun Microsystems, Inc. Sun Fire™ 6800/4800/4810/3800
Systems Site Planning Guide, part number 805-7365.
URL Resources – Public Web sites
• http://sunsolve.sun.com/handbook_pub/
• http://www.sun.com/software/solaris/
sunmanagementcenter/hwds/
• http://docs.sun.com
• http://www.sun.com/blueprints/0803/
817-3342.pdf
Operational Overview
Sun Fire midrange server architecture includes:
• New system bus architecture based on the Sun
Fireplane interconnect
• High-performance Sun Fireplane interconnect switch
technology
• An enhanced CPU architecture starting at speeds of
750 MHz and greater
• An industry standard I/O incorporating PCI and cPCI
technology
IOC IOC
Data Interconnect
Address
Functionality
• The Sun Fireplane interconnect provides a 288-bit data
path between the UltraSPARC III processors and the
PCI I/O bridge (IOC) with a high clock frequency of
150 MHz.
• The connection between the Sun Fireplane interconnect
devices (UltraSPARC III processors and PCI and
enhanced PCI [EPCI] bridges) and the data path uses
point-to-point connections.
• The UltraSPARC III processors are interfaced to the
data path using the dual CPU data switch (DCDS).
CPU
DX DX and
Memory
2
CPU
AR AR and
Memory
2
6
System boards
DX
DX
PCI IOC
AR AR
Data 2
Address
4 I/O boards
0
1
2
3
4
5
IB6
IB7
IB8
IB9
SB
SB
SB
SB
SB
SB
SB0
SB1
SB2
SB3
SB4
SB5
IB6
IB7
IB8
IB9
Crossbar configuration:
SB0, SB2, SB4, and IB6
SB1, SB3, SB5, and IB7
E-Cache E-Cache
Tag Tag
Core 0 Core 1
(US-III Cu) (US-III Cu) Memory
M Address (SDRAM)
C
U 15
75MHz 512+
Data 36ECC+
28MTag
Data
128+ DCDS
9ECC+
7MTag
UltraSPARC IV 256+
150MHz
18ECC+
Transaction Request Signals Data 14MTag
CPU 1
Data
DIMMs Address
SDC Data Route
SC0 SBBC0 Bootbus
SC1 SBBC1 Console Bus
8,9 0 Control Signals
DIMMs
6,7 1
5 2
4 3 CPU 2
4 2,3
DIMMs
Address
Gigabit Gigabit
Repeater
Connection to Fireplane
Ethernet Ethernet
Controller Controller
Data A
Controller IOC 1
B
A
Data
Switch IOC 0 B
Dual Channel
SCSI
PCI PCI Controller
Data PCI
IDE #0 #1 #5
Switch External
Controller Internal
SCSI SCSI
Internal Connection Connection
IDE
SBBC SRAM FPROM Connection PCI Riser
2 x 33MHz 1 x 66MHz
To 2 x SCSI disks
Slots Slot
To DVD and DDS-4 tape
0
8,9
0
8,9
0
8,9
Level 1:
Board Address Repeater Address Repeater
PCI PCI
Proc Proc Proc Proc
Controller Controller
Level 1:
Board Data Switch Data Switch
Console Arbitration
SC
PCI Service
CBH SBBC
proc
I/O
AR
PCI Fireplane Switch
IOC IOC SBBC SDC
SDC
AR
SRAM Misc.
Echip Registers Sensors
PCI (33 MHz)
16552
TOD NVRAM ScApp Serial
Clocks Controller
NVRAM FPROM FPROM
RIO
16552
Serial
Controller
DRAM
MicroSPARC IIep
boot
FPROM
Console Bus
CBH
PROMBus
SBBC
Echip SRAM Misc.
Registers
I 2C
MUXs
I 2 C Buses
2
Local I C
Ebus
RIO
vxworks
boot SC (MicroSPARC IIep) DRAM
FPROM
To PCI Controller
SRAM FPROM
PCI Bus
Console Bus PROM Bus
To Datapath 2
JTAG Boot Bus Controller I C Buses
Controller
Boot Bus
Sensors LEDs
To Processer To Processer
3 4
SDC
AR 2 Console Bus
Multiplexing Section
0 1
SBBC0 SBBC1
(SB only)
Fireplane
I/O Board Switch
Select 75 MHz Board
PLL
Local
150 MHz
Clock
Loads
lom> setupsc
System Controller Configuration
-------------------------------
SC POST diag Level [max]:
Host Watchdog [enabled]: enabled
Rocker Switch [disabled]: disabled
Secure Mode [off]: on
L2 ASIC
reporting errors
Centerplane Possible
pin damage
L1 ASIC L1 ASIC
reporting errors reporting errors
Centerplane Possible
pin damage
L1 ASIC L1 ASIC
reporting errors reporting errors
RP Sun Fire 48x0 and 3800 servers (Sun Fire 3800 0/1 SB0
repeaters are integrated as part of the
2/3 SB2
centerplane)
4/5 SB4
6/7 IB6
8/9 IB8
Parity
Data
Centerplane
CPU
(Parity Memory CPU
generated)
Parity
Centerplane
Data
L1AR L1AR
(Parity detected) (Parity detected)
CPU CPU
(Parity (Parity
generated) corrected)
L2DX
Centerplane
L1DX L1DX
ES ES
ED ED Data
CPU DCDS PCI IOC Parity
ED ES ECC checked by SC
ED ECC checked by
domain
Ecache DRAM
L2DX
Centerplane
L1DX L1DX
L1DX
(ECC detected) (ECC detected)
Ecache Ecache
Environmental Errors
Usually caused by faulty (or blocked) fan trays or power
supplies.
Sun-Fire-sc0:SC> showlogs
Dec 12 08:31:00 Sun-Fire-sc0 Chassis-Port.SC: Domain
A has a SYSTEM ERROR
Dec 12 08:31:07 Sun-Fire-sc0 Chassis-Port.SC: This
domain is still running because error pause is not enabled for
this domain
Dec 12 08:31:18 Sun-Fire-sc0 Chassis-Port.SC: Device
temperature problem: /N0/SB5 auto power off may occur due to
device: Cheetah 3 Temp. 0 Value: 127 Degrees C
Dec 12 08:31:19 Sun-Fire-sc0 Chassis-Port.SC: Device
temperature problem: Shutting down /N0/SB5 due to temperature of
device: Cheetah 3 Temp. 0 Value: 127 Degrees C
Dec 12 08:31:19 Sun-Fire-sc0 Chassis-Port.SC: /N0/SB5,
sensor status, over limit (7,1,0x201050603030000)
Dec 12 08:32:08 Sun-Fire-sc0 Chassis-Port.SC: ...board
successfully powered off.
Diagnosis Engines
The following automatic diagnosis engines (DEs) identify and
diagnose hardware errors that affect the availability of the
system and its domains:
• SMS DE
• Solaris OS DE
• POST DE
Domain is running.
Deconfiguration of appropriate
resources (next domain reboot)
Blacklisting Components
Component Component Name
System boards SB0, SB1, SB2, SB3, SB4, and SB5
Ports on the system board or I/O P0, P1, P2, and P3
assembly
Memory banks on system boards B0 and B1
I/O assemblies IB6, IB7, IB8, and IB9
Ports on the I/O assembly P0 (C0, C1, C2, and C3)
P1 (C4, C5, C6, and C7)
I/O cards in the I/O assemblies C0, C1, C2, C3, C4, C5, C6, and C7
Collecting Data
Apart from a Solaris OS core file, various other bits of
information might provide insight into the failure. Follow
these steps to collect this data:
1. Ensure you that have a record of what you did
during the previous verification step.
2. If you do not have a hardware error, assess current
conditions on the domain:
• Is there any output being printed on the domain
console?
• Does the domain console echo characters you type?
• Does the domain respond to the ping command?
• Does the domain respond to the rup command?
Sun Fire™ Midrange Server Maintenance Module 4, slide 107 of 114
Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision C
Sun Services
Collecting Data
3. Record the results of all the preceding tests.
4. Run the following commands from the system
controller domain shell, and collect the output in a
file:
• showlogs
• showenvironment
• showdomain
Collecting Data
5. Run the following commands from the system
controller platform shell, and collect the output in a
file:
• showsc
• showlogs
• showplatform
• history
Collecting Data
6. Run the Sun Explorer software utility to collect the
system configuration information.
If the domain is paused because of an error, nothing else can
be done after collecting all the previous information. Reboot
the domain with the following command in the domain shell:
setkey off ; setkey on
panic[cpu3]/thread=30002368580: zero
Obtaining Registers
If you cannot get a core file out of the Solaris OS, there should
still be CPU register information to collect.
The reset command causes all the CPUs in the target domain
to save their registers in a save area in CPU static random
access memory (SRAM).
showresetstate -v