SYSTEM BY MICHAEL M.
DUPAS AND THOMAS SIBLE
STRATEGIES
Making the Transition to
Parallel Sysplex at Boeing:
Part II — CMOS
Technology
This is the second of CMOS INSTALLATION
a three-part series of Packaging
articles on the transition Everything about the new CMOS
(Complimentary Metal-Oxide Semiconductor)
to Parallel Sysplex processors is different. To begin with, let’s
at Boeing Information focus on the terminology.
At today’s MIPS (22 this year), a CPC
& Support Services, (Central Processor Complex, formerly called
a CEC) is the equivalent of a mainframe.
Seattle, Wash. This A 9672 can contain multiple CPCs, so imme-
article discusses the new diately we are faced with the problem of
equating a machine/footprint (9672) with
CMOS technology used multiple mainframes. In addition, each 9672
comes with PS/2 PCs that run unique micro-
in a Parallel Sysplex. code and that must be treated as part of the
machine. Unfortunately, these packaging
differences caused some problems with our
current asset tracking software. Another
notable difference is that the CPCs are
controlled by one or more Hardware
Management Consoles (HMCs) running on
the PS/2 PCs. The communication is via
Token-Ring, so each CPC and HMC is cabled
to a Multi-Station Access Unit (MSAU) in the
processor. Each CPC and HMC is given an
SNA address. The major problem at our
installation was that the addresses assigned at
the factory were duplicates. So, when we
fired up the complex, the Token-Ring (and
we) became very confused.
The CMOS Configuration
Figure 1 examines our CMOS configuration.
There are three separate sysplexes. Each of
them has a partition on each of the two cou-
pling facilities. We’ve given these partitions
the name of the sysplex using them. All of the
Coupling Facility Sender (CFS) paths from
the MVS systems are shared (ESCON Multi-
Image Facility [EMIFd]).
On the 9674 side, the Coupling Facility
Receiver (CFR) paths cannot be EMIFd. We
have 9674-C01 CFs which have only two
CPs, so we share CPs in this configuration
between the three partitions. IBM does not
recommend doing this in a production data
sharing environment, but for the purposes of
testing and validating these test/development
systems, it’s performing acceptably so far.
The 9674-C01s have 512 MB of memory.
This memory can be designated central or
expanded, although it’s all the same. But, for
a coupling facility, you don’t want any
expanded memory, so define it all as central
memory. The 9674 is essentially the same
box as the 9672, which runs MVS, so this
capability to designate memory is from the
9672 design. The 9674 runs Coupling Facility
Control Code (CFCC), not an operating
system, but it does use 12 MB of Hardware
System Area (HSA). Next, we divided the
remaining 500 MB into three nearly equal
segments of approximately 166 MB. To date,
we have not used even close to half of the 166
MB on any of the partitions, but we have
plenty to learn yet about CF usage.
It should be noted that sufficient memory
must be free on each CF so that, in the event
of a failure, there is room for the structures
that were on the failing CF in addition to the
structures that normally reside on the remain-
ing CF. Also, for recovery, a production
configuration should have two links to each
CF from each partition to allow recovery to
the same CF in the event of a link failure.
CMOS Definition Using Hardware
Configuration Dialogue (HCD)
The first step in setting up an HCD CMOS
configuration is defining the 9674s to be used
as a coupling facility. There is also a capability
TECHNICAL SUPPORT FEBRUARY 1996

for defining partitions on the newer processors
to function as a coupling facility for testing,
but we did not explore this option. When a
processor with CF link support is set up
in HCD, certain CHPs (channel paths) are
designated for use as CF links. The key para-
meter is on the second screen display when
updating a processor in HCD. On this panel
you select the EC (Engineering Change) level
of the processor which supplies the CF support.
This change to the processor may or may not
require that all the CHPs in the processor
move to different addresses. Note the CHPs
used as CFS on the 9672 and 9021-942 in
Figure 1. Once the coupling facilities and
processors with CF links are defined in the
IODF, defining the CF links is easy. You simply
pick a CF link CHP on either the processor
side or coupling facility side, and HCD
prompts you to pick the CHP on the other
side to connect to. It then chooses a
CUNUMBR (Control Unit Number) starting
at FFFF and continues until it finds an unused
one. It does the same for the CF link devices.
You can then accept its choices and the defi-
nition is complete. Note that defining the
devices to the operating system (MVS) is not
necessary, so HCD doesn’t do it. Therefore, if
you run the HCD CSS/OS (Channel
SubSystem/Operating System) compare
reports, these devices will show up as not
being defined to the operating system. This
can be ignored, since it’s not a problem.
The final step was to create an IOCP deck
from HCD with which to initially load the
IOCDS on the 9672. Since the CMOS proces-
sors are controlled by PCs, there is a new
option to load the IOCDS the first time from
diskette rather than tape.
Initial Power Up
After we sorted out the duplicate SNA
address problem within the CMOS Token-
Ring, the initial power-up was uneventful. All
of the CMOS boxes PORd successfully, and
we were ready to IPL an MVS image on the
9672. Once MVS is up on one of the CMOS
processors, HCD has access to all of the
processors on the Token-Ring. HCD can turn
write protect on or off for the IOCDS slots,
and change the L1/L2 IPL parameters for a
partition. Unfortunately, there is no visibility
to the settings at the HMC. If you set the
switch in an IPL image profile that allows
HCD to change L1/L2, you cannot see what
the settings are, which, from our viewpoint,
negates some of its usefulness. We’ve asked
IBM to provide this visibility. If you need to
change the name of a 9672, note that to re-
establish the HCD connection after the name
is changed using the HMC, you must IPL
MVS. We encountered this problem because
we originally chose a large name for the 9672
(CPC9672A). We later found it was easier
Figure 1: Current Development Environment
9021-942
BGDPLEX DEVPLEX
BGDT BGD1 BGD2 BGD3 DOLP
EMIF 71 F1 70
12 11 10
BGDPLEX DEVPLEX BCSPLEX
LPAR3 LPAR1 LPAR2
9674-C01 CF9674A
Note: This is not a production configuration. For production there should be two CF links to each
coupling facility from each partition. This would allow recovery to the same CF in the event of a link
failure. Without a second link, recovery must be to the second CF! Also, we are sharing CPs in the
CFs. Not recommended for production.
working with the PROFILEs on the HMC if
the name was shorter, so we changed it to
A9672.
HARDWARE MANAGEMENT CONSOLE
(HMC)
A Radical Change to
Controlling Mainframes
One of the biggest challenges of making
the transition to the world of Parallel Sysplex
and CMOS is controlling the systems via an
OS/2-based Graphical User Interface (GUI).
It wasn’t that long ago that PCs were intro-
duced to system programmers. Some protested
replacing their perfectly good, trusty 3270
dumb terminals with a PC. However, the tran-
sition is more intimidating to the operations
staff. For years they have assimilated new
machine command languages in small doses
without any problem. However, moving to
HMC is unlike the relatively small increments
of change that have occurred in the past.
Additionally, the GUI is so powerful that one
can erroneously IPL multiple systems with a
double mouse click.
Training Issues
Much more training is required for this
transition than for previous generations of
mainframes. This training is needed for oper-
ations personnel, systems programmers, and
the IBM or other vendor Customer Engineers
(CEs). There is first a need to establish OS/2
knowledge, and knowledge of the use of a
mouse and GUIs in general.
SYSTEM STRATEGIES
9672-E01
BSCPLEX
DIMS TOST BOST TIMS
F0 EMIF 10 11
12 11 10
BGDPLEX DEVPLEX BCSPLEX
LPAR3 LPAR1 LPAR2
9674-C01 CF9674B
After a knowledge base has been estab-
lished, the HMC GUI, which is unique, needs
to be understood. Being OS/2 or Windows
adept does not translate to knowing HMC.
The HMC screen is divided into three
separate areas.
Figure 2 depicts an HMC screen. By invok-
ing (double-clicking) the GROUPS option in
the VIEWS area, you can select the type of
objects with which you want to work, i.e.,
hardware objects (CMOS boxes) or software
objects (IPL images). The WORK area is the
section of the screen that contains icons
representing the objects selected in GROUPS.
If the GROUP selected is for hardware
objects, then icons representing CMOS boxes
are displayed in the WORK area. If the
GROUP selected is for IPL images, then
icons representing MVS images are displayed.
The third area, the TASK area, is on the right
of the screen and is vertically oriented. There
are many types of tasks that can be chosen,
and one of the challenges is understanding
that a task, such as ACTIVATE, has different
consequences for different types of objects.
For example, if you ACTIVATE a hardware
object, you POR the box, quite effectively
bringing down all of the partitions and MVS
images. In contrast, if an IPL image is selected
and ACTIVATED, the IPL images selected
will be IPLed and the other images will be
unaffected. In spite of many warnings to the
staff, occasionally someone will erroneously
bring down all of the MVS images on the
9672 when he/she simply wanted to IPL one
of the images. This brief introduction to the
TECHNICAL SUPPORT FEBRUARY 1996
 SYSTEM STRATEGIES
Figure 2: HMC1 Hardware Management Console Workplace
Views
! Messages
Groups Exceptions Active
Tasks
MVS_IPL Work Area
BOST TOST TIMS
HMC desktop is meant to illustrate how
different it is from any tool used before, and
the need for education and hands-on training.
Fortunately, there are many avenues for
accomplishing this. There are video tapes,
HMC tutorial disks, and formal classes. One
highly recommended class is IBM’s H4017-
Parallel Sysplex Operations and Recovery.
This class is taught onsite, and includes labs
where CF link and other failures are simulat-
ed and proper recovery actions are taught.
It includes classroom and hands-on use of the
HMC. However, classes alone are not enough.
It is crucial that operators and systems pro-
grammers devote time to hands-on exercises.
PROFILES and HMC Settings
In addition to the formidable challenge of
operating a CMOS complex with the HMC,
controlling the complex is a separate signifi-
cant challenge. PROFILES on the HMC are
constructs stored in the processor that control
initialization and operation of the CMOS
complex. All of the information needed on
the previous generations of mainframes, such
as the amount of memory to give to an LPAR,
LPAR capping, IPL load address, etc., is
stored in PROFILES. In addition, there are
many new controls stored in PROFILES,
such as whether an MVS image will IPL
automatically when a CMOS box is PORd.
There are also PROFILES that define the
“exceptional” conditions that dictate what
color to paint an icon, and if the object
should be placed in the EXCEPTIONS work
area. There are also settings to be made to
TECHNICAL SUPPORT FEBRUARY 1996
Daily Tasks
Hardware
Console Books Operating
Actions System
Messages
Activation
Reset
Normal
Deactivate
CPC
Console
Grouping
? Help
control the HMC desktop. For example,
when a new MVS image is set up on a
CMOS processor, a new IPL image object
must be set up and a PROFILE associated
with it. This new object is stored on the
HMC itself, not in the processor, as the
PROFILES are. The HMC desktop changes
must be backed up and propagated to other
HMC consoles by loading the backup
diskette at the other HMC.
The point here is that
CMOS technology is
significantly different,
and these differences
need to be understood
when configuring
the Parallel Sysplexes
of the future.
Responsibilities
This brings us to the requirement of
defining who is responsible for what in the
CMOS world. There are levels of authority
that allow dividing up which users can do
what tasks on the HMC. Due to the com-
plexity and steep learning curve associated
with maintaining the PROFILES on the
HMC, we’ve given that responsibility to the
systems programming staff. The user ID for
operations is not allowed to update the
PROFILES. Since the PROFILES don’t
change much after they’re set up, this seems
to be working. This decision was made
considering the fact there is remote HMC
capability via the DCAF (Disabled Console
Access Facility), which will allow the sys-
tems programmers to run HMC console
sessions outside the data center. The system
programmers are also responsible for the
HMC desktop settings and for backing them
up. We have left the OS/2 configuration,
HMC microcode , as well as backing up the
entire workstation to optical disk, solely as
IBM’s responsibility.
Day-to-Day Operations
It’s a well known fact that it’s difficult to
diagnose a problem when you’re called at
home. The advent of the HMC does not
improve this. In fact, it will likely make it
more difficult. There are many more things
that can go wrong on the HMC GUI than on
the system consoles of the previous main-
frames. Additionally, walking an operator
around the HMC remotely will not likely be
viable. However, hope is on the horizon. The
DCAF facility has dial-in capability. The
performance of this is suspect, but if it will
decrease the need to drive to the data center
when problems occur during off hours, it is
worth exploring. Since system management
tools are generally migrating to GUIs, find-
ing a solution via dial-up will be needed.
CMOS HARDWARE RELIABILITY
Trade offs vs. Bi-Polar/ECL
(Emitter-Coupled Logic)
Technology
To accomplish the goal of providing
low-cost computing via mass produced-
commodity based CMOS chips, IBM had to
make some trade offs vs. the previous tec h-
nologies. Some of these were based upon the
higher reliability of CMOS systems which
have a MTBF (Mean Time Between Failure)
rate of one in 15 years vs. one in 11 years for
the previous mainframe technology.
For example, all of the CPs are on a single
board, and it is not possible to do concurrent
CP maintenance. This tradeoff shows up on
the positive side as significantly lower main-
tenance charges for CMOS, but the down-
side is significant. So, if you’re unlucky
enough to experience a failure as we have,
you’re down until the card is replaced.
For other types of failures, you may remain
up but degraded, and the degradation cannot
be relieved without bringing the entire
CPC down for repair. The technology will
continue to evolve. A new feature of CMOS
will be to reassign a normal CP as the SAP
 SYSTEM STRATEGIES

(System Assist Processor) and other

availability enhancements, such as automatic
CP restart. However, not being able to do
concurrent CP maintenance remains a notable
difference from the previous technolo gy that
needs to be considered. There are other dif-
ferences, such as multiple-bit error correc-
tion, which is not done on CMOS. The point
here is that CMOS technology is significant -
ly different, and these differences need to be
understood when configuring the Parallel
Sysplexes of the future.
Managing the Parallel Sysplex
This article presented an overview of the
new CMOS technology and some of its
challenges. While the potential benefits of
Parallel Sysplex are numerous, there is much
work required in positioning and establishing
new processes to manage it. The learning
curve is steep, so personnel training needs
must be given a high priority for a successful
transition to take place. The concluding arti-
cle will examine some of the software impli-
cations of implementing Parallel Sysplex
technology. ts
Michael M. DuPas is a Thomas Sible is a senior
senior systems program - systems programmer in
mer for Boeing the MVS design/build
Information & Support group at Boeing
Services. He has worked Information & Support
in the computing field for Services. One of his
25 years, 18 of them for current assignments
Boeing. He has been a speaker at IBM’s Data is the implementation of coupling facilities and
Centers of the 90s technical conference since its Parallel Sysplex. He has given presentations at
inception. He currently is a lead systems the SHARE conferences, the most recent being
programmer in the MVS Implementation Group, “MVS 5.2 and Parallel Sysplex User
responsible for migrating MVS maintenance Experiences” in Orlando last summer.
levels and hardware configuration definition.
©1996 Technical Enterprises, Inc. Reprinted with permission of Technical Support
magazine. For subscription information, email mbrship@naspa.net or call 414-768-8000, Ext. 116.