Sun - VERITAS Volume Manager - Student Guide

VERITAS Volume Manager
Troubleshooting
IES-410
Student Guide
Sun Microsystems, Inc.

UBRM05-104
500 Eldorado Blvd.
Broomfield, CO 80021
U.S.A.
Revision B
January 13, 2003 5:15 pm
Copyright 2003 Sun Microsystems, Inc., 901 San Antonio Road, Palo Alto, California 94303, U.S.A. All rights reserved.
This product or document is protected by copyright and distributed under licenses restricting its use, copying, distribution, and
decompilation. No part of this product or document may be reproduced in any form by any means without prior written authorization of
Sun and its licensors, if any.
Third-party software, including font technology, is copyrighted and licensed from Sun suppliers.
Sun, Sun Microsystems, the Sun Logo, Solaris, StorEdge, Sun Enterprise, SunSolve, Sun Enterprise Network Array, JumpStart, OpenBoot,
Solstice, Sun BluePrints, and Solstice DiskSuite are trademarks or registered trademarks of Sun Microsystems, Inc. in the U.S. and other
countries.
All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International, Inc. in the U.S. and
other countries. Products bearing SPARC trademarks are based upon an architecture developed by Sun Microsystems, Inc.
UNIX is a registered trademark in the U.S. and other countries, exclusively licensed through X/Open Company, Ltd.
U.S. Government approval might be required when exporting the product.
RESTRICTED RIGHTS: Use, duplication, or disclosure by the U.S. Government is subject to restrictions of FAR 52.227-14(g)(2)(6/87) and
FAR 52.227-19(6/87), or DFAR 252.227-7015 (b)(6/95) and DFAR 227.7202-3(a).
DOCUMENTATION IS PROVIDED AS IS AND ALL EXPRESS OR IMPLIED CONDITIONS, REPRESENTATIONS, AND
WARRANTIES, INCLUDING ANY IMPLIED WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE OR
NON-INFRINGEMENT, ARE DISCLAIMED, EXCEPT TO THE EXTENT THAT SUCH DISCLAIMERS ARE HELD TO BE LEGALLY
INVALID.
THIS MANUAL IS DESIGNED TO SUPPORT AN INSTRUCTOR-LED TRAINING

(ILT) COURSE AND IS INTENDED TO BE USED FOR REFERENCE PURPOSES IN
CONJUNCTION WITH THE ILT COURSE. THE MANUAL IS NOT A STANDALONE
TRAINING TOOL. USE OF THE MANUAL FOR SELF-STUDY WITHOUT CLASS
ATTENDANCE IS NOT RECOMMENDED.
Export Control Classification Number (ECCN) assigned: 17 July 2002
Please
Recycle
Copyright 2003 Sun Microsystems Inc., 901 San Antonio Road, Palo Alto, California 94303, Etats-Unis. Tous droits rservs.
Ce produit ou document est protg par un copyright et distribu avec des licences qui en restreignent lutilisation, la copie, la distribution,
et la dcompilation. Aucune partie de ce produit ou document ne peut tre reproduite sous aucune forme, par quelque moyen que ce soit,
sans lautorisation pralable et crite de Sun et de ses bailleurs de licence, sil y en a.
Le logiciel dtenu par des tiers, et qui comprend la technologie relative aux polices de caractres, est protg par un copyright et licenci
par des fournisseurs de Sun.
Sun, Sun Microsystems, the Sun Logo, Solaris, StorEdge, Sun Enterprise, SunSolve, Sun Enterprise Network Array, JumpStart, OpenBoot,
Solstice, Sun BluePrints, et Solstice DiskSuite sont des marques de fabrique ou des marques dposes de Sun Microsystems, Inc. aux EtatsUnis et dans dautres pays.
Toutes les marques SPARC sont utilises sous licence sont des marques de fabrique ou des marques dposes de SPARC International, Inc.
aux Etats-Unis et dans dautres pays. Les produits portant les marques SPARC sont bass sur une architecture dveloppe par Sun
Microsystems, Inc.
UNIX est une marques dpose aux Etats-Unis et dans dautres pays et licencie exclusivement par X/Open Company, Ltd.
Laccord du gouvernement amricain est requis avant lexportation du produit.
LA DOCUMENTATION EST FOURNIE EN LETAT ET TOUTES AUTRES CONDITIONS, DECLARATIONS ET GARANTIES
EXPRESSES OU TACITES SONT FORMELLEMENT EXCLUES, DANS LA MESURE AUTORISEE PAR LA LOI APPLICABLE, Y
COMPRIS NOTAMMENT TOUTE GARANTIE IMPLICITE RELATIVE A LA QUALITE MARCHANDE, A LAPTITUDE A UNE
UTILISATION PARTICULIERE OU A LABSENCE DE CONTREFAON.
CE MANUEL DE RFRENCE DOIT TRE UTILIS DANS LE CADRE DUN COURS

DE FORMATION DIRIG PAR UN INSTRUCTEUR (ILT). IL NE SAGIT PAS DUN
OUTIL DE FORMATION INDPENDANT. NOUS VOUS DCONSEILLONS DE
LUTILISER DANS LE CADRE DUNE AUTO-FORMATION.
Please
Recycle
Table of Contents
About This Course ........................................................... Preface--xiii
Course Goals...................................................................... Preface--xiii
Course Map......................................................................... Preface--xiv
Topics Not Covered.............................................................Preface--xv
How Prepared Are You?................................................... Preface--xvi
Introductions ..................................................................... Preface--xvii
How to Use Course Materials ........................................ Preface--xviii
Conventions .........................................................................Preface--xix
Icons .............................................................................Preface--xix
Typographical Conventions ......................................Preface--xx
Introducing the VERITAS Volume Manager Software
Architecture ......................................................................................1-1
Objectives ........................................................................................... 1-1
Relevance............................................................................................. 1-2
Additional Resources ........................................................................ 1-3
Introducing Storage Management................................................... 1-4
Host-Based Storage Management........................................... 1-4
Controller-Based Storage Management................................. 1-5
Comparison of Storage Management Methods.................... 1-6
Exploring VxVM Software and Storage Management ................. 1-7
Relationship to the Operating System Environment ........... 1-7
Configuration Database ........................................................... 1-8
Device Discovery Layer (DDL) ............................................... 1-9
Drivers and Daemons............................................................... 1-9
VxVM Software Support Files.............................................. 1-11
Examining VxVM Software Objects.............................................. 1-20
Physical Disks......................................................................... 1-21
VxVM Software Disks ............................................................ 1-21
Disk Groups ............................................................................. 1-23
Subdisks.................................................................................... 1-24
Plexes ........................................................................................ 1-25
Volumes.................................................................................... 1-26
VxVM Software Layered Volume Objects........................... 1-27
Sun Proprietary: Internal Use Only
v
Copyright 2003 Sun Microsystems, Inc. All Rights Reserved. Enterprise Services, Revision B
Resynchronizing Volumes.............................................................. 1-29

Dirty Flag................................................................................. 1-29
Resynchronization Process .................................................. 1-30
Introducing VxVM Software Logging .......................................... 1-31
RAID 5 Logs............................................................................. 1-31
Dirty Region Logs (DRLs)...................................................... 1-32
Examining Plex States ..................................................................... 1-34
Plex State Descriptions ........................................................... 1-34
Plex Kernel States.................................................................... 1-36
General Plex State Cycle ...................................................... 1-37
Other Plex State Transitions .................................................. 1-37
Introducing Supported VxVM Software Version 3.2
Features .......................................................................................... 1-40
Device Discovery .................................................................... 1-40
Device Naming........................................................................ 1-43
Sun StorEdge Traffic Manager Software Support.............. 1-46
Explicit LUN Failover............................................................. 1-46
Ordered Storage Allocation................................................... 1-46
Introducing Unsupported VxVM Software Version 3.2
Features .......................................................................................... 1-47
Disk Group Split, Move, and Join......................................... 1-47
Persistent Fast Resync Function............................................ 1-47
Encapsulating Disks........................................................................ 2-1
Objectives ........................................................................................... 2-1
Relevance............................................................................................. 2-2
Introducing Disk Encapsulation ...................................................... 2-4
Encapsulated Disk Types......................................................... 2-4
Reasons to Encapsulate Disks ................................................. 2-4
Encapsulating Data Disks ................................................................. 2-5
Pre-Encapsulation Configuration Data................................. 2-6
Data Disk Encapsulation Process ........................................... 2-8
Post-Encapsulation Configuration Data .............................. 2-13
Related Post-Encapsulation Files.......................................... 2-18
Encapsulating a Non-Conforming Disk .............................. 2-20
Unencapsulating Data Disks .......................................................... 2-23
Encapsulating Boot Disks ............................................................... 2-31
Booting root Volumes ........................................................... 2-32
Volume Restrictions................................................................ 2-33
Boot Disk Encapsulation Process.......................................... 2-33
Pre-Encapsulation Configuration Data............................... 2-35
Post-Encapsulation Configuration Data .............................. 2-38
Mirroring the Encapsulated Boot Disk ................................ 2-41
Related Post-Encapsulation Files.......................................... 2-45
vi

VERITAS Volume Manager Troubleshooting
Examining Sun Enterprise Services Best Practices for VxVM

Software-Managed Boot Disks.................................................... 2-48
Best-Practice Boot Disk Configuration Guidelines ........... 2-49
Manually Bringing the Boot Disk Under VxVM
Software Management ........................................................... 2-50
Scripted Process Using the EIS CD-ROM............................ 2-56
Unencapsulating Boot Disks .......................................................... 2-57
Unencapsulating a Boot Disk Using the vxunroot
Utility ..................................................................................... 2-57
Manually Unencapsulating a Boot Disk .............................. 2-60
Unencapsulating When Booted From the CD-ROM ........ 2-64
Performing a Basic or Functional Unencapsulation .......... 2-68
Exploring Unencapsulation Issues ................................................ 2-72
Data Disks ................................................................................ 2-72
Boot Disks................................................................................. 2-72
Exercise: Encapsulating Disks........................................................ 2-75
Preparation............................................................................... 2-75
Task 1 Installing the VxVM Software and
Encapsulating a Boot Disk ................................................. 2-76
Task 2 Unencapsulating a Boot Disk Using
the vxunroot Utility ........................................................... 2-78
Task 3 Encapsulating a Boot Disk Using
the vxdiskadm Utility.......................................................... 2-79
Task 4 Manually Unencapsulating a Boot Disk
When Booted From the CD-ROM...................................... 2-79
Task 5 Encapsulating a Data Disk Using
the vxdiskadm Utility (Optional) ...................................... 2-80
Task 6 Unencapsulating a Data Disk (Optional) ............. 2-82
Task 7 Encapsulating a Non-Conforming Data Disk
(Optional) .............................................................................. 2-83
Task 8 Lab House Cleaning ................................................ 2-84
Exercise Summary............................................................................ 2-85
Exercise: Encapsulating Disks........................................................ 2-86
Task 1 Solutions....................................................................... 2-86
Task 2 Solutions...................................................................... 2-88
Task 3 Solutions...................................................................... 2-89
Task 4 Solutions....................................................................... 2-89
Task 5 Solutions....................................................................... 2-90
Task 6 Solutions...................................................................... 2-92
Task 7 Solutions...................................................................... 2-93
Managing Dynamic Multi-Pathing ...................................................3-1
Objectives ........................................................................................... 3-1
Relevance............................................................................................. 3-2
Examining VxVM Software and Dynamic Multi-Pathing ........... 3-4

vii
VxVM Software Architecture and DMP................................ 3-4

Load Balancing .......................................................................... 3-5
Unique Disk Identifier............................................................. 3-6
DMP Device Paths .................................................................... 3-6
Solaris OE Drives and DMP Drives........................................ 3-6
DMP and Device Discovery .................................................... 3-8
Installing and Verifying DMP.......................................................... 3-9
Enabling and Disabling DMP......................................................... 3-10
Disabling DMP in Version 3.0 and Earlier .......................... 3-10
Disabling DMP in Version 3.1 and Later............................. 3-10
Files Related to Disabling and Suppression........................ 3-15
Administrating DMP With the vxdmpadm Command ................ 3-17
The listctrl Option............................................................. 3-17
Viewing Multi-Pathing Status............................................... 3-17
The getsubpaths Option ...................................................... 3-18
The getdmpnode Option ........................................................ 3-19
The disable Opt ion.............................................................. 3-19
The enable Option ................................................................. 3-20
The start restore and stop restore Options............. 3-20
Reviewing Common DMP Problems ............................................ 3-21
Disks Do Not Appear to Be Multi-Pathed........................... 3-21
Serial Number Problems........................................................ 3-21
The product_id and vendor_id Do Not Match .............. 3-22
VxVM Software Does Not See Disk Devices ...................... 3-22
Exercise: Operating DMP................................................................ 3-23
Preparation............................................................................... 3-23
Task 1 Enabling and Disabling DMP Operations .......... 3-24
Task 2 Administrating DMP .............................................. 3-25
Task 3 Using the /etc/vx/diag.d/vxdmping Script .... 3-26
Exercise: Operating DMP................................................................ 3-28
Task 1 Solutions....................................................................... 3-28
Task 2 Solutions....................................................................... 3-29
Task 3 Solutions....................................................................... 3-30
Troubleshooting Tools and Utilities............................................... 4-1
Objectives ........................................................................................... 4-1
Relevance............................................................................................. 4-2
Logging Errors.................................................................................... 4-4
Using the /var/vxvm/vxconfigd.log File ......................... 4-4
Interpreting /var/adm/messages File syslog
Messages.................................................................................. 4-8
The root Mail............................................................................ 4-9
Recovering From Errors.................................................................. 4-10
Using the Debugging Tools and Utilities ..................................... 4-11
viii

The vxexplorer Utility ......................................................... 4-18

The vxprivutil Utility ......................................................... 4-33
The vxdevwalk Utility............................................................ 4-38
The vxkprint Utility............................................................. 4-40
Using System-Level Debugging Utilities ..................................... 4-41
Exercise: Using the Error Logging and Debugging
Utilities............................................................................................ 4-42
Preparation............................................................................... 4-42
Task 1 Enabling vxconfigd Debug Logging................... 4-42
Task 2 Viewing the VxVM Software Messages ............... 4-43
Task 3 Using VxVM Software Debug Utilities ................ 4-44
Exercise: Using the Error Logging and Debugging
Utilities............................................................................................ 4-47
Task 1 Solutions....................................................................... 4-47
Task 2 Solutions....................................................................... 4-47
Task 3 Solutions....................................................................... 4-49
Recovering Boot and System Processes.......................................5-1
Objectives ........................................................................................... 5-1
Relevance............................................................................................. 5-2
Surveying VxVM Software System Recovery Processes.............. 5-4
Examining the VxVM Software Boot Process ................................ 5-5
Single-User Boot Processing.................................................... 5-6
Multi-User Startup Files......................................................... 5-17
Other Boot Process Failures................................................... 5-18
Troubleshooting Boot Process Failures......................................... 5-20
Bootable Boot Disk.................................................................. 5-20
Valid /etc/system File ......................................................... 5-21
Valid /etc/vfstab File ......................................................... 5-22
Valid rootdg Disk Group...................................................... 5-23
Startable Volumes ................................................................... 5-23
Non-Corrupted and Appropriate Binaries and
Libraries................................................................................. 5-24
Valid /etc/vx/volboot File ............................................... 5-27
VxVM Software Startup Scripts ............................................ 5-27
Troubleshooting VxVM Software Failures................................... 5-29
Exercise: Determining the VxVM Software Problem ................. 5-30
Preparation............................................................................... 5-30
Tasks ........................................................................................ 5-32
Exercise: Determining the VxVM Software Problem ................. 5-36
Task Solutions.......................................................................... 5-36

ix
Recovering Disk, Disk Group, and Volume Failures .................... 6-1

Objectives ........................................................................................... 6-1
Relevance............................................................................................. 6-2
Introducing Recovery Processes ...................................................... 6-4
Identifying Disk Errors ..................................................................... 6-5
The vxprint Command........................................................... 6-5
The vxdisk Command............................................................. 6-6
The vxdiskadm list Option ................................................. 6-7
The vxstat Command............................................................. 6-7
Disk Failure Categories and root Mail ................................. 6-8
Full Disk Failure........................................................................ 6-8
Replacing Disks ................................................................................ 6-10
Replacing Disks Using the vxdiskadm Utility.................... 6-10
Replacing Disks Using the Command Line ........................ 6-10
Recovering a Disk Which VxVM Software Cannot See .... 6-11
Troubleshooting Volume Errors .................................................... 6-13
Listing Unstartable Volumes................................................. 6-13
Restarting a Disabled Volume.............................................. 6-14
Recovering a Mirrored Volume ............................................ 6-14
Recovering a RAID 5 Volume ............................................... 6-15
Forcibly Starting RAID 5 Volumes ....................................... 6-16
Troubleshooting Disk Group Errors ............................................. 6-17
Exercise: Determining the VxVM Software Disk Problem ........ 6-18
Preparation............................................................................... 6-18
Tasks ......................................................................................... 6-20
Exercise: Determining the VxVM Software Disk Problem ........ 6-24
Task Solutions.......................................................................... 6-24
Upgrading the VxVM Software........................................................ 7-1
Objectives ........................................................................................... 7-1
Relevance............................................................................................. 7-2
Surveying the Upgrade Processes and Procedures ...................... 7-4
Upgrading With a Script (VxVM 3.1/3.2/3.5)...................... 7-4
Upgrading Manually (VxVM 3.1/3.2/3.5)............................ 7-6
Upgrade Using pkgadd (VxVM 3.5)...................................... 7-9
Upgrading a Disk Group ....................................................... 7-11
Release Notes.......................................................................... 7-12
Licensing .................................................................................. 7-12
Upgrading the Solaris OE ............................................................... 7-13
Exercise: Upgrading the VxVM Software..................................... 7-14
Preparation............................................................................... 7-14
Tasks ......................................................................................... 7-14

SunSolve INFODOCs....................................................................... A-1

INFODOC 16051 ............................................................................... A-2
INFODOC 24663 ............................................................................... A-4
Disk Encapsulation Processes ...................................................... B-1
Example Five-Slice Boot Disk Encapsulation............................... C-1
Pre-Encapsulation Configuration Data.......................................... C-3
Pre-Encapsulation prtvtoc Command................................ C-3
Pre-Encapsulation format Print Utility ........................... C-4
Pre-Encapsulation df -k Command .................................... C-4
Pre-Encapsulation swap -l Command................................ C-4
Post-Encapsulation Configuration Data ........................................ C-5
Post-Encapsulation prtvtoc /dev/rdsk/c1t2d0s2
Command............................................................................... C-5
Post-Encapsulation format print Utility.......................... C-6
Post-Encapsulation df -k Command................................... C-6
Post-Encapsulation vxprint Command .............................. C-6
Post-Encapsulation /etc/vfstab File.................................. C-7
Post-Encapsulation Mirror Disk prtvtoc Command ........ C-8
Post-Encapsulation Mirror Disk format Print
Utility ..................................................................................... C-9
Example Five-Slice Boot Disk Unencapsulation......................... C-10
Unencapsulating Using the vxunroot Command............ C-10
Manually Unencapsulating .................................................. C-10
The Boot Process ............................................................................ D-1
Configuring the VxVM Software..................................................... E-1
Objectives ........................................................................................... E-1
Relevance.............................................................................................E-2
Additional Resources ........................................................................E-3
Supported RAID Levels ....................................................................E-4
Layered Volumes ......................................................................E-4
Ordered Allocation of Storage ................................................E-4
VxVM Software States.......................................................................E-5
Plex States...................................................................................E-5
Plex and Volume Kernel States ...............................................E-5
Volume States ............................................................................E-5
Disk States ..................................................................................E-6
The vxprint Command....................................................................E-7
Output Header Information ....................................................E-7
Field Descriptions .................................................................... E-8
Stripe Width and Stripe Units ...............................................E-11
Complex RAID Levels.....................................................................E-12
Additional Layout Options ...................................................E-15
Additional Mirror Attributes ............................................... E-17
Disk Layout Practices ......................................................................E-18
xi
Guidelines for RAID Array Layouts ................................... E-19

Guidelines for Complex File System Layouts.....................E-19
Striping Considerations .........................................................E-20
Manipulating Disk Layouts............................................................E-22
Online Re-Layout ....................................................................E-22
Growing File Systems.............................................................E-23
Changing Disk Group Configurations..........................................E-27
Disk Group Reconfiguration Commands............................E-27
Disk Group Reconfiguration Recovery................................E-28
Reconfiguration Considerations ...........................................E-30
Hot-Relocation..................................................................................E-32
Hot-Relocation Process ..........................................................E-32
Hot-Relocation Configuration.............................................. E-36
Unrelocating ........................................................................... E-37
Hot-Spares.........................................................................................E-38
Comparison of Hot-Relocation and Hot-Spares.................E-38
Activating the Hot-Spare Function ..................................... E-39
xii

Preface
About This Course

Course Goals
The VERITAS Volume Manager Troubleshooting course introduces you to the
VERITAS Volume Manager (VxVM) software and its functions.
Upon completion of this course, you should be able to:
Define information technology (IT) storage management
Improve the availability of the VxVM software
Identify and repair common disk and disk group management

problems
Upgrade the VxVM software
Resolve VxVM software version and licensing problems

Preface-xiii
Course Map
Course Map
The following course map enables you to see the general topics and the
modules for that topic area in reference to the course goal.
Architecture
The VERITAS
Volume Manager
Software Architecture
Availability Management
Encapsulating
Disks
Managing Dynamic
Multi-Pathing
Problem Management
Troubleshooting
Tools and Utilities
Recovering
Boot and System
Processes
Recovering Disk,
Disk Group and
Volume Failures
Release Management
Upgrading
the VxVM
Software
Preface-xiv

Topics Not Covered
Topics Not Covered

This course does not cover the following topics. Many of these topics are
covered in other courses offered by Sun Educational Services:
Storage area networks Covered in ES-475: Design and Administration

of Storage Area Networks
Solaris Operating Environment (OE) administration Covered in

SA-288: Solaris 8 System Administration II
Basic VxVM software administration Covered in IES-310:

Sun StorEdge Volume Manager Administration
Hitachi Lightning SE9900 administration Covered in HDS-335:

Hitachi Lightning SE9900 Overview and Configuration
Sun StorEdge T3 administration Covered in ES-255: Sun

Hardware RAID and T3 Storage Systems Administration
Refer to the Sun Educational Services catalog for specific information and
registration.

About This Course
Preface-xv
How Prepared Are You?
How Prepared Are You?

To be sure you are prepared to take this course, can you answer yes to the
following questions?
Preface-xvi
Can you install the VxVM software?
Can you create the following VxVM software objects from the
command line and Storage Administrator graphical user interface
(GUI)?
Disks
Subdisks
Plexes
Volumes:
Striped
Simple
RAID 5
Can you use the vxprint utility to print VxVM software

configuration information?
Can you mirror the boot disk on a server?
Can you replace a failed disk using command line utilities and the
Storage Administrator GUI?

Introductions
Introductions
Now that you have been introduced to the course, introduce yourself to
the other students and the instructor, addressing the following items:
Name
Company affiliation
Title, function, and job responsibility
Experience related to topics presented in this course
Reasons for enrolling in this course
Expectations for this course

About This Course
Preface-xvii
How to Use Course Materials
How to Use Course Materials

To enable you to succeed in this course, these course materials employ a
learning model that is composed of the following components:
Preface-xviii
Goals You should be able to accomplish the goals after finishing

this course and meeting all of its objectives.
Objectives You should be able to accomplish the objectives after

completing a portion of instructional content. Objectives support
goals and can support other higher-level objectives.
Lecture The instructor will present information specific to the

objective of the module. This information will help you learn the
knowledge and skills necessary to succeed with the activities.
Activities The activities take on various forms, such as an exercise,

self-check, discussion, and demonstration. Activities are used to
facilitate mastery of an objective.
Visual aids The instructor might use several visual aids to convey a
concept, such as a process, in a visual form. Visual aids commonly
contain graphics, animation, and video.

Conventions
Conventions
The following icons and typographical conventions are used in this course
to represent various training elements and alternative learning resources.
Icons
Additional resources Indicates additional reference materials are
available.
Discussion Indicates a small-group or class discussion on the current

topic is recommended at this time.
Power user Indicates additional supportive topics, ideas, or other

optional information.
Note Indicates additional information that can help but is not crucial to
understanding of the concept being described. Examples of notational
information include keyword shortcuts and minor system adjustments.
Caution Indicates that there is a risk of personal injury from a
nonelectrical hazard, or risk of irreversible damage to data, software, or
the operating system. A caution indicates that the possibility of a hazard
(as opposed to certainty) might happen, depending on the action of the
user.
Caution Indicates that either personal injury or irreversible damage of
data, software, or the operating system will occur if the user performs this
action. A warning does not indicate potential events; if the action is
performed, catastrophic events will occur.

About This Course
Preface-xix
Conventions
Caution Indicates the risk of injury due to heat or hot surfaces will
result.
Typographical Conventions
Courier is used for the names of command, files, and directories, as well
as on-screen computer output. For example:
Use ls -al to list all files.
system% You have mail.
Courier bold is used for characters and numbers that you type. For
example:
system% su
Password:
Courier italic is used for variables and command-line place-holders

that are replaced with a real name or value. For example:
To delete a file, type the rm filename. command.
Courier italic bold is used to represent variables whose values are to

be entered by the student as part of an activity; for example:
Type chmod a+rwx filename to grant read, write, and execute
rights for filename to world, group, and users.
Palatino italics is used for book titles, new words or terms, or words that
are emphasized. For example:
Read Chapter 6 in the Users Guide.
You must be root to do this.
Preface-xx

Module 1
Introducing the VERITAS Volume Manager

Software Architecture
Objectives
Upon completion of this module, you should be able to:
Describe the two storage management methodologies
Describe the relationship between the VxVM software and the

Solaris Operating Environment
Identify and describe the major components of the VxVM software

configuration database
Identify and define all the VxVM software objects
Describe the resynchronization process
Describe how the VxVM software identifies disks under control
Describe the different plex states
List newly supported and unsupported features introduced in the

VxVM software version 3.2

1-1
Relevance
Relevance
!
?
1-2
Discussion The following questions are relevant to understanding how

the VxVM software internal architecture supports the management of
enterprise storage subsystems:
What VxVM software objects are used to build mountable volumes?
What files are used to control the VxVM software internal

configuration, and where are they located?
Where are the VxVM software driver and driver configuration files
stored?
How does device discovery enable adding disks to enterprise storage

subsystems without a reboot of the system?
What new features are included in the VxVM software version 3.2?

Additional Resources
Additional resources The following references provide additional
information on the topics described in this module:
VERITAS Volume Manager 3.2 Administrators Guide. Mountain

View, California: VERITAS Software Corporation, August 2001,
number 30-000392-011, TechPDF ID 240253.
VERITAS Volume Manager 3.2 Installation Guide. Mountain View,

California: VERITAS Software Corporation, August 2001,
VERITAS Volume Manager 3.2 Troubleshooting Guide. Mountain

VERITAS Volume Manager Storage Administrator 3.2 Administrators

Guide. Mountain View, California: VERITAS Software Corporation,
July 2001, number 30-000393-011, TechPDF ID 240257.
For additional information and resources, refer to:
http://storage.east, http://storage.central, and

http://storage.west

Introducing the VERITAS Volume Manager Software Architecture
1-3
Introducing Storage Management

Storage provided by local or remote storage subsystems can be managed
using one of two methods. These methods are host-based and controllerbased storage management. The method used depends on the capabilities
of the storage subsystem. This section describes both storage management
methodologies.
Host-Based Storage Management

Storage subsystems such as the Sun StorEdge A5200 array are enclosurebased systems known as just a bunch of disks (JBODs). These JBOD
storage enclosures must be managed by the VxVM software if additional
capacity and availability beyond single disks is desired. This is illustrated
in Figure 1-1.
Host System
Host Bus
Adapter
Device Drivers
Process
Data
User
Manager
9-GByte
VxVM Software
Virtual
Layer
Volume
JBOD Enclosure
Interface
Adapter Board
3-GByte
disk/slice/LUN
Figure 1-1
1-4
disk
disk
disk
disk
disk
disk
disk
disk
disk
disk
disk
disk
Host-Based Storage Management


JBOD disks also can be found in Suns midrange servers, such as the Sun
Enterprise 3500 and Sun Enterprise 250 servers. These disks are not
considered by the VxVM software to be enclosure-based JBODs because
they are configured differently. This difference is addressed later in this
module.
Controller-Based Storage Management

Storage subsystems such as the Sun StorEdge T3 array have intelligent
controllers which are capable of internally configuring logical units
(LUNs) for a redundant array of independent disks (RAID) layout. This is
illustrated in Figure 1-2.
Host System
Host Bus
Adapter
Data
Storage Subsystem
Manager
9-GByte
LUN
Controller
RAID
User
Manager
Process
Cache
RAID
Hardware
3-GByte
Disk/Slice
Figure 1-2
disk
disk
disk
disk
disk
disk
disk
disk
disk
disk
disk
disk
Controller-Based Storage Management
Intelligent controller storage can be managed either by the storage

subsystems controller, the VxVM software, or both.

1-5
Comparison of Storage Management Methods

The decision to use either host- or controller-based storage management
ultimately depends on the customers storage environment and the
processes and procedures used to manage that storage.
Host-based management has the advantage of providing a standard set of
utilities for configuring storage which is independent of the storage
provider. Host-based management does this at a slight cost to overall
system performance. Since the VxVM software executes in the Solaris OE,
it requires processor cycles to perform storage management operations.
This overhead, typically 12 percent, can be an issue for CPU cycle
constrained servers.
Controller-based storage management provides the best overall system
performance since no CPU cycles are used to manage a servers storage.
This method does have operational issues if the customer has multiple
vendors providing storage. If a customer has storage subsystems from
different vendors installed in the storage environment, the customers
administrative staff must learn multiple RAID management applications,
and must track which storage is managed by which RAID management
application on different servers. These issues can lead to confusion and to
mistakes that cause storage availability issues.
If a storage environment contains both intelligent arrays and JBOD
enclosures, administrators must track which storage subsystem is
managed by a RAID manager or by the VxVM software. In this situation,
it can be advantageous to use intelligent arrays to configure small
individual LUNs, which are then used to build larger LUNS using the
VxVM software. A storage management strategy of this type eases
operational issues by turning storage into a black box. This means that
intelligent arrays are configured once using a RAID manager, and the
LUNs are then managed using the VxVM software along with the JBOD
enclosures.
1-6

Exploring VxVM Software and Storage Management

The VxVM software provides the Solaris OE with the ability to manage
local and remote storage subsystems. The word manage describes the
Solaris OEs ability to use storage configured in such a fashion that the
Solaris OE capacity and availability is enhanced beyond single, nonmanaged storage devices. The VxVM software is one of two products
available for the Solaris OE designed specifically for storage management.
The other product is the Sun Microsystems Solstice DiskSuite software.
Solstice DiskSuite software concepts and facilities are not described as
part of this course.
This section describes the VxVM software use for storage management.
Relationship to the Operating System Environment

The VxVM software operates as a storage management subsystem
between the Solaris OE and hosted data management subsystems, such as
a database manager or the Solaris OE file system.
The VxVM software storage management subsystem is implemented as a
set of drivers and daemons that are tightly coupled with the Solaris OE.
The VxVM software drivers implement virtual volumes from physical
storage resources that are mounted and accessed by users and
applications hosted by the Solaris OE. Figure 1-3 on page 1-8 displays this
relationship.

1-7
Applications
Data Management Layer

File System
Database
VxVM
Configuration
Database
VxVM Storage Management Subsystem
DMP Driver
Storage Device Drivers
Kernel
vxconfigd
Enclosure/Array
DDL
Figure 1-3
The VxVM Software and Solaris OE Relationship
Configuration Database
The VxVM software configuration database stores all disk and volume
configuration data. The following apply to the configuration database:
1-8
Database access is managed through the /dev/vx/config device.
Accesses are executed serially.
Initial volume configurations are downloaded to the kernel through

this device.
The vxconfigd daemon updates the database to reflect changes to

the configuration of VxVM software objects.

The vxconfigd daemon is the exclusive owner of the

/dev/vx/config device.
Non-volatile copies of the database are stored in the private region of

a VM disk as follows:
Configuration copies are replicated within the disk group and

are available to prevent total loss of the database.
The kernel configuration database is created from the disk

group copies during the system boot process.
Device Discovery Layer (DDL)

The device discovery layer (DDL) was introduced in the VxVM software
version 3.2. It is used to discover all disk devices connected to a system
and their attributes. DDL then stores this information in the dynamic
multi-pathing (DMP) configuration database for use by DMP operations.
Note DDL concepts and administration are discussed in the section
Device Discovery on page 1-40.
Drivers and Daemons

The VxVM software storage management subsystem is implemented by
the drivers and daemons described in this section.
Kernel Drivers
VxVM software storage management drivers include the following:
vxio The vxio driver manages access to VxVM software virtual

devices. Prior to initiating an input/output (I/O) operation to one of
these virtual devices, the vxio driver consults the VxVM
configuration database. The vxio driver is also responsible for
reporting device errors.
vxdmp The vxdmp daemon performs DMP operations on multipathed storage subsystems.
vxspec The vxspec software control and status driver is used by

vxconfigd and other VxVM software utilities to communicate with
the Solaris OE kernel.

1-9
VxVM Software Daemons

VxVM software storage management daemons include the following:
vxconfigd The vxconfigd software configuration daemon is

responsible for maintaining disk and disk group configuration
information. The vxconfigd daemon performs the following:
It takes configuration change requests from VxVM software

utilities, communicates those to the kernel, and updates the
VxVM software configuration database.
During system boot processing, vxconfigd reads the kernel

log to determine the current state of VxVM software objects and
any recovery operations to be performed.
During disk group imports, vxconfigd scans the private

regions of the disk groups VM disks to find the most current
copy of its configuration database. The daemon then adds that
data to the systems VxVM software kernel configuration
database.
It receives cluster related information from the vxclust utility.

In a cluster environment, the different instances of vxconfigd
running on the cluster nodes communicate with each other
across the network.
It logs any VxVM software object errors.
Note The vxconfigd logging and error messages are discussed in detail
in Module 4, Troubleshooting Tools and Utilities.
1-10
vxrelocd The vxrelocd daemon performs hot-relocation to

restore redundancy. The vxrelocd daemon performs the following:
Data located on subdisks that are part of a failed VM disk is

relocated to spare disks that have sufficient free space
configured in the disk group.
When a relocation operation begins, vxrelocd sends mail to the

local root account.

VxVM Software Support Files

The files and directories described in this section are installed by the
vxinstall process to support the VxVM software storage management
operations. This discussion is organized by directory, as follows:
/kernel/drv
/kernel/drv/sparcv9
/sbin
/usr/sbin
/etc/init.d
/etc/rc*.d
/opt
/etc/vx
/var/vxvm
/dev/vx
The /kernel/drv Directory

The /kernel/drv directory contains the driver binaries and
configuration files for the 32-bit versions of vxiod, vxspec, and vxdmp.
The strategy Sun StorEdge employs to account for different VxVM
software versions of the Solaris OE is important. The driver binary for
vxio is the /kernel/drv/vxio file. There are three additional vxio driver
files, one each for Solaris OE versions 5.6, 5.7 and 5.8. During the
installation process the installer is prompted for the version of Solaris OE
in which this instance of the VxVM software is being installed. The
answer determines which driver version is copied, and that version
becomes the becomes /kernel/drv/vxio file. Notice that the byte count
for /kernel/dev/vxio is identical to /kernel/drv/vxio.SunOS_5.8 as
shown in the following example:
640
4
608
608
640
3776
2
-rw-r--r--rw-r--r--rw-r--r--rw-r--r--rw-r--r--rw-r--r--rw-r--r--
1
1
1
1
1
1
1
root
root
root
root
root
root
root
sys
sys
sys
sys
sys
sys
sys
314156 Nov 21 23:03 ./vxdmp

1026 Aug 15 2001 ./vxdmp.conf
297920 Aug 15 2001 ./vxdmp.SunOS_5.6
300980 Aug 15 2001 ./vxdmp.SunOS_5.7
314156 Nov 21 23:03 ./vxdmp.SunOS_5.8
1920316 Nov 21 23:03 ./vxio
991 Aug 15 2001 ./vxio.conf

1-11

3664
3696
3776
30
4
28
30
30
26
2
38
-rw-r--r--rw-r--r--rw-r--r--rw-r--r--rw-r--r--rw-r--r--rw-r--r--rw-r--r--rwxr-xr-x
-rw-r--r--rwxr-xr-x
1
1
1
1
1
1
1
1
1
1
1
root
root
root
root
root
root
root
root
root
root
root
sys
sys
sys
other
sys
sys
sys
sys
sys
sys
sys
1860500 Aug 15 2001 ./vxio.SunOS_5.6

1878432 Aug 15 2001 ./vxio.SunOS_5.7
1920316 Nov 21 23:03 ./vxio.SunOS_5.8
14968 Mar 29 15:41 ./vxspec
1315 Aug 15 2001 ./vxspec.conf
14060 Aug 15 2001 ./vxspec.SunOS_5.6
14388 Aug 15 2001 ./vxspec.SunOS_5.7
14968 Aug 15 2001 ./vxspec.SunOS_5.8
12384 Jan 8 2000 ./wc
129 Jan 5 2000 ./wc.conf
18544 Jan 8 2000 ./xbox
If the VxVM software driver binaries are corrupted, copy the proper
architecture version of the binary into the corrupted driver binary file to
correct the problem. This process is presented in Module 5, Recovering
Boot and System Processes.
The /kernel/drv/sparcv9 Directory

The /kernel/drv/sparcv9 directory contains the driver binaries for the
64-bit versions of vxio, vxspec, and vxdmp. Notice in this example list
output that the configuration files are not present. When installed in 64-bit
mode (the default for Solaris 8 OE and later), the drivers use the
configuration files
800
768
800
5328
5232
5328
38
36
38
-rw-r--r--rw-r--r--rw-r--r--rw-r--r--rw-r--r--rw-r--r--rw-r--r--rw-r--r--rw-r--r--
1
1
1
1
1
1
1
1
1
root
root
root
root
root
root
root
root
root
sys
sys
sys
sys
sys
sys
other
sys
sys
393968 Nov 21 23:03 vxdmp

380840 Aug 15 2001 vxdmp.SunOS_5.7
393968 Nov 21 23:03 vxdmp.SunOS_5.8
2714688 Nov 21 23:03 vxio
2666256 Aug 15 2001 vxio.SunOS_5.7
2714688 Nov 21 23:03 vxio.SunOS_5.8
18504 Mar 29 15:41 vxspec
17928 Aug 15 2001 vxspec.SunOS_5.7
18504 Aug 15 2001 vxspec.SunOS_5.8
The /sbin Directory

The /sbin directory holds the binaries for vxconfigd and additional
VxVM software commands. Notice the configuration files in this example
list output.
2960
2800
2832
2960
864
256
1-12
-r-xr-xr-x
-r-xr-xr-x
-r-xr-xr-x
-r-xr-xr-x
-r-xr-xr-x
-r-xr-xr-x
1
1
1
1
1
1
root
root
root
root
root
root
sys
sys
sys
sys
sys
sys
1499404 Nov 21 23:03 vxconfigd

1421748 Aug 15 2001 vxconfigd.SunOS_5.6
1439184 Aug 15 2001 vxconfigd.SunOS_5.7
1499404 Nov 21 23:03 vxconfigd.SunOS_5.8
432736 Nov 21 17:18 vxdg
116100 Nov 21 17:18 vxdmpadm


400 -r-xr-xr-x
400 -r-xr-xr-x
1 root
1 root
bin
bin
195272 Aug 15
189800 Aug 15
2001 vxliccheck
2001 vxlicense
Note The vxconfigd file has Solaris OE architectural requirements

similar to the driver binaries in the /kernel/drv directory.
The /usr/sbin Directory

The /usr/sbin directory contains all other VxVM software commands
and utilities, as shown in Table 1-1.
Table 1-1 VxVM Software Commands and Utilities
vxassist
vxclust
vxconfigd
vxdco
vxdctl
vxddladm
vxdg
vxdisk
vxdiskadd
vxdiskadm
vxdiskconfig
vxdmpadm
vxedit
vxibc
vxinfo
vxinstall
vxiod
vxliccheck
vxlicense
vxmake
vxmend
vxnetd
vxnotify
vxplex
vxprint
vxrecover
vxrecover.wrap
vxrelayout
vxrlink
vxrvg
vxsd
vxspcshow
vxstart_vvr
vxstat
vxtask
vxtrace
vxvol
The /etc/init.d Directory

The /etc/init.d directory contains the following VxVM software
startup scripts:
vmsa-server
volmgt
vxnm-host_infod

1-13
vxnm-vxnetd
vxvm-reconfig
vxvm-recover
vxvm-shutdown
vxvm-startup1
vxvm-startup2
vxvm-sysboot
The /etc/rc*.d Directories

The /etc/rc*.d directories contain the hard links to the VxVM software
startup scripts in /etc/init.d, as follows:
The /etc/rc0.d directory contains the following links:
K10vmsa-server
K99vxvm-shutdown
The /etc/rc1.d directory contains the K10vmsa-server link.
The /etc/rcS.d directory contains the following links:
S35vxvm-startup1
S85vxvm-startup2
S86vxvm-reconfig
The /etc/rc2.d directory contains the following links:
S94vxnm-host_infod
S94vxnm-vxnetd
S95vxvm-recover
S96vmsa-server
The VxVM software startup sequence during a system boot is covered in

Module 5, Recovering Boot and System Processes.
1-14

The /opt Directory

The /opt directory contains the VxVM software man pages, documents,
the Storage Administrator GUI binaries, and licensing utilities. The /opt
directory contains the following:
VRTS:
man
VRTS/man:
man1m man3x man4
man7
VRTSlic:
bin
VRTSlic/bin:
vxliccheck vxlicense
VRTSvxvm:
docs
VRTSvxvm/docs:
admin.pdf
hwnotes.pdf
igbook.pdf
tshoot.pdf
admin.ps
hwnotes.ps
igbook.ps
tshoot.ps
VRTSspt:
FS
README.VRTSspt VRTSexplorer
VVR
VRTSspt/FS:
MetaSave VxBench
VRTSspt/VRTSexplorer:
README
fw
salr
vcs
VRTSexplorer
gcm
samba
vfr
bin.SunOS
isis
spc
visnC
dbed
lib
spcs
visnS
dbed1
main.SunOS
spnas
vmsa
edition.dro
ndmp
sybed1
vnas
edition.sybed sal
txpt
vras
VRTSvmsa:
bin
jre
vmsa
VRTSvmsa/bin:
autostart
gen_params
pkg_params
server_params
VRTSvmsa/jre:
bin
COPYRIGHT
jre_config.txt lib
VRTSvmsa/vmsa:
java
os_properties properties
server
vmsaguide.pdf
vmsaguide.ps
vrtsisp
vsap
vvr
vxfs
vxld
vxvm
vmsa
vmsa_server
LICENSE.ps

1-15
The /etc/vx Directory

The /etc/vx directory holds hundreds of files and subdirectories that are
used to hold key information about the VxVM software and the server
configuration on which it is installed.
Note Due to the large number of files, this section lists those files and
directories of most importance to the recovery of the VxVM software. A
full list of files is available in Appendix A.
Some of the files and directories in the /etc/vx directory are:
The /etc/vx/reconfig.d directory holds the files and
subdirectories, shown in the following list output, that are used to
define the VxVM softwares present and prior configurations.
./vx/reconfig.d:
total 18
2 drwxr-xr-x
2 drwxr-xr-x
2 drwxr-xr-x
2 -rw-r--r-2 drwxr-xr-x
6
8
3
1
2
1
3
1
2
root
root
root
root
root
root
root
root
root
sys
other
other
other
sys
other
root
other
sys
512
512
512
18
512
7
512
54
512
Mar
Mar
Mar
Mar
Mar
Mar
Mar
Mar
Mar
29
29
29
29
29
29
29
29
29
16:06
16:08
16:01
16:01
15:41
16:02
17:15
16:02
17:16
.
..
disk.d
enclrs
org.d
OTHER_DISKS
saveconf.d
SENA0
state.d
There are three files in this directory that describe disks under VxVM
software control:
enclrs This file lists the files in the directory that identify
disks detected by the VxVM software. An example is shown
here.
::::::::::::::
enclrs
::::::::::::::
OTHER_DISKS
SENA0
OTHER_DISKS This file lists all non-enclosure disks, as seen in

this example.
::::::::::::::
OTHER_DISKS
::::::::::::::
c0t1d0
1-16

SENA0 This file lists all disks in the enclosure named SENA0
detected by the VxVM software device discovery. An example
of the SENA0 file is shown here.
::::::::::::::
SENA0
::::::::::::::
c2t25d0
c2t16d0
c2t26d0
c2t10d0
c2t5d0
c2t0d0
c2t19d0
The /etc/vx/reconfig.d/disk/d directory lists subdirectories that

hold pre- and post-encapsulation information for encapsulated disks,
as seen in these list outputs.
./vx/reconfig.d/disk.d:
total 6
2 drwxr-xr-x
3 root
2 drwxr-xr-x
6 root
2 drwxr-xr-x
2 root
other
sys
other
./vx/reconfig.d/disk.d/c0t0d0:
total 12
2 drwxr-xr-x
2 root
other
2 drwxr-xr-x
3 root
other
2 -rw-r--r-1 root
other
2 -rw-r--r-1 root
other
2 -rw-r--r-1 root
other
2 -rw-r--r-1 root
other
512 Mar 29 16:01 .

512 Mar 29 16:06 ..
512 Mar 29 16:01 c0t0d0
512 Mar
512 Mar
9 Mar
933 Mar
7 Mar 29
452 Mar
29 16:01 .
29 16:01 ..
29 16:01 dmname
29 16:01 newpart
16:01 primary_node
29 16:01 vtoc
The ./c0t0d0 directory contains files that describe present and prior
configuration information about the encapsulated boot disk. The file
named vtoc holds the boot disks pre-encapsulation vtoc and can be
used to recover the original configuration for the boot disk if
unencapsulation fails.
Note The procedure for using the vtoc file to unencapsulate the boot
disk is addressed in Module 2, Encapsulating Disks.

1-17
The /etc/vx/reconfig.d/saveconf.d directory has a single

subdirectory, ./etc, that contains dump device information as well
as a current copy of the /etc/system file. These directories are
shown in list output.
./vx/reconfig.d/saveconf.d:
total 6
2 drwxr-xr-x
3 root
2 drwxr-xr-x
6 root
2 drwxr-xr-x
2 root
root
sys
root
./vx/reconfig.d/saveconf.d/etc:
total 14
2 drwxr-xr-x
2 root
root
2 drwxr-xr-x
3 root
root
2 -rw-r--r-1 root
root
2 -rw-r--r-1 root
root
2 -rw-r--r-1 root
root
4 -rw-r--r-1 root
other
512 Mar 29 17:15 .

512 Mar 29 16:06 ..
512 Mar 29 16:08 etc
512
512
239
239
140
2002
Mar
Mar
Mar
Mar
Mar
Mar
29
29
29
29
29
29
16:08
17:15
16:08
16:08
16:08
15:41
.
..
dumpadm.conf.new
dumpadm.conf.orig
dumpadm.out.orig
system
The /etc/vx directory also holds a file called /etc/vx/volboot.

This is the VxVM software bootstrap file. It is an ASCII file that
adheres to a very strict format and should not be edited. This file has
the following characteristics:
It is 512 bytes in length, including padding.
Update this file using the vxdctl command.
The volboot file holds the VxVM software host identifier

hostid. This is usually the Solaris OE node name, not the
hardware hostid. Keep in mind the following concepts:
1-18
The VxVM software hostid does not have to match the

servers node name, which can cause confusion.
The hostid is used to establish disk and disk group

ownership.
If two or more servers can access the same disks using the
same bus, the VxVM software hostid ensures that the two
hosts do not interfere with each other when accessing the
VxVM software disks.
The volboot file can also contain a list of simple disks for
rootdg. Refer to the VxVM Software Disks on page 1-21 for
more information.

The /var/vxvm Directory

The /var/vxvm directory contains a subdirectory called tempdb that holds
copies of imported disk group configuration databases. There is a file for
each disk group, as shown in this list output.
/var/vxvm
total 6
2 drwxr-xr-x
2 drwxr-xr-x
2 drwxr-xr-x
/var/vxvm/tempdb
total 34
2 drwxr-xr-x
2 drwxr-xr-x
18 -rw-r--r-6 -rw-r--r-6 -rw-r--r--
3 root
31 root
2 root
2
3
1
1
1
root
root
root
root
root
sys
sys
root
root
sys
root
root
root
512 Mar 29 17:15 .

512 Mar 29 15:41 ..
512 Mar 29 17:15 tempdb
512
512
8704
2560
3072
Mar
Mar
Mar
Mar
Mar
29
29
29
29
29
17:15
17:15
17:15
17:15
17:15
.
..
DGa
DGb
rootdg
Caution This directory must be present at boot time or the VxVM

software does not start. If the boot disk is under VxVM control, the
system does not boot. Do not remove this directory.
The /dev/vx Directory

The /dev/vx directory contains the logical volume device files that are
used to access the VxVM software objects. These files are shown here in a
list output.
/dev/vx
clust
config
dmp
dmpconfig
dsk
info
iod
netiod
rdmp
rdsk
task
taskmon
trace
Further explanation of these device files is covered later in the

Examining VxVM Software Objects on page 1-20.

1-19
Examining VxVM Software Objects

The VxVM software manipulates physical and virtual devices which
enable the VxVM software storage management operations, as follows:
Physical objects Physical objects are physical storage devices that

present raw or block device interfaces to the Solaris OE.
Virtual objects The VxVM software builds virtual objects from

physical storage objects that are brought under the VxVM software
control. The VxVM software virtual objects are combined into
volumes which are then formatted and mounted for use by
applications and users.
All objects that are not physical objects are virtual objects.
Figure 1-4 illustrates the relationship between physical and virtual VxVM
software objects.
Volume
Physical Objects
Physical Disks
Virtual Objects
Disk Group
VxVM
Software
Private
Plex
Plex
Plex
Subdisk
Plex
Subdisk
Subdisk
Subdisk
Subdisk
Public
VxVM
Software
Private
Subdisk
Public
Figure 1-4
1-20
VxVM Software Objects

Physical Disks
Physical disks are storage devices where data is ultimately stored.
Physical disks, or physical objects, are identified by the Solaris OE using a
unique identifier called a ctd number. Valid ctd identifiers are:
c The system controller or host bus adapter number
t A Small Computer System Interface (SCSI) target identifier
d A device or logical unit number
s A slice or partition
The VxVM software uses a drive ctd number for identification of the
physical device when it is brought under VxVM software control.
VxVM Software Disks

When a physical disk is brought under VxVM software control, it is either
initialized or encapsulated.
Initialized Disks
Initialized disks are reformatted with either one or two partitions, and all
data is destroyed. The partitions are used to store the VxVM software
configuration and data areas called private and public regions.
The private region is a small partition where disk group configuration
information is stored. The private region has the following characteristics:
It is usually slice 3.
Region size starts at 1024 sectors in early versions of the VxVM

software, and 2048 sectors in version 3.2.
Use the vxdisksetup command privlen option to expand the size
of the private region. This is a complicated procedure and is not
recommended. The procedure for expanding the private region is
addressed in a lab exercise.
It is assigned vtoc tag number 15 for identification purposes.
This region is not used for data storage.

1-21
The private region contains the following information:
Disk name and indentifier
Disk group name and identifier
Disk group configuration copy
The public region uses the remaining space available on the physical disk
to store subdisks. The public region has the following characteristics:
It is usually slice 4.
It is used for data storage.
The region is maintained by the VxVM software commands.
It is assigned vtoc tag number 14 for identification purposes.
Initialized disks can be configured in one of three methods:
The sliced configuration Public and private regions are defined as

separate Solaris OE partitions. This is the preferred method for
initializing a VxVM software disk.
The simple configuration Public and private regions are defined as

a single Solaris OE partition.
These are volatile devices and disappear after a reboot if they are not
in use or defined in the /etc/vx/volboot file.
The nopriv configuration The disk is configured without a private

region. Configuration information is held and maintained by a sliced
disk acting as a proxy.
These disks are not automatically discovered at bootup unless
defined in the /etc/vx/volboot file.
Note The nopriv configuration is not supported in future releases.

Avoid using this configuration method.
1-22

Encapsulated Disks
Encapsulation brings a physical disk under the VxVM software control
and preserves the data. The /etc/vfstab file is modified to reflect the
new volume names on the disks file systems. Encapsulation has the
following characteristics:
The /etc/vfstab file requires free space on the disk, usually in the
private region, to store configuration information.
The file size starts at 1024 sectors to the VxVM software version 3.2,
and 2048 sectors in version 3.2 and later.
Encapsulation fails if there is not enough free space to build a private

region. If space is not available for a private region, use the nopriv
option.
All partitions on the disk are reassigned to a new public region,

which is usually slice 6.
Boot disk can be encapsulated and remain bootable.
Note Avoid nopriv configurations. Support for this configuration is

being phased out.
Encapsulation, including boot disk encapsulation, is covered later in this
course.
Disk Groups
A named collection of VxVM software disks that share a common
configuration is called a disk group. Common configuration refers to a set of
records that provide detailed information about related VxVM software
objects, their connections and attributes. This configuration information is
stored in the private region of the VxVM software disks. A backup copy
for each configured disk group is stored in /var/vxvm/tempdb.
Disk groups are virtual objects and have the following characteristics:
The default disk group is rootdg.
Additional disk groups can be created on the fly.
Disk group names are a maximum of 31 characters in length.
Disk groups can be renamed.
Disk groups are versioned.

1-23
Disk groups allow grouping of the VxVM software disks into logical
collections.
Disk groups can be moved from one system to another with an

import and deport process.
Volumes created within a specific disk group can use only the VxVM
software disks that are a member of that disk group.
Volumes and disks can be moved among disk groups.
Note Moving volumes and disks among disk groups using an early
version of the VxVM software was a risky procedure. The VxVM software
version 3.2 has new options for the vxdg command to move the VxVM
software objects among disk groups and to split and join disk groups.
These new options require a special license.
Subdisks
Subdisks are contiguous blocks of space. Subdisks provide the basic
building blocks for the VxVM software plexes and volumes, creating the
VxVM software basic unit of storage allocation. Subdisks are virtual
objects.
The following characteristics apply to subdisks:
Subdisk storage is allocated from the public region of a VxVM

software disk.
A VxVM software disk can be subdivided into one or more subdisks.

Multiple subdisks cannot overlap.
Space on a VxVM software disk not allocated to a subdisk is

considered free space.
Subdisk names are based on the VxVM software disk name where they
reside, appended with an incremental numeric identifier. Figure 1-5 on
page 1-25 illustrates how subdisk names are derived.
1-24

Physical Disk
VxVM Software Disk
Subdisks
Private Region
Public Region
disk01-01
disk01-01
/dev/rdsk/c2t3d5
disk01-02
disk01-02
disk01-03
Free Space
disk01-03
disk01
Figure 1-5
Subdisk Naming Scheme
Plexes
The VxVM software virtual objects built from subdisks are called plexes. A
plex consists of one or more subdisks located on one or more VxVM
software disks.
Plexes are:
Also known as submirrors
Mid-tier building blocks of the VxVM software volumes
Named based on the name of the volume for which it is a submirror,

plus an appended incremental numeric identifier
Organized using the following methods:
Concetanation
Stripe (RAID 0)
Mirror (RAID 1)
Striping with parity (RAID 5)

1-25

Figure 1-6 illustrates the architectural components of a plex.
VxVM Software Disk
Plex
Private Region
Public Region
disk01-01
disk01-01
disk01-02
disk01-02
disk01-03
disk01-03
vol01-01
Free Space
disk01
Figure 1-6
Subdisks
This is the first

plex of volume
vol01.
Volume name
This plex is a
sub-mirror.
Plex Components
Note Plex states are covered in detail in the section Resynchronizing

Volumes on page 1-29. Additional plex state information is found in
Appendix E, Configuring the VxVM Software.
Volumes
Volumes are virtual devices (virtual objects) that appear to be physical
disks to applications, databases and file systems. Volumes are the VxVM
softwares top-tier virtual objects. Although volumes appear to be
physical disks, they do not share the limitations of physical disks.
Volumes have the following characteristics:
1-26
Volumes are directly addressed by the Solaris OE.
They consist of one or more plexes. Each plex holds one copy of the
volumes data.
Volumes are not restricted to a single disk or specific areas of disks.

The configuration of a volume can be changed using the VxVM

software utilities. Configuration changes can be accomplished
without disruption to the volume operations.
Volume names can contain up to 31 characters.
Volumes can consist of up to 32 plexes.

Each plex can contain multiple subdisks, and all subdisks must be in
the same disk group.
Figure 1-7 illustrates a volume with two plexes.

Plex
Volume
disk01-01
disk01-02
disk01-03
vol01-01
disk02-01
disk02-02
disk01-01
disk01-02
disk02-01
disk02-02
disk01-03
disk02-03
vol01-01
vol01-02
vol01
disk02-03
vol01-02
Figure 1-7
Two-Plex Volume
Note Detailed volume layouts are described in Appendix E,

Configuring the VxVM Software.
VxVM Software Layered Volume Objects

With the VxVM software version 3.0, volumes can now be constructed by
mapping a volumes subdisks to underlying volumes and using those
volumes to build volumes. Volumes constructed in this manner are virtual
objects called layered or professional volumes.

1-27

Layered volumes are used to implement RAID 1 + 0 or RAID 10 virtual
devices. RAID 1 + 0 devices are striped, mirrored volumes and provide
for a more fault-resilient volume configuration. Implementing these
layered volumes require two new VxVM software objects: subplexes and
subvolumes.
Figure 1-8 illustrates the how a layered volume differs from a standard
volume.
Volume
Volume
Plex
Mirror
Plex
Mirror
Plex
SD
SD
SD
SD
SD
SD
SD
Subplex
Subplex
SD
SD
Subvolume
Stripe
Stripe
SD
Subvolume
Mirror
Subplex
Subplex
SD
SD
RAID 0+1
Subvolume
Mirror
Subplex
Subplex
SD
SD
RAID 1+0
Figure 1-8
Layered Volumes
Layered volumes require more VxVM software objects than standard

volumes. A standard volume requires 9 objects. A layered volume of the
same capacity requires 17 objects. Large disk groups using layered
volumes can exceed the alloted space for the disk groups private region.
1-28

Resynchronizing Volumes
The VxVM software ensures that all data stored redundantly using
mirrored or RAID 5 volumes remains in a consistent state. Data is written
in parallel to both mirrored and RAID volumes to ensure that data
remains consistent unless there is a system crash or physical disk failure.
If this occurs, data can become inconsistent or unsynchronized.
System failures are not the only reason data can become unsynchronized.
Data can become inconsistent during maintenance procedures when a
mirrored plex or RAID 5 element is taken offline. If data becomes
inconsistent between mirrored plexes or between a RAID 5 volumes data,
use a volume resynchronization to correct the problem.
Volume resynchronization happens for some volumes during a system
reboot. Other volumes undergo volume resynchronization when they are
started. Plexes that are taken offline for maintenance become stale. Stale
plexes are resynchronized when configured back into a volume.
The components of volume resynchronization are discussed in detail in
this section.
Dirty Flag
The VxVM software keeps track of data synchronization operations using
a flag called the dirty flag. When data is written to a volume, it is marked
as dirty until the volume stops or all writes are completed and the data in
the volume plexes are identical.
Volumes that do not have the dirty flag reset require volume
resynchronization when started or during a system reboot.

1-29
Resynchronization Process
The resynchronization (resync) process depends on the type of volume
started.
During resync of a RAID 5 volume, if a log is available, the log is
replayed. If a log is not available, the volume is placed in reconstructrecovery mode, and all parity is regenerated. This is a very time
consuming recovery process. Make sure that all RAID 5 volumes have a
log attached to reduce the time it takes to resync a RAID 5 volume.
During resync of a mirrored volume, the volume is placed into recovery
mode (or read-writeback recovery mode). Data is available for use by the
Solaris OE. This type of recovery is executed in the background and is
very time consuming. Attaching a dirty region log (DRL) to mirrored
volumes and using a fast resync operation can help speed up the recovery
process.
The resync process can be expensive and have an adverse impact on
system resources. The VxVM software reduces the impact multiple
resyncs have on the system by managing the recovery operations to avoid
placing stress on specific disks and controllers.
1-30

Introducing VxVM Software Logging

The VxVM software transaction log facility provides a method of
decreasing the amount of time it takes to perform volume
resynchronization. This is accomplished by attaching a log plex to a
volume and using that plex to save transaction recovery information.
There are two types of logs: RAID 5 and DRL. This section contains a
detailed description of these logs.
RAID 5 Logs
RAID 5 logs perform the following tasks:
RAID 5 logs protect data and parity calculations from system

crashes.
Degraded RAID 5 volumes are not restarted during a reboot; they
must be started manually. Attaching a RAID 5 log does not alter this
fact.
RAID 5 parity is only used when data needs to be rebuilt. It is only

written and never checked.
It is advised to run the vxr5check utility on a regular basis to check
parity.
RAID 5 logs appear as a plex with a plex state of Log.
Configure the RAID 5 logs as follows:
Create a RAID 5 log that is large enough to allow multiple

concurrent accesses to the volume. Make the log several times the
stripe size of a RAID 5 plex.
It is possible to configure a RAID 5 volume without a log, but it is

highly discouraged. It is not a a good practice to configure RAID 5
volumes without logs.
RAID 5 volumes without logs are subject to silent parity errors in the
event of a system crash.
Sun StorEdge documentation recommends using two RAID 5 logs

per RAID 5 volume.
Using two logs can have a significant performance impact on a
volume type that already has performance problems. Most sites only
use one log.

1-31

The following is an example of a RAID 5 volume with an attached log:
Disk group: ddg
DG
DM
V
PL
SD
SV
NAME
NAME
NAME
NAME
NAME
NAME
NCONFIG
DEVICE
USETYPE
VOLUME
PLEX
PLEX
NLOG
TYPE
KSTATE
KSTATE
DISK
VOLNAME
MINORS
PRIVLEN
STATE
STATE
DISKOFFS
NVOLLAYR
GROUP-ID
PUBLEN
LENGTH
LENGTH
LENGTH
LENGTH
STATE
READPOL
LAYOUT
[COL/]OFF
[COL/]OFF
PREFPLEX
NCOL/WID MODE
DEVICE
MODE
AM/NM
MODE
v
pl
sd
sd
sd
pl
sd
r0
r0-01
ddg01-01
ddg02-01
ddg03-01
r0-02
ddg04-01
RAID-5
r0
r0-01
r0-01
r0-01
r0
r0-02
ENABLED
ENABLED
ddg01
ddg02
ddg03
ENABLED
ddg04
ACTIVE
ACTIVE
0
0
0
LOG
0
20480
21568
10800
10800
10800
4320
4320
RAID
RAID
0/0
1/0
2/0
CONCAT
0
3/32
c1t2d0
c1t2d1
c1t3d0
c1t4d0
RW
ENA
ENA
ENA
RW
ENA
Dirty Region Logs (DRLs)

Dirty region logging provides a facility that helps speed the process of
resynchronizing a mirrored volume after a system crash. The DRL process
provides a bit map image of the data blocks available in a volume and
sets the dirty bit whenever data is written to a specific block. If a system
crash occurs, only the dirty blocks are resynchronized. If the DRL is not
used, all mirrors must be returned to a consistent state through a full
resynchronization of the volume.
Dirty region logging performs the following tasks:
It provides faster resynchronization in the event of a system crash or

abnormal system termination only.
In case of a system crash, only the dirty blocks are resynchronized.
The logging does not perform the same function as the Solstice
DiskSuite softwares dirty mirror region log. Dirty region logging
does not enable partial resynchronization if a plex is taken offline for
maintenance.
Partial resynchronization is handled by the fast mirror resync (FMR)
facility. FMR requires a separate license.
1-32
Dirty region logging requires additional system I/O overhead.
A DRL is relatively small.

DRLs are not supported on core system volumes, such as the /, /usr,
and /var volumes.
Dirty region logging is usually implemented as a log plex.
The following is an example of a mirrored plex with an attached log:

Disk group: ddg
DG
DM
V
PL
SD
SV
NAME
NAME
NAME
NAME
NAME
NAME
NCONFIG
DEVICE
USETYPE
VOLUME
PLEX
PLEX
NLOG
TYPE
KSTATE
KSTATE
DISK
VOLNAME
MINORS
PRIVLEN
STATE
STATE
DISKOFFS
NVOLLAYR
GROUP-ID
PUBLEN
LENGTH
LENGTH
LENGTH
LENGTH
STATE
READPOL
LAYOUT
[COL/]OFF
[COL/]OFF
PREFPLEX
NCOL/WID MODE
DEVICE
MODE
AM/NM
MODE
v
pl
sd
pl
sd
pl
sd
v
pl
sd
sd
pl
sd
m0
m0-01
ddg04-02
m0-02
ddg05-01
m0-03
ddg04-03
m1
m1-01
z
ddg01-02
m1-02
ddg02-02
fsgen
m0
m0-01
m0
m0-02
m0
m0-03
fsgen
m1
m1-01
m1-01
m1
m1-02
ENABLED
ENABLED
ddg04
ENABLED
ddg05
ENABLED
ddg04
ENABLED
ENABLED
ddg05
ddg01
ENABLED
ddg02
ACTIVE
ACTIVE
4320
ACTIVE
4320
ACTIVE
25920
ACTIVE
ACTIVE
0
10800
ACTIVE
10800
20480
21600
21600
21600
21600
LOGONLY
5
20480
21600
4320
21600
21600
21600
SELECT
CONCAT
0
CONCAT
0
CONCAT
LOG
SELECT
CONCAT
LOG
0
CONCAT
0
c1t4d0
c1t4d1
c1t4d0
c1t4d1
c1t2d0
c1t2d1

RW
ENA
RW
ENA
RW
ENA
RW
ENA
ENA
RW
ENA
1-33
Examining Plex States

Plex states change a part of the normal volume operations and do not
necessarily indicate a problem. When a plex is activated as part of a
volume being started or due to an error-recovery procedure, the targeted
plex progresses through a pre-defined set of plex states. When a plex is
taken offline due to an error and a volume resynchronization is initiated,
all plex data is resynchronized, allowing the volume to return to the
synchronized state. During the resynchronization process, the out-of-sync
plexes move through various plex states until the operation completes.
Plex states are automatically maintained by VxVM software utilities
which:
Indicate the state of the volumes contents
Determine if a plex content is valid
Track whether a plex was actively in use at the time of a system

failure
Monitor plex operations
Plex states are discussed in detail in this section.
Plex State Descriptions

Plex states refer to the states the plex transitions through to ensure they
contain a complete and consistent copy of a volumes contents. Plexes
associated with a volume can have one of the following states:
Empty When a volume is created and the plex is not initialized, the
plex is in an empty state.
Clean A plex is in a clean state when it is known to contain a good

copy (mirror) of the volume. If all the plexes of a volume are clean,
no action is required.
Active A plex is in the active state when the volume is started and
the plex fully participates in normal volume I/O (meaning the plex
contents change as the contents of the volume change).
A plex state is also active when the volume is stopped as a result of
a system crash and the plex was active at the moment of the crash. In
this case, a system failure can leave plex contents in an inconsistent
1-34


state. When a volume is started, the VxVM software performs a
recovery action to guarantee that the contents of the plexes are
identical and are marked as active.
Note The active state is the most common state for plexes on a wellrunning system.
Stale If there is a possibility a plex does not have the complete and
current volume contents, this plex is placed in a stale state.
Additionally, if I/O errors occur on a plex, the kernel stops using
and updating the plex, and the operation sets the state of the plex to
stale.
To reattach the stale plex to the volume, synchronize the data

and set the plex to the active state, use the following command:
*vxplex -g (diskgroup) att volumename plexname*
To force a plex into Stale state, use the following command:
*vxplex -g (diskgroup) det plexname*
Offline The following command detaches a plex from a volume

setting and changes the plex state to offline:
*vxmend -g (diskgroup) off plexname*
Although the detached plex is associated with the volume, the
changes to the volume are not reflected to the plex while it is in the
offline state.
To set the plex state to stale and start to recover data after the
vxvol start operation, use the following command:
*vxplex -g (diskgroup) att volumename plexname*
Temp A utility sets the plex state to temp at the start of an

operation, and also sets the plex to an appropriate state at the end of
the operation.
For example, attaching a plex to an enabled volume requires copying
volume contents to the plex before it can be fully attached. If the
system goes down for any reason, a temp plex state indicates the
operation is incomplete; a subsequent vxvol operation starts to
disassociate plexes in the temp state.

1-35
Temprm A temprm plex state resembles a temp state with the

exception that, upon completion of the operation, the temprm plex is
removed.
If the system goes down for any reason, a temprm plex state
indicates the operation is incomplete; a subsequent vxvol operation
starts to disassociate plexes and remove the temprm plex.
Temprmsd The temprmsd plex state is used by vxassist when

attaching new plexes. If the operation does not complete, the plex
and its subdisk are removed.
Iofail The iofail plex state is associated with persistent state

logging. On the detection of a failure of an active plex, the
vxconfigd operation places that plex in the iofail state to disqualify
it from the recovery selection process at volume start time.
Plex Kernel States

The plex kernel state indicates the accessibility of a plex to the volume
driver monitoring it. Valid plex kernel states are:
Disabled The plex may not be accessed.
Detached A write to the volume is not reflected to the plex. A read

request from the volume is never satisfied from the plex.
Plex operations and ioctl functions are accepted.
1-36
Enabled A write request to the volume is reflected to the plex. A

read request from the volume is satisfied from the plex.

General Plex State Cycle

Volumes are automatically started at system startup, and all clean plexes
are marked active by the vxvol start command. At system shutdown,
all plexes in the active state are marked clean by the vxvol stop
command. If all plexes are in the clean state, this is indicative of a
controlled system shutdown.
This cycle is illustrated in Figure 1-9.
Start Up (vxvol start)
PS: Clean
PKS: Disabled
PS
PS: Active
PKS: Enabled
= Plex state
PKS = Plex kernel state

Shut Down (vxvol stop)
Figure 1-9
General Plex State Cycle
Other Plex State Transitions

Additional plex states are needed to help transition plexes during
hardware error recovery, abnormal system shutdown, and system
administrator intervention.
Figure 1-10 on page 1-38 illustrates these state transitions.

1-37
Create plex
PS: Empty
PS: Active
PKS: Disabled
PKS: Disabled
After crash
and reboot
(vxvol start)
Initialize plex
Start up
Recover data
(vxvol init clean)
(vxvol start)
(vxvol resync)
Take plex offline

(vxmend off)
PS:Clean
PS: Active
PS: Offline
PKS: Disabled
PKS: Enabled
PKS: Disabled
Resync data
(vxplex att)
Shut down
(vxvol stop)
Put plex online

(vxmend on)
Uncorrectable
I/O failure
PS: IOFAIL
PKS: Detached
PS
= Plex state
PKS
= Plex kernel state
Resync
PS: Stale
fails
PKS: Detached
Figure 1-10 Additional Plex State Transitions

An example of how these additional states are used to manage the
integrity of plexes is as follows:
1-38
1.
A new plex is created leaving the plex state (PS) as empty and the
plex kernel state (PKS) as disabled.
2.
The plex is initialized and transitions to the clean PS. It remains in

the disabled PKS.
3.
Once the volume is started, the plex transitions to the active PS and
to the enabled PKS. At this time, the plex is available for use by the
volume and stays in this state until an error occurs or the volume is
stopped.


When an uncorrectable error happens, the following plex states are used
to manage recovery:
1.
An un-correctable I/O failure occurs and the PS transitions to the

iofail state. The PKS transitions to the detached state.
2.
Repairs are affected and the plex is reattached to the volume. This
causes the execution of a data resync. Once the data in the plex is
updated, the PS transitions to active, and the PKS transitions to
enabled.
The plex is now usable by the volume.
Use the vxprint utility to view plex states and their transitions. If a small
plex is re-synching, vxprint might not show the transition states as they
may happen too fast for vxprint to report.
The following example shows a VM disk (rootmirror) that has an I/O
failure. The plex rootvol-02 has transitioned to a detached PKS and an
iofail PS.
bash-2.03# vxprint -htg rootdg
DG NAME
NCONFIG
NLOG
MINORS
DM NAME
DEVICE
TYPE
PRIVLEN
RV NAME
RLINK_CNT
KSTATE
STATE
RL NAME
RVG
KSTATE
STATE
V NAME
RVG
KSTATE
STATE
PL NAME
VOLUME
KSTATE STATE
SD NAME
PLEX
DISK
DISKOFFS
SV NAME
PLEX
VOLNAME NVOLLAYR
DC NAME
PARENTVOL
LOGVOL
SP NAME
SNAPVOL
DCO
GROUP-ID
PUBLEN
PRIMARY
REM_HOST
LENGTH
LENGTH
LENGTH
LENGTH
STATE
DATAVOLS SRL
REM_DG
REM_RLNK
READPOL
PREFPLEX
LAYOUT
NCOL/WID
[COL/]OFF DEVICE
[COL/]OFF AM/NM
dg rootdg
default
default
1023554924.1025.lowtide
dm rootdisk
dm rootmirror
c1t0d0s2
-
sliced
-
3590
-
17678493 IOFAIL
v
pl
sd
sd
pl
sd
rootvol
ENABLED ACTIVE
4197879 ROUND
rootvol-01
rootvol
ENABLED ACTIVE
4197879 CONCAT
rootdisk-B0 rootvol-01
rootdisk 17678492 1
0
rootdisk-02 rootvol-01
rootdisk 0
4197878 1
rootvol-02 rootvol
DETACHED IOFAIL 4197879 CONCAT
rootmirror-01 rootvol-02 rootmirror 0
4197879 0
c1t0d0
c1t0d0
-

UTYPE
MODE
MODE
MODE
root
RW
ENA
ENA
RW
RLOC
1-39
Introducing Supported VxVM Software Version 3.2 Features
Introducing Supported VxVM Software Version 3.2

Features
The following section describes new features introduced with the release
of the VxVM software version 3.2 that are supported by Sun
Microsystems.
Device Discovery
Device discovery is separated from the base VxVM software functionality
into a separate layer. Previously, the VxVM software discovered block
storage devices by scanning the Solaris OE device tree using the vxiod
daemon. This strategy assumed that the Solaris OE device tree remained
static or changed very little. When changes occurred, command line
utilities were used to update the VxVM software configuration.
Additionally, dynamic multi-pathing (DMP) must be aware of all multipathed storage subsystems and know the type of storage subsystem to
enable DMP support. Prior to the VxVM software version 3.2, support for
new arrays would necessitate changes to the DMP device discovery code.
This required patching the VxVM software and undergoing a probable
system outage.
With the growth of disk subsystem vendors and storage area networks
implemented within storage environments, the previous strategy proved
to be inadequate. The current device discovery facility is designed to
allow the dynamic addition of new storage subsystems without
modification to the VxVM software modules. The VxVM software DDL
discovers all disks that are visible to the Solaris OE.
Device Discovery Layer

The DDL is part of the vxconfigd daemon, and performs the following:
1-40
Discovers all block storage devices connected to a host
Probes using SCSI commands from user space to determine the

following:
Type of disks
Number of paths

Attributes
Executes from the vxdisk utility to perform a vxdisk scan

command
Figure 1-11 illustrates DDL components and the relationship to the VxVM
software kernel.
Applications
Data Management Layer

File System
Database
VxVM
Configuration
Database
VxVM Storage Management Subsystem
DMP Driver
Storage Device Drivers
Kernel
Enclosure/Array
vxconfigd
DDL
Figure 1-11 Device Discovery Layer
Addition of New Devices

Libraries are included with the VxVM software for the most popular
brands of arrays. If a new array is created by a vendor, the VxVM
software only needs an updated device discovery library to recognize the
new array. The new library can be added dynamically.

1-41

Support can be added dynamically for the following types of disk arrays:
Active/Active
Active/Passive (Autorepass mode)
Active/Passive (LUN group failover)
JBODs
JBOD Support
The device discovery facility can detect multi-pathed disk storage devices
that do not belong to a disk array but are capable of being multi-pathed
by DMP. Use the vxddladm utility to add or remove JBODs.
Disks must have a unique serial number that can be read through a SCSI
inquiry or the mode_sense command to be detected correctly.
DDL Administration
The device discovery facility is managed by the vxddladm utility to
provide a high-level function for device discovery and configuration. It is
used to:
Add JBODs
List supported JBODs
Remove JBODs
Include arrays
Exclude arrays
List supported arrays
List excluded arrays
A sample output from the execution of the vxddladm command with the
listsupport option is shown here. This option lists all disk enclosures
currently supported by the systems DDL.
bash-2.03# vxddladm listsupport
LIB_NAME
ARRAY_TYPE
VID
PID
=======================================================================
libvxap.so
A/A
SUN
AP_NODES
libvxatf.so
A/A
VERITAS
ATFNODES
libvxeccs.so
A/A
ECCS
all
libvxemc.so
A/A
EMC
SYMMETRIX
libvxhds.so
A/A
HITACHI
OPEN-*
libvxhitachi.so
A/PG
HITACHI
DF350
1-42


libvxhitachi.so
libvxhitachi.so
libvxlsiinf.so
libvxnec.so
libvxnec.so
libvxnec.so
libvxnec.so
libvxnec.so
libvxnec.so
libvxnec.so
libvxnec.so
libvxnec.so
libvxnec.so
libvxnec.so
libvxnec.so
libvxnec.so
libvxnec.so
libvxnec.so
libvxnec.so
libvxnec.so
libvxpurple.so
libvxrdac.so
libvxsena.so
libvxshark.so
libvxssa.so
libvxstorcomp.so
libvxveritas.so
libvxxp256.so
libvxfujitsu.so
libvxfujitsu.so
libvxfujitsu.so
libvxfujitsu.so
libvxfujitsu.so
libvxfujitsu.so
A/PG
A/PG
A/P
A/A
A/A
A/A
A/A
A/A
A/A
A/A
A/A
A/A
A/A
A/A
A/A
A/A
A/A
A/A
A/A
A/A
A/P
A/A
A/A
A/A
A/A
A/A
A/PF
A/A
A/A
A/A
A/A
A/A
A/A
A/A
HITACHI
HITACHI
LSI
NEC
NEC
NEC
NEC
NEC
NEC
NEC
NEC
NEC
NEC
NEC
NEC
NEC
NEC
NEC
NEC
NEC
SUN
VERITAS
SENA
IBM
SSA
StorComp
VERITAS
HP
FUJITSU
FUJITSU
FUJITSU
FUJITSU
FUJITSU
FUJITSU
DF400
DF500
INF-01-00
DS1200
DS1200F
DS3000SL
DS3000SM
DS3001
DS3002
DS1000
DS1000F
DS1100
DS1100F
DS3011
DS1230
DS450
DS450F
iStorage
iStorage
iStorage
T300
RDACNODES
all
2105
all
OmniForce
all
OPEN-*
GR710
GR720
GR730
GR740
GR820
GR840
1000
2000
4000
Note See the vxddladm man page for command syntax options.
Device Naming
Enclosure-based and operating system (OS)-independent naming changes
the way storage devices are identified by the VxVM software and
provides the following benefits:
Naming is independent of the OS.

This mitigates device naming confusion in multi-OS environments.

1-43
OS-independent naming relieves problems with some third-party

drivers that do not use the standard Solaris OE c#t#d# device
naming schema.
The VxVM software is now independent of arbitrary device names.
Enclosure-based naming allows for better location identification of

disks.
This leads to faster fault isolation.
In cluster environments, disk array names are the same on all nodes
in the cluster.
Device Name Format

The device name format is based on the name of the enclosure. Use the
following format to name devices:
logical_enclosure_name_#
Default disk names are based on the vendor identification (ID). For
example, disks in a Sun StorEdge T3 multi-path array named purple0 by
the VxVM software have the following names:
purple0_1
purple0_2
purple0_3
Device names have the following characteristics:
Logical names persist across reboots.
Logical enclosure names can be customized.
Disk names within enclosures are persistent as long as the disks

position within an enclosure remains static.
All fabric devices are displayed using the enclosure format.

During installation, an option allows the installer to display all nonfabric devices using the native open profiling standard (OPS) format.
Enabling and Disabling Naming Schemes

Enclosure-based naming is enabled or disabled as follows:
1-44
Enabled or disabled during installation.
Enabled or disabled using vxdiskadm option 20.

Enabled by executing the following commands:

# touch /etc/vx/.newnames
# vxconfigd -k
Disabled by executing the following commands:

# rm /etc/vx/.newnames
# vxconfigd -k
This example shows a system with enclosure-based naming enabled.

bash-2.03# ls /dev/vx/dmp
SENA0_0
SENA0_13s4 SENA0_5s2
SENA0_0s0
SENA0_0s1
SENA0_0s2
SENA0_0s3
SENA0_1s0
SENA0_5s6
SENA0_0s4
SENA0_1s1
SENA0_5s7
SENA0_0s5
SENA0_1s2
SENA0_6
SENA0_0s6
SENA0_1s3
SENA0_6s0
SENA0_0s7
SENA0_1s4
SENA0_6s1
...
c1t0d0
c1t0d0s0
c1t0d0s1
c1t0d0s2
c1t0d0s3
c1t0d0s4
c1t0d0s5
c1t0d0s6
c1t0d0s7
c1t19d0s5
c1t19d0s6
c1t19d0s7
c1t1d0
c1t1d0s0
c1t1d0s1
c1t1d0s2
c1t1d0s3
c1t1d0s4
c1t2d0s2
c1t2d0s3
c1t2d0s4
c1t2d0s5
c1t2d0s6
c1t2d0s7
c1t3d0
c1t3d0s0
c1t3d0s1
Notice that the dmp metanodes now use devices named SENA0_xyz. A
long listing of the directory shows that c1t0d0 and SENA0_0 are the same
device by comparing the major and minor numbers.
bash-2.03# ls -las
total 12
10 drwxr-xr-x
2
2 drwxr-xr-x
6
0 brw------1
...
0 brw------1
...
/dev/vx/dmp
root
root
root
other
other
other
5120 Jul 5 09:17 .

512 Jun 8 09:49 ..
68, 2 Jul 5 09:05 SENA0_0
root
other
68,
2 Jun
8 09:41 c1t0d0
An experienced system administrator can take the c#t#d# number and

cross-reference that to a physical device.
A sample vxdisk command listing shows that the VxVM software is
using enclosure-based names.
bash-2.03# vxdisk list
DEVICE
TYPE
DISK
SENA0_0
sliced
rootdisk
SENA0_1
sliced
SENA0_2
sliced
SENA0_3
sliced
SENA0_4
sliced
SENA0_5
sliced
SENA0_6
sliced
rootmirror
SENA0_7
sliced
-
GROUP
rootdg
rootdg
-
STATUS
online
offline
error
online
online
offline
online
online

1-45

SENA0_8
SENA0_9
SENA0_10
SENA0_11
SENA0_12
SENA0_13
sliced
sliced
sliced
sliced
sliced
sliced
datadg02
datadg01
-
datadg
datadg
-
online
online
online
online
online
online
Sun StorEdge Traffic Manager Software Support

The VxVM software version 3.2 is the first version of this software to
support the Sun StorEdge Traffic Manager software.
Explicit LUN Failover

The LUN failover feature enables systems that are using DMP in place of
the Sun StorEdge Traffic Manager software to manage path and LUN
failover functions.
To enable this support within the Sun StorEdge T3 array, configure the
following modifications in the mp_support file:
Single-host environment Set mp_support to rw.
Multi-host environment Set mp_support to std.
Ordered Storage Allocation

Ordered storage allocation is a new option to the vxassist utility. This
feature allows a specification in which subdisks are allocated for use when
building a volume. In early versions of the VxVM software (before
version 3.2) and before ordered allocation, an administrator could specify
which disks to use, but the VxVM software would allocate the order in
which those disks were used based on the softwares proximity rules.
The following example command shows how to use the ordered storage
allocation option:
# vxassist -g testdg -o ordered make vol01 1g layout=mirror-stripe ncol=3 disk02 disk04
disk06 disk01 disk03 disk05
1-46

Introducing Unsupported VxVM Software Version 3.2 Features
Introducing Unsupported VxVM Software Version 3.2

Features
The following features, although available in the VxVM software
version 3.2, are not supported by Sun Microsystems. These features
require an additional license.
Disk Group Split, Move, and Join

The disk group split, move, and join facility allows greater flexibility in
disk group management functions. These actions are described as follows:
Split Removes objects from an established disk group and creates a

new disk group from those objects
Move Moves objects from one disk group to another
Join Joins two disk groups
Persistent Fast Resync Function

The fast resync and persistent fast resync functions allow for fast recovery
of a mirrored volume that contains a plex that was taken offline for
maintenance. Fast resync uses data change objects (DCO) to keep track of
changes to a mirror. Persistent fast resync stores resynchronized maps on
a disk rather than in memory, which enables the maps to survive a system
reboot or crash.

1-47
Module 2
Encapsulating Disks
Objectives
Describe the encapsulation and unencapsulation processes for data

and boot (system) disks
Encapsulate a data disk
Unencapsulate a data disk
Encapsulate a boot disk
Recover the loss of the encapsulated disk in a boot disk mirrored pair
Describe best practice partition configurations for boot disk

encapsulation
Unencapsulate a boot disk, including a compact disc read-only

memory (CD-ROM)
Perform a successful vxunroot process on a boot disk mirror,

including a mirror that has a replaced encapsulated disk
Recover an encapsulated boot disk that failed the vxunroot process

2-1
Relevance
Relevance
!
?
2-2

the VxVM software supports the preservation (encapsulation) of existing
data on disks brought under the softwares control:
What is disk encapsulation?
How is disk encapsulation implemented?
What are the differences between encapsulating a data disk as

opposed to a system (boot) disk?
How does a user reverse the encapsulation process without having

to restore data?
Should the VxVM software be used to encapsulate a system boot

disk?
What are the supported slice configurations for a boot disk?
What process recovers an encapsulated boot disk that fails the

vxunroot utilitys attempt at unencapsulation?






http://storage.west
SunSolveSM Online Free Info Docs and Free Symptoms and Resolutions,
[http://sunsolve.Sun.COM/pub-cgi/search.pl?mode=
advanced].
INFODOC 16051 How to Encapsulate Disks With No Free

Space Using Volume Manager 22 March 2002. (See
Appendix A, SunSolve INFODOCs.)
INFODOC 24663 Full and Basic/Functional Unencapsulation

of a Volume Manager Encapsulated Root Disk While Booted
CDROM 22 March 2002. (See Appendix A, SunSolve
INFODOCs.)
INFODOCS 13775, 13781, 15838, and 19245.
Symptom and Resolution Database (SRDB) 19245.
VERITAS Software Corporation support knowledge base

TechNote ID 244678,
[http://seer.support.veritas.com/nav_bar/index.asp?
content_sURL=%2Fsearch%5Fforms%2Ftechsearch%2Easp].
Sun BluePrints OnLine part number 806-6197-10, Toward a

Reference Configuration for VxVM Managed Boot Disks, August, 2000.
[http://www.sun.com/solutions/blueprints/online.html].

Encapsulating Disks
2-3
Introducing Disk Encapsulation
Introducing Disk Encapsulation

The VxVM software uses two methods to bring disk storage devices
under control: initialization and encapsulation. Initialization is the process
of partitioning a data disk into two partitions for use as public and private
regions. This process destroys all data on the initialized disk. Initialized
disks are then used to build volumes. The encapsulation process provides
the VxVM software with the ability to manage a disk without destroying
data. Each partition on the encapsulated disk becomes a unique VxVM
software volume. Once encapsulated, other initialized VxVM software
disks can be used to provide mirrors for these partitions.
Appendix B, Disk Encapsulation Processes, provides an overview of the
different processes which the VxVM software performs involving disk
encapsulation.
Encapsulated Disk Types

There are two types of encapsulated disks, data disks (non-root) and
system (boot or root) disks. Data disks do not contain any system data
needed to boot the operating system. Boot disks do contain data that is
needed to successfully boot the operating system. Rootability is the VxVM
softwares ability to encapsulate system partitions that are usable by the
system. The VxVM software manages data and boot disks differently, as
described in this module.
Reasons to Encapsulate Disks

There are at least four good reasons to encapsulate a disk:
2-4
Increase the fault resiliency of data through mirroring
Grow the data storage area
Convert a data disk to a striped volume to increase performance
Provide a disk to populate the rootdg disk group

Encapsulating Data Disks

Data disk encapsulation involves changes to the encapsulated disks
partition table, and has the following requirements:
Two unused partitions in the partition table
A small amount of unallocated disk space (2 cylinders) at the

beginning or end of the disk
Slice 2 representing the whole disk
If these requirements are not met, encapsulation is a more difficult

procedure as described in Encapsulating a Non-Conforming Disk on
page 2-20.
When the encapsulation process is finished, the encapsulated disk has two
partitions. These are usually partition 6, used for the public region, and
partition 7, used for the private region. Figure 2-1 illustrates this
partitioning.
Data Disk
Encapsulated Data Disk
Slice 0
Public
Slice 2
Slice 1
Slice 3
Region
(Slice 6)
Slice 2
and
Slice 6
Slice 4
Slice 7
Free Space
Figure 2-1
Private Region
Encapsulation Partition Scheme
Although the VxVM software re-partitions the disk, the original data
resides in the same blocks they occupied prior to encapsulation. The data
is not moved or overwritten. The VxVM software is responsible for
performing the translation necessary to make the data available as VxVM
software volumes.

Encapsulating Disks
2-5
Pre-Encapsulation Configuration Data

The following are example configuration data from a system prior to
installation of the VxVM software and encapsulation of a data disk. The
state of this data prior to the encapsulation process is discussed in this
section.
Note In some examples of output presented in this module, columns are
aligned and spaces added to aid clarity.
Pre-Encapsulation /etc/vfstab File

The /etc/vfstab file displays the file systems to be encapsulated on a
disk. This example of data disk encapsulation uses disk c1t3d0 as the
target disk.
#device
#to mount
#
#/dev/dsk/c1d0s2
fd
/proc
/dev/dsk/c1t0d0s1
/dev/dsk/c1t0d0s0
swap
#
# Storage
#
/dev/dsk/c1t3d0s0
/dev/dsk/c1t3d0s1
/dev/dsk/c1t3d0s3
/dev/dsk/c1t3d0s4
device
to fsck
mount
point
/dev/rdsk/c1d0s2
/dev/rdsk/c1t0d0s0
-
/dev/rdsk/c1t3d0s0
/dev/rdsk/c1t3d0s1
/dev/rdsk/c1t3d0s3
/dev/rdsk/c1t3d0s4
/usr
/dev/fd
/proc
/
/tmp
/fs1
/fs2
/fs3
/fs4
ufs
ufs
ufs
ufs
1
2
3
4
FS
type
fsck
pass
ufs
fd
proc
swap
ufs
tmpfs
1
1
-
yes
yes
yes
yes
logging
logging
logging
logging
mount
mount
at boot options
yes
no
no
no
no
yes
logging
-
<--\
<- Disk to be encapsulated.
<- /
<--/
Pre-Encapsulation df -k Command
This example output from the df -k command displays the file systems
currently mounted from disk c1t3d0.
# df -k
Filesystem
/dev/dsk/c1t0d0s0
/proc
fd
mnttab
swap
swap
2-6
kbytes
used
avail capacity
7670973 1536449 6057815
21%
0
0
0
0%
0
0
0
0%
0
0
0
0%
1209520
16 1209504
1%
1209520
16 1209504
1%
Mounted on
/
/proc
/dev/fd
/etc/mnttab
/var/run
/tmp


/dev/dsk/c1t3d0s1
/dev/dsk/c1t3d0s0
/dev/dsk/c1t3d0s4
/dev/dsk/c1t3d0s3
2055705
2055705
2055705
2055705
2073
2073
2073
2073
1991961
1991961
1991961
1991961
1%
1%
1%
1%
/fs2
/fs1
/fs4
/fs3
<-- Mount Points

<--/ / /
<---/ /
<----/
Pre-Encapsulation prtvtoc Command

Executing the prtvtoc command on the target disk shows if there is
space available at the beginning or end of the disk for a private region. If
there is not sufficient space available, the alternative method of
encapsulation must be used. The alternative disk encapsulation process is
described in Encapsulating a Non-Conforming Disk on page 2-20.
#
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
prtvtoc /dev/rdsk/c1t3d0s2
/dev/rdsk/c1t3d0s2 partition map
Dimensions:
512 bytes/sector
133 sectors/track
27 tracks/cylinder
3591 sectors/cylinder
4926 cylinders
4924 accessible cylinders
Flags:
1: unmountable
10: read-only
Unallocated space:
First
Sector
Last
Sector
Count
Sector
16791516
890568 17682083 <---- Unallocated space at end of disk.
Partition
0
1
2
3
4
Tag
0
0
5
0
0
First
Sector
Last
Flags
Sector
Count
Sector
00
0
4197879
4197878
00
4197879
4197879
8395757
01
0 17682084 17682083
00
8395758
4197879 12593636
00
12593637
4197879 16791515
Mount Directory
/fs1 <-- Mount Points
/fs2 <-/fs3 <-/fs4 <--
Notice that the beginning sector of unallocated space is the last sector of
partition 4 plus 1.

Encapsulating Disks
2-7
Pre-Encapsulation format Utility Partition Print

The print option on the format utility partition menu shows the present
partition configuration of the disk. Print this information and save a copy
in a safe place to use if the disk needs to be unencapsulated. Note that
there are at least two unused partitions available in the partition table
(partitions 6 and 7).
partition> print
Current partition table (original):
Total disk cylinders available: 4924 + 2 (reserved cylinders)
Part
Tag
0 unassigned
1 unassigned
2
backup
3 unassigned
4 unassigned
5 unassigned
6 unassigned
7 unassigned
Flag
wm
wm
wu
wm
wm
wm
wm
wm
Cylinders
0 - 1168
1169 - 2337
0 - 4923
2338 - 3506
3507 - 4675
0
0
0
Size
2.00GB
2.00GB
8.43GB
2.00GB
2.00GB
0
0
0
Blocks
(1169/0/0) 4197879
(1169/0/0) 4197879
(4924/0/0) 17682084
(1169/0/0) 4197879
(1169/0/0) 4197879
(0/0/0)
0
(0/0/0)
0
(0/0/0)
0
Data Disk Encapsulation Process

Following is an example of the encapsulation process for disk c1t3d0.
This process uses the vxdiskadm utility to perform the encapsulation, as
follows:
1.
Start the vxdiskadm utility from the shell command line while
logged in as the superuser. Type the following:
# vxdiskadm
Volume Manager Support Operations
Menu: VolumeManager/Disk
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
2-8
Add or initialize one or more disks

Encapsulate one or more disks
Remove a disk
Remove a disk for replacement
Replace a failed or removed disk
Mirror volumes on a disk
Move volumes from a disk
Enable access to (import) a disk group
Remove access to (deport) a disk group
Enable (online) a disk device
Disable (offline) a disk device
Mark a disk as a spare for a disk group
Turn off the spare flag on a disk
Unrelocate subdisks back to a disk
Exclude a disk from hot-relocation use


16 Make a disk available for hot-relocation use
17 Prevent multipathing/Suppress devices from VxVM's view
18 Allow multipathing/Unsuppress devices from VxVM's view
19 List currently suppressed/non-multipathed devices
20 Change the disk naming scheme
21 Get the newly connected/zoned disks in VxVM view
listList disk information
?
??
q
Display help about menu

Display help about the menuing system
Exit from menus
2.
Select menu item 2, Encapsulate one or more disks, to begin the

encapsulation process. Type the following:
Select an operation to perform: 2

Menu: VolumeManager/Disk/Encapsulate
Use this operation to convert one or more disks to use the Volume Manager.
This adds the disks to a disk group and replaces existing partitions
with volumes. Disk encapsulation requires a reboot for the changes
to take effect.
More than one disk or pattern may be entered at the prompt.
some disk selection examples:
Here are
all: all disks

c3 c4t2:all disks on both controller 3 and controller 4, target 2
c3t4d2:a single disk (in the c#t#d# naming scheme)
xyz_0 :a single disk (in the enclosure based naming scheme)
xyz_ :all disks on the enclosure whose name is xyz
3.
Enter the disk to be encapsulated using c#t#d# notation, and answer

yes to the Continue operation? prompt. Type the following:
Select disk devices to encapsulate:

[<pattern-list>,all,list,q,?] c1t3d0
Here is the disk selected.
Output format: [Device_Name]
c1t3d0
Continue operation? [y,n,q,?] (default: y) y
You can choose to add this disk to an existing disk group or to
a new disk group. To create a new disk group, select a disk group
name that does not yet exist.

Encapsulating Disks
2-9

4.
Enter the name of the disk group for this encapsulated disk to join.
In the following example the disk group name is storagedg.
Answer all prompts as appropriate for this particular encapsulation
procedure:
Which disk group [<group>,list,q,?] (default: rootdg) storagedg

There is no active disk group named storagedg.
Create a new group named storagedg? [y,n,q,?] (default: y) y
Use a default disk name for the disk? [y,n,q,?] (default: y) n
A new disk group will be created named storagedg and the selected
disks will be encapsulated and added to this disk group with
disk names that will be specified interactively.
c1t3d0
Continue with operation? [y,n,q,?] (default: y) y
The following disk has been selected for encapsulation.
Output format: [Device_Name]
c1t3d0
Continue with encapsulation? [y,n,q,?] (default: y) y
5.
Select a name for the newly encapsulated disk. Although the name is
arbitrary, it is a good idea to follow some naming convention. In the
following example, the disk name is storage01. Answer the
remaining prompts as appropriate for this specific encapsulation
procedure:
Enter disk name for c1t3d0 [<name>,q,?] (default: storage01) storage01

A new disk group storagedg will be created and the disk device c1t3d0 will
be encapsulated and added to the disk group with the disk name storage01.
Use a default private region length for this disk?
[y,n,q,?] (default: y) y
The typical response to the prompt Use a default private

region length for this disk? is yes. If the size of the private
region in the selected disk group was modified, then answer no, and
follow the prompts to adjust the size.
Caution The size of the private region of all member disks in a disk
group must be the same.
2-10


The c1t3d0 disk has been configured for encapsulation.
The first stage of encapsulation has completed successfully. You
should now reboot your system at the earliest possible opportunity.
The encapsulation will require two or three reboots which will happen
automatically after the next reboot. To reboot execute the command:
shutdown -g0 -y -i6
This will update the /etc/vfstab file so that volume devices are
used to mount the file systems on this disk device. You will need
to update any other references such as backup scripts, databases,
or manually created swap devices.
Encapsulate other disks? [y,n,q,?] (default: n) n
[H[2J
1
2
3
Remove a disk
4
5
6
7
8
9
10 Enable (online) a disk device
11 Disable (offline) a disk device
12 Mark a disk as a spare for a disk group
13 Turn off the spare flag on a disk
14 Unrelocate subdisks back to a disk
15 Exclude a disk from hot-relocation use
17 Prevent multipathing/Suppress devices from VxVM's view
18 Allow multipathing/Unsuppress devices from VxVM's view
20 Change the disk naming scheme
?
??
q

Exit from menus

Encapsulating Disks
2-11

6.
A reboot is necessary to finish the encapsulation process.
This example illustrates how the VxVM software views this disk prior to
completeion of the encapsulation procedure. After the encapsulation
process is completed, compare this example to the same information for
the encapsulated disk.
Select an operation to perform: list
List disk information

Menu: VolumeManager/Disk/ListDisk
Use this menu operation to display a list of disks. You can
also choose to list detailed information about the disk at
a specific disk device address.
Enter disk device or "all" [<address>,all,q,?] (default: all) all
DEVICE
c1t0d0
c1t1d0
c1t2d0
c1t3d0
c1t4d0
c1t5d0
c1t6d0
c1t16d0
c1t17d0
c1t18d0
c1t19d0
c1t20d0
c1t21d0
c1t22d0
DISK
rootdisk
rootmirror
GROUP
rootdg
rootdg
STATUS
online
error
error
error <--- Disk is in the error state.
online
online
error
error
error
error
error
online
online
online
Device to list in detail [<address>,none,q,?] (default: none) c1t3d0

Device:
c1t3d0s2
devicetag: c1t3d0
type:
sliced
flags:
online error private autoconfig
errno:
Disk is not usable <--- Disk is not usable
Multipathing information:
numpaths:
2
c1t3d0s2state=enabled
c2t3d0s2state=enabled
List another disk device? [y,n,q,?] (default: n) n
[H[2J
2-12


1
2
3
4
5
6
7
8
9
10

Remove a disk
.
.
?
??
q

Exit from menus
Select an operation to perform: q

Goodbye.
A system reboot is required before the disk is available for use by the
VxVM software.
Post-Encapsulation Configuration Data

The following are configuration data from example disk c1t3d0 after
encapsulation.
Post-Encapsulation /etc/vfstab File

In the following example of the /etc/vfstab file, the mount statements
for fs1 through fs4 are modified to use VxVM software volumes.
Additionally, the VxVM software annotated this file with information
about the disks prior state. This information is useful if the disk is
unencapsulated.
#device
#to mount
#
#/dev/dsk/c1d0s2
fd
/proc
/dev/vx/dsk/swapvol
/dev/vx/dsk/rootvol
swap
device
to fsck
mount
point
FS
type
fsck
pass
mount
at boot
mount
options
/dev/rdsk/c1d0s2
/dev/vx/rdsk/rootvol
-
/usr
/dev/fd
/proc
/
/tmp
ufs
fd
proc
swap
ufs
tmpfs
1
1
-
yes
no
no
no
no
yes
logging
-

Encapsulating Disks
2-13

#
# Storage
#
/dev/vx/dsk/storagedg/fs1 /dev/vx/rdsk/storagedg/fs1 /fs1 ufs
#NOTE: volume rootvol (/) encapsulated partition c1t0d0s0
#NOTE: volume swapvol (swap) encapsulated partition c1t0d0s1
#NOTE: volume fs1 (/fs1) encapsulated partition c1t3d0s0
1
2
3
4
yes
yes
yes
yes
logging
logging
logging
logging
Post-Encapsulation /etc/vfstab.prevm File

A copy of the /etc/vfstab file before it was modified by the VxVM
software is saved as /etc/vfstab.prevm, as shown in the following
example.
#to mount
#
#/dev/dsk/c1d0s2
fd
/proc
/dev/dsk/c1t0d0s1
/dev/dsk/c1t0d0s0
swap
#
# Storage
#
/dev/dsk/c1t3d0s0
/dev/dsk/c1t3d0s1
/dev/dsk/c1t3d0s3
/dev/dsk/c1t3d0s4
to fsck
point
/dev/rdsk/c1d0s2
/dev/rdsk/c1t0d0s0
-
/dev/rdsk/c1t3d0s0
/dev/rdsk/c1t3d0s1
/dev/rdsk/c1t3d0s3
/dev/rdsk/c1t3d0s4
type
/usr
/dev/fd
/proc
swap
/
/tmp
/fs1
/fs2
/fs3
/fs4
ufs
ufs
ufs
ufs
pass
ufs
fd
proc
ufs
tmpfs
1
2
3
4
yes
yes
yes
yes
1
no
1
-
at boot options
yes
no
no
no
yes
logging
-
logging
logging
logging
logging
Post-Encapsulation df -k Command
The VxVM software volumes in the following example of the df -k
command are displayed as the file system for fs1 through fs4.
bash-2.03# df -k
Filesystem
kbytes
used
avail capacity
/dev/vx/dsk/rootvol 7670973 1646146 5948118
22%
/proc
0
0
0
0%
fd
0
0
0
0%
mnttab
0
0
0
0%
swap
1175152
16 1175136
1%
swap
1175160
24 1175136
1%
/dev/vx/dsk/storagedg/fs2
2055705
2073 1991961
1%
2-14
Mounted on
/
/proc
/dev/fd
/etc/mnttab
/var/run
/tmp
/fs2


2055705
2055705
2055705
2073 1991961
1%
/fs1
2073 1991961
1%
/fs3
2073 1991961
1%
/fs4
Post-Encapsulation prtvtoc Command

The following example of the prtvtoc command shows that the c1t3d0
partition table was modified to have only two partitions (partitions 6 and
7). Partition 6 is the public region and partition 7 is the private region.
This partitioning scheme is indicative of encapsulated disks. Even though
this disk partition table was modified, the data from the original four
partitions occupy the same blocks as they did prior to the encapsulation
process. The data was not moved or otherwise disturbed. The data
remains in this configuration unless the volumes are grown or converted
to a striped layout. This is an important distinction to remember if this
disk is ever unencapsulated.
bash-2.03# prtvtoc /dev/rdsk/c1t3d0s2
* /dev/rdsk/c1t3d0s2 partition map
*
* Dimensions:
*
512 bytes/sector
*
133 sectors/track
*
27 tracks/cylinder
*
*
4926 cylinders
*
*
* Flags:
*
1: unmountable
* 10: read-only
*
* Unallocated space:
*
First
Sector
Last
*
Sector
Count
Sector
*
17682084 4294963705 17678492
*
*
First
Sector
Last
* Partition Tag Flags
Sector
Count
Sector
2
5
01
0 17682084 17682083
6
14
01
0 17682084 17682083
7
15
01
17678493
3591 17682083
Mount Directory
The unencapsulation process for data disks is described in

Unencapsulating Data Disks on page 2-23.

Encapsulating Disks
2-15
Post-Encapsulation format Utility Partition Table

The following example of the format utility prints the new partition table
for c1t3d0. Notice that partition 6 overlaps the entire disk, including
partition 7, which is the private region.
partition> print
Part
Tag
0 unassigned
1 unassigned
2
backup
3 unassigned
4 unassigned
5 unassigned
6
7
-
Flag
wm
wm
wu
wm
wm
wm
wu
wu
Cylinders
0
0
0 - 4923
0
0
0
0 - 4923
4923 - 4923
Size
0
0
8.43GB
0
0
0
8.43GB
1.75MB
Blocks
(0/0/0)
0
(0/0/0)
0
(4924/0/0) 17682084
(0/0/0)
0
(0/0/0)
0
(0/0/0)
0
(4924/0/0) 17682084
(1/0/0)
3591
Post-Encapsulation vxdisk list Command

The vxdisk list command in the following example shows that c1t3d0
is under the management of the VxVM software.
DEVICE
TYPE
c1t0d0s2
sliced
c1t1d0s2
sliced
c1t2d0s2
sliced
c1t3d0s2
sliced
c1t4d0s2
sliced
c1t5d0s2
sliced
c1t6d0s2
sliced
c1t16d0s2
sliced
c1t17d0s2
sliced
c1t18d0s2
sliced
c1t19d0s2
sliced
c1t20d0s2
sliced
c1t21d0s2
sliced
c1t22d0s2
sliced
2-16
DISK
rootdisk
storage01
rootmirror
GROUP
rootdg
storagedg
rootdg
STATUS
online
error
error
online
online
online
error
error
error
error
error
online
online
online

Post-Encapsulation vxdisk list c1t3d0 Command

This example of the vxdisk list command provides detailed
information about the encapsulated disk. Prior to the encapsulation, the
VxVM software was unaware of this disk.
bash-2.03# vxdisk list c1t3d0
Device:
c1t3d0s2
devicetag: c1t3d0
type:
sliced
hostid:
lowtide
disk:
name=storage01 id=1019688339.1134.lowtide
group:
name=storagedg id=1019688341.1137.lowtide
info:
privoffset=1
flags:
online ready private autoconfig autoimport imported
pubpaths: block=/dev/vx/dmp/c1t3d0s6 char=/dev/vx/rdmp/c1t3d0s6
privpaths: block=/dev/vx/dmp/c1t3d0s7 char=/dev/vx/rdmp/c1t3d0s7
version:
2.2
iosize:
min=512 (bytes) max=2048 (blocks)
public:
slice=6 offset=1 len=17678493
private:
update:
time=1019688342 seqno=0.5
headers:
0 248
configs:
count=1 len=2630
logs:
count=1 len=398
Defined regions:
config
priv 000017-000247[000231]: copy=01 offset=000000 enabled
config
log
numpaths:
2
c1t3d0s2
state=enabled
c2t3d0s2
state=enabled
Post-Encapsulation vxprint -ht -g storagedg Command

The vxprint utility in this example displays detailed information about
the encapsulated partitions and their associated VxVM software objects.
bash-2.03# vxprint -ht -g storagedg
DG NAME
NCONFIG
NLOG
DM NAME
DEVICE
TYPE
RV NAME
RLINK_CNT
KSTATE
RL NAME
RVG
KSTATE
V NAME
RVG
KSTATE
PL NAME
VOLUME
KSTATE
SD NAME
PLEX
DISK
SV NAME
PLEX
VOLNAME
DC NAME
PARENTVOL
LOGVOL
SP NAME
SNAPVOL
DCO
dg storagedg
default
default
MINORS
PRIVLEN
STATE
STATE
STATE
STATE
DISKOFFS
NVOLLAYR
GROUP-ID
PUBLEN
PRIMARY
REM_HOST
LENGTH
LENGTH
LENGTH
LENGTH
127000
1019688341.1137.lowtide
STATE
DATAVOLS
REM_DG
READPOL
LAYOUT
[COL/]OFF
[COL/]OFF
SRL
REM_RLNK
PREFPLEX
NCOL/WID
DEVICE
AM/NM

Encapsulating Disks
UTYPE
MODE
MODE
MODE
2-17
dm storage01
c1t3d0s2
sliced
v
pl
sd
sd
fs1
fs1-01
fs1-01
ENABLED ACTIVE
4197879
ENABLED ACTIVE
4197879
storage01 17678492 1
storage01 0
4197878
ROUND
CONCAT
0
1
c1t3d0
c1t3d0
gen
RW
ENA
ENA
v fs2
pl fs2-01
fs2
sd storage01-03 fs2-01
ENABLED ACTIVE
4197879
ENABLED ACTIVE
4197879
storage01 4197878 4197879
ROUND
CONCAT
0
c1t3d0
gen
RW
ENA
v fs3
pl fs3-01
fs3
ENABLED ACTIVE
4197879
ENABLED ACTIVE
4197879
storage01 8395757 4197879
ROUND
CONCAT
0
c1t3d0
gen
RW
ENA
v fs4
pl fs4-01
fs4
ENABLED ACTIVE
4197879 ROUND
ENABLED ACTIVE
4197879 CONCAT
storage01 12593636 4197879 0
c1t3d0
gen
RW
ENA
fs1
fs1-01
storage01-B0
storage01-04
3590
17678493 -
Related Post-Encapsulation Files

When a disk is encapsulated, a subdirectory under
/etc/vx/reconfig.d/disk.d is created which contains files that hold
information related to the encapsulation. The following is an example of
files that relate to the encapsulation of c1t3d0.
The File List

A directory list of files specific to c1t3d0 is shown in the following
example.
bash-2.03# pwd
/etc/vx/reconfig.d/disk.d/c1t3d0
bash-2.03# ls -las
total 16
2 drwxr-xr-x
2 root
other
2 drwxr-xr-x
4 root
other
2 -rw-r--r-1 root
other
2 -rw-r--r-1 root
other
4 -rw-r--r-1 root
other
2 -rw-r--r-1 root
other
2 -rw-r--r-1 root
other
512
512
10
10
1156
7
476
Apr
Apr
Apr
Apr
Apr
Apr
Apr
24
24
24
24
24
24
24
15:29
15:29
15:29
15:29
15:29
15:29
15:29
.
..
dg
dmname
newpart
primary_node
vtoc
The dg File
The dg file in the following example delineates the disk group of which
this encapsulated disk is a member.
2-18


bash-2.03# more *
::::::::::::::
dg
::::::::::::::
storagedg
::::::::::::::
The dmname File

The dmname file for the following example lists the VxVM software disk
name of the encapsulated disk.
::::::::::::::
dmname
::::::::::::::
storage01
::::::::::::::
The newpart File

The following example of the newpart file displays the new partition
information for this encapsulated disk. Additionally, it lists all the VxVM
software commands executed to successfully complete the encapsulation
process.
::::::::::::::
newpart
::::::::::::::
# volume manager partitioning for drive c1t3d0
0 0x0 0x000
0
0
1 0x0 0x000
0
0
2 0x5 0x201
0 17682084
3 0x0 0x000
0
0
4 0x0 0x000
0
0
5 0x0 0x000
0
0
6 0xe 0x201
0 17682084
7 0xf 0x201 17678493
3591
#vxmake vol fs1 plex=fs1-%%00 usetype=gen
#vxmake plex fs1-%%00 sd=storage01-B0,storage01-%%00
#vxmake sd storage01-%%00 disk=storage01 offset=0 len=4197878
#vxmake sd storage01-B0 disk=storage01 offset=17678492 len=1 putil0=Block0
comment="Remap of block 0
#vxvol start fs1
#rename c1t3d0s0 fs1
#vxmake plex fs2-%%01 sd=storage01-%%01
#vxvol start fs2

Encapsulating Disks
2-19

#vxvol start fs3
#vxvol start fs4
The primary_node File

The primary_node file lists the original c#t#d# name of the encapsulated
disk in the following example.
::::::::::::::
primary_node
::::::::::::::
c1t3d0
::::::::::::::
The vtoc File

The following example of the vtoc file saves the original partition table of
the encapsulated disk. In earlier versions of the VxVM software, this file
was modified and used with the /etc/fmthard command to re-partition
disks during the unencapsulation process. The vxmksdpart command
now performs this function.
::::::::::::::
vtoc
::::::::::::::
#THE PARTITIONING OF /dev/rdsk/c1t3d0s2 IS AS FOLLOWS:
#SLICE
TAG FLAGS
START
SIZE
0
0x0 0x200
0
4197879
1
0x0 0x200
4197879
4197879
2
0x5 0x201
0
17682084
3
0x0 0x200
8395758
4197879
4
0x0 0x200
12593637
4197879
5
0x0 0x000
0
0
6
0x0 0x000
0
0
7
0x0 0x000
0
0
Encapsulating a Non-Conforming Disk

To encapsulate a disk into the VxVM software, the disk is required to have
enough free space for the VxVM software to write a private region to the
disk. The private region is generally smaller than 2 megabytes.
2-20


If there is not enough free space on the disk and additional free space
cannot be configured, the situation can be alleviated by temporarily
encapsulating the non-conforming disk without a private region, then
mirroring the encapsulated disk volumes to another disk. Once the data is
mirrored to a real VxVM software disk, the encapsulated plexes are
detached, leaving the data on the mirror. At that point, mirror the mirror
disk to another VxVM software disk to complete the operation.
Only use this procedure if the data on a non-conforming disk must be
brought under the VxVM software control.
The following procedure is based on SunSolve INFODOC 16051. Refer to
Appendix A, SunSolve INFODOCs, for a complete copy of this
document. Check the SunSolve Online Web site to determine if the
INFODOC has been updated.
To encapsulate a non-conforming disk:
1.
For each slice on the disk (excluding slice 2), run the following
command to prepare the disk for encapsulation without a private
region. In this example, only slices 5 and 6 have data on them. Type
the following:
vxdisk define c#t#d#s5 type=nopriv

2.
Use the vxdg -g command to add each of the slices of the target disk
to a disk group as though each slice is a disk, assigning the slice a
name. In this example the names NPdisk05 and NPdisk06 are used,
as follows:
vxdg -g <diskgroup> adddisk NPdisk05=c#t#d#s5

3.
On each of the new disks, create a simple volume (not a file system)
that spans the entire disk. Then use the vxdisk list command to
determine the maximum size for the volumes to be created:
vxdisk list NPdisk05 | grep

public:
slice=0 offset=0
vxdisk list NPdisk06 | grep
public:
slice=0 offset=0
4.
public
len=8196096
public
len=9400320
Use the information derived from the vxdisk list command len
value and the vxassist command to create the volumes. The
following example names the volumes NPdisk05vol and
NPdisk06vol.
vxassist -g <diskgroup> make NPdisk05vol 8196096 layout=nostripe alloc="NPdisk05"


Encapsulating Disks
2-21

5.
Mirror the volumes to a disk that has enough space to mirror both
volumes. In the following example, the volumes are mirrored to a
disk named disk01.
vxassist -g <diskgroup> mirror NPdisk05vol layout=nostripe alloc="disk01"

6.
When the mirroring process is complete, remove the original side of

the mirror. This removes the disk that does not have a private region
configured. Type the following:
vxplex -g <diskgroup> -o rm dis NPdisk05vol-01

7.
Remove the old disks from the disk group and return them to their
original state:
vxdg -g <diskgroup> rmdisk NPdisk05

vxdisk rm c0t5d10s5 vxdisk rm c0t5d10s6
This leaves two concatenated volumes named NPdisk05vol and

NPdisk06vol. These volumes contain the data that was originally located
on c#t#d#s5 and c#t#d#s6.
2-22

Unencapsulating Data Disks

After a data disk is encapsulated there is no easy method to reverse the
process. From a VxVM software point of view, once a disk is
encapsulated, there is no reason to reverse the process. Additionally, if the
encapsulated volume is grown or converted to a stripe volume,
unencapsulation is impossible. The only method to reverse the
encapsulation process after a volume is modified in this way is to back up
the data and restore it to a non-VxVM software-managed disk.
Caution Be sure to back up all data on the encapsulated disk prior to
performing this procedure.
The following example of the unencapsulation process uses an
encapsulated disk that is a member of the storagedg disk group with a
physical address of c1t3d0.
1.
Make sure that no volumes from the encapsulated disk were grown
or had their basic structure modified. Use the vxprint command
and the format utility to verify format and condition of the
encapsulated volumes, as follows:
bash-2.03# vxprint -g storagedg -ht

DG NAME
NCONFIG
NLOG
DM NAME
DEVICE
TYPE
RV NAME
RLINK_CNT
KSTATE
RL NAME
RVG
KSTATE
V NAME
RVG
KSTATE
PL NAME
VOLUME
KSTATE
SD NAME
PLEX
DISK
SV NAME
PLEX
VOLNAME
DC NAME
PARENTVOL
LOGVOL
SP NAME
SNAPVOL
DCO
MINORS
PRIVLEN
STATE
STATE
STATE
STATE
DISKOFFS
NVOLLAYR
GROUP-ID
PUBLEN
PRIMARY
REM_HOST
LENGTH
LENGTH
LENGTH
LENGTH
STATE
DATAVOLS
REM_DG
READPOL
LAYOUT
[COL/]OFF
[COL/]OFF
SRL
REM_RLNK
PREFPLEX
NCOL/WID
DEVICE
AM/NM
UTYPE
MODE
MODE
MODE
dg storagedg
default
default
127000
1019688341.1137.lowtide
dm storage01
dm storage02
c1t3d0s2
c1t4d0s2
sliced
sliced
3590
3590
17678493 17674902 -
v
pl
sd
sd
pl
sd
fs1
fs1-01
fs1-01
fs1
fs1-02
ENABLED ACTIVE
4197879
ENABLED ACTIVE
4197879
storage01 17678492 1
storage01 0
4197878
ENABLED ACTIVE
4197879
storage02 0
4197879
ROUND
CONCAT
0
1
CONCAT
0
c1t3d0
c1t3d0
c1t4d0
gen
RW
ENA
ENA
RW
ENA
v fs2
pl fs2-01
fs2
ENABLED ACTIVE
4197879
ENABLED ACTIVE
4197879
storage01 4197878 4197879
ROUND
CONCAT
0
c1t3d0
gen
RW
ENA
fs1
fs1-01
storage01-B0
storage01-04
fs1-02
storage02-01

Encapsulating Disks
2-23

pl fs2-02
fs2
ENABLED ACTIVE
4197879
storage02 4197879 4197879
CONCAT
0
c1t4d0
RW
ENA
v
pl
sd
pl
sd
ROUND
CONCAT
0
CONCAT
0
c1t3d0
c1t4d0
gen
RW
ENA
RW
ENA
ROUND
CONCAT
0
CONCAT
0
c1t3d0
c1t4d0
gen
RW
ENA
RW
ENA
fs3
fs3-01
storage01-02
fs3-02
storage02-03
fs3
fs3-01
fs3
fs3-02
ENABLED ACTIVE
ENABLED ACTIVE
storage01 8395757
ENABLED ACTIVE
storage02 8395758
4197879
4197879
4197879
4197879
4197879
v fs4
pl fs4-01
sd storage01-01
pl fs4-02
sd storage02-04
bash-2.03#
fs4
fs4-01
fs4
fs4-02
ENABLED ACTIVE
4197879
ENABLED ACTIVE
4197879
storage01 12593636 4197879
ENABLED ACTIVE
4197879
storage02 12593637 4197879
Note that all volumes on c1t3d0 (fs1fs4) have a value of

LAYOUT=CONCAT, which indicates that the volumes were not grown
or converted to a striped volume. This means that the
unencapsulation process should work on this disk.
Each volume has two plexes, fs#-01, the primary, and fs#-02, the
mirror. This is verified by printing the partition table of each disk.
Additionally, the B0 subdisk is used as a safety mechanism to protect
the bootblock of encapsulated disks. It can be safely ignored during
this process.
The partition tables for these example disks are as follows
Encapsulated disk The following capture is of disk c1t3d0.

Notice that it has slicing indicative of an encapsulated disk.
This is the target disk for the unencapsulation example.
partition> print
Part
Tag
0 unassigned
1 unassigned
2
backup
3 unassigned
4 unassigned
5 unassigned
6
7
-
Flag
wm
wm
wu
wm
wm
wm
wu
wu
Cylinders
0
0
0 - 4923
0
0
0
0 - 4923
4923 - 4923
2-24
Size
0
0
8.43GB
0
0
0
8.43GB
1.75MB
Blocks
(0/0/0)
0
(0/0/0)
0
(4924/0/0) 17682084
(0/0/0)
0
(0/0/0)
0
(0/0/0)
0
(4924/0/0) 17682084 <-- Public
(1/0/0)
3591 <-- Private
Mirror disk This is a partition print of disk c1t4d0. It is the

mirror of c1t3d0 and has the partitioning scheme of a VxVM
software disk. This is the disk from which the plexes are
detached (unmirrored) during the unencapsulation process.


partition> print
Part
Tag
0 unassigned
1 unassigned
2
backup
3
4
5 unassigned
6 unassigned
7 unassigned
Flag
wm
wm
wu
wu
wu
wm
wm
wm
2.
bash-2.03#
bash-2.03#
bash-2.03#
bash-2.03#
vxplex
vxplex
vxplex
vxplex
-o
-o
-o
-o
Cylinders
0
0
0 - 4923
0 0
2 - 4923
0
0
0
Size
0
0
8.43GB
1.75MB
8.43GB
0
0
0
Blocks
(0/0/0)
0
(0/0/0)
0
(4924/0/0) 17682084
(1/0/0)
3591 <-- Private
(4922/0/0) 17674902 <-- Public
(0/0/0)
0
(0/0/0)
0
(0/0/0)
0
Unmirror all volumes to be unencapsulated, using either the

vxassist or vxplex command. The following example uses the
vxplex command:
rm
rm
rm
rm
dis
dis
dis
dis
fs1-02
fs2-02
fs3-02
fs4-02
Power user Alternatively, for disks with multiple mirrors, the vxplex
command can be looped:
bash-2.03# for i in 1 2 3 4
> do
> vxplex -o rm dis fs$i-02
> done
3.
bash-2.03#
DG NAME
DM NAME
RV NAME
RL NAME
V NAME
PL NAME
SD NAME
SV NAME
DC NAME
SP NAME
Use the vxprint command to verify that all mirrors are detached:
vxprint -g storagedg -ht

NCONFIG
NLOG
DEVICE
TYPE
RLINK_CNT
KSTATE
RVG
KSTATE
RVG
KSTATE
VOLUME
KSTATE
PLEX
DISK
PLEX
VOLNAME
PARENTVOL
LOGVOL
SNAPVOL
DCO
MINORS
PRIVLEN
STATE
STATE
STATE
STATE
DISKOFFS
NVOLLAYR
GROUP-ID
PUBLEN
PRIMARY
REM_HOST
LENGTH
LENGTH
LENGTH
LENGTH
STATE
DATAVOLS
REM_DG
READPOL
LAYOUT
[COL/]OFF
[COL/]OFF
SRL
REM_RLNK
PREFPLEX
NCOL/WID
DEVICE
AM/NM
dg storagedg
default
default
127000
1019688341.1137.lowtide
dm storage01
dm storage02
c1t3d0s2
c1t4d0s2
sliced
sliced
3590
3590
17678493 17674902 -
v
pl
sd
sd
fs1
fs1-01
fs1-01
ENABLED ACTIVE
4197879
ENABLED ACTIVE
4197879
storage01 17678492 1
storage01 0
4197878
fs1
fs1-01
storage01-B0
storage01-04
ROUND
CONCAT
0
1
c1t3d0
c1t3d0

Encapsulating Disks
UTYPE
MODE
MODE
MODE
gen
RW
ENA
ENA
2-25

v fs2
pl fs2-01
fs2
ENABLED ACTIVE
4197879
ENABLED ACTIVE
4197879
storage01 4197878 4197879
ROUND
CONCAT
0
c1t3d0
gen
RW
ENA
v fs3
pl fs3-01
fs3
ENABLED ACTIVE
4197879
ENABLED ACTIVE
4197879
storage01 8395757 4197879
ROUND
CONCAT
0
c1t3d0
gen
RW
ENA
v fs4
pl fs4-01
fs4
ENABLED ACTIVE
4197879 ROUND
ENABLED ACTIVE
4197879 CONCAT
storage01 12593636 4197879 0
c1t3d0
gen
RW
ENA
Notice that the -02 plexes are missing from the vxprint output for
these volumes. This is a clear indication that the mirrors no longer
exist. Also, note the subdisk numbers for the surviving plex of each
volume. This information is used in the next step of the
unencapsulation process.
4.
For normal partitions (non-private), recreate the original partitions on

the disk by running the vxmksdpart command on each subdisk, as
shown in this example.
Caution The following steps must be performed exactly as shown or

loss of data may occur. It is very important that a full understanding of
the pre-encapsulation partitioned layout of the disk is known, or the postencapsulation partition layout could be incorrect.
To make sure that the mapping of subdisks to encapsulated
partitions is correct, cross-reference the output of the vxprint
command with the commented lines of the /etc/vfstab file. These
comments were added to the file by the VxVM software during the
encapsulation process. This method provides a one-for-one match
between subdisks and encapsulated partitions.
bash-2.03# /etc/vx/bin/vxmksdpart -g storagedg storage01-04 0 0x00 0x00
partition> print
Part
Tag
0 unassigned
1 unassigned
2
backup
3 unassigned
4 unassigned
5 unassigned
6
7
-
2-26
Flag
wm
wm
wu
wm
wm
wm
wu
wu
Cylinders
0 - 1168
0
0 - 4923
0
0
0
0 - 4923
4923 - 4923
Size
2.00GB
0
8.43GB
0
0
0
8.43GB
1.75MB
Blocks
(1169/0/0) 4197879
(0/0/0)
0
(4924/0/0) 17682084
(0/0/0)
0
(0/0/0)
0
(0/0/0)
0
(4924/0/0) 17682084
(1/0/0)
3591


Notice that after execution of the vxmksdpart command for the
storage01-04 subdisk, the partition table of c1t3d0 is modified to
reflect the original sliced location of that subdisks data.
The process continues by creating partitions for the remaining
subdisks.
When the process is completed, a printout of the partition table for

the c1t3d0 disk shows that all the original partitions were recreated
and in the correct locations.
partition> print
Part
Tag
0 unassigned
1 unassigned
2
backup
3 unassigned
4 unassigned
5 unassigned
6
7
-
Flag
wm
wm
wu
wm
wm
wm
wu
wu
5.
Cylinders
0 - 1168
1169 - 2337
0 - 4923
2338 - 3506
3507 - 4675
0
0 - 4923
4923 - 4923
Size
2.00GB
2.00GB
8.43GB
2.00GB
2.00GB
0
8.43GB
1.75MB
Blocks
(1169/0/0) 4197879
(1169/0/0) 4197879
(4924/0/0) 17682084
(1169/0/0) 4197879
(1169/0/0) 4197879
(0/0/0)
0
(4924/0/0) 17682084
(1/0/0)
3591
Unmount the volumes. Type the following:
bash-2.03# df -k
Filesystem
kbytes
used
avail capacity
/dev/vx/dsk/rootvol 7670973 1646148 5948116
22%
/proc
0
0
0
0%
fd
0
0
0
0%
mnttab
0
0
0
0%
swap
1174480
16 1174464
1%
swap
1174496
32 1174464
1%
2055705
2073 1991961
1%
2055705
2073 1991961
1%
2055705
2073 1991961
1%
2055705
2073 1991961
1%
bash-2.03#
bash-2.03#
bash-2.03#
bash-2.03#
umount
umount
umount
umount
Mounted on
/
/proc
/dev/fd
/etc/mnttab
/var/run
/tmp
/fs2
/fs1
/fs3
/fs4
/fs1
/fs2
/fs3
/fs4

Encapsulating Disks
2-27

When this step is completed, all volumes hosted by this disk are
unmounted, as shown in the following example:
bash-2.03# df -k
Filesystem
/dev/vx/dsk/rootvol
/proc
fd
mnttab
swap
swap
kbytes
used
avail capacity
7670973 1646148 5948116
22%
0
0
0
0%
0
0
0
0%
0
0
0
0%
1176080
16 1176064
1%
1176096
32 1176064
1%
Mounted on
/
/proc
/dev/fd
/etc/mnttab
/var/run
/tmp
6.
Stop all applications using data on these volumes prior to executing

the remaining steps.
7.
Edit the /etc/vfstab file by changing the mount statements to

reflect partitions instead of volumes:
#device
device
mount
FS
fsck
#to mount
to fsck
point
type
pass
#
#/dev/dsk/c1d0s2 /dev/rdsk/c1d0s2 /usr
ufs
1
fd
/dev/fd fd
no
/proc
/proc
proc
no
/dev/vx/dsk/swapvol
swap
no
/dev/vx/dsk/rootvol
/
ufs
swap
/tmp
tmpfs
yes
#
# Storage
#
/dev/dsk/c1t3d0s0
/dev/rdsk/c1t3d0s0
/fs1
ufs
/dev/dsk/c1t3d0s1
/dev/rdsk/c1t3d0s1
/fs2
ufs
/dev/dsk/c1t3d0s3
/dev/rdsk/c1t3d0s3
/fs3
ufs
/dev/dsk/c1t3d0s4
/dev/rdsk/c1t3d0s4
/fs4
ufs
8.
mount
mount
at boot options
yes
no
logging
1
1
1
1
yes
yes
yes
yes
logging
logging
logging
logging
Mount the partitions. Use the mountall command:
bash-2.03# mountall
bash-2.03# df -k
Filesystem
/dev/vx/dsk/rootvol
/proc
fd
mnttab
swap
swap
/dev/dsk/c1t3d0s1
/dev/dsk/c1t3d0s3
/dev/dsk/c1t3d0s4
/dev/dsk/c1t3d0s0
kbytes
used
avail capacity Mounted on
7670973 1646148 5948116
22%
/
0
0
0
0%
/proc
0
0
0
0%
/dev/fd
0
0
0
0%
/etc/mnttab
1174552
16 1174536
1%
/var/run
1174568
32 1174536
1%
/tmp
2055705
2073 1991961
1%
/fs2
2055705
2073 1991961
1%
/fs3
2055705
2073 1991961
1%
/fs4
2055705
2073 1991961
1%
/fs1
It is now safe to start applications that use data contained on these

partitions.
2-28


9.
bash-2.03#
bash-2.03#
bash-2.03#
bash-2.03#
vxedit
vxedit
vxedit
vxedit
-r
-r
-r
-r
Remove each volume (recursively), as shown in the following

example:
-f
-f
-f
-f
rm
rm
rm
rm
fs1
fs2
fs3
fs4
bash-2.03# vxprint -g storagedg -ht

DG NAME
NCONFIG
NLOG
DM NAME
DEVICE
TYPE
RV NAME
RLINK_CNT
KSTATE
RL NAME
RVG
KSTATE
V NAME
RVG
KSTATE
PL NAME
VOLUME
KSTATE
SD NAME
PLEX
DISK
SV NAME
PLEX
VOLNAME
DC NAME
PARENTVOL
LOGVOL
SP NAME
SNAPVOL
DCO
MINORS
PRIVLEN
STATE
STATE
STATE
STATE
DISKOFFS
NVOLLAYR
GROUP-ID
PUBLEN
PRIMARY
REM_HOST
LENGTH
LENGTH
LENGTH
LENGTH
STATE
DATAVOLS
REM_DG
READPOL
LAYOUT
[COL/]OFF
[COL/]OFF
SRL
REM_RLNK
PREFPLEX
NCOL/WID
DEVICE
AM/NM
dg storagedg
default
default
127000
1019688341.1137.lowtide
dm storage01
dm storage02
c1t3d0s2
c1t4d0s2
sliced
sliced
3590
3590
17678493 17674902 -
UTYPE
MODE
MODE
MODE
Notice that all volumes are removed.

10. Remove the disk from the disk group. Type the following:
bash-2.03# vxdg rmdisk storage01
bash-2.03# vxdisk rm c1t3d0
DEVICE
TYPE
c1t0d0s2
sliced
c1t1d0s2
sliced
c1t2d0s2
sliced
c1t4d0s2
sliced
c1t5d0s2
sliced
c1t6d0s2
sliced
c1t16d0s2
sliced
c1t17d0s2
sliced
c1t18d0s2
sliced
c1t19d0s2
sliced
c1t20d0s2
sliced
c1t21d0s2
sliced
c1t22d0s2
sliced
DISK
rootdisk
storage02
rootmirror
GROUP
rootdg
storagedg
rootdg
STATUS
online
error
error
online
online
error
error
error
error
error
online
online
online
The vxdisk list output shows that c1t3d0 was removed from
VxVM software control.

Encapsulating Disks
2-29

11. Remove the public and private partitions using the format utility.
Type the following:
partition> print
Current partition table (unnamed):
Part
Tag
0 unassigned
1 unassigned
2
backup
3 unassigned
4 unassigned
5 unassigned
6 unassigned
7 unassigned
Flag
wm
wm
wu
wm
wm
wm
wu
wu
Cylinders
0 - 1168
1169 - 2337
0 - 4923
2338 - 3506
3507 - 4675
0
0
0
Size
2.00GB
2.00GB
8.43GB
2.00GB
2.00GB
0
0
0
Blocks
(1169/0/0) 4197879
(1169/0/0) 4197879
(4924/0/0) 17682084
(1169/0/0) 4197879
(1169/0/0) 4197879
(0/0/0)
0
(0/0/0)
0
(0/0/0)
0
The unencapsulation process is now complete.
2-30

Encapsulating Boot Disks

The VxVM software is able to bring system partitions, such as root, usr,
var, opt, or home, under management. This is called rootability. Rootability
is accomplished through encapsulation. The encapsulation procedure for
system partitions is similar to the procedure used for data (non-root) disks
discussed in the section Encapsulating Data Disks on page 2-5.
Encapsulation of system partitions is the process of converting standard
UNIX file system (UFS) partitions to VxVM software volumes. Once
converted to VxVM software volumes, the encapsulated volumes can be
mirrored to increase the fault resiliency of the system areas of a servers
file system. A volume that is configured as a swap area is called a swap
volume. A volume that is configured as a root file system is called a root
volume.
Figure 2-2 illustrates partition mapping for an encapsulated boot disk
with two system partitions.
Boot Disk
Encapsulated Boot Disk
rootvol
Overlay PartitionSlice 0 - /
/ (0)
and
Slice 2
Overlay Partition Slice 1 -
Slice 2
swap
Free Space
swap
(1)
Slice 4 Private Region
Slice 3
Public Region
swapvol
Mirror Disk
rootvol
Slice 3 Public
Overlay Partition -
(0)
Slice 2
Region
Overlay Partition -
swap (1)
Figure 2-2
swapvol
Two-Slice Boot Disk

Encapsulating Disks
2-31

Partition mapping for an encapsulated boot disk with four system
partitions is shown in Figure 2-3.
Boot Disk
Slice 0 -
Slice 1 -
swap
Slice 3 -
/usr
Slice 4 -
/var
Slice 2
Free Space
Encapsulated Boot Disk
/(0)
Overlay Partition - swap(1)
Overlay Partition -
/usr(3)
Overlay Partition - /var(4)
Overlay Partition -
rootvol
swapvol
Slice 2
and Slice 6
Public
Region
Mirror Disk
/ (0)
Overlay Partition - swap (1)
Overlay Partition - /usr (6)
Overlay Partition - /var (7)
Overlay Partition -
Slice 4
Public Region
Figure 2-3
rootvol
swapvol
Slice 2
Four-Slice Boot Disk
Encapsulating the boot disk brings increased fault resiliency to a server

but also brings with it additional system management issues that must be
addressed by system administrators. These issues are addressed in this
section on boot disk encapsulation.
Booting root Volumes

During boot processing, the root and swap volume areas must be available
prior to the loading of the vxconfigd daemon. This means that these
areas of an encapsulated boot disk must be accessible from contiguous
blocks of space as ordinary partitions. Due to this restriction, root and
swap volumes cannot be striped, concatenated, or spanned. Also, the
mirrors (plexes) of these volumes cannot be striped, concatenated, or
spanned.
2-32

Volume Restrictions
The rootvol, swapvol, and usr volumes have restrictions that other
encapsulated volumes do not have. These restrictions are:
The root volume rootvol must be a member of the rootdg disk

group.
The rootvol, swapvol, and usr volumes must have the following
specific minor numbers:
rootvol = 0
swapvol = 2
usr = 3
The rootvol, swapvol, usr, and var volumes use restricted mirrors
that have overlay partitions created for them. Overlay partitions
occupy the same disk blocks as the restricted mirror and are used to
boot the system prior to the availability of the vxconfigd daemon.
The rootvol, swapvol, usr, var, and opt volumes cannot be grown,
spanned or occupy a plex that has multiple non-contiguous
subdisks. All data associated with encapsulated system partitions
must reside in contiguous blocks of space.
When mirroring the boot disk, the mirror disk must be large enough
to hold all plexes on that disk or mirroring fails for one or more
volumes on the encapsulated boot disk.
The rootvol, swapvol, and usr volumes cannot have DRL attached
to their volumes.
Boot Disk Encapsulation Process

There are two methods which can be used to encapsulate a boot disk. One
method performs the encapsulation during the vxinstall process. The
second method performs the encapsulation process after the VxVM
software is installed. This requires the use of either the vxdiskadm
(preferred) or vxdiskadd, vxmake, and vxvol utilities. This section uses
the vxdiskadm utility for all encapsulation and mirroring example
procedures.
Appendix C, Example Five-Slice Boot Disk Encapsulation, provides a
detailed example of the procedures used to encapsulate a boot disk.

Encapsulating Disks
2-33
Note The following processes and procedures are standard VERITAS

Software Corporation recommendations for encapsulating and mirroring
boot disks. Sun best practices for boot disk management are discussed in
Examining Sun Enterprise Services Best Practices for VxVM SoftwareManaged Boot Disks on page 2-48.
A detailed boot disk encapsulation example is not presented in this
section due to the similarity of that process to the encapsulation process
for data disks.
Encapsulating During the VxVM Software Installation

To encapsulate the boot disk during VxVM software installation,
answering yes to the encapsulation prompt during the installation script
process. Once vxinstall is finished, the system is rebooted twice. The
first reboot performs the encapsulation. The second reboot boots the
system using the encapsulated system volumes.
Encapsulating Using the vxdiskadm Utility

Encapsulating the boot disk using the vxdiskadm command requires that
a minimum of one disk be configured into rootdg during the vxinstall
processing. This disk is initalized for use as the mirror for the
encapsulated boot disk. Boot disk encapsulation using the vxdiskadm
command is identical to the process outlined for data disks described in
Data Disk Encapsulation Process on page 2-8. The difference between
the two processes is in the number of reboots. Data disk encapsulation
requires a single reboot, while boot disk encapsulation performs multiple
reboots.
Note For more information on boot disk encapsulation see the
following: SunSolve INFODOCs 15838, 19245, 13781, 13775, and 24663,
and SRDBs 19245.
2-34


The following section displays disk partition tables and files associated
with non-VxVM software management of a system boot disk and
associated file systems. This example set depicts a two-slice boot disk.
These examples were captured prior to the installation of the VxVM
software and encapsulation of the boot disk. This system has a boot disk
that is configured to use two slices, / and swap.
Pre-Encapsulation /etc/vfstab File

The following example shows a standard /etc/vfstab file prior to the
installation of the VxVM software and encapsulation of the boot disk.
Notice that the boot disk is configured to use two system partitions, /
and swap.
#device
device
mount
#to mount
to fsck
point
#
fd
/dev/fd fd
no
/proc
/proc
proc
no
/dev/dsk/c1t0d0s1
swap
/dev/dsk/c1t0d0s0
/dev/rdsk/c1t0d0s0
swap
/tmp
tmpfs
yes
FS
type
fsck
pass
mount
mount
at boot options
ufs
/
-
yes
no
ufs
no
logging
A df -k command delineates the example systems mounted file systems.
At this time, this server does not have any application or data file system
configured.
bash-2.03# df -k
Filesystem
/dev/dsk/c1t0d0s0
/proc
fd
mnttab
swap
swap
kbytes
used
avail capacity
7670973 1536449 6057815
21%
0
0
0
0%
0
0
0
0%
0
0
0
0%
1209520
16 1209504
1%
1209520
16 1209504
1%
Mounted on
/
/proc
/dev/fd
/etc/mnttab
/var/run
/tmp

Encapsulating Disks
2-35

The prtvtoc command output of the example system boot disk shows
the disk having two partitions and no free unpartitioned space. Boot
disks, unlike data disks, do not require any unpartitioned free space for
encapsulation. In this situation, space for the private region is taken from
the swap partition.
This is a less than ideal configuration for encapsulation. The disk
encapsulates, but it will probably fail mirroring if the mirror disk is the
exact same size as this encapsulated disk. This is because the mirror disk
is an initialized VxVM software disk with 2048 bytes or greater of space
consumed for the private region. This leaves a public region with
insufficient space to mirror both the rootvol and swapvol volumes.
*
* Dimensions:
*
512 bytes/sector
*
133 sectors/track
*
27 tracks/cylinder
*
*
4926 cylinders
*
*
* Flags:
*
1: unmountable
* 10: read-only
*
*
First
Sector
Last
Sector
Count
Sector
0
2
00
0 15581349 15581348
1
3
01
15581349
2100735 17682083
2
5
00
0 17682084 17682083
Mount Directory
/
In this example, the space problem was rectified by reducing the size of
the swap partition, leaving sufficient unpartitioned free space for use by
the encapsulation process. Additional swap space can be configured using
swap files to supplement the reduced swap partition size. The corrected
partition map now shows unallocated space.
*
* Dimensions:
*
512 bytes/sector
*
133 sectors/track
*
27 tracks/cylinder
*
*
4926 cylinders
2-36


*
*
* Flags:
*
1: unmountable
* 10: read-only
*
<-- Notice this
*
First
Sector
Last
*
Sector
Count
Sector
*
15581349 4279385947 4294967295
*
17682084 4292870152 15584939
*
16637103
1041390 17678492
*
First
Sector
Sector
Count
0
2
00
0 15581349
1
3
01
15584940
1052163
2
5
00
0 17682084
disk now has unallocated space.
Last
Sector
15581348
16637102
17682083
Mount Directory
Note It is a VxVM software best practice to always configure boot disks

with free unpartitioned space at the beginning or end of the disk in the
event that the disk is encapsulated at another time.
Pre-Encapsulation eeprom Command

The following truncated example output from the eeprom command
shows the contents of the systems nvramrc area. Once the boot disk is
encapsulated and mirrored, new device aliases are created to delineate the
root disk and mirror for booting purposes.
bash-2.03# eeprom
disabled-memory-list: data not available.
disabled-board-list: data not available.
memory-interleave=max
.
.
.
boot-device=test1
local-mac-address?=false
ansi-terminal?=true
screen-#columns=80
screen-#rows=34
silent-mode?=false
use-nvramrc?=true
nvramrc=devalias test1 /sbus@3,0/SUNW,socal@0,0/sf@0,0/ssd@w210000203713fc9f,0:a
devalias test3 /sbus@3,0/SUNW,socal@0,0/sf@0,0/ssd@w21000020372d5917,0:a
devalias test2 /sbus@3,0/SUNW,socal@0,0/sf@1,0/ssd@w220000203713f643,0:a
.
.
.

Encapsulating Disks
2-37

hardware-revision: data not available.
last-hardware-update: data not available.
diag-switch?=false

The following examples were captured after the VxVM software was
installed and the boot disk encapsulated. This example depicts a two-slice
boot disk.

The /etc/vfstab file displays the new devices to mount and to fsck
after the encapsulation in this example. Notice that comments at the end
of this example file describe the pre-encapsulation device configuration of
the / and swap partitions.
#device
device
mount
FS
fsck
#to mount
to fsck
point
type
pass
#
ufs
1
fd
/dev/fd fd
no
/proc
/proc
proc
no
/dev/vx/dsk/swapvol
swap
no
/dev/vx/dsk/rootvol
/
ufs
swap
/tmp
tmpfs
yes
mount
mount
at boot options
yes
no
logging
The following example of the df -k command shows that / is mounted
using the new encapsulated root volume.
bash-2.03# df -k
Filesystem
/dev/vx/dsk/rootvol
/proc
fd
mnttab
swap
swap
2-38
kbytes
used
avail capacity
7670973 1646089 5948175
22%
0
0
0
0%
0
0
0
0%
0
0
0
0%
1175352
16 1175336
1%
1175360
24 1175336
1%
Mounted on
/
/proc
/dev/fd
/etc/mnttab
/var/run
/tmp

Post-Encapsulation swap -l Command

An example of the swap -l command shows that the encapsulated
swapvol is used as the swap device.
bash-2.03# swap -l
swapfile
dev
/dev/vx/dsk/swapvol 75,5
swaplo blocks
free
16 1052144 1052144
Post-Encapsulation prtvtoc Command

Output from the following example of the prtvtoc command clearly
shows the encapsulation slicing scheme. Unlike that of the data disk
encapsulation slicing, encapsulated boot disk partitioning is more
complex. A two-slice boot disk private region uses slice 4, and the public
region is slice 3, which overlaps the entire disk.
Additionally, there are two overlay partitions for the / and swap partitions
that block-for-block match the original / and swap partition locations.
This is an important distinction. The overlay partitions are used during
initial boot processing prior to the initialization of the vxconfigd
daemon. Because the data in an encapsulated disk is not overwritten, the
overlay partitions map to the proper data blocks. This allows you access
to them without having to perform an active vxconfigd process to access
the rootvol and swapvol volumes.
*
* Dimensions:
*
512 bytes/sector
*
133 sectors/track
*
27 tracks/cylinder
*
*
4926 cylinders
*
*
* Flags:
*
1: unmountable
* 10: read-only
*
*
First
Sector
Last
*
Sector
Count
Sector
*
15581349 4279385947 4294967295
*
17682084 4292866561 15581348
*
*
First
Sector
Last
Sector
Count
Sector Mount Directory
0
2
00
0 15581349 15581348 <-- Overlay partition for /

Encapsulating Disks
2-39

1
2
3
4
3
5
14
15
01
00
01
01
15584940
0
0
15581349
2097144
17682084
17682084
3591
17682083
17682083
17682083
15584939
<-<-<-<--
Overlay partition for swap

Backup
Public Region
Private Region
Post-Encapsulation format Utility print Option

The following example of the print selection from the format utility
shows another view of the encapsulated disk partitioning scheme.
partition> print
Part
Tag
0
root
1
swap
2
backup
3
4
5 unassigned
6 unassigned
7 unassigned
Flag
wm
wu
wm
wu
wu
wm
wm
wm
Cylinders
0 - 4338
4340 - 4923
0 - 4923
0 - 4923
4339 - 4339
0
0
0
Size
7.43GB
1024.00MB
8.43GB
8.43GB
1.75MB
0
0
0
Blocks
(4339/0/0) 15581349
(584/0/0)
2097144
(4924/0/0) 17682084
(4924/0/0) 17682084
(1/0/0)
3591
(0/0/0)
0
(0/0/0)
0
(0/0/0)
0
<-- Overlay
<-- Overlay
<-- Public
<-- Private
Post-Encapsulation vxprint -ht (No Mirror) Command

A vxprint command example shows the following:
A VxVM software disk called rootdisk The rootdisk is the

default disk name for an encapsulated disk when using the
vxinstall or the vxdiskadm utility to perform the encapsulation.
Two plexes for the rootvol volume, as follows:
rootdisk-B0 A shadow subdisk used to preserve an

encapsulated disk boot block.
rootdisk-02 The encapsulated / partition.
One plex for the swapvol volume Subdisk rootdisk-01 is the

encapsulated swap partition.
bash-2.03# vxprint -ht

Disk group: rootdg
DG
DM
RV
RL
V
PL
SD
NAME
NAME
NAME
NAME
NAME
NAME
NAME
2-40
NCONFIG
DEVICE
RLINK_CNT
RVG
RVG
VOLUME
PLEX
NLOG
TYPE
KSTATE
KSTATE
KSTATE
KSTATE
DISK
MINORS
PRIVLEN
STATE
STATE
STATE
STATE
DISKOFFS
GROUP-ID
PUBLEN
PRIMARY
REM_HOST
LENGTH
LENGTH
LENGTH
STATE
DATAVOLS
REM_DG
READPOL
LAYOUT
[COL/]OFF
SRL
REM_RLNK
PREFPLEX UTYPE
NCOL/WID MODE
DEVICE
MODE


SV NAME
DC NAME
SP NAME
PLEX
PARENTVOL
SNAPVOL
VOLNAME
LOGVOL
DCO
NVOLLAYR LENGTH
[COL/]OFF AM/NM
dg rootdg
default
default
1019673916.1025.lowtide
dm rootdisk
c1t0d0s2
sliced
3590
17682083 -
MODE
sd rootdiskPriv -
rootdisk 15581349 3590
PRIVATE
c1t0d0
ENA
v
pl
sd
sd
ROUND
CONCAT
0
1
c1t0d0
c1t0d0
root
RW
ENA
ENA
ROUND
CONCAT
0
c1t0d0
swap
RW
ENA
rootvol
rootvol-01
rootdisk-B0
rootdisk-02
rootvol
rootvol-01
rootvol-01
ENABLED
ENABLED
rootdisk
rootdisk
ACTIVE
ACTIVE
15581348
0
15581349
15581349
1
15581348
v swapvol
pl swapvol-01
sd rootdisk-01
swapvol
swapvol-01
ENABLED ACTIVE
2097144
ENABLED ACTIVE
2097144
rootdisk 15584939 2097144
Mirroring the Encapsulated Boot Disk

Mirroring an encapsulated boot disk is accomplished using either the
vxdiskadm or vxrootmir utility. The boot disk mirror used in this
example was created using the vxdiskadm utility.
Post-Encapsulation vxprint -ht Command With Mirror

Notice in the following example output that each system volume (root
and swap) is now mirrored using a VxVM software disk called
rootmirror. The example rootmirror disk provides the following
plexes:
rootvol - plex = rootvol-02 / sd = rootmirror-01
swapvol - plex = swapvol-02 / sd = rootmirror-02
bash-2.03# vxprint -ht -g rootdg

DG NAME
NCONFIG
NLOG
DM NAME
DEVICE
TYPE
RV NAME
RLINK_CNT
KSTATE
RL NAME
RVG
KSTATE
V NAME
RVG
KSTATE
PL NAME
VOLUME
KSTATE
SD NAME
PLEX
DISK
SV NAME
PLEX
VOLNAME
DC NAME
PARENTVOL
LOGVOL
SP NAME
SNAPVOL
DCO
dg rootdg
default
default
MINORS
PRIVLEN
STATE
STATE
STATE
STATE
DISKOFFS
NVOLLAYR
GROUP-ID
PUBLEN
PRIMARY
REM_HOST
LENGTH
LENGTH
LENGTH
LENGTH
1019673916.1025.lowtide
STATE
DATAVOLS
REM_DG
READPOL
LAYOUT
[COL/]OFF
[COL/]OFF
SRL
REM_RLNK
PREFPLEX
NCOL/WID
DEVICE
AM/NM

Encapsulating Disks
UTYPE
MODE
MODE
MODE
2-41

dm rootdisk
dm rootmirror
c1t0d0s2
c1t22d0s2
sliced
sliced
3590
3590
17678493 17674902 -
v
pl
sd
sd
pl
sd
rootvol
rootvol-01
rootvol
rootvol-02
rootvol
rootmirror-01 rootvol-02
ENABLED ACTIVE
ENABLED ACTIVE
rootdisk 17678492
rootdisk 0
ENABLED ACTIVE
rootmirror 0
15581349
15581349
1
15581348
15581349
15581349
ROUND
CONCAT
0
1
CONCAT
0
c1t0d0
c1t0d0
c1t22d0
root
RW
ENA
ENA
RW
ENA
v
pl
sd
pl
sd
swapvol
swapvol-01
swapvol
rootdisk-01 swapvol-01
swapvol-02
swapvol
rootmirror-02 swapvol-02
ENABLED ACTIVE
1052163 ROUND
ENABLED ACTIVE
1052163 CONCAT
rootdisk 15584939 1052163 0
ENABLED ACTIVE
1052163 CONCAT
rootmirror 15581349 1052163 0
c1t0d0
c1t22d0
swap
RW
ENA
RW
ENA
Post-Encapsulation vxdisk list Command

Example output from the vxdisk list command shows both the
encapsulated boot disk (rootdisk) and its mirror (rootmirror). Both
disks must be members of the rootdg disk group.
DEVICE
TYPE
c1t0d0s2
sliced
c1t1d0s2
sliced
c1t2d0s2
sliced
c1t3d0s2
sliced
c1t6d0s2
sliced
c1t16d0s2
sliced
c1t17d0s2
sliced
c1t18d0s2
sliced
c1t19d0s2
sliced
c1t20d0s2
sliced
c1t21d0s2
sliced
c1t22d0s2
sliced
DISK
rootdisk
rootmirror
GROUP
rootdg
rootdg
STATUS
online
error
error
error
error
error
error
error
error
online
online
online
Post-Encapsulation Boot Disk prtvtoc Command

The following example of the prtvtoc command output for the boot disk
displays an encapsulation partition scheme with overlay partitions for /
and swap. Notice the starting and stopping sectors for each overlay
partition.
*
* Dimensions:
*
512 bytes/sector
*
133 sectors/track
*
27 tracks/cylinder
2-42


*
*
4926 cylinders
*
*
* Flags:
*
1: unmountable
* 10: read-only
*
*
First
Sector
Last
*
Sector
Count
Sector
*
15581349 4279385947 4294967295
*
17682084 4292870152 15584939
*
16637103
1041390 17678492
*
*
First
Sector
Sector
Count
0
2
00
0 15581349
1
3
01
15584940
1052163
2
5
00
0 17682084
3
14
01
0 17682084
4
15
01
17678493
3591
Last
Sector
15581348
16637102
17682083
17682083
17682083
Mount Directory
Post-Encapsulation Boot Disk Mirror prtvtoc Command

The following example shows the output from the prtvtoc command for
the root mirror disk. Notice that it not only mirrors the public and private
regions of the boot disk, but additionally creates overlay partitions for /
and swap. The partitions are configured in this manner, so the mirror disk
can be used to boot the system if the primary fails. The difference in the
starting sector for overlay partition 0 is because this is a VxVM softwareinitialized disk which has a private region configured within the first two
cylinders of the disk. This causes the offset to sector 7182 for the overlay
of partition 0.
*
* Dimensions:
*
512 bytes/sector
*
133 sectors/track
*
27 tracks/cylinder
*
*
4926 cylinders
*
*
* Flags:
*
1: unmountable
* 10: read-only
*

Encapsulating Disks
2-43

*
First
Sector
Last
*
Sector
Count
Sector
*
3591
3591
7181
*
15588531 4279385947
7181
*
17682084 4292873743 15588530
*
16640694
1041390 17682083
*
*
First
Sector
Sector
Count
0
2
00
7182 15581349
1
3
01
15588531
1052163
2
5
00
0 17682084
3
15
01
0
3591
4
14
01
7182 17674902
Last
Sector
15588530
16640693
17682083
3590
17682083
Mount Directory
Post-Encapsulation Boot Disk format print Command

The format utility print option for the example rootdisk shows the
boot disk slicing on cylinder boundaries.
partition> print
Part
Tag
0
root
1
swap
2
backup
3
4
5 unassigned
6 unassigned
7 unassigned
Flag
wm
wu
wm
wu
wu
wm
wm
wm
Cylinders
0 - 4338
4340 - 4632
0 - 4923
0 - 4923
4923 - 4923
0
0
0
Size
7.43GB
513.75MB
8.43GB
8.43GB
1.75MB
0
0
0
Blocks
(4339/0/0) 15581349
(293/0/0)
1052163
(4924/0/0) 17682084
(4924/0/0) 17682084
(1/0/0)
3591
(0/0/0)
0
(0/0/0)
0
(0/0/0)
0
partition>
Post-Encapsulation Boot Disk Mirror format print Command

The following example output from the format utility print option is for
the rootmirror disk. Notice how the private region (slice 3) occupies
cylinder 0. The private region (slice 4) and overlay partition 0 start at
cylinder 2. Contrast this with the output for the encapsulated boot disk.
partition> print
Part
0
1
2
2-44
Tag
root
swap
backup
Flag
wm
wu
wm
Cylinders
2 - 4340
4341 - 4633
0 - 4923
Size
7.43GB
513.75MB
8.43GB
Blocks
(4339/0/0) 15581349
(293/0/0)
1052163
(4924/0/0) 17682084


3
4
5 unassigned
6 unassigned
7 unassigned
partition>
wu
wu
wm
wm
wm
0 0
2 - 4923
0
0
0
1.75MB
8.43GB
0
0
0
(1/0/0)
3591
(4922/0/0) 17674902
(0/0/0)
0
(0/0/0)
0
(0/0/0)
0
Related Post-Encapsulation Files

When a disk is encapsulated, a subdirectory under
/etc/vx/reconfig.d/disk.d is created which contains files that hold
information related to the encapsulation. The following is an example of
files that relate to the encapsulation of c1t0d0.
File List
A directory list of files specific to c1t0d0 is shown in the following
example.
bash-2.03# pwd
/etc/vx/reconfig.d/disk.d/c1t0d0
bash-2.03# ls -las
total 12
2 drwxr-xr-x
2
2 drwxr-xr-x
4
2 -rw-r--r-1
2 -rw-r--r-1
2 -rw-r--r-1
2 -rw-r--r-1
root
root
root
root
root
root
other
other
other
other
other
other
512
512
9
827
7
452
Apr
Apr
Apr
Apr
Apr
Apr
26
24
26
26
26
26
19:17
15:29
19:17
19:17
19:17
19:17
.
..
dmname
newpart
primary_node
vtoc
The dmname File

The dmname file for this example lists the VxVM software disk name of the
encapsulated disk.
bash-2.03# more *
::::::::::::::
dmname
::::::::::::::
rootdisk
::::::::::::::

Encapsulating Disks
2-45
The newpart File

The following example of the newpart file displays the new partition
information for this encapsulated disk. Additionally, it lists all the VxVM
software commands executed to successfully complete the encapsulation
process.
::::::::::::::
newpart
::::::::::::::
# volume manager partitioning for drive c1t0d0
0 0x2 0x200
0 15581349
1 0x3 0x201 15584940 1052163
2 0x5 0x200
0 17682084
3 0xe 0x201
0 17682084
4 0xf 0x201 17678493
3591
5 0x0 0x000
0
0
6 0x0 0x000
0
0
7 0x0 0x000
0
0
#vxmake vol rootvol plex=rootvol-%%00 usetype=root logtype=none
#vxmake plex rootvol-%%00 sd=rootdisk-B0,rootdisk-%%00
#vxmake sd rootdisk-%%00 disk=rootdisk offset=0 len=15581348
#vxmake sd rootdisk-B0 disk=rootdisk offset=17678492 len=1 putil0=Block0 comment="Remap
of block 0
#vxvol start rootvol
#rename c1t0d0s0 rootvol
#vxmake vol swapvol plex=swapvol-%%01 usetype=swap
#vxmake plex swapvol-%%01 sd=rootdisk-%%01
#vxmake sd rootdisk-%%01 disk=rootdisk offset=15584939 len=1052163
#vxvol start swapvol
#rename c1t0d0s1 swapvol
The primary_node File

The primary_node file lists the original c#t#d# name of the encapsulated
disk in this example.
::::::::::::::
primary_node
::::::::::::::
c1t0d0
::::::::::::::
2-46

The vtoc File

The following example of the vtoc file saves the original partition table of
the encapsulated disk. In earlier versions of the VxVM software, this file
was modified and used with the /etc/fmthard command to re-partition
disks during the unencapsulation process. The vxmksdpart command
now performs this function.
::::::::::::::
./vtoc
::::::::::::::
#THE PARTITIONING OF /dev/rdsk/c1t0d0s2 IS AS FOLLOWS:
#SLICE
TAG FLAGS
START
SIZE
0
0x2 0x200
0
15581349
1
0x3 0x201
15584940
1052163
2
0x5 0x200
0
17682084
3
0x0 0x000
0
0
4
0x0 0x000
0
0
5
0x0 0x000
0
0
6
0x0 0x000
0
0
7
0x0 0x000
0
0

Encapsulating Disks
2-47
Examining Sun Enterprise Services Best Practices for VxVM Software-Managed
Examining Sun Enterprise Services Best Practices for

VxVM Software-Managed Boot Disks
The previous sections describe the processes and procedures to
encapsulate and mirror a systems boot disk using standard VxVM
software recommendations from the VERITAS Software Corporation. This
type of encapsulation is found in many sites that have not contracted with
Sun Enterprise Services to install the VxVM software and bring the
systems boot disk under VxVM software management.
The main difference between the standard VxVM software-encapsulated
disk and the Sun Enterprise Services management process is that the Sun
Enterprise Services method does not leave the root disk as an
encapsulated disk. Once the boot disk is mirrored, the encapsulated root
disk is detached, initialized, and mirrored to the original boot disk mirror.
This removes the B0 subdisk and makes the two VxVM disks used to
manage the rootability of the system identical. The complexity of the boot
disk configuration is reduced by providing a consistent slice structure
between the boot disk and its mirrors.
This section addresses the following two methods of bringing a boot disk
under VxVM software management using the Sun Enterprise Services
best practice recommendations:
Manual process using the command line
Scripted process using the Sun Enterprise Installation Services (EIS)

CD-ROM
Note The Sun Enterprise Services best practices for boot disk
management are based on guidelines from the Sun BluePrints OnLine
document Toward a Reference Configuration for VxVM Managed Boot Disks,
part number 806-6197-10.
2-48

Best-Practice Boot Disk Configuration Guidelines

Use the following design goals to configure boot disk media:
Simple Keep the configuration simple. A system administrator

with moderate levels of experience should be able to view the boot
disk and understand the configuration.
Consistent Boot disk configurations that are alike across an

enterprise result in simpler recovery operations and installation
procedures. If system administrators can successfully troubleshoot a
boot disk failure on one system, consistency increases the possibility
that the administrator can do so on other systems.
Resilient Design boot media so that a single hardware, driver or

device failure does not cause a failure. Do not tolerate a single point
of failure.
The following are recommended guidelines for configuring VxVM

software-managed boot media:
Use a maximum of three slices for the boot disk.

The suggested best practice is use only the / and swap slices. If
necessary, also configure the /var slice as a discreet slice.
Never configure /usr as a separate slice.

All the support files and utilities for the VxVM software are located
in the /usr directory. If the /usr volume cannot be mounted, these
support files are unavailable.
Attach mirrors in geographical, not alphabetical, order.

The vxdiskadm process mirrors volumes in alphabetical order. Do
not use vxdiskadm to mirror the boot media. Mirroring the disk in
geographical order ensures that the mirror disk overlay slicing looks
identical to the original boot disk. Depending on the release of the
VxVM software, this is only an issue if the boot disk is sliced with
partitions other than /, swap, and /var.
Convert the boot disk from an encapsulated disk to an initialized

disk.
An encapsulated disk is a special case and may be the only
encapsulated device in the system. Replacing the encapsulated boot
disk with an initialized disk insures that the mirror disk and the boot
disk are exact clones of each other. This reduces the complexity of the
VxVM software-managed boot disk configuration.

Encapsulating Disks
2-49
Map core operating system volumes to disk slices or partitions.

This ensures that the system is bootable with minimal effort if the
VxVM software does not start.
Ensure that all mirrors of the boot disk are bootable and that a device
alias is present in the OpenBoot programmable read-only memory
(PROM).
Create a clone disk.

The clone disk must be bootable from slices, not volumes, and be
able to run VxVM software utilities. This disk is used if there is a
complete failure of a VxVM software-managed boot disk which
makes the system un-bootable.
Manually Bringing the Boot Disk Under VxVM Software

Management
Use the following procedure to configure a boot disk according to the
Sun Enterprise Services best practices.
This example uses the following disks and slicing:
/dev/dsk/c0t0d0 rootdisk
/dev/dsk/c1t1d0 rootmirror
The boot disk has three partitions:
/ Slice 0
swap Slice 1
/var Slice 6
Perform the following:

1.
Save the boot disks vtoc to a file for later reference, if needed. Type
the following:
# mkdir /var/tmp/sysdocs
# prtvtoc /dev/rdsk/c0t0d0s2 > /var/tmp/sysdocs/rootdisk_prevm.vtoc
2.
2-50
Encapsulate the boot disk, using either the vxinstall or vxdiskadm

utility. Boot disk encapsulation is usually completed during the
installation and initial setup of the VxVM software by using the
vxinstall utility.


3.
Initialize the disk to be used as the rootmirror and add it to the

rootdg disk group. Type the following:
# /etc/vx/bin/vxdisksetup -i c1t1d0
# vxdg -g rootdg adddisk rootmirror=c1t1d0
4.
Attach the mirrors in the order of the slices on the encapsulated boot
disk.
Note If the boot disk was sliced with swap as the first slice, reverse the
order of mirroring for the / and swap slices.
Perform the following steps:
a.
Use the following command to start the procedure to mirror /

to the rootmirror disk:
# /etc/vx/bin/vxrootmir rootmirror&
b.
Use the following command to start the procedure to mirror the

swap slice:
# vxassist -g rootdg mirror swapvol rootmirror&
c.

/var slice:
# vxassist -g rootdg mirror var rootmirror&
Use the following procedure to display the progress of the mirroring

process:
#while true
> do
> vxtask list
> sleep 15
> echo ##################
> done
TASKID PTID TYPE/STATE
PCT
160
ATCOPY/R 84.39%
163
ATCOPY/R 24.95%
###############
TASKID PTID TYPE/STATE
PCT
160
ATCOPY/R 86.00%
163
ATCOPY/R 26.57%
PROGRESS
0/4197879/3542680 PLXATT rootvol rootvol-02
0/4197879/1047304 PLXATT var var-02
PROGRESS
0/4197879/3610384 PLXATT rootvol rootvol-02
0/4197879/1115256 PLXATT var var-02
Wait for the mirroring procedure to complete before continuing this

procedure.
5.
#
#
#
#
vxplex
vxplex
vxplex
vxedit
Dissociate the rootdisk plexes. Type the following:
-g rootdg dis rootvol-01

-g rootdg dis swapvol-01
-g rootdg dis var-01
-fr rm rootvol-01 swapvol-01 var-01

Encapsulating Disks
2-51

6.
Remove rootdisk from the rootdg disk group:
# vxdg -g rootdg rmdisk rootdisk
7.
Initialize the rootdisk and add it back to the rootdg disk group:
# /etc/vx/bin/vxdisksetup -i c0t0d0
# vxdg -g rootdg adddisk rootdisk=c0t0d0
8.
Attach the mirrors in the order of the slices on the encapsulated boot
disk. This is similar to the process outlined in Step 4.
Note If the boot disk was sliced with swap as the first slice, reverse the
order of mirroring for the / and swap slices.
Perform the following steps:
a.
Use the following command to start the procedure to mirror /

to the rootmirror disk:
# /etc/vx/bin/vxrootmir rootdisk&
b.

swap slice:
# vxassist -g rootdg mirror swapvol rootdisk&
c.

/var slice:
# vxassist -g rootdg mirror var rootdisk&
Use the following procedure to display the progress of the mirroring

process:
#while true
> do
> vxtask list
> sleep 15
> echo ##################
> done
Wait for the mirroring procedure to complete before continuing this

procedure.
9.
If needed, create the underlying partitions by creating overlay

partitions for each partition on the boot disk. Use the vxmksdpart
command to create the overlay partitions; vxmksdpart requires the
partition name, flags, and tags as input.
Note Depending on the version of the VxVM software used, building

overlay partitions for /, swap and /var might not be necessary.
2-52


A list of valid flags is shown in Table 2-1.
Table 2-1 Partition Flags
Name
Flag
Mountable, Read and Write
0x00
Not Mountable
0x01
Mountable, Read Only
0x10
Valid tags are listed in Table 2-2.

Table 2-2 Partition Tags
Name
Tag
UNASSIGNED
0x00
BOOT
0x01
ROOT
0x02
SWAP
0x03
USR
0x04
BACKUP
0x05
STAND
0x06
VAR
0x07
HOME
0x08
ALTSCTR
0x09
CACHE
0x0a
VxVM PRIVATE REGION
0x15
VxVM PUBLIC REGION
0x14

Encapsulating Disks
2-53

Use vxprint and prtvtoc to print partitioning and subdisk
information to identify which overlay partitions must be built. These
commands are shown in the following example.
# vxprint -qhtg rootdg
dg rootdg
default
default
1023554924.1025.lowtide
dm rootdisk
dm rootmirror
sliced
sliced
3590
3590
17678493 17674902 -
c1t0d0s2
c1t1d0s2
v
pl
sd
sd
pl
sd
rootvol
rootvol-01
rootvol
rootvol-02
rootvol
ENABLED ACTIVE
ENABLED ACTIVE
rootdisk 17678492
rootdisk 0
ENABLED ACTIVE
rootmirror 0
v
pl
sd
pl
sd
swapvol
swapvol-01
swapvol
swapvol-02
swapvol
ENABLED ACTIVE
1052163
ENABLED ACTIVE
1052163
rootdisk 4197878 1052163
ENABLED ACTIVE
1052163
rootmirror 4197879 1052163
v
pl
sd
pl
sd
var
var-01
var
rootdisk-03 var-01
var-02
var
rootmirror-03 var-02
ENABLED ACTIVE
4197879 ROUND
ENABLED ACTIVE
4197879 CONCAT
rootdisk 9447920 4197879 0
ENABLED ACTIVE
4197879 CONCAT
rootmirror 5250042 4197879 0
#
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
4197879
4197879
1
4197878
4197879
4197879
ROUND
CONCAT
0
1
CONCAT
0
c1t0d0
c1t0d0
c1t1d0
root
RW
ENA
ENA
RW
ENA
ROUND
CONCAT
0
CONCAT
0
c1t0d0
c1t1d0
swap
RW
ENA
RW
ENA
fsgen
RW
c1t0d0
ENA
RW
c1t1d0
ENA
Dimensions:
512 bytes/sector
133 sectors/track
27 tracks/cylinder
4926 cylinders
Flags:
1: unmountable
10: read-only
Unallocated space:
First
Sector
Last
Sector
Count
Sector
3591
3591
7181
4205061 4290769417
7181
17682084 4281490273
4205060
9455103
8226981 17682083
2-54


*
* Partition
0
1
2
3
4
Tag
2
3
5
15
14
Flags
00
01
00
01
01
First
Sector
Last
Sector
Count
Sector
7182
4197879
4205060
4205061
1052163
5257223
0 17682084 17682083
0
3591
3590
7182 17674902 17682083
Mount Directory
Note In this example, the capture was modified to show the /var slice
as not built. This was done to help show how to use the vxmksdpart
command to create overlay partitions. The VxVM software version 3.2
creates overlay partitions for /, swap and /var, and thus you do not need
to execute any additional commands. If /opt or other non-system
partitions are defined on the boot disk, use vxmksdpart to define those
partitions.
In this example, an overlay partition must be built for the /var slice.
The subdisk information listed in Table 2-3 is needed as input to the
vxmksdpart command.
Table 2-3 Required subdisk Information
Subdisk
Slice
Tag
Flag
rootdisk-03
0x07
0x00
rootmirror-03
0x07
0x00
To create the underlying partitions for /var, type the following:

# /usr/vx/bin/vxmksdpart -g rootdg rootdisk-03 6 0x07 0x00
# /usr/vx/bin/vxmksdpart -g rootdg rootmirror-03 6 0x07 0x00
This example output from the prtctoc command now shows the
/var partition as an overlay partition.
#
*
*
*
*
*
*
*
*
*
*
*
*
*
Dimensions:
512 bytes/sector
133 sectors/track
27 tracks/cylinder
4926 cylinders
Flags:
1: unmountable
10: read-only

Encapsulating Disks
2-55

*
*
First
Sector
Last
*
Sector
Count
Sector
*
3591
3591
7181
*
4205061 4290769417
7181
*
17682084 4281490273
4205060
*
9455103
8226981 17682083
*
*
First
Sector
Last
Sector
Count
0
2
00
7182
4197879
4205060
1
3
01
4205061
1052163
5257223
2
5
00
0 17682084 17682083
3
15
01
0
3591
3590
4
14
01
7182 17674902 17682083
6
7
00
5257224
4197879
9455102
10. Set the dump device to a non-VxVM software disk, if available. If

such a disk is not available, use the boot disk. Type the following:
# dumpadm -d /dev/dsk/c0t0d0s1
11. Create the OpenBoot PROM device aliases, if needed. Build the
aliases using the eeprom commands nvedit or nvalias at the
OpenBoot PROM prompt.
Note Depending on the version of VxVM software installed, this step
might not be necessary.
Scripted Process Using the EIS CD-ROM

The EIS CD-ROM contains a script that mirrors the boot disk using the
same process as the manual best practices procedures. It is located on the
EIS CD-ROM at the following location:
/cdrom/eis-cd2/sun-internal/tools/VXTOOLS/K.Vietmeier.scripts/mirrorboot_<rev>.ksh
Caution This scripts is distributed as is and is not supported by Sun

Microsystems. To gain familiarity with the process, try performing the
manual procedure multiple times prior to using this script.
2-56

Unencapsulating Boot Disks

To unencapsulate boot disks (remove rootability), use the vxunroot utility
or perform the process manually. Use the manual method only if the
vxunroot utility fails to unencapsulate the boot disk. The manual method
is similar to the process used to unencapsulate data disks, described in
This section addresses multiple methods of boot disk unencapsulation.
Additional information about boot disk unencapsulation is located in
SunSolve INFODOC 24663 (see Appendix A, SunSolve INFODOCs).
Boot disk unencapsulation always requires a reboot of the system.
Caution Back up all volumes to be unencapsulated. Use these backups
for recovery purposes if the process fails and the encapsulated volumes
are corrupted.
Unencapsulating a Boot Disk Using the vxunroot

Utility
The following process is based on best practice recommendations from the
VERITAS Software Corporation TechNote ID 244678. This example
depicts a two-slice boot disk:
1.
bash-2.03# df -n
/
/proc
/dev/fd
/etc/mnttab
/var/run
/tmp
Unmount all file systems that are under the control of the VxVM
software except the encapsulated boot disk file systems (/, usr, var,
and opt). Type the following:
:
:
:
:
:
:
2.
ufs
proc
fd
mntfs
tmpfs
tmpfs
Record and save the current configuration of rootdg using the

vxprint -qhtg rootdg command. This preserves a copy of the
rootdg configuration prior to unencapsulation, as shown in the
following example.
bash-2.03# vxprint -qhtg rootdg >/rootdg.print

bash-2.03# more /rootdg.print
dg rootdg
default
default
1019673916.1025.lowtide

Encapsulating Disks
2-57
dm rootdisk
dm rootmirror
c1t0d0s2
c1t22d0s2
sliced
sliced
3590
3590
17678493 17674902 -
v
pl
sd
sd
pl
sd
rootvol
rootvol-01
rootvol
rootvol-02
rootvol
ENABLED ACTIVE
ENABLED ACTIVE
rootdisk 17678492
rootdisk 0
ENABLED ACTIVE
rootmirror 0
ROUND
CONCAT
0
1
CONCAT
0
c1t0d0
c1t0d0
c1t22d0
root
RW
ENA
ENA
RW
ENA
v
pl
sd
pl
sd
swapvol
swapvol-01
swapvol
swapvol-02
swapvol
ENABLED ACTIVE
1052163 ROUND
ENABLED ACTIVE
1052163 CONCAT
rootdisk 15584939 1052163 0
ENABLED ACTIVE
1052163 CONCAT
rootmirror 15581349 1052163 0
c1t0d0
c1t22d0
swap
RW
ENA
RW
ENA
3.
15581349
15581349
1
15581348
15581349
15581349
Capture the current configuration of the disk using the vxdisk

utility:
bash-2.03# vxdisk -o alldgs list > /vxdisk_alldgs.list
4.
Detach all plexes associated with the rootmirror disk. Type the
following:
bash-2.03# vxprint -qhtg rootdg -s | grep -i rootmirror | awk {print $3}> /rmsub.plex
bash-2.03# more /rmsub.plex
rootvol-02
swapvol-02
bash-2.03# for i in cat /rmsub.plex
> do
> vxplex -g rootdg dis $i
> vxprint -qhtg rootdg -p $i
> done
pl rootvol-02
sd rootmirror-01 rootvol-02
pl swapvol-02
sd rootmirror-02 swapvol-02
bash-2.03#
DISABLED 15581349 CONCAT

rootmirror 0
15581349 0
rootmirror 15581349 1052163 0
c1t22d0
c1t22d0
RW
ENA
RW
ENA
Adding the rm option to the vxplex command removes the mirror

plexes in addition to performing the disable operation. The
command syntax is as follows:
# vxplex -g rootdg -o rm dis plex_name
If executed within a loop, the command syntax is
> vxplex -g rootdg -o rm dis $i
5.
2-58
Verify that all rootmirror plexes were detached. Use the vxprint
command as follows:


bash-2.03# vxprint -qhtg rootdg
dg rootdg
default
default
1019673916.1025.lowtide
dm rootdisk
dm rootmirror
3590
3590
17678493 17674902 -
c1t0d0s2
c1t22d0s2
sliced
sliced
pl rootvol-02
sd rootmirror-01 rootvol-02
DISABLED rootmirror 0
15581349 CONCAT
15581349 0
c1t22d0
RW
ENA
pl swapvol-02
sd rootmirror-02 swapvol-02

rootmirror 15581349 1052163 0
c1t22d0
RW
ENA
v
pl
sd
sd
ROUND
CONCAT
0
1
c1t0d0
c1t0d0
root
RW
ENA
ENA
ROUND
CONCAT
0
c1t0d0
swap
RW
ENA
rootvol
rootvol-01
rootdisk-B0
rootdisk-02
rootvol
rootvol-01
rootvol-01
ENABLED
ENABLED
rootdisk
rootdisk
ACTIVE
ACTIVE
17678492
0
15581349
15581349
1
15581348
v swapvol
pl swapvol-01
sd rootdisk-01
swapvol
swapvol-01
ENABLED ACTIVE
1052163
ENABLED ACTIVE
1052163
rootdisk 15584939 1052163
Successful completion of the previous steps is critical to the success

of this process. If all plexes from the mirror are not disabled, the
vxunroot utility fails.
6.
Remove rootability using the vxunroot utility as follows:
bash-2.03# /etc/vx/bin/vxunroot
This operation will convert the following file systems from
volumes to regular partitions: root swap usr var opt home
Replace volume rootvol with c1t0d0s0.
This operation will require a system reboot. If you choose to
continue with this operation, system configuration will be updated
to discontinue use of the volume manager for your root and swap
devices.
Do you wish to do this now [y,n,q,?] (default: y) y
Restoring kernel configuration...
A shutdown is now required to install the new kernel.
You can choose to shutdown now, or you can shutdown later, at your
convenience.
Do you wish to shutdown now [y,n,q,?] (default: n) n
Please shutdown before you perform any additional volume manager
or disk reconfiguration. To shutdown your system cd to / and type
shutdown -g0 -y -i6
bash-2.03# init 6

Encapsulating Disks
2-59

7.
bash-2.03# df -k
Filesystem
/dev/dsk/c1t0d0s0
/proc
fd
mnttab
swap
swap
bash-2.03# swap -l
swapfile
/dev/dsk/c1t0d0s1
After the reboot completes, check the devices used for the / and
swap partitions. Verify that these partitions use non-VxVM software
objects:
kbytes
used
avail capacity
7670973 1697985 5896279
23%
0
0
0
0%
0
0
0
0%
0
0
0
0%
655784
16 655768
1%
655792
24 655768
1%
Mounted on
/
/proc
/dev/fd
/etc/mnttab
/var/run
/tmp
dev swaplo blocks

free
118,145
16 1052144 1052144
Manually Unencapsulating a Boot Disk

The following procedure has much in common with the unencapsulation
process used for data disks. Only use this procedure if the vxunroot
utility fails to unencapsulate the boot disk.
The following procedure is based on VERITAS Software Corporation
TechNote ID 244678:
1.
bash-2.03# df -n
/
/proc
/dev/fd
/etc/mnttab
/var/run
/tmp
Unmount all the file systems that are under the control of the VxVM
software, except the encapsulated boot disk file systems (/, usr, var,
and opt). Type the following:
:
:
:
:
:
:
2.
ufs
proc
fd
mntfs
tmpfs
tmpfs
Record the current configuration of rootdg using the

vxprint -qhtg rootdg command. This preserves a copy of the
rootdg configuration prior to unencapsulation. Type the following:
bash-2.03# vxprint -qhtg rootdg >/rootdg.print
3.
Capture the current disk configuration using the vxdisk utility:
bash-2.03# vxdisk -o alldgs list > vxdisk_alldgs.list
2-60


4.
Detach all plexes associated with the rootmirror disk. Use the
following script:
bash-2.03# vxprint -qhtg rootdg -s | grep -i rootmirror | awk {print $3}> /rmsub.plex
bash-2.03# for i in cat ./rmsub.plex
> do
> vxplex -g rootdg -o rm dis $i
> done
5.
Verify that all rootmirror plexes were detached using the vxprint
command as follows:

dg rootdg
default
default
1019673916.1025.lowtide
dm rootdisk
dm rootmirror
c1t0d0s2
c1t22d0s2
sliced
sliced
3590
3590
17678493 17674902 -
v
pl
sd
sd
rootvol
rootvol-01
rootdisk-B0
rootdisk-02
rootvol
rootvol-01
rootvol-01
ENABLED
ENABLED
rootdisk
rootdisk
ACTIVE
ACTIVE
17678492
0
15581349
15581349
1
15581348
v swapvol
pl swapvol-01
sd rootdisk-01
swapvol
swapvol-01
ENABLED ACTIVE
1052163
ENABLED ACTIVE
1052163
rootdisk 15584939 1052163
ROUND
CONCAT
0
1
c1t0d0
c1t0d0
root
RW
ENA
ENA
ROUND
CONCAT
0
c1t0d0
swap
RW
ENA
Successful completion of the previous steps are critical to the success

of this process. If all plexes from the mirror are not disabled, the
vxunroot utility fails.
6.
Remove rootability using the following manual procedure:

a.
Remove the /etc/vx/reconfig.d/state.d/root-done file.

Typethe following:
bash-2.03# rm /etc/vx/reconfig.d/state.d/root-done
Removal of this file tells the VxVM software that the root disk is
no longer encapsulated.
b.
Edit the /etc/system and /etc/vfstab files back to their preencapsulation state. Be sure to back up each file prior to editing.
Perform the following tasks:
Remove the following lines from the /etc/system file

rootdev:/pseudo/vxio@0:0
set vxio:vol_rootdev_is_volume=1
Caution Do not comment out these lines, remove them.

Encapsulating Disks
2-61
Restore the /etc/vfstab file to its pre-encapsulation state.

Edit the /etc/vfstab file and modify the mount
statements for /, usr, var, and swap to use the original
physical devices. The original partitions were preserved
through the use of overlay partitions.
Alternatively, copy the /etc/vfstab.prevm file to
/etc/vfstab. This option is valid only if there were no
changes to the system storage configuration since the boot
disk was encapsulated.
The edited file looks like the following example:
#device
device
mount
#to mount
to fsck
point
#
fd
/dev/fd fd
no
/proc
/proc
proc
no
/dev/dsk/c1t0d0s1
swap
/dev/dsk/c1t0d0s0
/dev/rdsk/c1t0d0s0
swap
/tmp
tmpfs
yes
c.
FS
type
fsck
pass
mount
mount
at boot options
ufs
/
-
yes
no
ufs
no
logging
If non-system (/, usr, var, and swap) partitions exist on the

boot disk and do not have an overlap partition already
configured, use the vxmksdpart command to recreate the
partitions.
Note This process is similar to the one used to restore partitions for data
disk unencapsulation.
7.
Reboot the system.
8.
Recursively remove boot volumes from the rootdg disk group:
bash-2.03# vxedit -r -f rm rootvol

bash-2.03# vxedit -r -f rm swapvol
9.
Verify that the volumes are gone. Type the following:

dg rootdg
default
default
1019673916.1025.lowtide
dm rootdisk
dm rootmirror
3590
3590
17678493 17674902 -
c1t0d0s2
c1t22d0s2
sliced
sliced
10. Remove the boot disk from VxVM software control. Type the
following:
bash-2.03# vxdg -g rootdg rmdisk rootdisk
2-62


Caution There must be one disk in rootdg, or the VxVM software does
not start. Do not remove the disk that is used as the root mirror.
11. Verify that the boot disk is removed from VxVM software control:
dg rootdg
default
default
1019673916.1025.lowtide
dm rootmirror
3590
17674902 -
c1t22d0s2
sliced
12. Delete public and private region partitions by using the format
utility. Type the following:
partition> print
Current partition table (unnamed):
Part
Tag
0
root
1
swap
2
backup
3 unassigned
4 unassigned
5 unassigned
6 unassigned
7 unassigned
Flag
wm
wu
wm
wu
wu
wm
wm
wm
Cylinders
0 - 4338
4340 - 4632
0 - 4923
0
0
0
0
0
Size
7.43GB
513.75MB
8.43GB
0
0
0
0
0
Blocks
(4339/0/0) 15581349
(293/0/0)
1052163
(4924/0/0) 17682084
(0/0/0)
0
(0/0/0)
0
(0/0/0)
0
(0/0/0)
0
(0/0/0)
0
13. Run the vxdctl enable command to update the VxVM software
view of installed disks. Type the following:
bash-2.03# vxdctl enable
DEVICE
TYPE
c1t0d0s2
sliced
c1t1d0s2
sliced
c1t2d0s2
sliced
c1t3d0s2
sliced
c1t4d0s2
sliced
c1t5d0s2
sliced
c1t6d0s2
sliced
c1t16d0s2
sliced
c1t17d0s2
sliced
c1t18d0s2
sliced
c1t19d0s2
sliced
c1t20d0s2
sliced
c1t21d0s2
sliced
c1t22d0s2
sliced
DISK
rootmirror
GROUP
rootdg
STATUS
error
error
error
online
online
online
online invalid
error
error
error
error
online
online
online

Encapsulating Disks
2-63
Unencapsulating When Booted From the CD-ROM

If it is not possible to boot a system into single-user mode, booting from
CD-ROM may be the only option for successful system recovery. The
procedure outlined in this section describes the process of booting a
system from the CD-ROM and unencapsulating the boot disk.
document. Check the SunSolve Online web site to determine if the
Note This is a recovery procedure.
To unencapsulate a boot disk when the system is booted from CD-ROM:
1.
Bring the system to the ok prompt and insert a Solaris OE compact

disc (CD) into the CD-ROM drive.
2.
Boot the system to single-user mode from the CD-ROM, as follows:
3.
After the system is booted from the CD-ROM, set the terminal type
so that the vi utility works correctly. Type the following:
ok boot cdrom -s
# TERM=vt100;export TERM
If TERM=sun does not work, try TERM=vt100.

4.
Execute an fsck on the root file system. Type the following:
# fsck -y /dev/rdsk/c#t#d#s0
5.
If the fsck response is clean, mount slice 0 to /a.

If fsck cannot repair the root file system, determine the source of the
problem and correct it. This guide does not contain explanations of
file system corruption or how to repair it. The fsck response must be
clean to continue this procedure.
6.
Mount the root file system to /a. Type the following:
# mount /dev/dsk/c#t#d#s0 /a
7.
Make a backup of /a/etc/system.
# cp /a/etc/system /a/etc/system.orig
8.
Edit the /etc/system file.
# vi /a/etc/system
2-64


9.
Completely remove the following lines from the system file:
Note If the disk is re-encapsulated, these lines are added correctly by the
process, so there is no harm done by removing them.
10. Make a backup of /a/etc/vfstab. Type the following:
# cp /a/etc/vfstab /a/etc/vfstab.orig
11. Edit the vfstab file back to its original state, pointing /, swap, /usr,
and /var to hard partitions on the disk, such as /dev/dsk and
/dev/rdsk, rather than to /dev/vx/ entries. Type the following:
# vi /a/etc/vfstab
12. Temporarily comment out all other /dev/vx volumes from the
/a/etc/vfstab file by using the # character. This includes file
systems like /opt and /export, if they exist.
The original /etc/vfstab looks like the following, assuming root is
c0t0d0.
--------------------------------------------------------------------------/dev/vx/dsk/swapvol
swap
no /dev/vx/dsk/rootvol
/dev/vx/rdsk/rootvol /
ufs
1
no /dev/vx/dsk/usr
/dev/vx/rdsk/usr
/usr
ufs
1
no /dev/vx/dsk/var
/dev/vx/rdsk/var
/var
ufs
1
no /dev/vx/dsk/export
/dev/vx/rdsk/export
/export ufs
2
yes swap
/tmp
tmpfs
yes /dev/vx/dsk/datadg/somevol
/dev/vx/rdsk/datadg/somevol
/somevol
ufs
2 yes -

#NOTE: volume usr (/usr) encapsulated partition c0t0d0s5
#NOTE: volume var (/var) encapsulated partition c0t0d0s6
#NOTE: volume export (/export) encapsulated partition c0t0d0s7
---------------------------------------------------------------------------
Once edited, the vfstab file should look like the following:
--------------------------------------------------------------------------/dev/dsk/c1t0d0s1
swap
no /dev/dsk/c1t0d0s0
/dev/rdsk/c0t0d0s0
/
ufs
1
/dev/rdsk/c0t0d0s5
/usr
ufs
1
/dev/rdsk/c0t0d0s6
/var
ufs
1
no #/dev/dsk/c1t0d0s7
/dev/rdsk/c0t0d0s7
/export ufs
2
yes swap
/tmp
tmpfs
yes #/dev/vx/dsk/datadg/somevol
/somevol
ufs
2 yes -

Encapsulating Disks
2-65

---------------------------------------------------------------------------
13. Make sure that the VxVM software does not start during the next
boot.
# touch /a/etc/vx/reconfig.d/state.d/install-db
This is important because if the root disk contains mirrors, and the
system boots, the mirrors are synced; this corrupts the changes just
made.
14. Remove the flag that tells the VxVM software that the root disk is
encapsulated.
# rm /a/etc/vx/reconfig.d/state.d/root-done
15. Reboot the system for changes to take effect. Type the following:
# reboot
Once rebooted, the system comes up in a partially unencapsulated

state with /, /usr, /var, and swap mounted.
Note The VxVM software does not start. It can be started manually once
the system is booted.
16. Start the VxVM software. Execute the following commands:
#
#
#
#
rm /etc/vx/reconfig.d/state.d/install-db
vxiod set 10
vxconfigd -m disable
vxdctl enable
17. Remove the volumes that existed on the encapsulated boot disk.
These are generally rootvol, swapvol, usr, and var. This might also
include home, opt, or other non-standard root partitions. Use the
command vxprint -htg rootdg to list the volumes in rootdg
before removing them.
Then, for each volume, run the following command:
# /usr/sbin/vxedit -rf rm volume_name
18. Remove the rootdisk from the rootdg disk group. The disk name is
usually rootdisk. Type the following:
# /usr/sbin/vxdg -k rmdisk disk_name
2-66


19. Re-write the vtoc of the disk so that hard partitions are again
defined for the root file systems.
There are several ways to put the hard partitions back into the vtoc
file, including using the fmthard command on a modified
/etc/vx/reconfig.d/disk.d/c#t#d#/vtoc file, using the format
utility to partition the disk manually, or using the vxmksdpart
command. The simplest method, however, is to use the vxedvtoc
command.
a.
When the VxVM software encapsulates a disk, it makes a record

of the old vtoc of the disk. This file is stored for each disk in
/etc/vx/reconfig.d/disk.d/c#t#d#. It is stored in a VxVM
software-specific format, so it cannot be used as an argument
for the fmthard command unless it is modified. The vxedvtoc
command is similar to the fmthard command except that it can
read this vtoc file and write that vtoc to a disk. The command
takes the following form:
vxedvtoc -f filename devicename
b.
Assuming that the boot disk is c0t0d0, run the command as

follows:
# /etc/vx/bin/vxedvtoc -f /etc/vx/reconfig.d/disk.d/c0t0d0/vtoc /dev/rdsk/c0t0d0s2

# THE ORIGINAL PARTITIONING IS AS FOLLOWS:
#SLICE
TAG FLAGS
START
SIZE
0
0x0 0x200
0
0
1
0x0 0x200
0
0
2
0x5 0x201
0 8794112
3
0x0 0x200
0
0
4
0x0 0x200
0
0
5
0x0 0x200
0
0
6
0xe 0x201
0 8794112
7
0xf 0x201 8790016
4096
# THE NEW PARTITIONING WILL BE AS FOLLOWS :
#SLICE
TAG FLAGS
START
SIZE
0
0x0 0x200
0 2048000
1
0x0 0x200 2048000 2048000
2
0x5 0x201
0 8794112
3
0x0 0x201 4096000 2048000
4
0x0 0x201 6144000 2048000
5
0x0 0x200
0
0
6
0x0 0x200
0
0
7
0x0 0x200
0
0
DO YOU WANT TO WRITE THIS TO THE DISK ? [Y/N] :y
WRITING THE NEW VTOC TO THE DISK
This partitions the disk back to a pre-encapsulation state.

20. Uncomment the entries for any of the non-root partitions (/, /usr,
/var, and swap) from /etc/vfstab, as well as any data volumes.

Encapsulating Disks
2-67

In this example, comments were removed from /export and the data
volume /somevol:
# vi /etc/vfstab
/dev/dsk/c0t0d0s1
/dev/dsk/c0t0d0s0
/dev/dsk/c0t0d0s5
/dev/dsk/c0t0d0s6
/dev/dsk/c0t0d0s7
swap
/dev/vx/dsk/datadg/somevol
/dev/rdsk/c0t0d0s0
/
/dev/rdsk/c0t0d0s5
/usr
/dev/rdsk/c0t0d0s6
/var
/dev/rdsk/c0t0d0s7
/export
/tmp
swap
no ufs
1
no ufs
1
no ufs
1
no ufs
2
yes tmpfs
yes /somevol ufs 2 yes -
21. Start all volumes.

# /usr/sbin/vxvol startall
22. Issue a mountall command to mount the now uncommented

volumes.
# mountall
At this point the root disk is completely free of the VxVM software
control. The VxVM software daemons are started, and all system file
systems should be mounted.
Performing a Basic or Functional Unencapsulation

This section describes the process for performing a temporary
unencapsulation of a system boot disk. Use this procedure only to attempt
recovery of a system that does not boot due to problems with the
encapsulated boot disk. This procedure unencapsulates the boot disk and
allows the system to boot from standard Solaris OE partitions while
leaving the root disk under control of the VxVM software.
This process is a useful boot-issue troubleshooting tool because it
performs a temporary unencapsulation of the boot disk, which can be
reversed once the underlying problem is resolved.
document. Check the SunSolve Online Web site to determine if the
Note This is a recovery procedure.
2-68


To perform a basic or functional unencapsulation:
1.
Bring the system to the ok prompt, and insert a Solaris OE CD into

the CD-ROM drive.
2.
Boot system to single-user mode from CD-ROM. Type the following:
ok boot cdrom -s
3.
After the system is booted from the CD-ROM, set the terminal type
so that the vi utility works correctly.
# TERM=vt100;export TERM
If TERM=sun does not work, try TERM=vt100.

4.
Run the fsck utility on the root file system. Type the following:
# fsck -y /dev/rdsk/c#t#d#s0
5.
If the fsck response is clean, mount slice 0 to /a.

If fsck cannot repair the root file system, determine the source of the
problem and correct it. This guide does not contain explanations of
file system corruption or how to repair it. The fsck response must be
clean to continue this procedure.
6.
Mount root file system to /a. Type the following:
# mount /dev/dsk/c#t#d#s0 /a
7.
Make a backup of /a/etc/system.
# cp /a/etc/system /a/etc/system.orig
8.
Edit the /etc/system file. Type the following:
# vi /a/etc/system
9.
Comment out the following lines by using double asterisks (**):
**rootdev:/pseudo/vxio@0:0
**set vxio:vol_rootdev_is_volume=1
10. Make a backup of /a/etc/vfstab. Type the following:

# cp /a/etc/vfstab /a/etc/vfstab.orig
11. Edit the vfstab file back to its original state, pointing /, swap, /usr,
and /var to hard partitions on the disk, such as /dev/dsk and
/dev/rdsk, rather than to /dev/vx/ entries. Type the following:
# vi /a/etc/vfstab
12. Temporarily comment out all other /dev/vx volumes from the
/a/etc/vfstab file by using the # character. This includes file
systems like /opt and /export, if they exist.

Encapsulating Disks
2-69

The original /etc/vfstab looks like the following, assuming root is
c0t0d0:
swap
ufs
1
no /dev/vx/dsk/usr
/dev/vx/rdsk/usr
/usr
ufs
1
no /dev/vx/dsk/var
/dev/vx/rdsk/var
/var
ufs
1
/dev/vx/rdsk/export
/export ufs
2
yes swap
/tmp
tmpfs
/somevol
ufs
2 yes -

---------------------------------------------------------------------------
Once edited, the vfstab looks like the following:

--------------------------------------------------------------------------/dev/dsk/c1t0d0s1
swap
/dev/rdsk/c0t0d0s0
/
ufs
1
/dev/rdsk/c0t0d0s5
/usr
ufs
1
/dev/rdsk/c0t0d0s6
/var
ufs
1
/dev/rdsk/c0t0d0s7
/export ufs
2
yes swap
/tmp
tmpfs
/somevol
ufs
2 yes -

---------------------------------------------------------------------------
13. Make sure the VxVM software does not start during the next boot.
Type the following:
# touch /a/etc/vx/reconfig.d/state.d/install-db
This is important because if the root disk contains mirrors and the
system boots, the mirrors are synced; this corrupts the changes just
made.
14. Remove the flag that tells the VxVM software that the root disk is
encapsulated.
# rm /a/etc/vx/reconfig.d/state.d/root-done
2-70


15. Reboot the system for changes to take effect. Type the following:
# reboot
Once rebooted, the system comes up in an unencapsulated state with

/, /usr, /var, and swap mounted.
At this point a basic or functional unencapsulation is complete. Do not
leave the system in this state system permanently. It is a state that is
useful for troubleshooting and system maintenance. When problems with
the system are resolved and it is ready to be re-encapsulated, perform the
following series of commands:
#
#
#
#
#
touch /etc/vx/reconfig.d/state.d/root-done
cp /a/etc/vfstab.orig /a/etc/vfstab
cp /a/etc/system.orig /a/etc/system
reboot

Encapsulating Disks
2-71
Exploring Unencapsulation Issues

This section addresses various issues that can affect ununcapsulation of
data and boot disks.
Data Disks
Data disks fail unencapsulation if the following is true:
Any encapsulated partition was grown or has a modified layout.
The encapsulated disk failed and was replaced.
The encapsulated disk was non-conforming and was unencapsulated

using the procedure in Encapsulating a Non-Conforming Disk on
page 2-20.
If the data on the disk must be removed from the VxVM software control,
back up the data and restore it to a non-VxVM software disk. Data must
be restored because the encapsulated disks original mapping of partitions
to blocks within the public region changes when the disk is replaced and
synced. This prevents the vxmksdpart command from properly mapping
subdisks within the public region to physical partitions.
Boot Disks
The issues described in this section affect boot disk encapsulation.
Grown File Systems

Do not grow any encapsulated boot disk file systems. This causes
unencapsulation to fail. The only recovery option is to back up and restore
the boot disk from tape.
2-72

Encapsulated Disk Was Replaced

Unlike data disks, if an encapsulated boot disk fails and is replaced, the
disk can be unencapsulated. This is because the /, /usr, /var, and swap
partitions and any other system partitions (such as /opt or /home)
encapsulated on the boot disk are preserved by the mirror. When the
failed boot disk is replaced and re-synced, these preserved and overlay
partitions are copied to the replacement disk and are then available to
successfully unencapsulate the disk.
Depending on the original configuration of the boot disk and the method
of mirroring, the final unencapsulated partition scheme can change from
the pre-encapsulation configuration, as follows:
The /, /usr, /var, and swap partitions There should be no

problems with these partitions; the disk unencapsulates successfully
using both scripted and manual unencapsulation methods. In
particular
On a two-slice boot disk (/ and swap partitions) The

unencapsulated partition scheme is identical to preencapsulation except that slice 0 is moved back one or two
cylinders from the start of the disk. Modifications to the
/etc/vfstab file are unnecessary if manually unencapsulated.
On three- or four-slice boot disks (/, /usr, /var, and swap

partitions) The unencapsulated partition scheme is different
from the pre-encapsulated scheme. The /usr and /var
partitions are relocated to partition 6 and 7 if originally
configured in partitions 3 and 4.
Scripted unencapsulation using the vxunroot command
successfully unencapsulates boot disk and modifies the
/etc/vfstab file to reflect the new location of the /usr and
/var partitions. Manual unencapsulation requires manual
modification of the /etc/vfstab file to reflect the new
locations of the /usr and /var partitions.
Caution If the required modifications to the /etc/vfstab are not made,

the reboot fails. Recovery from this error requires booting from CD-ROM,
mounting the root file system, and editing the /etc/vfstab file.

Encapsulating Disks
2-73
The /, /usr, /var, and swap partitions plus /opt or /home The
partition scheme can change depending on:
Scripted unencapsulation using vxunroot works with no

problems. The final partition scheme is changed from the
original, which is reflected in the /etc/vfstab file. The system
boots with no problems.
Manual unencapsulation can be confusing if the original (preencapsulation) partition scheme is not known. Determining
where the original /, /usr, /var, and swap partitions were
located is easy. Output from the format utility delineates those
partitions. The confusing part is determining which of the
preserved system partitions (such as /opt and /home) is which
if the encapsulated boot disk has both.
Additionally, manual unencapsulation procedures must be
modified to reflect the following:
The vxmksdpart and the vxedvtoc commands are not

necessary because the disk is already partitioned. All
system partitions (/, /usr, /var, and swap) and
encapsulated system partitions (/opt and /home) are
visible. Additionally, the data these commands use is
invalid.
Use the format utility only to remove the public or private

region partitions.
Edit the /etc/vfstab file to reflect the new locations for

any relocated partitions. If /usr was originally in slice 4
and now occupies slice 6, this must be reflected in the
/etc/vfstab file prior to a system reboot.
Note If the Sun Enterprise Services best practices boot disk management
processes are followed, the partitioning concerns described in
Encapsulated Disk Was Replaced on page 2-73 are eliminated.
2-74

Exercise: Encapsulating Disks

In this exercise, you perform the processes required to encapsulate and
unencapsulate disks using the VxVM software commands and utilities.
You will complete the following tasks:
Install the VxVM software
Encapsulate a boot disk
Unencapsulate a boot disk using the vxunroot command
Unencapsulate a boot disk using manual methods
Unencapsulate a boot disk that has a replaced primary mirror
Encapsulate a data disk
Unencapsulate a data disk
Encapsulate a non-conforming data disk
Preparation
To prepare for this exercise:
Identify four disks in addition to the boot disk to use as mirror and
data disks.
Make sure that the boot disk has the /, swap, /usr, /var, and /opt
partitions.
If the boot disk is not configured this way, have the instructor
perform a re-flash installation on your system using the proper boot
disk configuration for this lab.
Ask your instructor for the location of the VxVM software packages,
patches, and supporting Solaris OE software.
Have paper and writing instruments for taking notes.

Encapsulating Disks
2-75
Task 1 Installing the VxVM Software and

Encapsulating a Boot Disk
Complete the following steps:
1.
Open a text editor such as vi and capture the following preencapsulation boot disk information:
The prtvtoc value
The format utility partition print
The df -k output
Contents of the /etc/vfstab file
Save the file as /bootdisk_capture. The information in this file is

used later in this lab exercise.
2.
Install the VxVM software release 3.2 packages.
3.
Install all VxVM software release 3.2 patches as indicated by the

instructor.
4.
Use vxinstall to configure the system as follows:

a.
Do not use enclosure-based naming.
b.
Select custom install.
c.
Encapsulate the boot disk as follows:
d.
2-76
1.
Assign it a VxVM software disk name of rootdisk

(default).
2.
Accept the default private region size.
Leave all other disks alone.
5.
Reboot the system.
6.
Open /bootdisk_capture using a text editor and capture the

following post-encapsulation boot disk information:
The prtvtoc value
The df -k value
The vxprint -qhtg rootdg output


7.
Mirror the encapsulated boot disk using Sun Enterprise Services best
practices procedure (see Examining Sun Enterprise Services Best
Practices for VxVM Software-Managed Boot Disks on page 2-48).
Assign the mirror disk a VxVM software disk name of rootmirror.
8.
While the boot disk is being mirrored, answer the following

questions:
a.
Describe the difference between the pre- and post-encapsulation

boot disk partition configuration. Use the contents of
/bootdisk_capture as a guide.
________________________________________________________
________________________________________________________
________________________________________________________
________________________________________________________
________________________________________________________
b.
What directory contains both pre- and post-encapsulation

configuration information about the boot disk?
________________________________________________________
________________________________________________________
________________________________________________________
c.
State the purpose of the following files:
/etc/vx/reconfig.d/state.d/root-done
_____________________________________________________
_____________________________________________________
/etc/vx/reconfig.d/state.d/install-db
_____________________________________________________
_____________________________________________________
/etc/vfstab.prevm
_____________________________________________________
_____________________________________________________
9.
After the boot disk is mirrored, use vxprint to capture the postmirror configuration. Copy this information to the
/bootdisk_capture file for use later in this lab.

Encapsulating Disks
2-77
Task 2 Unencapsulating a Boot Disk Using the

vxunroot Utility
1.
Unencapsulate the boot disk by using the vxunroot command.

The procedure for this process is found in Examining Sun
Enterprise Services Best Practices for VxVM Software-Managed Boot
Disks on page 2-48, but can be successfully used for a five-slice boot
disk as described in Unencapsulating a Boot Disk Using the
vxunroot Utility on page 2-57.
To unencapsulate a five-slice boot disk, use the following command
syntax in place of that listed in the procedure to execute the loop to
remove the mirrors:
# for i in cat /rmsub.plex
>do
>vxplex -g rootdg -o rm dis $i
>done
This substitution results in full removal of the plexes and their
subdisks.
2.
Was the ununcapsulation successful?

_____________________________________________________________
_____________________________________________________________
3.
How do you know, and what commands do you use to verify this?
_____________________________________________________________
_____________________________________________________________
_____________________________________________________________
_____________________________________________________________
4.
Compare the post-unencapsulation partition configuration of the

boot disk with the pre-unencapsulation and describe the differences.
_____________________________________________________________
_____________________________________________________________
_____________________________________________________________
_____________________________________________________________
2-78

Task 3 Encapsulating a Boot Disk Using the

vxdiskadm Utility
1.
2.
Encapsulate the boot disk using the vxdiskadm utility menu

selection 2. Use the following configuration:
a.
Assign it a VxVM software disk name of rootdisk (default).
b.
Do not configure it as a spare disk.
c.
Accept all other defaults.
One the encapsulation process is complete and the system has

rebooted, mirror the boot disk using the Sun Enterprise Services best
practice procedure (see Examining Sun Enterprise Services Best
Task 4 Manually Unencapsulating a Boot Disk When

Booted From the CD-ROM
1.
Unencapsulate the boot disk using the manual procedure when

booted from CD-ROM described in Unencapsulating When Booted
From the CD-ROM on page 2-64. Be sure to recover the
encapsulated /opt partition using the vxedvtoc command outlined
in this procedure.
Be patientthis procedure reboots the system multiple times.
2.

_____________________________________________________________
_____________________________________________________________
3.
If the unencapsulation was not successful, what do you think went

wrong?
_____________________________________________________________
_____________________________________________________________
_____________________________________________________________
_____________________________________________________________
_____________________________________________________________

Encapsulating Disks
2-79

4.
Is the mistake recoverable?

_____________________________________________________________
_____________________________________________________________
5.
Describe the process used to recover and successfully unencapsulate

the boot disk.
_____________________________________________________________
_____________________________________________________________
_____________________________________________________________
_____________________________________________________________
_____________________________________________________________
_____________________________________________________________
6.
Re-encapsulate the boot disk using the vxdiskadm utility.
7.
After the system reboots, re-mirror the boot disk using the Sun
Enterprise Services best practices procedure (see Examining Sun
Disks on page 2-48). Assign the mirror disk a VxVM software disk
name of rootmirror.
Caution Do not continue the lab without re-mirroring the boot disk. The
next task requires that the boot disk be mirrored.
Task 5 Encapsulating a Data Disk Using the

vxdiskadm Utility (Optional)
1.
Select a disk to be used as a data (non-root) disk for encapsulation.
2.
Use the format utility to create the following partition configuration

for this disk
3.
2-80
Create two partitions minimum (512 megabyte maximum size).
Leave partitions 6 and 7 unallocated.
Build file systems on each of the partitions.


4.
Create mount points for each of the partitions on the data disk and
verify that the new file systems successfully mounts.
5.
Update the /etc/vfstab file to auto-mount these file systems

during system reboot.
6.
Verify that the file systems mount from the /etc/vfstab file.
7.
Use prtvtoc, the df command, and the format utility to capture

pre-encapsulation and mount information to the
/datadisk_capture file. Also capture the contents of the
/etc/vfstab file. This information is used later in this lab
exercise.
8.
Encapsulate this disk using the vxdiskadm utility.

While encapsulating this disk, you are asked for a disk group for this
disk to join. Create a new disk group called datadg, or use rootdg.
9.
After the system reboots, verify that the encapsulation was

successful using the df -k and vxprint commands.
10. Mirror the encapsulated data disk using a disk you identified for this
purpose and the vxdiskadm utility.
11. While the data disk is being mirrored, answer the following
questions:
a.
What are the differences between the pre- and postencapsulation partition configuration for this disk?
________________________________________________________
________________________________________________________
________________________________________________________
b.
How many reboots did the system execute?

________________________________________________________
________________________________________________________
________________________________________________________
c.
Using output from the execution of a df -k command, contrast

the differences in the devices used to mount the newly
encapsulated file systems.
________________________________________________________
________________________________________________________
________________________________________________________

Encapsulating Disks
2-81

d.
View the /etc/vfstab file and describe the differences

between the pre- and post-data disk encapsulation content.
________________________________________________________
________________________________________________________
________________________________________________________
12. Wait for the mirror process to complete before proceeding to the next
task.
Task 6 Unencapsulating a Data Disk (Optional)

1.
Unencapsulate the data disk using the procedure outlined in

2.
Was the unencapsulation successful?

_____________________________________________________________
_____________________________________________________________
3.

wrong?
_____________________________________________________________
_____________________________________________________________
_____________________________________________________________
_____________________________________________________________
4.

_____________________________________________________________
_____________________________________________________________
5.

the data disk.
_____________________________________________________________
_____________________________________________________________
_____________________________________________________________
_____________________________________________________________
_____________________________________________________________
_____________________________________________________________
2-82


6.
Once the unencapsulation successfully completes, contrast the preencapsulation and post-unencapsulation partition scheme of this
disk.
Are there any differences?
_____________________________________________________________
_____________________________________________________________
If yes, list them.
_____________________________________________________________
_____________________________________________________________
_____________________________________________________________
_____________________________________________________________
7.
Unmount and remove all references to the file systems on this disk
from the /etc/vfstab file.
Task 7 Encapsulating a Non-Conforming Data Disk

(Optional)
1.
2.

for this disk:
Create two partitions, minimum, using all the available space

on the disk. Do not leave any free space.
Leave at least one partition unused.
3.
4.
Create mount points for each of the partitions on the data disk, and
verify that the new file systems successfully mount.
5.

6.

Encapsulating Disks
2-83

7.

/etc/vfstab file. This information is used later in this lab exercise.
8.
Encapsulate this disk using the procedure in Encapsulating a NonConforming Disk on page 2-20.
Task 8 Lab House Cleaning

Complete the following
1.
Unencapsulate and remove all references, mount points, and

/etc/vfstab entries for the data disk if the data disk encapsulation
lab tasks were completed.
2.
Encapsulate the boot disk if it is not already encapsulated.
3.
Mirror the boot disk if it is not already mirrored using the Sun
Disks on page 2-48).
Caution It is critical for the successful completion of labs in other

modules that you complete the lab house-cleaning tasks. When you are
finished, only the rootdg disk group should exist with the boot disk
encapsulated and mirrored.
2-84

Exercise Summary
Exercise Summary
!
?
Discussion Take a few minutes to discuss what experiences, issues, or

discoveries you had during the lab exercise.
Experiences
Interpretations
Conclusions
Applications

Encapsulating Disks
2-85

Following are the answers to the lab exercises.
Task 1 Solutions
1.
Open a text editor such as vi and capture the following preencapsulation boot disk information:
The prtvtoc value
The df -k output
Save the file as /bootdisk_capture. The information in this file is

used later in this lab exercise.
2.
Install the VxVM software release 3.2 packages.
3.
Install all VxVM software release 3.2 patches as indicated by the

instructor.
4.
Use vxinstall to configure the system as follows:

a.
Do not use enclosure-based naming.
b.
Select custom install.
c.
Encapsulate the boot disk as follows:
d.
2-86
1.
Assign it a VxVM software disk name of rootdisk

(default).
2.
Accept the default private region size.
Leave all other disks alone.
5.
Reboot the system.
6.
Open /bootdisk_capture using a text editor and capture the

following post-encapsulation boot disk information:
The prtvtoc value
The df -k value

The vxprint -qhtg rootdg output
7.
Mirror the encapsulated boot disk using Sun Enterprise Services best
practices procedure (see Examining Sun Enterprise Services Best
8.
While the boot disk is being mirrored, answer the following

questions:
a.
Describe the difference between the pre- and post-encapsulation

boot disk partition configuration. Use the contents of
/bootdisk_capture as a guide.
There are two new disk partitions for the private and public regions. Additionally,
/opt was encapsulated into the disks public region and is no longer visible. The
/, swap, /usr, and /var partitions are still visible as overlay partitions.
b.
What directory contains both pre- and post-encapsulation

configuration information about the boot disk?
The /etc/vx/reconfig.d/disk.d/c#t#d# directory.

c.
State the purpose of the following files:
/etc/vx/reconfig.d/state.d/root-done
This file defines that the root disk was encapsulated.
/etc/vx/reconfig.d/state.d/i/etc/vx/reconfig.d/sta
te.d/install-db
This file defines by its presence that vxinstall was not executed. It prevents the
VxVM software daemons from starting.
/etc/vfstab.prevm
This file holds a copy of the pre-boot disk encapsulation /etc/vfstab file
contents.
9.
After the boot disk is mirrored, use vxprint to capture the postmirror configuration. Copy this information to the
/bootdisk_capture file for use later in this lab.

Encapsulating Disks
2-87
Task 2 Solutions
1.
Unencapsulate the boot disk by using the vxunroot command.

The procedure for this process is found in Examining Sun
Disks on page 2-48, but can be successfully used for a five-slice boot
disk as described in Unencapsulating a Boot Disk Using the
vxunroot Utility on page 2-57.
To unencapsulate a five-slice boot disk, use the following command
syntax in place of that listed in the procedure to execute the loop to
remove the mirrors:
# for i in cat /rmsub.plex
>do
>done
This substitution results in full removal of the plexes and their
subdisks.
2.
It should be.
3.
How do you know, and what commands do you use to verify this?
The system reboots. Use the following commands:
df -k
vxprint
mount
4.
Compare the post-unencapsulation partition configuration of the

boot disk with the pre-unencapsulation and describe the differences.
Partitions were restored to pre-encapsulation configuration. The public and

private region partitions were removed.
2-88

Task 3 Solutions
1.
2.
Encapsulate the boot disk using the vxdiskadm utility menu

selection 6. Use the following configuration:
a.
Assign it a VxVM software disk name of rootdisk (default).
b.
Do not configure it as a spare disk.
c.
Accept all other defaults.
One the encapsulation process is complete and the system has

rebooted, mirror the boot disk using the Sun Enterprise Services best
practice procedure (see Examining Sun Enterprise Services Best
Task 4 Solutions
1.
Unencapsulate the boot disk using the manual procedure when

booted from CD-ROM described in Unencapsulating When Booted
From the CD-ROM on page 2-64. Be sure to recover the
encapsulated /opt partition using the vxedvtoc command outlined
in this procedure.
Be patientthis procedure reboots the system multiple times.
2.
It should be.

Encapsulating Disks
2-89

3.

wrong?
It depends on the problem.

4.

5.

the boot disk.

6.
Re-encapsulate the boot disk using the vxdiskadm utility.
7.
After the system reboots, re-mirror the boot disk using the Sun
Disks on page 2-48). Assign the mirror disk a VxVM software disk
name of rootmirror.
Caution Do not continue the lab without re-mirroring the boot disk. The
next task requires that the boot disk be mirrored.
Task 5 Solutions
2-90
1.
2.

for this disk
Create two partitions minimum (512 megabyte maximum size).
Leave partitions 6 and 7 unallocated.
3.
4.
Create mount points for each of the partitions on the data disk and
verify that the new file systems successfully mounts.
5.



6.
7.

/etc/vfstab file. This information is used later in this lab
exercise.
8.
Encapsulate this disk using the vxdiskadm utility.

While encapsulating this disk, you are asked for a disk group for this
disk to join. Create a new disk group called datadg, or use rootdg.
9.
After the system reboots, verify that the encapsulation was

successful using the df -k and vxprint commands.
10. Mirror the encapsulated data disk using a disk you identified for this
purpose and the vxdiskadm utility.
11. While the data disk is being mirrored, answer the following
questions:
a.
What are the differences between the pre- and postencapsulation partition configuration for this disk?
The post-encapsulated disk has only partitions 6 and 7. The original partitions
were encapsulated in slice 6, the public region. Slice 7 is the private region.
b.
How many reboots did the system execute?
c.
Using output from the execution of a df -k command, contrast

the differences in the devices used to mount the newlyencapsulated file systems.
One.
Original devices were normal Solaris OE devices which use

/dev/dsk/c#t#d#s# addresses. Post-encapsulation devices use the VxVM
software volume names. These are /dev/vx/dsk/data1 or other volume names.
d.
View the /etc/vfstab file and describe the differences

between the pre- and post-data disk encapsulation content.
The /etc/vfstab file now uses the VxVM software volumes as devices to
mount and fsck. The original device information was saved as comments at the
end of the file.
12. Wait for the mirror process to complete before proceeding to the next
task.

Encapsulating Disks
2-91
Task 6 Solutions
1.
Unencapsulate the data disk using the procedure outlined in

2.
Was the unencapsulation successful?
It should be.
3.

wrong?

4.

5.

the data disk.

6.
Once the unencapsulation successfully completes, contrast the preencapsulation and post-unencapsulation partition scheme of this
disk.
Are there any differences?
No.
If yes, list them.
There should not be any differences.
7.
2-92
Unmount and remove all references to the file systems on this disk
from the /etc/vfstab file.

Task 7 Solutions
1.
2.

for this disk:
Create two partitions, minimum, using all the available space

on the disk. Do not leave any free space.
Leave at least one partition unused.
3.
4.
Create mount points for each of the partitions on the data disk, and
verify that the new file systems successfully mount.
5.

6.
7.

/etc/vfstab file.
8.
Encapsulate this disk using the procedure in Encapsulating a NonConforming Disk on page 2-20.

Encapsulating Disks
2-93
Module 3
Managing Dynamic Multi-Pathing

Objectives
Define and explain how the dynamic multi-pathing (DMP) functions

enhance the availability and accessibility of VxVM softwaremanaged storage devices
Explain how DMP identifies disks in both pre- and post-version 3.2
of the VxVM software
Describe how to install and verify DMP
Enable and disable multi-pathing to selected disks and controllers
Administrate DMP by using the vxdmpadm utility
Perform start restore and stop restore functions
Identify common DMP problems

3-1
Relevance
Relevance
!
?
3-2
Discussion The following questions are relevant to understanding what

DMP and DMP administration is all about:
What are the key features of DMP?
How does DMP manage load balancing?
How does DMP improve the fault resiliency of system storage?
How is DMP functionality enabled and disabled on a controller and

on individual disks?
How does DMP recognize disks?
How does DMP interface with the DDL function introduced in the
VxVM software version 3.2?
Is DMP compatible with other Sun Microsystems multi-pathing

applications?
Is DMP compatible with the Sun Microsystems dynamic

reconfiguration (DR) software?
How does DMP interface with Sun StorEdge T3 storage arrays, and
how is this different than DMP support of the Sun StorEdge A5000
series of array storage subsystems?

details on the topics discussed in this module:





http://storage.west
SunSolveSM Online SRDBs and INFODOC 18314,

[http://sunsolve.Sun.COM/pub-cgi/search.pl?mode=
advanced].
VERITAS Software Corporation support knowledge base,

[http://seer.support.veritas.com/nav_bar/index.asp?
content_sURL=%2Fsearch%5Fforms%2Ftechsearch%2Easp].
Man pages for the vxdmpadm command.

3-3
Examining VxVM Software and Dynamic Multi-Pathing

DMP allows the Solaris OE to see only disks and not the multiple paths to
the disks. It handles the I/O across these multiple paths transparently to
the Solaris OE and the user. Additionally, DMP manages load balancing
for I/Os across the multiple paths.
This feature is only available for Active/Active DMP arrays; it is not
available for active/passive DMP arrays. DMP also handles the failover
from one path to another, transparent to the Solaris OE and the user.
DMP is an important part of the VxVM softwares overall storage fault
resiliency strategy and is a mandatory part of the installation of the VxVM
software version 3.1.x and later. This section describes the VxVM software
and how it implements DMP functions.
VxVM Software Architecture and DMP

Figure 3-1 illustrates the relationship of the DMP driver within the system
software stack, in which the DMP driver is a component of the kernel
level of the software stack.
Applications
User
I/O
Kernel
File System
VxVM software (vxio)
/dev/vx/dmp/c1t1d0s0
(DMP meta node)
DMP
/dev/dsk/c1t1d0s0
/dev/dsk/c2t1d0s0
Target Disk Driver
Target HBA Driver
Physical
Path 1
Figure 3-1
3-4
SD
H i ta c h i D a t a S y s te m s
Path 2
R E A D YW A R N I N G A L A R MP O W E R
System Software Stack

Load Balancing
Load balancing is the function which attempts to maximize I/O
throughput by using the full bandwidth of all paths. Although the goal is
the same, load balancing is implemented differently depending on which
version of the VxVM software is used.
Load Balancing in Software Version 2.5.x

Load balancing in the VxVM software version 2.5.x is achieved by routing
random I/O across all available paths in a round-robin fashion. For the
Sun StorEdge A5000 series of array storage subsystems, any I/O that is
within 64 kilobytes on either side of the last I/O is sent down the same
path, which results in more effective use of cache.
Early versions of the VxVM software (before 2.5.4 for example) allow the
user to disable DMP. Version 2.5.4 of the VxVM software does not allow
the user to disable DMP. While some systems can disable DMP, this causes
vxconfigd to fail.
Load Balancing in Software Version 3.0

Load balancing in the VxVM software version 3.0 is achieved by using the
balanced path policy for the active/active-type disk arrays only, and not for
active/passive-type disk arrays.
The method used to balance random I/Os across all the available paths is
to send all I/Os that start within a single 64-kilobyte block range down
the same path. I/Os that start within other 64-kilobyte ranges on the disk
are routed through a different path.
If uneven distribution of I/Os between the controllers occurs, it may be
caused by a contiguous read of data from the disk within a single
64-kilobyte block range. The term load balancing is misleading in this case
because it refers to I/O distribution, depending on the location of the data
on the disk.

3-5
Unique Disk Identifier

The Solaris OE requires a unique means to identify each disk drive. To
that end, disk manufacturers provide a serial number along with other
information in the standard inquiry data for each disk. It is possible to
view this information by using the vxdmpinq script. The following
example shows output from the vxdmpinq script.
bash-2.03# cd /etc/vx/diag.d
bash-2.03# ./vxdmpinq /dev/rdsk/c1t0d0s2
Vendor id
: SEAGATE
Product id
: ST19171FCSUN9.0G
Revision
: 177E
Serial Number
: 9831X06256
DMP Device Paths

The DMP implemented in the VxVM software version 2.5.x supports up
to 42 paths. On the VxVM software version 3.0.x and 3.2, DMP supports
an unlimited number of paths.
For each device path, the Solaris OE creates an entry in the dev_info tree.
DMP scans the dev_info tree and creates metanodes in the /dev/vx tree.
The VxVM software uses the metanodes to access the disk.
Solaris OE Drives and DMP Drives

The Solaris OE sees all the paths to each drive and lists them separately in
the /dev/dsk tree. An example illustrates two links, c1t1d0s0 and
c2t1d0s0, each pointing to the same drive (3f643).
bash-2.03# ls -las /dev/dsk/*t0d0s0
2 lrwxrwxrwx
1 root
root
70 Apr 21 15:22 /dev/dsk/c1t0d0s0 ->
../../devices/sbus@3,0/SUNW,socal@0,0/sf@0,0/ssd@w210000203713f643,0:a
2 lrwxrwxrwx
1 root
root
70 Apr 21 15:22 /dev/dsk/c2t0d0s0 ->
Conversely, DMP creates one metanode per LUN. That is, DMP identifies
LUNs and creates a metanode for each LUN. The result is that a disk with
multiple paths is seen as a single disk by the VxVM software.
3-6


A LUN can be a physical disk as in the individual disks of a Sun StorEdge
A5000 series of array storage subsystems, or it may be a more complex
configuration of multiple physical disks as in a software RAID device
such as a Sun StorEdge T3 storage array. The operating system (and
therefore the user) sees no difference between a LUN and a disk.
For example, an array with five LUNs has five metanodes. If there are
four paths from the host to the array, the host has 20 entries under
/dev/dsk tree, and only five entries under /dev/vx/dmp tree.
The following example illustrates multiple paths to a single disk in a Sun
StorEdge A5000 Sun Network Storage Array (SENA), with two interface
boards (IBs) and two gigabit interface converters (GBICs).
bash-2.03# ls -las /dev/rdsk/*t0d0s0
2 lrwxrwxrwx
1 root
root
74 Apr 21 15:22 /dev/rdsk/c1t0d0s0 ->
../../devices/sbus@3,0/SUNW,socal@0,0/sf@0,0/ssd@w210000203713f643,0:a,raw
2 lrwxrwxrwx
1 root
root
74 Apr 21 15:22 /dev/rdsk/c2t0d0s0 ->
../../devices/sbus@3,0/SUNW,socal@0,0/sf@1,0/ssd@w220000203713f643,0:a,raw
In the following example, the same disks are managed by DMP, which
arbitrarily chooses one of the two paths and creates a single device entry.
To see the multiple paths to individual disks, use the vxdmpadm command.
bash-2.03# ls -las /dev/vx/rdmp/*t0d0s0
0 crw------1 root
other
74,
0 May 21 15:34 /dev/vx/rdmp/c1t0d0s0
The following example shows output from the vxdisk list command to
illustrate how the VxVM software sees the same disk shown in the
previous examples.
DEVICE
TYPE
c1t0d0s2
sliced
c1t3d0s2
sliced
c1t4d0s2
sliced
c1t5d0s2
sliced
c1t6d0s2
sliced
c1t16d0s2
sliced
c1t18d0s2
sliced
c1t19d0s2
sliced
c1t22d0s2
sliced
DISK
rootdisk
rootmirror
GROUP
rootdg
rootdg
STATUS
online
error
online
online
error
error
error
error
online

3-7
DMP and Device Discovery

In early versions of the VxVM software, DMP was responsible for device
discovery. With the release of the VxVM software version 3.2 and
implementation of the DDL feature, DMP is no longer responsible for
device discovery. This is now handled by the DDL feature. This process is
discussed in Appendix E, Configuring the VxVM Software."
3-8

Installing and Verifying DMP
Installing and Verifying DMP

The DMP driver is installed into the in /kernel/drv directory during the
installation of the VRTSvxvm package. During installation, a force load is
added to the /etc/system file which force loads the drivers at boot time to
allow the system to boot from a disk under VxVM software control.
With the release of the VxVM software version 3.1, DMP is always
installed and cannot be disabled or the VxVM software does not start. The
process of selectively disabling DMP on a controller or drive basis is
described in Enabling and Disabling DMP on page 3-10.
To verify the presence of the drivers, use the modinfo | grep vx
command as seen in this example.
bash-2.03# modinfo |
19 101e9025 fe998
21 102d43c8 17308
22 102e96f0
837
grep
75
74
79
vx
1 vxio (VxVM 3.2t_p1.6 I/O driver)
1 vxdmp (VxVM 3.2t_p1.6: DMP Driver)
1 vxspec (VxVM 3.2t control/status driver)
One way to verify that the DMP driver is installed correctly and that it is
not corrupt is to compare the size of the DMP driver file to that of the
driver_OSversion, which is in the same directory. Both drivers should
be the same size as seen in this example. The /kernel/drv file holds the
32-bit drivers and /kernel/drv/sparcv9 holds the 64-bit driver.
bash-2.03# pwd
/kernel/drv
bash-2.03# ls -las vxdmp*
640 -rw-r--r-1 root
608 -rw-r--r-1 root
608 -rw-r--r-1 root
640 -rw-r--r-1 root
4 -rw-r--r-1 root
sys
sys
sys
sys
sys
314156
297920
300980
314156
1026
bash-2.03# pwd
/kernel/drv/sparcv9
bash-2.03# ls -las vxdmp*
800 -rw-r--r-1 root
768 -rw-r--r-1 root
800 -rw-r--r-1 root
sys
sys
sys
393968 Nov 21
380840 Aug 15
393968 Nov 21
Nov
Aug
Aug
Nov
Aug
21
15
15
21
15
2001
2001
2001
2001
2001
vxdmp
vxdmp.SunOS_5.6
vxdmp.SunOS_5.7
vxdmp.SunOS_5.8
vxdmp.conf
2001 vxdmp
2001 vxdmp.SunOS_5.7
2001 vxdmp.SunOS_5.8

3-9
Enabling and Disabling DMP

In early versions of the VxVM software, DMP was disabled for a number
of different reasons. The most notable reason for disabling DMP was that
the Sun Microsystems alternate pathing (AP) application was installed.
With the release of the VxVM software version 3.1, the incompatibilities
between DMP and AP are resolved, and DMP is no longer disabled.
Caution AP must be at level 2.3.1 with patch 110722-01 installed.
Additionally, with the release of the VxVM software version 3.1, DMP is a
mandatory component of the VxVM software and must be operational or
the VxVM software is disabled. This is due to new features which are
introduced in the VxVM software version 3.2, such as enclosure-based
naming, which require DMP to function.
Disabling DMP in Version 3.0 and Earlier

With early versions of the VxVM software, DMP could be disabled by
locating the /kernel/drv/ap file prior to the installation of the VxVM
software. If the VxVM software identified the /kernel/drv/ap file, it
would not install. The procedure to disable DMP after installing the
VxVM software is more complex. Refer to SunSolve INFODOC 18314 for
more information.
The disadvantage of this all or none implementation of DMP is that disks
that are not under AP or other third-party multi-pathing software cannot
take advantage of DMP. The release of the VxVM software version 3.1
implementation of DMP changed this to allow selective disabling of the
function for controllers or individual disk devices.
Disabling DMP in Version 3.1 and Later

Disabling DMP in version 3.1 and 3.2 of the VxVM software is
accomplished during the software installation process using the
vxinstall utility or, post-installation, using the vxdiskadm utility. The
disabling or enabling of multi-pathing through DMP is achieved on a
controller-by-controller or disk-by-disk basis. This is a granular, efficient
process that addresses the all or none strategy of previous versions of the
VxVM software.
3-10


One of the challenges of implementing DMP in this fashion is that, if
multi-pathing is disabled or prevented for a disk, the VxVM software sees
both paths for that disk. This can lead to administrative confusion. To
resolve this problem the VxVM software suppresses the second path
through options provided in the vxinstall and vxdiskadm utilities.
DMP Terminology
Both vxinstall and vxdiskadm use the following terminology to
describe enabling and disabling DMP on selected devices:
Exclude Excludes a disk or controller from DMP control. Excludes

have the following options:
Prevent Prevents multi-pathing operations
Suppress Suppresses the VxVM software operations on the

secondary paths to a device
Include Enables DMP operations. Includes have the following

options:
Allow Allows multi-pathing operations
Un-Suppress Makes secondary paths visible to the VxVM

software operations
When you are disabling DMP on a device or group of devices, the prevent
operation executes first, and the suppress operation executes second.
When you are enabling DMP, the un-suppress operation occurs first and
the allow operation occurs second.
Disabling DMP Using the vxinstall Utility

When you are installing the VxVM software, the vxinstall utility
performs the initial VxVM software setup tasks. One of these tasks is to
prevent multi-pathing and suppress devices from the VxVM softwares
view.
To disable DMP using the vxinstall utility, use option 3, Prevent Multipathing/Suppress devices from VxVMs view, from the main vxinstall
menu. An example shows the process used in vxinstall.
Volume Manager Installation
Menu: VolumeManager/Install

3-11

You will now be asked if you wish to use Quick Installation or Custom
Installation. Custom Installation allows you to select how the Volume
Manager will handle the installation of each disk attached to your
system.
Quick Installation examines each disk attached to your system and

attempts to create volumes to cover all disk partitions that might be
used for file systems or for other similar purposes.
If you want to exclude any devices from being seen by VxVM or not be
multipathed by VxDMP then use the Prevent multipathing/Suppress
devices from VxVMs view option, before you choose Custom
Installation
or Quick Installation.
If you do not wish to use some disks with the Volume Manager, or if
you wish to reinitialize some disks, use the Custom Installation
option. Otherwise, we suggest that you use the Quick Installation
option.
Hit RETURN to continue.
Press Return to continue.

1 Quick Installation
2 Custom Installation
3 Prevent multipathing/Suppress devices from VxVMs view
? Display help about menu
?? Display help about menuing system
q Exit
from menus
When option 3 is selected, the following menu is displayed:

Volume Manager Device Operations
Menu: VolumeManager/Install/Exclude Devices
1 Suppress all paths through a controller from VxVMs view
2 Suppress a path from VxVMs view
3 Suppress disks from VxVMs view by specifying a VID:PID combination
4 Suppress all but one path to a disk
5 Prevent multipathing of all disks on a controller by VxVM
6 Prevent multipathing of a disk by VxVM
7 Prevent multipathing of disks by specifying a VID:PID combination
? Display help about menu
??Display help about the menuing system
q Exit from menus
3-12


Options 1 through 4 on the vxinstall utility main menu are used for
suppression operations. Options 6 through 8 are used for prevention
operations. Option 8 lists suppressed or non-multi-pathed devices. To use
this options, select the menu option for the specific operation and follow
the scripted prompts.
Whenever multi-pathing is prevented and secondary paths are
suppressed, the system must be rebooted for these changes to take effect.
Disabling DMP Using the vxdiskadm Utility

To disable multi-pathing or secondary paths after installing the software,
use the vxdiskadm utility. The terminology and procedures used to
exclude and include multi-pathing in the vxinstall utility apply to
vxdiskadm utility.
This example shows the menus used to accomplish multi-pathing and
suppression activities from the vxdiskadm utility.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
list

Remove a disk
Disable (offline) a disk device
Mark a disk as a spare for a disk group
Turn off the spare flag on a disk
Unrelocate subdisks back to a disk
Exclude a disk from hot-relocation use
Make a disk available for hot-relocation use
Prevent multipathing/Suppress devices from VxVMs view
Allow multipathing/Unsuppress devices from VxVMs view
List currently suppressed/non-multipathed devices
Change the disk naming scheme
Get the newly connected/zoned disks in VxVM view

3-13
?
??
q

Exit from menus
Select an operation to perform:
Use the vxdiskadm utility menu items 17 and 18 to include and exclude
DMP operations. Selecting option 17, Prevent multipathing/Suppress
devices from VxVMs view, produces the following output:
Exclude Devices
Menu: VolumeManager/Disk/ExcludeDevices
This operation might lead to some devices being suppressed from VxVMs view
or prevent them from being multipathed by vxdmp (This operation can be
reversed using the vxdiskadm command).
Do you want to continue ? [y,n,q,?] (default: y)

Menu: VolumeManager/Disk/ExcludeDevices
1
2
3
4
5
6
7
8
?
??
q
Suppress all paths through a controller from VxVMs view

Suppress a path from VxVMs view
Suppress disks from VxVMs view by specifying a VID:PID combination
Suppress all but one paths to a disk
Prevent multipathing of all disks on a controller by VxVM
Prevent multipathing of a disk by VxVM
Prevent multipathing of disks by specifying a VID:PID combination
Exit from menus
To use these options, select the menu option for the specific operation and
follow the scripted prompts. A system reboot is required to activate the
changes.
Selecting the vxdiskadm utility menu option 18, Allow
multi-pathing/Unsuppressed devices from VxVMs view, produces the
following output:
Include Devices
Menu: VolumeManager/Disk/IncludeDevices
The devices selected in this operation will become visible to VxVM and/or
will be multipathed by vxdmp again. Only those devices which were previously
excluded can be included again.
3-14

Do you want to continue ? [y,n,q,?] (default: y)

Menu: VolumeManager/Disk/IncludeDevices
1
2
3
4
5
6
7
8
?
??
q
Unsuppress all paths through a controller from VxVMs view

Unsuppress a path from VxVMs view
Unsuppress disks from VxVMs view by specifying a VID:PID combination
Remove a pathgroup definition
Allow multipathing of all disks on a controller by VxVM
Allow multipathing of a disk by VxVM
Allow multipathing of disks by specifying a VID:PID combination
Exit from menus
To use these options, select the menu option for the specific operation and
follow the scripted prompts. A system reboot is required to activate the
changes.
Files Related to Disabling and Suppression

The files in the following list are updated by vxinstall and vxdiskadm
when multi-pathing is disabled and secondary paths suppressed:
bash-2.03# ls -las /etc/vx/*exclude
2 -rw-r--r-1 root
other
2 -rw-r--r-1 root
other
59 May 21 15:34 /etc/vx/vxdmp.exclude

59 May 21 15:34 /etc/vx/vxvm.exclude
When vxinstall or vxdiskadm performs any exclude or include

operations, these files are modified. The /etc/vx/vxdmp.exclude file is
updated for DMP prevent or allow operations. The
/etc/vx/vxvm.exclude file is updated for suppress and un-suppress
operations.
The following example shows the /etc/vx/vxdmp.exclude after multipathing is disabled on disk c1t0d0.
vxdmp.exclude
:::::::::::::::
exclude_all 0
paths
c1t0d0s2 /sbus@2,0/SUNW,socal@d,10000/sf@0,0/ssd@w210000203713fc9f,0
c2t0d0s2 /sbus@2,0/SUNW,socal@d,10000/sf@1,0/ssd@w220000203713fc9f,0

3-15

#
controllers
#
product
#
pathgroups
#
The following example shows the /etc/vx/vxvm.exclude file when the

secondary path (c2t0d0) for disk c1t0d0 is suppressed.
vxvm.exclude
::::::::::::::
exclude_all 0
paths
c2t0d0s2 /sbus@3,0/SUNW,socal@0,0/sf@1,0/ssd@w220000203713fc9f,0
#
controllers
#
product
#
pathgroups
#
Caution Do not edit these files manually. Use vxdiskadm to make all
updates to these files. If it is necessary to edit these file manually be very
careful and only delete the lines needed to get an inoperable system into
service.
3-16

Administrating DMP With the vxdmpadm Command

The vxdmpadm command is the primary administrative interface to the
DMP facility. Use the vxdmpadm command to enable and disable DMP on
an interface-by-interface basis, display multi-pathing information, and
perform many other functions that affect the operation of DMP or display
DMP status.
To view help for the vxdmpadm command, use the man pages or enter:
# vxdmpadm help
This section describes vxdmpadm options.
The listctrl Option

The vxdmpadm command listctrl option lists all controllers seen by the
Solaris OE. This allows the user to determine which controller is not
communicating with the system. An example follows.
# vxdmpadm listctlr all
CTLR-NAME
ENCLR-TYPE
STATE
ENCLR-NAME
=====================================================
c1
SENA
ENABLED
SENA0
c2
SENA
ENABLED
SENA0
Viewing Multi-Pathing Status

Use the vxdmpadm listexclude command to list disks that are excluded
from DMP operations.
An example of the vxdmpadm listexclude command shows that path
c2t0d0 (the secondary path to c1t0d0) is suppressed, and paths c1t0d0
and c2t0d0 are excluded from DMP operations.
# vxdmpadm listexclude
Devices excluded from VxVM:
-------------------------Paths : c2t0d0
Controllers : None
VID:PID : None

3-17

Devices excluded from multipathing by vxdmp:
------------------------------------------Paths : c1t0d0 c2t0d0
VID:PID : None
Pathgroups : None
----------
An example of the vxdisk list command shows that c1t0d0s2 is still

usable by the VxVM software but the secondary path c2t1d0s2 is
suppressed and not visible to the software.
.bash-2.03# vxdisk list
DEVICE
TYPE
DISK
c1t0d0s2
sliced
rootdisk
c1t3d0s2
sliced
c1t4d0s2
sliced
c1t5d0s2
sliced
c1t6d0s2
sliced
c1t16d0s2
sliced
c1t18d0s2
sliced
c1t19d0s2
sliced
c1t22d0s2
sliced
rootmirror
GROUP
rootdg
rootdg
STATUS
online
error
online
online
error
error
error
error
online
The getsubpaths Option

The getsubpaths ctlr option of the vxdmpadm command lists all
subpaths through a specific controller. This allows the system
administrator to determine which DMP nodes (devices) on the path are
not functioning. An example is:
# vxdmpadm getsubpaths ctlr=c1
NAME
STATE
PATH-TYPE DMPNODENAME ENCLR-TYPE
ENCLR-NAME
======================================================================
c1t0d0s2
ENABLED
c1t0d0s2
SENA
SENA0
c1t21d0s2
ENABLED
c1t21d0s2
SENA
SENA0
c1t17d0s2
ENABLED
c1t17d0s2
SENA
SENA0
c1t3d0s2
ENABLED
c1t3d0s2
SENA
SENA0
c1t4d0s2
ENABLED
c1t4d0s2
SENA
SENA0
c1t6d0s2
ENABLED
c1t6d0s2
SENA
SENA0
c1t1d0s2
ENABLED
c1t1d0s2
SENA
SENA0
c1t18d0s2
ENABLED
c1t18d0s2
SENA
SENA0
c1t20d0s2
ENABLED
c1t20d0s2
SENA
SENA0
c1t22d0s2
ENABLED
c1t22d0s2
SENA
SENA0
c1t16d0s2
ENABLED
c1t16d0s2
SENA
SENA0
c1t2d0s2
ENABLED
c1t2d0s2
SENA
SENA0
c1t5d0s2
ENABLED
c1t5d0s2
SENA
SENA0
c1t19d0s2
ENABLED
c1t19d0s2
SENA
SENA0
3-18


It is also possible to list the subpaths to a particular node using the
getsubpaths dmpnodename option. This helps to focus on a specific node
by eliminating unwanted information and showing the status of the paths
for that node, as seen in this example.
# vxdmpadm getsubpaths dmpnodename=c1t1d0s2
NAME
STATE
PATH-TYPE CTLR-NAME ENCLR-TYPE
ENCLR-NAME
====================================================================
c1t1d0s2
ENABLED
c1
SENA
SENA0
c2t1d0s2
ENABLED
c2
SENA
SENA0
The getdmpnode Option

To obtain DMP node information about disks in a specific enclosure, use
the vxdmpadm command getdmpnode option. The getdmpnode option
shows devices within a specific enclosure and the status of the paths to
each node within that enclosure. This example shows how to further
refine the information to a specific node.
# bash-2.03# vxdmpadm getdmpnode enclosure=SENA0
NAME
STATE
ENCLR-TYPE
PATHS ENBL DSBL ENCLR-NAME
=========================================================================
c1t0d0s2
ENABLED
SENA
2
2
0
SENA0
c1t21d0s2
ENABLED
SENA
2
2
0
SENA0
c1t17d0s2
ENABLED
SENA
2
2
0
SENA0
c1t3d0s2
ENABLED
SENA
2
2
0
SENA0
c1t4d0s2
ENABLED
SENA
2
2
0
SENA0
c1t6d0s2
ENABLED
SENA
2
2
0
SENA0
c1t1d0s2
ENABLED
SENA
2
2
0
SENA0
c1t18d0s2
ENABLED
SENA
2
2
0
SENA0
c1t20d0s2
ENABLED
SENA
2
2
0
SENA0
c1t22d0s2
ENABLED
SENA
2
2
0
SENA0
c1t16d0s2
ENABLED
SENA
2
2
0
SENA0
c1t2d0s2
ENABLED
SENA
2
2
0
SENA0
c1t5d0s2
ENABLED
SENA
2
2
0
SENA0
c1t19d0s2
ENABLED
SENA
2
2
0
SENA0
The disable Opt ion

The vxdmpadm command disable option disables one of the paths to a
disk. This can be advantageous when you are using DMP for multipathing disks on a DR-capable system to disable a path when a system
board is removed. An example shows the results of using the disable
option.
# vxdmpadm disable ctlr=c1
Jul 8 12:11:54 lowtide vxdmp: NOTICE: vxvm:vxdmp: disabled controller
/sbus@3,0/SUNW,socal@0,0/sf@0,0 connected to disk array 5080020000034908

3-19

An example of a vxdmpadm listctlr all command shows the path
disabled.
bash-2.03# vxdmpadm listctlr all
CTLR-NAME
ENCLR-TYPE
STATE
ENCLR-NAME
=====================================================
c1
SENA
DISABLED
SENA0
c2
SENA
ENABLED
SENA0
The enable Option

The enable option of the vxdmpadm command enables a controller to a
disk. This option can be used on a DR-capable system after a system
board is replaced, to re-enable a path disabled during the DR remove
operation. The following example shows the result of using the enable
option.
# vxdmpadm enable ctlr=c1
Jul 8 12:15:30 lowtide vxdmp: NOTICE: vxvm:vxdmp: enabled controller
/sbus@3,0/SUNW,socal@0,0/sf@0,0 connected to disk array 5080020000034908
An example of a vxdmpadm listctlr command all shows the paths

enabled.
CTLR-NAME
ENCLR-TYPE
STATE
ENCLR-NAME
=====================================================
c1
SENA
ENABLED
SENA0
c2
SENA
ENABLED
SENA0
The start restore and stop restore Options

The DMP restore daemon thread is a background process that monitors
paths that fail. If the paths are available for use, the daemon thread can be
stopped and re-started using the following syntax:
vxdmpadm start restore interval=seconds
The interval option specifies the time interval after which the daemon
thread checks the failed paths, as specified in seconds. The default is
300 seconds (5 minutes). To not wait for the command to time out, use the
following command:
# vxdctl enable
3-20

Reviewing Common DMP Problems

This section provides information on some common problems that might
occur with DMP functions.
Disks Do Not Appear to Be Multi-Pathed

If disks on a system do not appear to be multi-pathed, there can be a
licensing problem. Check the following:
For the VxVM software version 2.5.x, EMC Corporation disk arrays
require a license.
Use vxdmpdebug and check the following:

DEBUG:
DEBUG:
DEBUG:
DEBUG:
DEBUG:
DEBUG:
DEBUG:
DEBUG:
DEBUG:
is_module_present: vxdmp module is present

is_module_present: vxdmp module is loaded
is_module_present: vxdmp module is installed
dmp_is_link: /dev/vx/dmp/ is NOT a link
dmp_is_link: /dev/vx/rdmp/ is NOT a link
Have Simple disk DMP license
No SSA DMP license
dmp_set_lic: Start
dmp_set_lic: Done
Serial Number Problems

If a disk manufacturer does not provide unique serial numbers for disks
and a simple disk license is present on a system running the VxVM
software version 2.5.x, DMP does not see the disks correctly on the
system. This causes unexpected behavior, such as seeing multiple
independent paths to multiple disks.
The vxdmpinq command shows serial number information about DMP
disks. See the Unique Disk Identifier on page 3-6 for more information.

3-21
The product_id and vendor_id Do Not Match

Check the following characteristics in the values for product_id and
vendor_id do not match:
The disks are not multi-pathed.
The OTHER category is used.
VxVM Software Does Not See Disk Devices

Check the following if the VxVM software does not see disks:
Make sure that the operating system can see these disks.
Make sure that I/O can be performed on the disks.
Use these DMP utilities to gather more information:
vxdmpinq
vxdmpdebug
Note Problems seeing disks are addressed in Module 6, Recovering

Disk, Disk Group, and Volume Failures. DMP debugging utilities and
troubleshooting suggestions are discussed in Module 4, Troubleshooting
Tools and Utilities.
3-22

Exercise: Operating DMP

In this exercise, you complete the following tasks:
Configure DMP to exclude a disk device from multi-pathing

operations using the vxdiskadm utility.
Suppress a path from a DMP-excluded disk using the vxdiskadm

utility.
Unsuppress a path from a DMP-excluded disk using the vxdiskadm

utility.
Configure DMP to include a disk device for DMP operations using

the vxdiskadm utility.
View DMP status using the vxdmpadm utility.
Locate files used to disable and enable DMP.
Preparation
The VxVM software 3.2 must be installed and operational.
The boot disk must be encapsulated and mirrored.
A second disk group must exist other than rootdg with a configured
volume that is formatted and mounted.
Open a window, and execute an iostat while performing the lab

exercises to see the effect on pathing as paths are removed and
added. Use the following script to run the iostat command:
#
>
>
>
>
while true
do
iostat -xcnm
sleep 2
done
Note Re-size the window to fit the output of the iostat display.
Generate looping I/Os to one of the non-root volumes created in the

exercises in Module 2, Encapsulating Disks.

3-23
Task 1 Enabling and Disabling DMP Operations

1.
Use the vxdiskadm utility to disable multi-pathing to one VxVM

software disk on your lab teams system. Do not use the boot disk for
this task.
What vxdiskadm main menu option did you use?
_____________________________________________________________
How did you activate the change you made?
_____________________________________________________________
_____________________________________________________________
_____________________________________________________________
2.
Use the vxdiskadm utility to suppress the secondary paths to the

disk on which you just prevented DMP operations.
_____________________________________________________________
_____________________________________________________________
_____________________________________________________________
Did you have to reboot to activate this operation?
_____________________________________________________________
_____________________________________________________________
_____________________________________________________________
3.
View the DMP multi-pathing status to verify that you were

successful in the execution of the previous tasks.
What commands did you use?
_____________________________________________________________
_____________________________________________________________
_____________________________________________________________
4.
What two files were modified to enable the previous operation?

_____________________________________________________________
_____________________________________________________________
_____________________________________________________________
3-24


View the contents of these files to see the changes that were made.
_____________________________________________________________
5.
Un-suppress and allow DMP operations on the disk excluded in the

first two tasks of this lab exercise.
What commands did you use, and in what order did you execute
them?
_____________________________________________________________
_____________________________________________________________
_____________________________________________________________
View the contents of the two exclude files to see what changes were
made.
_____________________________________________________________
Task 2 Administrating DMP

1.
Use the vxdmpadm utility to list all controllers seen by the Solaris OE.
What format of the vxdmpadm command did you use?
_____________________________________________________________
_____________________________________________________________
_____________________________________________________________
2.
Use the vxdmpadm utility to list all subpaths to a specific node. The
DMP node you select depends on the configuration of your system.
_____________________________________________________________
_____________________________________________________________
_____________________________________________________________
3.
Disable a storage controller using the vxdmpadm utility.

_____________________________________________________________
_____________________________________________________________
_____________________________________________________________

3-25

4.
Verify that you were successful.

_____________________________________________________________
_____________________________________________________________
_____________________________________________________________
_____________________________________________________________
5.
Enable the storage controller you just disabled using the vxdmpadm
utility.
_____________________________________________________________
_____________________________________________________________
_____________________________________________________________
6.

_____________________________________________________________
_____________________________________________________________
_____________________________________________________________
Task 3 Using the /etc/vx/diag.d/vxdmping Script

Use the vxdmping script to view the serial number of a disk of your
choice.
What command syntax did you use?
__________________________________________________________________
__________________________________________________________________
__________________________________________________________________
What is the serial number of the disk you selected?
__________________________________________________________________
__________________________________________________________________
3-26

Exercise Summary
Exercise Summary
!
?

Experiences
Interpretations
Conclusions
Applications

3-27

Following are the solutions to this lab exercise.
Task 1 Solutions
1.
Use the vxdiskadm utility to disable multi-pathing to one VxVM

software disk on your lab teams system. Do not use the boot disk for
this task.
_____________________________________________________________
How did you activate the change you made?
Reboot.
2.
Use the vxdiskadm utility to suppress the secondary paths to the

disk on which you just prevented DMP operations.
Use option 17, Prevent multi-pathing/Suppress devices from VxVMs view.

Did you have to reboot to activate this operation?
No.
3.
View the DMP multi-pathing status to verify that you were

successful in the execution of the previous tasks.
What commands did you use?
Use the following:
Issue a vxdisk list command.
Issue a vxdmpadm listexclude command.
4.
What two files were modified to enable the previous operation?
The files /etc/vx/vxdmp.exclude and /etc/vx/vxvm.exclude.

View the contents of these files to see the changes that were made.
3-28


5.
Un-suppress and allow DMP operations on the disk excluded in the

first two tasks of this lab exercise.
What commands did you use and in what order did you execute
them?
Use vxdiskadm menu option 18 to unsuppress the secondary path.

Use vxdiskadm menu option 18 to include both disk paths in DMP operations.
View the contents of the two exclude files to see what changes were
made.
Task 2 Solutions
1.
Use the vxdmpadm utility to list all controllers seen by the Solaris OE.
Use vxdmpadm listctlr all.

2.
Use the vxdmpadm utility to list all subpaths to a specific node. The
DMP node you select depends on the configuration of your system.
Use vxdmpadm getsubpaths dmpnodename=c#t#d#s2.

3.
Disable a storage controller using the vxdmpadm utility.

Use vxdmpadm disable ctlr=full_Solaris_OE_device_path-name.

4.

_____________________________________________________________

5.
Enable the storage controller you just disabled using the vxdmpadm
utility.
Use vxdmpadm enable ctlr=full_Solaris_OE_device_path-name.

3-29

6.

Task 3 Solutions
Use the vxdmping script to view the serial number of a disk of your
choice.
What command syntax did you use?
Use /etc/vx/diag.d/vxdmpinq /dev/rdsk/c#t#d#s2.
What is the serial number of the disk you selected?
Depends on disk selected.
3-30

Module 4
Troubleshooting Tools and Utilities

Objectives
Describe the troubleshooting tools and utilities available for the

VxVM software
Enable vxconfigd logging from the command line and the VxVM
software startup scripts
Reference a VxVM failure to the error messages section of the VxVM

software troubleshooting manual
Use the VxVM software debugging and information-gathering tools

and utilities

4-1
Relevance
Relevance
!
?
4-2

to use the VxVM software tools and utilities to troubleshoot VxVM
software failures:
What tools and utilities are available to system administrators to use

in debugging VxVM software problems?
How is vxvonfigd logging enabled?
What system-level information gathering tools are available for

capturing VxVM software-specific information to use in debugging
problems?

Service Support (IT Infrastructure Library Series). Stationary Office

Books, February 2002, ISBN 0113300158.
CCTA Staff. Service Delivery. Stationary Office Books, April 2001,

ISBN 0113300174.
Ivor Macfarlane and Colin Rudd. IT Service Management Version 2.

United Kingdom: itSMF Ltd, March 2001, ISBN 0-9524706-1-6.





http://storage.west.

4-3
Logging Errors
Logging Errors
The VxVM software provides extensive logging capabilities that include
the following error logging mechanisms:
vxconfigd logging
/var/adm/messages file
root mail
Using the /var/vxvm/vxconfigd.log File

The VxVM software provides extensive logging capabilities, including the
ability to redirect all the VxVM software-related console messages to a
VxVM software-specific log file and to syslog. This saves for post-crash
analysis any messages generated by the VxVM software prior to a system
crash.
All logging is managed by the vxconfigd daemon. The vxconfigd
logging reports errors detected by the vxconfigd daemon; these errors
are not limited to disk errors. Any error detected by the vxconfigd
daemon can be logged. Logging to the vxconfigd.log is disabled by
default.
There are nine levels of debug logging. Level 1 is minimal logging, and
level 9 logging involves writing all debug and error messages to the
selected log file. Logging is enabled by running the vxconfigd command
from the command line, or by modifying the
/etc/init.d/vxvm-sysboot script.
Caution Enabling vxconfigd logging at the most verbose level, level 9,
produces excessive amounts of data and can negatively impact system
performance. Disable this type of logging until it is needed to debug a
VxVM software problem.
To view output of the log in real time, tail the /var/vxvm/configd.log
file. Use the following syntax:
# tail -f /var/vxvm/vxconfigd.log
4-4

Logging Errors
Manually Enabling and Disabling Logging

To enable logging from the command line, vxconfigd must first be
stopped. To stop the vxconfigd daemon, use the following command
syntax:
# pkill -9 vxconfigd; vxconfigd -x 9 -x log -x syslog > /dev/null 2>&1 &
This enables maximum vxconfigd logging while suppressing vxconfigd

log messages to the standard out and standard error outputs.
To enable logging selectively from the command line, do one of the
following:
Enable maximum logging of error and debug messages to

/var/vxvm/vxconfigd.log. Type the following:
# vxconfigd -x 9 -x log
Enable syslog logging of the VxVM software console messages. Run

the following command:
# vxconfigd -x syslog
This version of the vxconfigd command logs all Error, Fatal Error,
Warning and Notice messages to the syslog. Debug messages are
not logged.
Enable maximum logging to the vxconfigd log and the syslog file.
Type the following:
# vxconfigd -x 9 -x log -x syslog
To disable vxconfigd logging after it is enabled, use the following syntax:

# pkill -9 vxconfigd; vxconfigd -m enable
Automatically Enabling Logging

To enable logging automatically during the boot process, edit the
/etc/init.d/vxvm-sysboot file and un-comment the lines that
correspond to the logging desired for a specific server or group of servers,
as shown in the following example:
# comment-out or uncomment any of the following lines to enable or
# disable the corresponding feature in vxconfigd.
#opts="$opts
#opts="$opts
#opts="$opts
#opts="$opts
-x
-x
-x
-x
syslog"
log"
logfile=/foo/bar"
timestamp"
#
#
#
#
use syslog for console messages

messages to /var/vxvm/vxconfigd.log
specify an alternate log file
timestamp console messages

4-5
Logging Errors
# to turn on debugging console output, uncomment the following line.

# The debug level can be set higher for more output. The highest debug
# level is 9.
#debug=1
# enable debugging console ouptut
To activate logging after the vxvm-sysboot file is modified, you must

reboot the system.
Example vxconfigd Logging Output

This example of the /var/vxvm/vxconfigd.log shows maximum
logging enabled and a fiber cable disconnected.
07/03 09:01:16: DEBUG: dmp_get_events: Start
07/03 09:01:17: DEBUG: dmp_get_events: Number of events = 2
07/03
07/03
07/03
07/03
07/03
07/03
07/03
07/03
09:01:17:
09:01:17:
09:01:17:
09:01:17:
09:01:17:
09:01:17:
09:01:17:
09:01:17:
DEBUG:
DEBUG:
DEBUG:
DEBUG:
DEBUG:
DEBUG:
DEBUG:
DEBUG:
dmp_get_events: Event type = 0x1

dmp_get_events: event type = DMP_PATH_DISABLE_EVENT
dmp_get_events: path 118/48, dmpnode 68/48
dmp_get_events: Event type = 0x1
dmp_get_events: event type = DMP_PATH_DISABLE_EVENT
dmp_get_events: path 118/32, dmpnode 68/0
dmp_get_events: Done
request_loop: No. of DMP events = 2
07/03 09:01:17: DEBUG: request_loop: notify_clients 0

07/03 09:01:17: DEBUG: request_loop: notify_clients 1
The vxconfigd logging Debug messages are not described in the

VERITAS Volume Manager 3.2 Troubleshooting Guide. To make these
messages useful, decode any device-related major and minor number
information included in the Debug message.
4-6

Logging Errors
In the previous example, the following error is displayed:
07/03 09:01:17: DEBUG: dmp_get_events: event type = DMP_PATH_DISABLE_EVENT
07/03 09:01:17: DEBUG: dmp_get_events: path 118/48, dmpnode 68/48
This message is decoded as follows:

1.
Look at the event type:

event type = DMP_PATH_DISABLE_EVENT
This indicates a path error to one or more disk devices.
2.
The part of the message following the event type defines the disk
device affected. The major and minor number of that device are
listed.
path 118/48
To find the cxtxdx number, do a long list on the /dev/dsk directory
and search for the major and minor number pair listed in the error
message. Use the following form of the ls command:
# ls -laRL /dev/dsk | grep 118, 48
brw-r----- 1 root sys 118, 48 Jun 7 23:47 c2t1d0s0
3.
Using this information, the experienced system administrator maps

this to a failing piece of hardware. From the following example and
the ls -l /dev/dsk/c2t1d0s0 command, the hardware address for
device c2t1d0s0s is:
/sbus@3,0/SUNW,socal@0,0/sf@1,0/ssd@w210000203713f96d,0
This address can now be cross-references to a specific I/O bus, card
slot, port and cable. Information about bus mapping is available on
the SunSolve Online Web site.
4.
The remaining part of the message following the event type points to
the DMP metanode. The major and minor number of that device are
listed:
dmpnode 68/48
This device can also be found by using the ls command to list the
/dev/vx/dmp directory:
# ls -laRL /dev/vx/dmp | grep 68, 48
brw------- 1 root other 68, 48 Jun 8 09:41 c1t1d0s0
This clearly defines the DMP metanode device as c1t1d0s0.
Note DMP devices, metanodes, and other information about DMP were
described in Module 3, Managing Dynamic Multi-Pathing.

4-7
Logging Errors
Interpreting /var/adm/messages File syslog

Messages
The VxVM software logging to the system syslog facility is enabled
automatically. The software writes errors detected by vxconfigd to the
/var/adm/messages file. Use the -x sys log option when starting
vxconfigd to write all console messages created by vxconfigd to the
syslog as well.
The following shows vxconfigd messages logged to the
/var/adm/messages file with a failed Fiber Channel cable to a disk array:
Jul 3 11:11:56 lowtide
ssdrestart transport failed (fffffffe)
Jul 3 11:11:56 lowtide vxdmp: [ID 619769 kern.notice] NOTICE: vxdmp: Path failure on
118/0x48
Jul 3 11:11:56 lowtide vxdmp: [ID 997040 kern.notice] NOTICE: vxvm:vxdmp: disabled
path 118/0x48 belonging to the dmpnode 68/0x48
Cross-reference this message to the section in the VERITAS Volume

Manager 3.2 Troubleshooting Guide on error messages generated by the
VxVM software. Use the following procedure to troubleshoot this
problem:
1.
Go to section 3 in the VERITAS Volume Manager 3.2 Troubleshooting

Guide.
2.
Go to the section on vxdmp NOTICE messages.
3.
Look up the NOTICE message for the failure listed in the messages
log. The error message description is as follows:
Description: A path under the control of the DMP driver failed. The
failed device major and minor numbers are supplied in the message.
Action: None
Note The action recommendation is None because the error was not
caused by the VxVM software. This is a hardware problem, and there are
no VxVM software commands that can fix the error.
4.
Find the major and minor numbers, and use the following version of
the ls command to find the cxtxdx number:
# ls -laRL /dev/dsk | grep 118, 48
brw-r----- 1 root sys 118, 48 Jun 7 23:47 c2t1d0s0
The failed path is c2t1d0s0. Map this path to a specific piece of
hardware using step 3 of the procedure described on page 4-7.
4-8

Logging Errors
The root Mail

When the VxVM software detects volume errors, it sends mail to the root
account on the local system. This example shows mail sent by the VxVM
software after a relocation error.
From root Sat Jun 8 11:56:12 2002
Date: Sat, 8 Jun 2002 11:56:12 -0700 (PDT)
From: Super-User <root>
Message-Id: <200206081856.g58IuCs05735@lowtide.localdomain>
To: root
Subject: Attempting VxVM relocation on host lowtide
Content-Length: 420
Relocation was not successful for subdisks on disk test1dg05 in
volume test1_vol-L02 in disk group test1dg. No replacement was made and the
disk is still unusable.
The following volumes have storage on test1dg05:
test1_vol-L02
These volumes are still usable, but the the redundancy of
those volumes is reduced. Any RAID-5 volumes with storage on
the failed disk may become unusable in the face of further
failures.
Read mail messages using either the mail or mailx utility.

4-9
Recovering From Errors
Recovering From Errors

The VxVM software is fault resilient and capable of resolving most errors
without work by the system administrator. The vxconfigd daemon can
handle any error as long as it recognizes the error. If vxconfigd cannot
correct an error, then the system administrator must debug and resolve
the problem.
To debug errors, use the messages generated by the VxVM software. A list
of error messages is located in the VERITAS Volume Manager 3.2
Troubleshooting Guide in the Error Messages section for the version of the
VxVM software installed on the system.
4-10

Using the Debugging Tools and Utilities

To install many of the VxVM software debugging tools, add the VRTSspt
package. This package is located in the VxVM software support
directory:
bash-2.03# ls
copyright
hwnotes.pdf
hwnotes.ps
pkgs
scripts
support
vr_notes.pdf
vxvm_notes.ps
vr_notes.ps
win32
vxvm_notes.pdf
The debugging tools are in the /opt/VRTSspt directory. A list of the

contents of this directory is as follows:
bash-2.03# ls -las /opt/VRTSspt
total 14
2 dr-x-----5 root
other
2 drwxr-xr-x 12 root
sys
2 dr-xr-xr-x
4 root
other
4 -r--r--r-1 root
other
2 dr-xr-xr-x
4 root
other
2 dr-xr-xr-x
3 root
other
/opt/VRTSspt/FS:
total 8
2 dr-xr-xr-x
4 root
other
2 dr-x-----5 root
other
2 dr-x-----2 root
other
2 dr-x-----2 root
other
/opt/VRTSspt/VRTSexplorer:
total 200
2 dr-xr-xr-x
4 root
other
2 dr-x-----5 root
other
26 -r--r--r-1 root
other
4 -r-xr--r-1 root
other
2 dr-xr-xr-x
4 root
other
8 -r-xr--r-1 root
other
4 -r-xr--r-1 root
other
6 -r-xr--r-1 root
other
6 -r-xr--r-1 root
other
2 -r-xr--r-1 root
other
2 -r-xr--r-1 root
other
2 -r-xr--r-1 root
other
2 dr-xr-xr-x
2 root
other
14 -r-xr--r-1 root
other
2 -r-xr--r-1 root
other
6 -r-xr--r-1 root
other
4 -r-xr--r-1 root
other
8 -r-xr--r-1 root
other
8 -r-xr--r-1 root
other
16 -r-xr--r-1 root
other
6 -r-xr--r-1 root
other
6 -r-xr--r-1 root
other
512
512
512
1444
1024
512
Jul
Jul
Jul
Aug
Jul
Jul
2
2
2
8
2
2
08:19
08:19
08:19
2001
08:19
08:19
.
..
FS
README.VRTSspt
VRTSexplorer
VVR
512
512
512
512
Jul
Jul
Jul
Jul
2
2
2
2
08:19
08:19
08:19
08:19
.
..
MetaSave
VxBench
1024
512
12645
1880
1024
3833
1611
2452
2701
955
343
764
512
6491
809
2658
1692
3192
4022
7349
2078
2960
Jul
Jul
Aug
Aug
Jul
Aug
Aug
Aug
Aug
Aug
Aug
Aug
Jul
Aug
Aug
Aug
Aug
Aug
Aug
Aug
Aug
Aug
2
2
21
21
2
21
21
21
21
21
21
21
2
21
21
21
21
21
21
21
21
21
08:19
08:19
2001
2001
08:19
2001
2001
2001
2001
2001
2001
2001
08:19
2001
2001
2001
2001
2001
2001
2001
2001
2001
.
..
README
VRTSexplorer
bin.SunOS
dbed
dbed1
edition.dro
edition.sybed
fw
gcm
isis
lib
main.SunOS
ndmp
sal
salr
samba
spc
spcs
spnas
sybed1

4-11

6 -r-xr--r-1
6 -r-xr--r-1
2 -r-xr--r-1
4 -r-xr--r-1
8 -r-xr--r-1
2 -r-xr--r-1
4 -r-xr--r-1
2 -r-xr--r-1
6 -r-xr--r-1
4 -r-xr--r-1
4 -r-xr--r-1
6 -r-xr--r-1
2 -r-xr--r-1
6 -r-xr--r-1
/opt/VRTSspt/VVR:
total 26
2 dr-xr-xr-x
3
2 dr-x-----5
4 -r-x-----1
2 -r-------1
4 -r-x-----1
2 dr-xr-xr-x
2
10 -r-x-----1
root
root
root
root
root
root
root
root
root
root
root
root
root
root
other
other
other
other
other
other
other
other
other
other
other
other
other
other
2834
2384
349
1538
3342
229
1220
663
2355
1584
1434
2250
486
2278
Aug
Aug
Aug
Aug
Aug
Aug
Aug
Aug
Aug
Aug
Aug
Aug
Aug
Aug
21
21
21
21
21
21
21
21
21
21
21
21
21
21
2001
2001
2001
2001
2001
2001
2001
2001
2001
2001
2001
2001
2001
2001
txpt
vcs
vfr
visnC
visnS
vmsa
vnas
vras
vrtsisp
vsap
vvr
vxfs
vxld
vxvm
root
root
root
root
root
root
root
other
other
other
other
other
other
other
512
512
1592
388
1517
512
5081
Jul 2 08:19 .
Jul 2 08:19 ..
Jul 3 2001 Client.pl
Jun 29 2001 README.Client_Server
Jun 29 2001 Server.pl
Jul 2 08:19 VolSig
May 21 2001 vvrmemstat
Note The FS and VVR tools require additional licenses and are not part
of the basic VxVM software. They are not described in this guide.
The VxVM software debugging tools are listed in Table 4-1.
Table 4-1 The VRTSspt Debugging Tools
Tool
Use
Tool License
metasave
Gathers metadata from a VxVM file

system
FS
vxbench
Generates various I/Os for

benchmarking file system
performance
FS
VRTSexplorer*
Gathers configuration information

about the system and any installed
Sun StorEdge products
VxVM
software
vvrmemstat
Gathers Sun StorEdge Volume

Replicator (VVR) memory usage
statistics
VVR
* Some of the information generated by VRTSexplorer is redundant with the

information generated by the Sun Explorer 3.5.0 data collector.
4-12


Table 4-1 The VRTSspt Debugging Tools (Continued)
Tool
Use
Tool License
volsig
Determines the data consistency

between the primary and secondary
volumes in a VVR configuration
VVR
Client.pl and
Server.pl
Simulates traffic between the primary

and secondary nodes in a VVR
configuration
VVR
* Some of the information generated by VRTSexplorer is redundant with the

information generated by the Sun Explorer 3.5.0 data collector.
There are additional diagnostic and debugging tools installed as part of

the VRTSvxvm package. These tools and utilities are in the
/etc/vx/diag.d directory. A list of the VRTSvxvm tools and utilities
available for use by system administrators is as follows:
bash-2.03# ls /etc/vx/diag.d
config.d
vxaslkey
vxdevwalk
macros.d
vxautoconfig vxdmpdbprint
scripts
vxconfigdump vxdmpdebug
bash-2.03# ls -las /etc/vx/diag.d/*
544 -r-xr-xr-x
1 root
sys
14 -r-xr-xr-x
1 root
sys
464 -r-xr-xr-x
1 root
sys
14 -r-xr-xr-x
1 root
sys
168 -r-xr-xr-x
1 root
sys
16 -r-xr-xr-x
1 root
sys
12 -r-xr-xr-x
1 root
sys
124 -r-xr-xr-x
1 root
sys
114 -r-xr-xr-x
1 root
sys
240 -r-xr-xr-x
1 root
sys
/etc/vx/diag.d/config.d:
total 8
2 drwxr-xr-x
4 root
sys
2 drwxr-xr-x
5 root
other
2 drwxr-xr-x
2 root
sys
2 drwxr-xr-x
2 root
sys
/etc/vx/diag.d/config.d/sparcv7:
total 418
2 drwxr-xr-x
2 root
sys
2 drwxr-xr-x
4 root
sys
114 -r-xr-xr-x
1 root
sys
114 -r-xr-xr-x
1 root
sys
114 -r-xr-xr-x
1 root
sys
24 -r-xr-xr-x
1 root
sys
vxdmpinq
vxdmptp
vxkprint
262172
6788
221192
6808
85396
7356
5932
62472
58164
114640
vxprivutil
Apr 4 19:36 /etc/vx/diag.d/vxaslkey

Aug 15 2001 /etc/vx/diag.d/vxautoconfig
Apr 4 19:37 /etc/vx/diag.d/vxconfigdump
Aug 15 2001 /etc/vx/diag.d/vxdevwalk
Apr 4 19:36 /etc/vx/diag.d/vxdmpdbprint
Apr 4 19:36 /etc/vx/diag.d/vxdmpdebug
Apr 4 19:36 /etc/vx/diag.d/vxdmpinq
Apr 4 19:36 /etc/vx/diag.d/vxdmptp
Apr 4 19:37 /etc/vx/diag.d/vxkprint
Aug 15 2001 /etc/vx/diag.d/vxprivutil
512
512
512
512
Jun
Jun
Jun
Jun
512
512
57440
57708
57824
11960
Jun
Jun
Aug
Aug
Aug
Aug
8
8
8
8
09:19
09:31
09:19
09:19
.
..
sparcv7
sparcv9
8 09:19 .
8 09:19 ..
15 2001 vxautoconfig.SunOS_5.6
15 2001 vxdevwalk.SunOS_5.6

4-13

24 -r-xr-xr-x
24 -r-xr-xr-x
1 root
1 root
sys
sys
/etc/vx/diag.d/config.d/sparcv9:
total 348
2 drwxr-xr-x
2 root
sys
2 drwxr-xr-x
4 root
sys
0 -r-xr-xr-x
1 root
sys
140 -r-xr-xr-x
1 root
sys
140 -r-xr-xr-x
1 root
sys
0 -r-xr-xr-x
1 root
sys
32 -r-xr-xr-x
1 root
sys
32 -r-xr-xr-x
1 root
sys
/etc/vx/diag.d/macros.d:
total 58
2 drwxr-xr-x
3 root
2 drwxr-xr-x
5 root
4 -rwxrwxr-x
1 root
2 -rwxrwxr-x
1 root
2 -rwxrwxr-x
1 root
0 -rwxrwxr-x
1 root
2 -rwxrwxr-x
1 root
2 -rwxrwxr-x
1 root
2 -rwxrwxr-x
1 root
2 -rwxrwxr-x
1 root
2 -rwxrwxr-x
1 root
2 -rwxrwxr-x
1 root
2 -rwxrwxr-x
1 root
2 -rwxrwxr-x
1 root
2 -rwxrwxr-x
1 root
2 -rwxrwxr-x
1 root
2 -rwxrwxr-x
1 root
2 -rwxrwxr-x
1 root
2 -rwxrwxr-x
1 root
2 -rwxrwxr-x
1 root
2 -rwxrwxr-x
1 root
2 -rwxrwxr-x
1 root
2 -rwxrwxr-x
1 root
2 -rwxrwxr-x
1 root
2 -rwxrwxr-x
1 root
2 -rwxrwxr-x
1 root
2 -rwxrwxr-x
1 root
2 -rwxrwxr-x
1 root
2 drwxr-xr-x
2 root
other
other
sys
sys
sys
sys
sys
sys
sys
sys
sys
sys
sys
sys
sys
sys
sys
sys
sys
sys
sys
sys
sys
sys
sys
sys
sys
sys
other
/etc/vx/diag.d/macros.d/sparcv9:
total 56
2 drwxr-xr-x
2 root
other
2 drwxr-xr-x
3 root
other
4 -rwxrwxr-x
1 root
sys
2 -rwxrwxr-x
1 root
sys
2 -rwxrwxr-x
1 root
sys
4-14
12032 Aug 15
11968 Aug 15
2001 vxdevwalk.SunOS_5.7
2001 vxdevwalk.SunOS_5.8
512
512
0
71408
71560
0
15416
15368
Jun
Jun
Aug
Aug
Aug
Aug
Aug
Aug
8 09:19 .
8 09:19 ..
1024
512
1975
94
99
0
435
69
72
285
82
364
448
78
104
52
63
66
50
150
40
117
582
289
298
193
121
25
1024
Jun
Jun
Aug
Apr
Apr
Apr
Apr
Apr
Apr
Apr
Apr
Apr
Apr
Apr
Apr
Aug
Aug
Aug
Aug
Aug
Aug
Apr
Apr
Apr
Aug
Apr
Apr
Apr
Jun
8
8
15
4
4
4
4
4
4
4
4
4
4
4
4
15
15
15
15
15
15
4
4
4
15
4
4
4
8
1024
1024
1964
97
99
Jun 8 09:31 .
Jun 8 09:31 ..
Aug 15 2001 dmp
Apr 4 19:36 dmp_cpuiocount
Apr 4 19:36 dmp_cpuiocount_next
09:31
09:31
2001
19:36
19:36
19:36
19:36
19:36
19:36
19:36
19:36
19:36
19:36
19:36
19:36
2001
2001
2001
2001
2001
2001
19:36
19:36
19:36
2001
19:36
19:36
19:36
09:31
.
..
dmp
dmp_cpuiocount
dmp_cpuiocount_next
dmp_cpuiocount_zero
dmp_ctlr
dmp_ctlr_list_next
dmp_ctlr_path_next
dmp_dev_list
dmp_dev_list_next_dmpnode
dmp_dmpnode
dmp_dmpnode_next
dmp_dmpnode_next_ptr
dmp_dmpnode_path_next
dmp_dmpopencount
dmp_end_dev_list_ctlrs
dmp_end_dev_list_dmpnodes
dmp_end_dmp_nodes
dmp_errq_buf
dmp_opath_next
dmp_opaths
dmp_path
dmp_print_dev_list_ctlrs
dmp_print_dev_list_dmpnodes
dmp_print_errq
dmp_print_errq_next
dmp_print_errq_null
sparcv9


0
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
-rwxrwxr-x
-rwxrwxr-x
-rwxrwxr-x
-rwxrwxr-x
-rwxrwxr-x
-rwxrwxr-x
-rwxrwxr-x
-rwxrwxr-x
-rwxrwxr-x
-rwxrwxr-x
-rwxrwxr-x
-rwxrwxr-x
-rwxrwxr-x
-rwxrwxr-x
-rwxrwxr-x
-rwxrwxr-x
-rwxrwxr-x
-rwxrwxr-x
-rwxrwxr-x
-rwxrwxr-x
-rwxrwxr-x
-rwxrwxr-x
-rwxrwxr-x
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
root
root
root
root
root
root
root
root
root
root
root
root
root
root
root
root
root
root
root
root
root
root
root
0
436
69
72
282
84
388
428
81
104
52
63
66
50
150
40
117
590
289
298
195
121
25
Apr
Apr
Apr
Apr
Apr
Apr
Apr
Apr
Apr
Apr
Aug
Aug
Aug
Aug
Aug
Aug
Apr
Apr
Apr
Aug
Apr
Apr
Apr
4
4
4
4
4
4
4
4
4
4
15
15
15
15
15
15
4
4
4
15
4
4
4
sys
other
sys
sys
sys
sys
sys
512
512
512
3541
2382
4897
5409
Jun
Jun
Jun
Aug
Aug
Aug
Aug
8 09:20 .
8 09:31 ..
8 09:20 fix_lib
15 2001 fixmountroot
15 2001 fixsetup
15 2001 fixstartup
15 2001 fixunroot
/etc/vx/diag.d/scripts/fix_lib:
total 16
2 drwxr-xr-x
2 root
sys
2 drwxr-xr-x
3 root
sys
8 -r-xr-xr-x
1 root
sys
4 -r-xr-xr-x
1 root
sys
512
512
3437
1409
Jun 8 09:20 .
Jun 8 09:20 ..
Aug 15 2001 fixdevsetup
Aug 15 2001 fixgetmajor
/etc/vx/diag.d/scripts:
total 42
2 drwxr-xr-x
3 root
2 drwxr-xr-x
5 root
2 drwxr-xr-x
2 root
8 -r-xr-xr-x
1 root
6 -r-xr-xr-x
1 root
10 -r-xr-xr-x
1 root
12 -r-xr-xr-x
1 root
sys
sys
sys
sys
sys
sys
sys
sys
sys
sys
sys
sys
sys
sys
sys
sys
sys
sys
sys
sys
sys
sys
sys
19:36
19:36
19:36
19:36
19:36
19:36
19:36
19:36
19:36
19:36
2001
2001
2001
2001
2001
2001
19:36
19:36
19:36
2001
19:36
19:36
19:36
dmp_cpuiocount_zero
dmp_ctlr
dmp_ctlr_list_next
dmp_ctlr_path_next
dmp_dev_list
dmp_dev_list_next_dmpnode
dmp_dmpnode
dmp_dmpnode_next
dmp_dmpnode_next_ptr
dmp_dmpnode_path_next
dmp_dmpopencount
dmp_end_dev_list_ctlrs
dmp_end_dev_list_dmpnodes
dmp_end_dmp_nodes
dmp_errq_buf
dmp_opath_next
dmp_opaths
dmp_path
dmp_print_dev_list_ctlrs
dmp_print_dev_list_dmpnodes
dmp_print_errq
dmp_print_errq_next
dmp_print_errq_null

4-15

Table 4-2 contains a partial list of the VRTSvxvm tools.
Note For information on how to use utilities in the /etc/vx/diag.d
directory that is not available in this guide or the man pages, on SunSolve,
from the command line prompt or from the VERITAS Software
Corporation support site, call support at Sun Microsystems.
Table 4-2 The VRTSvxvm Debugging Tools
4-16
Tool
Use
vxprivutil
Allows system administrators to list and modify the

private region of disks.
vxdevwalk
Traverses the dev_info tree in kernel space. It can be

used to correlate user space entries with those in the
dev_info tree.
vxkprint
Prints kernel space information about VxVM objects

such as disk groups, disks, and volumes.
vxdmpdebug
Creates a DMP configuration dump suitable for

sending to Sun second-level support. Used in
debugging DMP problems.
vxdmpinq
Provides output that displays a disks unique serial

number and vendor information. Used in debugging
DMP problems.
vxautoconfig
Prints device, driver, and controller information about

a servers attached storage. This is very low-level
information and might not be useful to system
administrators.
vxdmpdbprint
Prints contents of the DMP database. Useful for

troubleshooting DMP path and device errors.


Table 4-3 lists scripts found in the /etc/vx/diag.d/scripts directory.
Use these scripts to recover a VxVM software configuration that is
corrupted and prevents a system from booting. These scripts help
reconfigure the VxVM software by defining alternative locations for
binaries which can be located on a CD-ROM, and mounting boot disks
that are VxVM software volumes.
Caution The /etc/vx/diag.d/scripts scripts are designed to help
recover a corrupted VxVM software configuration that prevents a system
from booting or mounting the VxVM software devices. Module 2,
Encapsulating Disks, describes how to recover problems with booting
from encapsulated disks if the VxVM software does not render a system
un-bootable. Use the /etc/vx/diag.d/scripts scripts with caution or
under the direction of Sun support.
Table 4-3 The /etc/vx/diag.d/scripts Script Utilities
Script
Use
fixmountroot
Mounts a root file system that is encapsulated as a

volume. The file system is mounted as read-only,
using a file system slice that underlies the root file
system volume.
fixsetup
Configures the VxVM software to run from an

alternate directory. This allows the system
administrator to make changes to a VxVM software
configuration when the VxVM software binaries are
not accessible.
Run this script prior to running the fixmountroot
script.
fixstartup
Starts the VxVM software running from the VxVM

CD-ROM. The script requires some information that
the VxVM software normally gets from the root file
system. Run the fixstartup command if the device
containing the root file system or one of the mirrors of
the root file system volume is known.
The fixstartup command asks for a disk containing
a copy of the root file system, and uses the file system
on that disk to get the necessary startup files.

4-17

Table 4-3 The /etc/vx/diag.d/scripts Script Utilities (Continued)
Script
Use
fixunroot
Converts system files so that the files no longer

require the VxVM software to boot the root file
system. Also disables startup of the VxVM software,
so that future recovery of a mirrored root volume does
not cause corruption.
Caution After running this script, use caution when
bringing up the VxVM software again. If the VxVM
software configuration retains a mirrored root
volume, starting the VxVM software can cause severe
corruption to the root file system.
The vxexplorer Utility

The vxexplorer utility is an information-gathering tool that provides
detailed information about key system and VxVM software files and
objects. It is one of the most common VxVM software debugging tools
used. This section describes the information collected by vxexplorer.
Note Not all files and commands described in this section are relevant to
the Solaris OE. Some of the information collected by the vxexplorer
utility reflects the UNIX cross-platform capabilities of the VxVM software.
Some files and command output reference versions of UNIX other than
the Solaris OE.
General System Information

The vxexplorer utility general system information gathering options
include:
uname -a Displays host and OS revision information.
ioscan -f Scans and lists system I/O hardware.
4-18
model Prints system model (on systems that do not use SPARC
technology).
isainfo -v Displays supported instruction sets.
prtdiag Displays system configuration and diagnostic data

generated by power-on self-test (POST).

prtconf -v Displays general system-hardware configuration

information.
showrev -p Displays system patch list.
ps -elf Shows running processes, including the command line

options used to start them.
pkginfo -l Displays installed packages.
modinfo Displays loadable modules that are active on the system.
/etc/system, fstab, bootconf vfstab, and

/etc/name_to_major Captures key system information files from
/etc. (The fstab and bootconf files are not Solaris OE files.)
/etc/services Lists Internet services ports and protocols.
/var/adm/messages Contains messages showing the VxVM

software and Solaris OE error messages.
ls -l /kernel/drv/vx*, and /kernel/drv/sparcv9/vx*

Collects a list of installed VxVM software drivers.
/kernel/drv/*.conf Collects a complete list of driver

configuration files.
vxlicense -p, vxfsserial -p, and vxliccheck -vp Displays

VxVM software licensing data.
la -laR /dev/dsk, ls -laR /dev/rdsk, ls -laRL /dev/dsk,

and ls -laRL /dev/rdsk Displays physical device information,
including major and minor numbers.
ls -laR /dev/es, ls -laRL /dev/es, luxadm (for each es device),

and luxadm fcal_s_download Displays information about
installed Sun Enterprise Network Array arrays.
prtvtoc information Displays the vtoc for each disk.
diskinfo A command (not Solaris OE) to show a disks

characteristics.
df -k and mount -v Lists basic file system information.
ifconfig -a, netstat -in, netstat -rn, arp -a, and lanscan
Lists basic network information.
vgdisplay -v A command (not Solaris OE) to display information

about logical volume manager (LVM) volume groups.
ls -l /etc/rc* and cp -pR /etc/rc* Lists and copies boot and

startup scripts.

4-19
cp -pR /etc/inet, cp -pR /etc/*router, cp -pR /etc/host*,

cp -pR /etc/*nodename*, and cp -pR /etc/nsswitch.conf
Obtains copies of network configuration files.
cp -pR /var/spool/cron Obtains information on cron jobs.
eeprom command output Displays OpenBoot PROM information.
cp -pR /etc/dfs Obtains information on exported NFS files

systems.
VxVM Software Information

The vxexplorer utility options for gathering VxVM software-specific
information include:
vxdctl mode Displays the current operating mode of vxconfigd.

Effectively displays the state of the VxVM software.
vxprint outputs Displays the configuration of all VxVM software

objects.
vxkprint output Displays the VxVM softwares configuration as

seen by the kernel. This output can occasionally show useful
information.
vxdisk list output Shows state and configuration information

for VxVM software disks.
vxdg list output Displays disk group ownership and state

information. Shows which disk groups are imported.
/var/vxvm contents Lists and displays the contents of the temp_db

files. These files can be useful in troubleshooting odd disk group
problems.
ls -laR /dev/vx/* Displays the VxVM software device tree.
/etc/vx contents Contains various VxVM software configuration

and library files.
/VXVM*UPGRADE* contents Contains information pertaining to the

VxVM software configuration before upgrade. This file is created by
the upgrade_start script.
DMP Information
The vxexplorer utility options for gathering DMP-specific information
are:
4-20

vxautoconfig Captures the disks on a system that are visible to

the VxVM software without DMP. Mimics the VxVM software device
discovery routines.
vxdevwalk Captures disk device information by walking the kernel

device tree.
SCSI inquiry output DMP requires a unique serial number for each
disk on the system. It uses SCSI inquiries to read the serial number.
The vxexplorer utility uses the vxdmpinq command to capture this
information.
vxconfigd -x 9 output Starting vxconfigd with the -x 9 option

enables vxconfigd logging and displays all the steps used to start
vxconfigd. This includes DMP initialization and the display of all
the disks that are or are not visible to DMP. If the Sun StorEdge
Cluster Server (VCS) or EMC PowerPath software is running, a
storage or node failover might occur.
DMP kernel table variables Various kernel table information is

captured using the adb utility.
Volume Manager Storage Administrator (VMSA) Information

The contents of /var/opt/vmsa/logs captures system access, server, and
command information.
Other Non-VxVM Software Information

The vxexplorer utility collects output for other non-VxVM software
products, including the following. For more information about these nonVxVM products, see the documentation for the product:
SRVM
VxFS
VxLD (NFS Accellerator)
First Watch
VCS
Sybase Edition
Oracle Edition
Sanbox
Global Cluster Manager
VFR

4-21
VRAS
SANPoint Control
Information left by VERITAS Software Corporation support
Running VRTSexplorer Utility

To run VRTSexplorer, perform the following:
1.
Change to the /opt/VRTSspt/VRTSexplorer directory. Type the

following:
cd /opt/VRTSspt/VRTSexplorer
2.
Start VRTSexplorer. Type the following:
./VRTSexplorer
VRTSexplorer: Initializing.
VRTSexplorer: Please enter case number, or just hit enter: 12345 <---- note
VRTSexplorer: Please select a destination directory (default: /tmp):
VRTSexplorer: Using /tmp as destination directory.
VRTSexplorer: Collecting system configuration information for SunOS system.
VRTSexplorer: Collecting VERITAS package version information.
VRTSexplorer: Collecting loadable module information.
VRTSexplorer: Collecting VMSA configuration information.
VRTSexplorer: Determining current VxVM operating mode.
VRTSexplorer: Collecting VxVM configuration information.
VRTSexplorer: Collecting DMP configuration information.
NOTICE:
This section will stop and restart the VxVM Configuration Daemon,
vxconfigd. This may cause your VxVA and/or VMSA session to exit.
This may also cause a momentary stoppage of any VxVM configuration
actions. This should not harm any data; however, it may cause some
configuration operations (e.g. moving subdisks, plex
resynchronization) to abort unexpectedly. Any VxVM configuration
changes should be completed before running this section.
If you are using EMC PowerPath devices with VERITAS Volume Manager,
you must run the EMC command(s) powervxvm setup (or safevxvm
setup) and/or powervxvm online (or safevxvm online) if this
script terminates abnormally.
Restart VxVM Configuration Daemon? [y,n] (default: n) y <---- note

VRTSexplorer: Script finished.
VRTSexplorer: Please ftp /tmp/VRTSexplorer_12345_lowtide.tar.Z to
ftp.veritas.com:/incoming
When VRTSexplorer executes, the program prompts for an optional case

number and whether to stop and restart vxconfigd to enable vxconfigd
logging. Answer the prompts as appropriate for each system.
4-22


The VRTSexplorer output file is by default stored in /tmp using the
following file name format:
VRTSexplorer_<case_number>_<VxVM_hostid>.tar.Z
Open this file and use the contents to aid in troubleshooting VxVM
software problems.
Interpreting VRTSexplorer Output

Use the following procedure to open and interpret a vxexplorer output
file:
1.
Locate the VRTSexplorer_<case_number>_<VxVM_hostid>.tar.Z

file. The default location is /tmp.
All files in /tmp are removed when the system reboots. Either move
this file to /var/tmp, or instruct VRTSexplorer to store the file in
/var/tmp when you are asked for an output location during the
program execution.
2.
Uncompress and untar the file. Type the following:

# uncompress VRTSexplorer _<case_number>_<VxVM_hostid>.tar.Z
# tar xf VRTSexplorer _<case_number>_<VxVM_hostid>.tar
This creates the following directory:
./VRTSexplorer_<case_number>_<VxVM_hostid>
3.
Change directories to the new directory:

./VRTSexplorer_<case_number>_<VxVM_hostid>
The following example shows the contents of this directory:
bash-2.03# ls -las
total 1360
16 drwxr-xr-x
9
16 drwxrwxrwt
7
16 -rw-r--r-1
16 drwxr-xr-x
4
16 drwxr-xr-x
3
16 -rw-r--r-1
16 -rw-r--r-1
16 -rw-r--r-1
16 -rw-r--r-1
16 drwxr-xr-x 12
16 -rw-r--r-1
16 -rw-r--r-1
16 -rw-r--r-1
16 drwxr-xr-x
3
16 -rw-r--r-1
16 -rw-r--r-1
16 -rw-r--r-1
root
root
root
root
root
root
root
root
root
root
root
root
root
root
root
root
root
other
sys
other
sys
other
other
other
other
other
other
other
other
other
other
other
other
other
2294
676
359
309
3308
529
4380
1152
1503
2426
9
477
54
177
7132
1759
542
Jul
Jul
Jul
Jun
Jul
Jul
Jul
Jul
Jul
Jul
Jul
Jul
Jul
Jul
Jul
Jul
Jul
3
3
3
7
3
3
3
3
3
3
3
3
3
3
3
3
3
06:27
06:27
06:00
23:11
06:01
06:00
06:00
06:00
06:00
06:01
05:59
06:00
05:59
06:00
06:00
06:00
06:00
.
..
arp_a
cron
dev
df_klaV
df_klag
df_klat
eeprom
etc
hostid
ifconfig_a
isainfo_v
kernel
modinfo
mount_v
netstat_i

4-23

16
16
64
576
112
16
16
16
96
16
16
16
16
16
16
16
16
16
16
-rw-r--r--rw-r--r--rw-r--r--rw-r--r--rw-r--r--rw-r--r--rw-r--r--rw-r--r--rw-r--r--rw-r--r--rw-r--r-drwxr-xr-x
drwxr-xr-x
-rw-r--r--rw-r--r--rw-r--r--rw-r--r--rw-r--r-drwxr-xr-x
1
1
1
1
1
1
1
1
1
1
1
4
3
1
1
1
1
1
3
4.
root
root
root
root
root
root
root
root
root
root
root
root
root
root
root
root
root
root
root
other
other
other
other
other
other
other
other
other
other
other
other
other
other
other
other
other
other
other
921
1322
29532
292733
51497
1708
4832
580
43673
70
66
302
242
233
834
69
41
240
1953
Jul
Jul
Jul
Jul
Jul
Jul
Jul
Jul
Jul
Jul
Jul
Jul
Jul
Jul
Jul
Jul
Jul
Jul
Jul
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
06:00
06:00
05:59
06:00
05:59
05:59
05:59
05:59
06:00
05:59
05:59
06:01
06:00
05:59
05:59
05:59
06:00
06:00
06:01
netstat_r
pkgchk
pkginfo
pkginfo_l
prtconf
prtdiag
ps_elf
reboot
showrev
uname_a
uptime
var
vmsa
vmstat
vmstat_s
vxexplore_version
vxliccheck_vp
vxlicense_p
vxvm
From information stored in the preceding files, determine the

systems configuration as follows:
a.
Verify the hostid.

# more ./hostid
807d5d60
b.
Verify the Solaris OE revision and hardware architecture.

# more uname_a
SunOS lowtide 5.8 Generic_108528-14 sun4u sparc
SUNW,Ultra-Enterprise
c.
Determine the version of the VxVM software installed by

viewing the contents of the pkginfo file. If the VxVM software
was upgraded on the system, there may be more than one
pkginfo file. To determine which is the most current listing,
look at the PKGINST and INSTDATE fields, as shown in the
following example.
# more ./pkginfo_l
... skipping
PKGINST: VRTSvxvm
NAME: VERITAS Volume Manager, Binaries
CATEGORY: system
ARCH: sparc
VERSION: 3.2,REV=08.15.2001.23.27
BASEDIR: /
VENDOR: VERITAS Software
DESC: Virtual Disk Subsystem
PSTAMP: VERITAS-3.2t_p2.5:23-May-2002
INSTDATE: Jun 08 2002 09:30
HOTLINE: 800-342-0652
EMAIL: support@veritas.com
4-24


STATUS:
FILES:
completely installed
479 installed pathnames
16 shared pathnames
8 linked files
59 directories
306 executables
1 setuid/setgid executables
116150 blocks used (approx)
Note The ./pkginfo_l file is a long list of all packages installed on the
system. Search through the file to find the VRTS packages.
d.
Verify that all correct modules are loaded for the VRTS
packages installed. If the modules are not loaded, there may
have been problems upgrading the VxVM software, or the
system was not rebooted after the modules were installed. The
following example shows a list of installed modules.
bash-2.03# grep vx ./modinfo

19 101e9005 ffa88 74
1 vxio (VxVM 3.2t_p2.5 I/O driver)
21 102d5440 17428 68
1 vxdmp (VxVM 3.2t_p2.5: DMP Driver)
22 102ea888
83f 75
1 vxspec (VxVM 3.2t_p2.5 control/status
e.
Verify the correct licenses are installed by viewing the

vxlicense_p file.
bash-2.03# more ./vxlicense_p

vrts:vxlicense:
vrts:vxlicense:
vrts:vxlicense:
vrts:vxlicense:
vrts:vxlicense:
INFO:
INFO:
INFO:
INFO:
INFO:
Feature name: PHOTON [98]

Number of licenses: 99
Expiration date: No expiration date
Release Level: 20
Machine Class: 334147806
f.
Check the OpenBoot PROM information by opening the eeprom

file. Some OpenBoot PROM information is shown in the
following example.
# more eeprom
.
.
.
use-nvramrc?=true
nvramrc=devalias test2 /sbus@3,0/SUNW,socal@0,0/sf@0,0/ssd@w220000203713df01,0:a
devalias test1 /sbus@3,0/SUNW,socal@0,0/sf@0,0/ssd@w210000203713fc9f,0:a
devalias vx-rootdisk /sbus@3,0/SUNW,socal@0,0/sf@0,0/ssd@w220000203713fc9f,0:a

devalias vx-rootmirror /sbus@3,0/SUNW,socal@0,0/sf@0,0/ssd@w220000203713f96d,0:a

4-25

g.
Viewing the prtdiag displays system POST diagnostic

information.
bash-2.03# more prtdiag

System Configuration: Sun Microsystems
System clock frequency: 84 MHz
Memory size: 256Mb
sun4u 8-slot Sun Enterprise 4000/5000
========================= CPUs =========================
Brd
--0
0
CPU
--0
1
Module
------0
1
Run
MHz
----168
168
Ecache
MB
-----1.0
1.0
CPU
Impl.
-----US-I
US-I
CPU
Mask
---4.0
4.0
========================= Memory =========================
Brd
--0
Bank
----0
MB
---256
Status
------Active
Condition
---------OK
Speed
----60ns
Intrlv.
Factor
------1-way
Intrlv.
With
-------
========================= IO Cards =========================
Brd
---
Bus
Type
----
Freq
MHz
----
SBus
25
SBus
Slot
----------
Name
--------------------------------
Model
-----------------
SUNW,socal/sf (scsi-3)
501-5266
25
SUNW,hme
SUNW,501-2919
SBus
25
DOLPHIN,sci
SBus
25
SUNW,hme
SBus
25
SUNW,fas/sd (block)
SBus
25
13
SUNW,soc
501-2069
No failures found in System

===========================
h.
bash-2.03# ls -las
total 880
16 drwxr-xr-x
3
16 drwxr-xr-x
9
64 -rw-r--r-1
4-26
The ./dev directory contains a copy of the contents of the /dev

directory of the live system. Use this information to verify
which devices that Solaris OE sees, including major and minor
numbers, prtvtoc output and more. An example follows:
./dev
root
root
root
other
other
other
3308 Jul
2294 Jul
31869 Jul
3 06:01 .
3 06:27 ..
3 06:00 dsk


32
16
16
16
16
16
16
16
16
16
16
16
16
16
16
16
16
-rw-r--r--rw-r--r--rw-r--r--rw-r--r--rw-r--r--rw-r--r--rw-r--r--rw-r--r--rw-r--r--rw-r--r--rw-r--r--rw-r--r--rw-r--r--rw-r--r--rw-r--r--rw-r--r--rw-r--r--
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
root
root
root
root
root
root
root
root
root
root
root
root
root
root
root
root
root
other
other
other
other
other
other
other
other
other
other
other
other
other
other
other
other
other
14860
663
367
1779
1779
1779
1779
143
41
930
648
648
648
648
929
481
481
Jul
Jul
Jul
Jul
Jul
Jul
Jul
Jul
Jul
Jul
Jul
Jul
Jul
Jul
Jul
Jul
Jul
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
06:00
06:00
06:00
06:00
06:00
06:00
06:00
06:00
06:00
06:00
06:00
06:00
06:00
06:00
06:00
06:00
06:00
bash-2.03# more ./dev/dsk

/dev/dsk:
total 482
drwxr-xr-x
2 root
sys
5120 Jun 7 23:47
drwxr-xr-x 14 root
sys
3584 Jul 3 05:29
lrwxrwxrwx
1 root
root
50 Jun 7 23:47
../../devices/sbus@3,0/SUNW,fas@3,8800000/sd@6,0:a
lrwxrwxrwx
1 root
root
50 Jun 7 23:47
../../devices/sbus@3,0/SUNW,fas@3,8800000/sd@6,0:b
lrwxrwxrwx
1 root
root
50 Jun 7 23:47
../../devices/sbus@3,0/SUNW,fas@3,8800000/sd@6,0:c
lrwxrwxrwx
1 root
root
50 Jun 7 23:47
../../devices/sbus@3,0/SUNW,fas@3,8800000/sd@6,0:d
lrwxrwxrwx
1 root
root
50 Jun 7 23:47
../../devices/sbus@3,0/SUNW,fas@3,8800000/sd@6,0:e
lrwxrwxrwx
1 root
root
50 Jun 7 23:47
../../devices/sbus@3,0/SUNW,fas@3,8800000/sd@6,0:f
dsk_device
es
es_device
es_luxadm_display_ses0
es_luxadm_fcal_s_download
prtvtoc_c0t6d0s2
prtvtoc_c1t0d0s2
prtvtoc_c1t16d0s2
prtvtoc_c1t17d0s2
prtvtoc_c1t18d0s2
prtvtoc_c1t19d0s2
prtvtoc_c1t1d0s2
prtvtoc_c1t20d0s2
prtvtoc_c1t21d0s2
.
..
c0t6d0s0 ->
c0t6d0s1 ->
c0t6d0s2 ->
c0t6d0s3 ->
c0t6d0s4 ->
c0t6d0s5 ->
bash-2.03# more prtvtoc_c1t16d0s2

*
* Dimensions:
*
512 bytes/sector
*
133 sectors/track
*
27 tracks/cylinder
*
*
4926 cylinders
*
*
* Flags:
*
1: unmountable
* 10: read-only
*
*
First
Sector
Last
*
Sector
Count
Sector
*
3591
3591
7181
4-27

*
*
* Partition
2
3
4
Tag
5
15
14
Flags
00
01
01
i.
bash-2.03# ls -las
total 640
16 drwxr-xr-x 12
16 drwxr-xr-x
9
16 -rw-r--r-1
16 drwxr-xr-x
2
16 -rw-r--r-1
16 -rw-r--r-1
16 -rw-r--r-1
16 -r--r--r-1
16 drwxr-xr-x
2
32 -rw-r--r-1
16 -rw-r--r-1
16 -rw-r--r-1
16 -rw-r--r-1
16 -r--r--r-1
16 -rwxr--r-1
16 drwxr-xr-x
2
16 -rwxr--r-1
16 drwxr-xr-x
2
16 -rwxr--r-1
16 drwxr-xr-x
2
16 -rwxr--r-1
16 drwxrwxr-x
2
16 -rwxr--r-1
16 -rwxr--r-1
32 -rwxr--r-1
16 drwxr-xr-x
2
16 drwxr-xr-x
3
16 -r--r--r-1
16 -r--r--r-1
16 -rw-r--r-1
16 drwxr-xr-x
2
16 -rw-r--r-1
16 -rw-r--r-1
16 -rw-r--r-1
16 -rw-r--r-1
16 -rw-r--r-1
16 -rw-r--r-1
16 drwxr-xr-x
9
First
Sector
Last
Sector
Count
Sector
0 17682084 17682083
0
3591
3590
7182 17674902 17682083
Mount Directory
Check the ./etc directory, which contains a copy of the

contents of the /etc directory of the live system. An example is
as follows:
./etc
root
root
root
root
root
root
root
root
root
root
root
root
root
root
root
root
root
root
root
root
root
root
root
root
root
root
root
root
root
root
root
root
root
root
root
root
root
root
other
other
other
sys
other
root
root
sys
sys
other
sys
root
sys
root
sys
sys
sys
sys
sys
sys
sys
sys
sys
sys
sys
sys
sys
sys
sys
sys
other
root
root
other
other
root
other
other
2426
2294
12
308
12
8
1
97
1223
12200
1731
8
780
5036
2792
2558
3177
2347
2885
3456
2341
650
2792
2792
9973
3275
181
184
3701
1001
182
2161
2161
2161
2161
728
415
1483
Jul 3 06:01 .
Jul 3 06:27 ..
Jun 8 05:43 defaultrouter
Jun 7 23:12 dfs
Jul 3 06:01 err.out
Jun 7 23:47 hostname.hme0
Jun 7 23:47 hostname6.hme0
Jun 8 05:43 hosts
Jun 8 04:52 inet
Jul 3 06:00 ls_l_rc
Jun 8 09:20 name_to_major
Jun 7 23:47 nodename
Jun 8 07:34 nsswitch.conf
Jun 8 09:28 path_to_inst
Jan 5 2000 rc0
Jun 8 09:32 rc0.d
Jan 5 2000 rc1
Jun 8 09:23 rc1.d
Jan 5 2000 rc2
Jun 8 09:23 rc2.d
Jan 5 2000 rc3
Jun 8 08:11 rc3.d
Jan 5 2000 rc5
Jan 5 2000 rc6
Jan 5 2000 rcS
Jun 8 09:32 rcS.d
Jun 7 23:11 rcm
Dec 18 2001 release
Jun 8 06:50 services
Jun 7 23:12 syslog.conf
Jul 3 06:00 syslog.d
Jun 8 09:49 system
Jun 8 09:49 system.GOOD
Jun 21 17:15 system.sav
Jun 24 14:39 system_06242002
Jun 8 11:23 vfstab
Jun 8 09:41 vfstab.prevm
Jul 3 06:01 vx
bash-2.03# ls -las ./etc/vx

total 320
4-28


16 drwxr-xr-x
9 root
16 drwxr-xr-x 12 root
0 -rw-r--r-1 root
16 -rw-rw-rw1 root
16 drwxr-xr-x
2 root
16 lrwxrwxrwx
1 root
16 -rw-r--r-1 root
16 lrwxrwxrwx
1 root
/usr/lib/vxvm/diag.d
16 drwxr-xr-x
2 root
16 -rwxrwxrwx
1 root
16 -rw-r--r-1 root
16 drwxr-xr-x
3 root
16 drwxr-xr-x
6 root
16 drwxr-xr-x
2 root
16 drwxr-xr-x
2 root
16 -rw-r--r-1 root
16 drwxr-xr-x
2 root
16 lrwxrwxrwx
1 root
/usr/lib/vxvm/voladm.d
16 -rw-r--r-1 root
0 prw------1 root
0 prw------1 root
16 -rw-r--r-1 root
16 -rw-r--r-1 root
j.
bash-2.03# ls -las ./var

total 64
16 drwxr-xr-x
4 root
16 drwxr-xr-x
9 root
16 drwxr-xr-x
2 root
0 -rw-r--r-1 root
16 drwxr-xr-x
3 root
other
other
root
other
other
other
other
other
1483
2426
0
30
1319
17
1261
20
Jul
Jul
Jun
Jul
Jun
Jul
Jun
Jul
sys
root
other
other
sys
root
other
other
sys
other
180
16
22
185
498
201
1101
5
117
22
Jul 3 06:00 elm

Jun 21 16:56 guid.state
Jun 8 09:33 jbod.info
Jun 8 09:19 lib
Jun 8 09:49 reconfig.d
Jun 8 11:56 saveconfig.d
Jun 8 09:33 slib
Jun 8 09:22 sr_port
Jun 8 09:19 type
Jul 3 06:27 voladm.d ->
root
other
other
other
other
512
0
0
59
59
Jun
Jun
Jun
Jun
Jun
3
3
8
3
8
3
8
3
8
8
8
8
8
06:01
06:01
09:48
05:29
09:31
06:27
09:33
06:27
09:48
09:20
09:20
09:41
09:41
.
..
.dumpadm
array.info
aslkey.d
bin -> /usr/lib/vxvm/bin
ddl.support
diag.d ->
volboot
vold_diag
vold_request
vxdmp.exclude
vxvm.exclude
The ./var directory contains the captured messages files. Use

the information in this directory to detect failure messages
logged by the VxVM software. An example is as follows:
other
other
other
other
sys
302
2294
308
0
180
Jul
Jul
Jul
Jul
Jul
3
3
3
3
3
06:01
06:27
06:00
06:01
05:29
.
..
adm
err.out
vxvm
bash-2.03# more ./var/adm/messages

.
.
.
Jun 8 09:56:55 lowtide pseudo: [ID 129642 kern.info] pseudo-device: devinfo0
Jun 8 09:56:55 lowtide genunix: [ID 936769 kern.info] devinfo0 is /pseudo/devinfo@0
Jun 8 11:55:40 lowtide vxdmp: [ID 619769 kern.notice] NOTICE: vxdmp: Path failure on
118/0x84
Jun 8 11:55:40 lowtide vxdmp: [ID 997040 kern.notice] NOTICE: vxvm:vxdmp: disabled path
118/0x80 belonging to the
dmpnode 68/0x10

4-29

The temp_db files are also captured under the ./var/vxvm
directory. An example is as follows:
bash-2.03# ls -las
total 48
16 drwxr-xr-x
2
16 drwxr-xr-x
3
16 -rw-r--r-1
./var/vxvm/tempdb
root
root
root
k.
root
sys
root
180 Jul
180 Jul
5120 Jul
3 05:29 .
3 05:29 ..
3 05:29 rootdg
The ./vxvm directory contains information about the VxVM

softwares disks and disk groups. There is also a ./vxvm/dmp
directory that contains DMP debugging information. The
./dmp/dmp.out file contains the output from execution of the
autoconfig, vxdevwalk, and vxdmpinq commands, among
others.
An example of the ./vxvm directory contents is as follows:
bash-2.03# ls -las
total 464
16 drwxr-xr-x
3
16 drwxr-xr-x
9
16 drwxr-xr-x
2
16 -rw-r--r-1
16 -rw-r--r-1
16 -rw-r--r-1
16 -rw-r--r-1
16 -rw-r--r-1
16 -rw-r--r-1
16 -rw-r--r-1
16 -rw-r--r-1
16 -rw-r--r-1
16 -rw-r--r-1
16 -rw-r--r-1
16 -rw-r--r-1
16 -rw-r--r-1
16 -rw-r--r-1
16 -rw-r--r-1
16 -rw-r--r-1
16 -rw-r--r-1
16 -rw-r--r-1
16 -rw-r--r-1
32 -rw-r--r-1
16 -rw-r--r-1
16 -rw-r--r-1
32 -rw-r--r-1
16 -rw-r--r-1
bash-2.03# ls -las
total 848
16 drwxr-xr-x
2
16 drwxr-xr-x
3
816 -rw-r--r-1
4-30
root
root
root
root
root
root
root
root
root
root
root
root
root
root
root
root
root
root
root
root
root
root
root
root
root
root
root
other
other
other
other
other
other
other
other
other
other
other
other
other
other
other
other
other
other
other
other
other
other
other
other
other
other
other
1953
2294
181
14
78
491
841
925
891
889
889
890
927
219
222
222
929
218
883
883
215
2107
13555
1985
2585
15366
432
Jul
Jul
Jul
Jul
Jul
Jul
Jul
Jul
Jul
Jul
Jul
Jul
Jul
Jul
Jul
Jul
Jul
Jul
Jul
Jul
Jul
Jul
Jul
Jul
Jul
Jul
Jul
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
06:01
06:27
06:01
06:00
06:01
06:01
06:00
06:00
06:00
06:00
06:01
06:01
06:00
06:01
06:01
06:01
06:00
06:00
06:00
06:00
06:00
06:00
06:00
06:00
06:00
06:01
06:01
.
..
dmp
vxdctl_mode
vxdg_list
vxdg_list_rootdg
vxdisk_list
vxdisk_list_c1t0d0s2
vxdisk_s_list
vxkprint
vxprint
vxprint_ht
vxprint_mpvshr_rootdg
vxstat_g_rootdg
./dmp
root
root
root
other
other
other
181 Jul
1953 Jul
414252 Jul
3 06:01 .
3 06:01 ..
3 06:02 dmp.out


bash-2.03# more vxdg_list
NAME
STATE
ID
rootdg
enabled 1023554924.1025.lowtide
bash-2.03# more vxdisk_list
DEVICE
TYPE
DISK
c1t0d0s2
sliced
rootdisk
c1t1d0s2
sliced
rootmirror
c1t2d0s2
sliced
disk01
c1t3d0s2
sliced
c1t4d0s2
sliced
c1t5d0s2
sliced
c1t6d0s2
sliced
c1t16d0s2
sliced
c1t17d0s2
sliced
c1t18d0s2
sliced
c1t19d0s2
sliced
c1t20d0s2
sliced
c1t21d0s2
sliced
c1t22d0s2
sliced
-
GROUP
rootdg
rootdg
rootdg
-
STATUS
online
online
online spare
error
online
online
error
online
online
online
online
error
error
error
bash-2.03# more vxprint_ht

Disk group: rootdg
DG
DM
RV
RL
V
PL
SD
SV
DC
SP
NAME
NAME
NAME
NAME
NAME
NAME
NAME
NAME
NAME
NAME
NCONFIG
DEVICE
RLINK_CNT
RVG
RVG
VOLUME
PLEX
PLEX
PARENTVOL
SNAPVOL
NLOG
TYPE
KSTATE
KSTATE
KSTATE
KSTATE
DISK
VOLNAME
LOGVOL
DCO
MINORS
PRIVLEN
STATE
STATE
STATE
STATE
DISKOFFS
NVOLLAYR
GROUP-ID
PUBLEN
PRIMARY
REM_HOST
LENGTH
LENGTH
LENGTH
LENGTH
dg rootdg
default
default
1023554924.1025.lowtide
dm disk01
dm rootdisk
dm rootmirror
c1t2d0s2
c1t0d0s2
c1t1d0s2
sliced
sliced
sliced
3590
3590
3590
17674902 SPARE
17678493 17678493 -
v
pl
sd
sd
pl
sd
rootvol
rootvol-01
rootvol
rootvol-02
rootvol
ENABLED ACTIVE
ENABLED ACTIVE
rootdisk 17678492
rootdisk 0
ENABLED ACTIVE
rootmirror 0
4197879
4197879
1
4197878
4197879
4197879
v
pl
sd
pl
sd
swapvol
swapvol-01
swapvol
swapvol-02
swapvol
ENABLED ACTIVE
1052163
ENABLED ACTIVE
1052163
rootdisk 4197878 1052163
ENABLED ACTIVE
1052163
rootmirror 4197879 1052163
STATE
DATAVOLS
REM_DG
READPOL
LAYOUT
[COL/]OFF
[COL/]OFF
SRL
REM_RLNK
PREFPLEX
NCOL/WID
DEVICE
AM/NM
UTYPE
MODE
MODE
MODE
ROUND
CONCAT
0
1
CONCAT
0
c1t0d0
c1t0d0
c1t1d0
root
RW
ENA
ENA
RW
ENA
ROUND
CONCAT
0
CONCAT
0
c1t0d0
c1t1d0
swap
RW
ENA
RW
ENA

4-31

v
pl
sd
pl
sd
usr
usr-01
usr
rootdisk-04 usr-01
usr-02
usr
rootmirror-03 usr-02
ENABLED ACTIVE
4197879
ENABLED ACTIVE
4197879
rootdisk 5250041 4197879
ENABLED ACTIVE
4197879
rootmirror 5250042 4197879
ROUND
CONCAT
0
CONCAT
0
c1t0d0
c1t1d0
fsgen
RW
ENA
RW
ENA
v
pl
sd
pl
sd
var
var-01
var
rootdisk-03 var-01
var-02
var
ENABLED ACTIVE
4197879
ENABLED ACTIVE
4197879
rootdisk 9447920 4197879
ENABLED ACTIVE
4197879
rootmirror 9447921 4197879
ROUND
CONCAT
0
CONCAT
0
c1t0d0
c1t1d0
fsgen
RW
ENA
RW
ENA
bash-2.03# more ./dmp/dmp.out

DMP Debugging information
testautoconfig output
binding_name:
node_name:
node name:
node addr:
parent_name:
instance:
minor:
binding_name:
node_name:
node name:
node addr:
parent_name:
instance:
minor:
binding_name:
node_name:
node name:
node addr:
.
.
.
ssd
ssd
ssd
w220000203713f582,0
sf
14
a, ddi_block:wwn
ssd
ssd
ssd
w220000203713f643,0
sf
15
a, ddi_block:wwn
ssd
ssd
ssd
w220000203713e0b9,0
vxdevwalk output
SUNW,Ultra-Enterprise :: id=-268264420
driver properties:
pm-hardware-state value=6e6f2d73 75737065 6e642d72 6573756d 6500
ascii=no-suspend-resume.
system properties:
relative-addressing value=00000001
MMU_PAGEOFFSET value=00001fff
MMU_PAGESIZE value=00002000
PAGESIZE value=00002000
Driver packages :: id=-268251420
Driver packages/terminal-emulator :: id=-268212908
4-32


Driver packages/deblocker :: id=-268201280
Driver packages/obp-tftp :: id=-268199524
scsi inquiry command outputs <--- vxdmpinq exexution
/dev/rdsk/c1t0d0s2
Vendor id
: SEAGATE
Product id
: ST19171FCSUN9.0G
Revision
: 177E
Serial Number
: 9831X06256
/dev/rdsk/c1t16d0s2
Vendor id
: SEAGATE
Product id
: ST19171FCSUN9.0G
Revision
: 177E
Serial Number
: 9831X07536
.
.
.
The dmp.out file is large and contains the output of all DMP debugging
and information gathering l commands executed by vxexplorer. Use
shell commands to parse this file for the information needed to verify the
existence of a DMP problem.
The vxprivutil Utility

System administrators use the vxprivutil command to list and modify
the private region of disks. Use this command with care, because it is
capable of damaging the VxVM software configuration database beyond
repair.
Use the following syntax for this utility:
# /etc/vx/bin/vxprivutil
vxvm:vxprivutil: TO FIX: Usage:
vxprivutil [-o privoffset] scan device-pathname
vxprivutil [-o privoffset] list device-pathname
vxprivutil [-o privoffset] set device-pathname attribute ...
vxprivutil [-o poffs] [-l] dumplog device-pathname [copy-number]
vxprivutil [-o poffs] [-cf] [-C path-to-vxconfigdump] dumpconfig
device-pathname [copy-number]
Caution The set option can damage the configuration database (private
region) beyond repair if not used properly. Some of the information
presented in this section is for reference only.

4-33
The dumpconfig Option

The dumpconfig option dumps any enabled configuration copy stored in
the specified private region (slice 3). The most common usage of this
option is to pipe the dumpconfig into a vxprint command as shown in
the following example:
bash-2.03# vxprivutil dumpconfig /dev/rdsk/c1t2d0s3 | vxprint -D - -ht
Disk group: rootdg
DG
DM
RV
RL
V
PL
SD
SV
DC
SP
NAME
NAME
NAME
NAME
NAME
NAME
NAME
NAME
NAME
NAME
NCONFIG
DEVICE
RLINK_CNT
RVG
RVG
VOLUME
PLEX
PLEX
PARENTVOL
SNAPVOL
NLOG
TYPE
KSTATE
KSTATE
KSTATE
KSTATE
DISK
VOLNAME
LOGVOL
DCO
MINORS
PRIVLEN
STATE
STATE
STATE
STATE
DISKOFFS
NVOLLAYR
GROUP-ID
PUBLEN
PRIMARY
REM_HOST
LENGTH
LENGTH
LENGTH
LENGTH
dg rootdg
default
default
1023554924.1025.lowtide
dm disk01
dm rootdisk
dm rootmirror
SPARE
-
4197879
4197879
1
4197878
4197879
4197879
ROUND
CONCAT
0
1
CONCAT
0
root
RW
DIS
DIS
RW
DIS
STATE
DATAVOLS
REM_DG
READPOL
LAYOUT
[COL/]OFF
[COL/]OFF
SRL
REM_RLNK
PREFPLEX
NCOL/WID
DEVICE
AM/NM
UTYPE
MODE
MODE
MODE
v
pl
sd
sd
pl
sd
rootvol
rootvol-01
rootvol
rootvol-02
rootvol
DISABLED ACTIVE
DISABLED ACTIVE
rootdisk 17678492
rootdisk 0
DISABLED ACTIVE
rootmirror 0
v
pl
sd
pl
sd
swapvol
swapvol-01
swapvol
swapvol-02
swapvol
DISABLED ACTIVE
1052163
DISABLED ACTIVE
1052163
rootdisk 4197878 1052163
DISABLED ACTIVE
1052163
rootmirror 4197879 1052163
ROUND
CONCAT
0
CONCAT
0
swap
RW
DIS
RW
DIS
v
pl
sd
pl
sd
usr
usr-01
usr
rootdisk-04 usr-01
usr-02
usr
DISABLED ACTIVE
4197879
DISABLED ACTIVE
4197879
rootdisk 5250041 4197879
DISABLED ACTIVE
4197879
rootmirror 5250042 4197879
ROUND
CONCAT
0
CONCAT
0
fsgen
RW
DIS
RW
DIS
v
pl
sd
pl
sd
var
var-01
var
rootdisk-03 var-01
var-02
var
DISABLED ACTIVE
4197879
DISABLED ACTIVE
4197879
rootdisk 9447920 4197879
DISABLED ACTIVE
4197879
rootmirror 9447921 4197879
ROUND
CONCAT
0
CONCAT
0
fsgen
RW
DIS
RW
DIS
4-34

The dumplog Option

The dumplog option lists the group log copy information stored in the
private region. An example of the dumplog option is shown here.
# ./vxprivutil dumplog /dev/rdsk/c0t3d0s3
LOG #01
BLOCK 0: KLOG 0 :
COMMIT tid=0.1352
BLOCK 0: KLOG 1 :
DIRTY rid=0.1154
The list Option

The list option provides output similar in format and content to the
vxdisk list command. This is a useful option to use when vxconfigd is
not running. An example is as follows:
# ./vxprivutil list /dev/rdsk/c0t3d0s3
diskid: 921260771.1030.stoli.veritas.com
group:
name=rootdg id=921260769.1025.stoli.veritas.com
flags:
private autoimport
hostid: stoli.veritas.com
version: 2.1
iosize: 512
public: slice=4 offset=0 len=673536
private: slice=3 offset=1 len=1151
update: time: 921263222 seqno: 0.17
headers: 0 248
configs: count=1 len=825
logs:
count=1 len=125
tocblks: 0
tocs:
1/1150
Defined regions:
config
config
log
The scan Option

The scan option produces an abbreviated disk information listing.
# ./vxprivutil scan /dev/rdsk/c0t3d0s3
diskid: 921260771.1030.stoli.veritas.com
group:
name=rootdg id=921260769.1025.stoli.veritas.com
flags:
private autoimport
hostid: stoli.veritas.com
version: 2.1
iosize: 512
public: slice=4 offset=0 len=673536
private: slice=3 offset=1 len=1151
update: time: 921263222 seqno: 0.17
headers: 0 248

4-35

configs: count=1 len=825
logs:
count=1 len=125
The set Option

Use the set option to change the attributes of a disk or disk group
without using vxconfigd.
Caution Use the set option carefully. This option can cause irreversible
damage to the configuration database in a disks private region and
damage it beyond repair.
Use the set option to modify the following attributes:
{ "diskid", optsetstr, DTAG(dka_diskid) },
{ "hostid", optsetstr, DTAG(dka_hostid) },
{ "dgid", optsetstr, DTAG(dka_dgid) },
{ "dg_name", optsetstr, DTAG(dka_dg_name) },
{ "dgname", optsetstr, DTAG(dka_dg_name) },
{ "dg", optsetstr, DTAG(dka_dg_name) },
{ "shared", optsetbool, OFFS(struct disk_attrs, dka_flags), DKA_FLAG_SHARED},
{ "autoimport", optsetbool, OFFS(struct disk_attrs,dka_flags),
DKA_FLAG_AUTOIMPORT },
Example Use of the set Option

See the following for how to use the set option. In this example, a new
disk group initialization failed because disk c1t15d0 is owned by rootdg.
# vxdg init newdg disk05=c1t15d0s2
vxvm:vxdg: ERROR: Device c1t15d0s2 appears to be owned by disk group rootdg.
# vxdisk list c1t15d0

Device:
c1t15d0s2
devicetag: c1t15d0
type:
sliced
hostid:
plstr04.veritas.com
disk:
name= id=968346728.1264.plstr04.veritas.com
group:
name=rootdg id=968426567.1289.plstr04.veritas.com
flags:
online ready private autoconfig noautoimport
version:
2.1
iosize:
public:
private:
update:
time=968446320 seqno=0.18
headers:
0 248
4-36


configs:
count=1 len=2226
logs:
count=1 len=337
Defined regions:
config
config
log
numpaths:
1
c1t15d0s2
state=enabled
An attempt to import a disk group locks clearing import (-C), as shown in

the following output:
# vxdg -C import 968426567.1289.plstr04.veritas.com
vxvm:vxdg: ERROR: Disk group 968426567.1289.plstr04.veritas.com: import failed:
Record already exists in disk group
Use the vxprivutil command to clear the dgname, as follows:

# ./vxprivutil set /dev/rdsk/c1t15d0s3 dgname=""
# vxdisk list c1t15d0
Device:
c1t15d0s2
devicetag: c1t15d0
type:
sliced
hostid:
plstr04.veritas.com
disk:
group:
flags:
online ready private autoconfig noautoimport
version:
2.1
iosize:
public:
private:
update:
time=968446579 seqno=0.21
headers:
0 248
configs:
count=1 len=2226
logs:
count=1 len=337
Defined regions:
config
config
log
numpaths:
1
c1t15d0s2
state=enabled
Next, import the disk group again:

# vxdg import 968426567.1289.plstr04.veritas.com
# vxdisk list
DEVICE
TYPE
DISK
GROUP
STATUS
c0t0d0s2
sliced
rootdisk
rootdg
online
c1t8d0s2
sliced
online

4-37

c1t9d0s2
c1t10d0s2
c1t11d0s2
c1t12d0s2
c1t13d0s2
c1t14d0s2
c1t15d0s2
sliced
sliced
sliced
sliced
sliced
sliced
sliced
disk01
disk02
disk03
disk04
m9dg
m9dg
m9dg
m9dg
online
online
online
online
online
online
online
The vxdevwalk Utility

The vxdevwalk utility scans the kernel dev_info tree and prints detailed
information about disk devices. This information includes physical device
paths, instance information, driver properties, and major and minor
numbers for each possible slice on the disk. For example:
Driver sbus@3,0/SUNW,socal@0,0/sf@1,0/ssd@w210000203713f579,0 :: id=18 instance=11
driver properties:
inquiry-revision-id value=31373745 00
ascii=177E.
inquiry-product-id value=53543139 31373146 4353554e 392e3047 00
ascii=ST19171FCSUN9.0G.
inquiry-vendor-id value=53454147 41544500
ascii=SEAGATE.
pm-hardware-state value=6e656564 732d7375 7370656e 642d7265 73756d65
00
ascii=needs-suspend-resume.
ddi-kernel-ioctl
device nodes:
a :: dev=118,88:block nodetype=ddi_block:wwn
b :: dev=118,89:block nodetype=ddi_block:wwn
c :: dev=118,90:block nodetype=ddi_block:wwn
d :: dev=118,91:block nodetype=ddi_block:wwn
e :: dev=118,92:block nodetype=ddi_block:wwn
f :: dev=118,93:block nodetype=ddi_block:wwn
g :: dev=118,94:block nodetype=ddi_block:wwn
h :: dev=118,95:block nodetype=ddi_block:wwn
a,raw :: dev=118,88:char nodetype=ddi_block:wwn
b,raw :: dev=118,89:char nodetype=ddi_block:wwn
c,raw :: dev=118,90:char nodetype=ddi_block:wwn
d,raw :: dev=118,91:char nodetype=ddi_block:wwn
e,raw :: dev=118,92:char nodetype=ddi_block:wwn
f,raw :: dev=118,93:char nodetype=ddi_block:wwn
g,raw :: dev=118,94:char nodetype=ddi_block:wwn
h,raw :: dev=118,95:char nodetype=ddi_block:wwn
4-38


Use this information to correlate the kernel space information with user
space entries by performing a long list on the /dev/dsk directory and
grep on the wwn or major and minor numbers of the listed device. For
example:
bash-2.03# ls -las /dev/dsk | grep 3f579
2 lrwxrwxrwx
1 root
root
70 Jun 7 23:47 c1t2d0s0 ->
2 lrwxrwxrwx
1 root
root
70 Jun 7 23:47 c1t2d0s1 ->
../../devices/sbus@3,0/SUNW,socal@0,0/sf@0,0/ssd@w220000203713f579,0:b
2 lrwxrwxrwx
1 root
root
70 Jun 7 23:47 c1t2d0s2 ->
../../devices/sbus@3,0/SUNW,socal@0,0/sf@0,0/ssd@w220000203713f579,0:c
2 lrwxrwxrwx
1 root
root
70 Jun 7 23:47 c1t2d0s3 ->
../../devices/sbus@3,0/SUNW,socal@0,0/sf@0,0/ssd@w220000203713f579,0:d
2 lrwxrwxrwx
1 root
root
70 Jun 7 23:47 c1t2d0s4 ->
../../devices/sbus@3,0/SUNW,socal@0,0/sf@0,0/ssd@w220000203713f579,0:e
2 lrwxrwxrwx
1 root
root
70 Jun 7 23:47 c1t2d0s5 ->
../../devices/sbus@3,0/SUNW,socal@0,0/sf@0,0/ssd@w220000203713f579,0:f
2 lrwxrwxrwx
1 root
root
70 Jun 7 23:47 c1t2d0s6 ->
../../devices/sbus@3,0/SUNW,socal@0,0/sf@0,0/ssd@w220000203713f579,0:g
2 lrwxrwxrwx
1 root
root
70 Jun 7 23:47 c1t2d0s7 ->
../../devices/sbus@3,0/SUNW,socal@0,0/sf@0,0/ssd@w220000203713f579,0:h
2 lrwxrwxrwx
1 root
root
70 Jun 7 23:47 c2t2d0s0 ->
2 lrwxrwxrwx
1 root
root
70 Jun 7 23:47 c2t2d0s1 ->
../../devices/sbus@3,0/SUNW,socal@0,0/sf@1,0/ssd@w210000203713f579,0:b
2 lrwxrwxrwx
1 root
root
70 Jun 7 23:47 c2t2d0s2 ->
../../devices/sbus@3,0/SUNW,socal@0,0/sf@1,0/ssd@w210000203713f579,0:c
2 lrwxrwxrwx
1 root
root
70 Jun 7 23:47 c2t2d0s3 ->
../../devices/sbus@3,0/SUNW,socal@0,0/sf@1,0/ssd@w210000203713f579,0:d
2 lrwxrwxrwx
1 root
root
70 Jun 7 23:47 c2t2d0s4 ->
../../devices/sbus@3,0/SUNW,socal@0,0/sf@1,0/ssd@w210000203713f579,0:e
2 lrwxrwxrwx
1 root
root
70 Jun 7 23:47 c2t2d0s5 ->
../../devices/sbus@3,0/SUNW,socal@0,0/sf@1,0/ssd@w210000203713f579,0:f
2 lrwxrwxrwx
1 root
root
70 Jun 7 23:47 c2t2d0s6 ->
../../devices/sbus@3,0/SUNW,socal@0,0/sf@1,0/ssd@w210000203713f579,0:g
2 lrwxrwxrwx
1 root
root
70 Jun 7 23:47 c2t2d0s7 ->
../../devices/sbus@3,0/SUNW,socal@0,0/sf@1,0/ssd@w210000203713f579,0:h
If the device is multi-pathed, this output shows both paths. In the

previous example, controllers c1 and c2 are multiple paths for a single
device.

4-39
The vxkprint Utility

The vxkprint utility prints kernel space information about VxVM
software objects, such as disk groups, disks, and volumes. Output from
this utility can be useful if other VxVM software commands are not
providing useful information about a VxVM software object.
Note The output of this command can be very large. Consider redirecting the command output to a file and using a text editor to search
the file contents.
The following example shows truncated output from a vxkprint
command:
.
.
.
# Groups: (cnt: 2)
#
Group NULLDG: iid=0.0 id=
version: 0
configtid=0.0 logsize=0
kflag=()
vflag=(created|notempdb|noautoreimport)
diskdetpolicy= 0
# Group-Objects: (cnt: 0)
# End-group: NULLDG
Group rootdg: iid=0.1 id=1023554924.1025.lowtide
version: 90
configtid=0.1351 logsize=398
kflag=(activated_ew)
vflag=(created|tempdb|autoreimport)
diskdetpolicy= 1
# Group-Objects: (cnt: 21)
.
.
.
Disk group information presented by this command shows that disk

group rootdg is imported by system lowtide. The disk group revision
level is 90.
4-40

Using System-Level Debugging Utilities
Using System-Level Debugging Utilities

The Solaris OE has system level debugging utilities that help debug
problems which cause the system to panic. When the system panics, a
panic dump file is usually written. Some system-level utilities include:
adb
kadb
crash
System administrators use these utilities to analyze panic dump files. The
VxVM software could at some time cause a system panic. It is helpful to
be able to diagnose a panic dump file to determine whether the VxVM
software was responsible for the crash and what component of the VxVM
software failed.
Note Analyzing panic dump files is beyond the scope of this class. Sun
Education provides classes that teach the use of system-level debugging
tools to analyze crash dump files.

4-41
Exercise: Using the Error Logging and Debugging Utilities

In this exercise, you will complete the following tasks:
Enable vxconfigd debugging
Generate simple errors
View error messages in the messages log and the vxconfigd log
Use the VERITAS Volume Manager 3.2 Troubleshooting Guide for an

explanation of error message
Run VRTSexplorer and view data
Use vxprivutil to display the contents of a disks private region
Use vxdevwalk to display dev_info tree data
Preparation
To prepare for this exercise, make sure that the VxVM software is installed
and operational.
Task 1 Enabling vxconfigd Debug Logging

1.
Enable maximum vxconfigd logging from the command line. What

command did you use?
_____________________________________________________________
2.
What is the location of the log?

_____________________________________________________________
3.
View the contents of the log. Is there data present?

_____________________________________________________________
_____________________________________________________________
_____________________________________________________________
4.
Disable vxconfigd logging. What command did you use?

_____________________________________________________________
5.
4-42
Remove the vxconfigd log.

Task 2 Viewing the VxVM Software Messages

1.
Enable maximum vxconfigd logging. What command did you use?

_____________________________________________________________
2.
Open a new window, and view the contents of the vxconfigd log in
real time. What command did you use?
_____________________________________________________________
3.
Open a new window, and view the current messages file in real time.
What command did you use?
_____________________________________________________________
4.
Pull a fiber cable.
5.
Perform an I/O operation to the attached disks by executing the

following command:
# find / -name foo.txt
6.
View the messages and vxconfigd log as the DMP notices that a
path is missing.
7.
After the logging completes, find the error message logged by DMP
in the messages file, and locate the appropriate message in the
VERITAS Volume Manager 3.2 Troubleshooting Guide.
Answer the following questions:
What error messages from the messages file were you able find
in the VERITAS Volume Manager 3.2 Troubleshooting Guide?
________________________________________________________
________________________________________________________
________________________________________________________
________________________________________________________
________________________________________________________
What is the cxtxdx address of the failed path?

________________________________________________________
________________________________________________________
What is the physical path of the failed path?

________________________________________________________
________________________________________________________

4-43
Power user The following task is for students who have the skills to
map a device tree entry to a physical hardware slot.
If you have access to SunSolve Online, pull the hardware mapping
for the system used by your lab group and map the physical device
address of the failing path to the cable that was disconnected.
What is the physical Peripheral Component Interconnect (PCI) or
SBus slot of the pulled cable?
_____________________________________________________________
8.
Replace the pulled cable.
9.
View any messages logged by the VxVM software as the path is

brought online.
_____________________________________________________________
_____________________________________________________________
10. Disable vxconfigd logging.

11. Close the real-time viewing window for the vxconfigd log.
12. Remove the vxconfigd log.
Task 3 Using VxVM Software Debug Utilities

4-44
1.
Install the VRTSspt package, if it is not already installed.
2.
Run the VRTSexplorer utility using the procedure described in The

vxexplorer Utility on page 4-18.
3.
Browse the output, and view the following files:
hostid
uname_a
pkginfo_l Search for the VxVM software packages and

verify the package levels and installation dates.
vxlicense_p
prtdiag
eeprom

4.
Contents of various files under the ./dev subdirectory
Contents of the various files under the ./vxvm subdirectory.
Run the vxprivutil command with the dumpconfig option on a

selected VxVM software disk. Pipe the output to the vxprint
command to display a vxprint -ht type of printout.
_____________________________________________________________
5.
Run the vxprivutil command with the list option on a selected

VxVM software disk, and view the output.
6.
Run the vxdevwalk command, and view the output.
7.
Run the vxkprint command, and view the output.

4-45
Exercise Summary
Exercise Summary
!
?
4-46

Experiences
Interpretations
Conclusions
Applications


The solutions to the lab exercises are provided in this section.
Task 1 Solutions
1.
Enable maximum vxconfigd logging from the command line. What

command did you use?
# pkill -9 vxconfigd; vxconfigd -x 9 -x log -x syslog&

2.
What is the location of the log?
In /var/vxvm/vxconfigd.log.
3.
View the contents of the log. Is there data present?

_____________________________________________________________
_____________________________________________________________
4.
Disable vxconfigd logging. What command did you use?
# pkill -9 vxconfigd; vxconfigd -m enable&

5.
Remove the vxconfigd log.
Task 2 Solutions
1.
Enable maximum vxconfigd logging. What command did you use?
# pkill -9 vxconfigd; vxconfigd -x 9 -x log -x syslog&

2.
Open a new window, and view the contents of the vxconfigd log in
real time. What command did you use?
# tail -f /var/vxvm/vxconfigd.log
3.
Open a new window and view the current messages file in real time.
# tail -f /var/adm/messages

4-47

4.
Pull a fiber cable.
5.
Perform an I/O operation to the attached disks by executing the

following command:
# find / -name foo.txt
6.
View the messages and vxconfigd log as the DMP notices that a
path is missing.
7.
After the logging completes, find the error message logged by DMP
in the messages file and locate the appropriate message in the
VERITAS Volume Manager 3.2 Troubleshooting Guide.
Answer the following questions:
What error messages from the messages file were you able find
in the VERITAS Volume Manager 3.2 Troubleshooting Guide?
________________________________________________________
________________________________________________________
________________________________________________________
________________________________________________________
________________________________________________________
What is the cxtxdx address of the failed path?

________________________________________________________
________________________________________________________
What is the physical path of the failed path?

________________________________________________________
________________________________________________________
Power user The following task is for students who have the skills to
map a device tree entry to a physical hardware slot.
If you have access to SunSolve Online, pull the hardware mapping
for the system used by your lab group and map the physical device
address of the failing path to the cable that was disconnected.
What is the physical Peripheral Component Interconnect (PCI) or
SBus slot of the pulled cable?
_____________________________________________________________
8.
4-48
Replace the pulled cable.


9.
View any messages logged by the VxVM software as the path is

brought online.
_____________________________________________________________
_____________________________________________________________
10. Disable vxconfigd logging.

11. Close the real-time viewing window for the vxconfigd log.
12. Remove the vxconfigd log.
Task 3 Solutions
1.
Install the VRTSspt package, if it is not already installed.
2.
Run the VRTSexplorer utility using the procedure described in The

vxexplorer Utility on page 4-18.
3.
Browse the output, and view the following files:
4.
hostid
uname_a
pkginfo_l Search for the VxVM software packages and

verify the package levels and installation dates.
vxlicense_p
prtdiag
eeprom
Contents of various files under the ./dev subdirectory
Contents of the various files under the ./vxvm subdirectory.
Run the vxprivutil command with the dumpconfig option on a

selected VxVM software disk. Pipe the output to the vxprint
command to display a vxprint -ht type of printout.
# vxprivutil dumpconfig /var/rdsk/cxtxdxs3 | vxprint -D - ht

4-49
4-50
5.
Run the vxprivutil command with the list option on a selected

VxVM software disk, and view the output.
6.
Run the vxdevwalk command, and view the output.
7.
Run the vxkprint command, and view the output.

Module 5
Recovering Boot and System Processes

Objectives
Describe the VxVM software system recovery processes
Describe how the VxVM software is initialized during system boot
Successfully troubleshoot boot problems that prevent the VxVM

software from starting
Identify errors that prevent the VxVM software from functioning
Use the correct recovery procedures to resolve initialization and

operational problems
Correctly determine when to reinstall the VxVM software
Identify the VxVM software errors, match these errors to a list of

known errors and successfully repair the problems

5-1
Relevance
Relevance
!
?
5-2
Discussion The following questions are relevant to understanding what

the VxVM software boot and recovery processes are:
What files are accessed during system boot to initialize the VxVM
software?
What entries in the /etc/system file are necessary for the VxVM
software initialization?
What recovery procedure is used on a system that has a problem

preventing the VxVM software from starting?
What procedures can be used to boot a system without starting the

the VxVM software?
What recovery procedure is used on a system that has corrupted

VxVM software binaries?
What VxVM software configuration allows multiple rootdgs to coexist on a single system?





SunSolveSM Online SRDB 24657,

[http://sunsolve.Sun.COM/pub-cgi/search.pl?mode=advanced].


5-3
Surveying VxVM Software System Recovery Processes
Surveying VxVM Software System Recovery Processes

The VxVM software utilities, commands and devices are only available
when the VxVM software is functional. Recovery of the VxVM software
and the ability to improve the fault resiliency of storage devices under
VxVM software management is critical for maintaining servers in
production environments. The VxVM software recovery processes and
procedures presented in this guide are organized as follows:
Boot process failures that prevent the VxVM software from starting
Failures of the VxVM software which prevent it from functioning

once the system is booted
Storage device errors
This module addresses errors that occur in the boot process and the
VxVM software functionality. Storage device errors are addressed in
Module 6, Recovering Disk, Disk Group, and Volume Failures.
5-4

Examining the VxVM Software Boot Process

The VxVM software initializes as part of the Solaris OE boot processing.
This section lists the processes initiated by the startup scripts used to
initialize the VxVM software. Startup script processing is divided into
actions initiated by scripts executing in single-user mode and those
executing in multi-user mode. Figure 5-1 illustrates when these scripts
execute.
/etc/rcS.d
Single-User Start-Script
Execution
System
Ready
Boot
/etc/rc2.d
Multi-User Start-Script
Execution
Figure 5-1
Boot Scripts
Scripts which execute in single-user mode that affect the VxVM software
initialization are:
/etc/rcS.d/S25vxvm-sysboot
/etc/rcS.d/S35vxvm-startup1
/etc/rcs.d/S40standardmounts
/etc/rcS.d/S50drvconfig
/etc/rcS.d/S60devlinks
/etc/rcS.d/S86vxvm-reconfig

5-5

Scripts which execute in multi-user mode that affect the VxVM software
initialization are:
/etc/rc2.d/S20sysetup
/etc/rc2.d/S94vxnm-host_infod
/etc/rc2.d/S94vxnm-vxnetd
/etc/rc2.d/S95vxvm-recover
/etc/rc2.d/S96vmsa-server
Note Appendix D, The Boot Process, contains an example of the

boot -v process, with annotations which indicate when each of these
scripts execute. Refer to this appendix for more information on processes
that execute during boot processing.
Single-User Boot Processing

This section describes the startup scripts that execute during the singleuser mode of boot processing.
The S25vxvm-sysboot Script

Figure 5-2 illustrates the actions started during the execution of the
/etc/rcS.d/S25vxvm-sysboot script.
Disk access records are created.
The rootdg ownership is

determined.
/etc/rcS.d/S25vxvmsysboot
VxVM software is started in boot
mode.
If the boot disk is on a stale plex, a

new mirror is suggested.
Figure 5-2
5-6
The /etc/rcS.dvxvm-sysboot Script


Exclusion of certain DMP devices was implemented in the VxVM
software version 3.1. Information about excluded drivers is downloaded
to DMP by invoking vxdmpadm with the doioctl option prior to the start
of vxconfigd by the S25vxvm-sysboot script. The VxVM software
version 3.2 does not use the vxdmpadm doioctl command; this
functionality is now incorporated into the vxconfigd process.
Disk Access Record Creation

Disk access records are created for all devices, as follows:
The dev_info tree is scanned for new devices.

If the Solaris OE does not see a disk using format, the VxVM
software does not see the disk.
Kernel and user-space entries are matched.

These are the /dev/dsk links. If a user-space entry does not exist, the
VxVM software guesses what it should be.
New names are given to unmatched entries.
The dev_info tree is a kernel structure that is built from device tree
information in the boot PROM. To view the dev_info tree, use the
/etc/vx/diag.d/vxdevwalk command.
An example shows an excerpt from the execution of vxdevwalk
command. The following example shows the dev_info tree data for device
ssd@w220000203713fc9f.
Driver sbus@3,0/SUNW,socal@0,0/sf@0,0/ssd@w220000203713fc9f,0 :: id=25 instance=18
driver properties:
inquiry-revision-id value=31373745 00
ascii=177E.
inquiry-product-id value=53543139 31373146 4353554e 392e3047 00
ascii=ST19171FCSUN9.0G.
inquiry-vendor-id value=53454147 41544500
ascii=SEAGATE.
pm-hardware-state value=6e656564 732d7375 7370656e 642d7265 73756d65
00
ascii=needs-suspend-resume.
ddi-kernel-ioctl
device nodes:
a :: dev=118,144:block nodetype=ddi_block:wwn
b :: dev=118,145:block nodetype=ddi_block:wwn
c :: dev=118,146:block nodetype=ddi_block:wwn
d :: dev=118,147:block nodetype=ddi_block:wwn
e :: dev=118,148:block nodetype=ddi_block:wwn
f :: dev=118,149:block nodetype=ddi_block:wwn
g :: dev=118,150:block nodetype=ddi_block:wwn

5-7

h :: dev=118,151:block nodetype=ddi_block:wwn
a,raw :: dev=118,144:char nodetype=ddi_block:wwn
b,raw :: dev=118,145:char nodetype=ddi_block:wwn
c,raw :: dev=118,146:char nodetype=ddi_block:wwn
d,raw :: dev=118,147:char nodetype=ddi_block:wwn
e,raw :: dev=118,148:char nodetype=ddi_block:wwn
f,raw :: dev=118,149:char nodetype=ddi_block:wwn
g,raw :: dev=118,150:char nodetype=ddi_block:wwn
h,raw :: dev=118,151:char nodetype=ddi_block:wwn
Matching Driver String Entries

User-space entries are located in the /dev/dsk directory. For example:
bash-2.03# ls -las /dev/dsk/c1t0d0s2
2 lrwxrwxrwx
1 root
root
70 Jun 7 23:47 /dev/dsk/c1t0d0s2 ->
../../devices/sbus@3,0/SUNW,socal@0,0/sf@0,0/ssd@w220000203713fc9f,0:c
The entries are matched by the dev_info tree entry, shown in the
following example.
Driver sbus@3,0/SUNW,socal@0,0/sf@0,0/ssd@w220000203713fc9f,0 :: id=25 instance=18
Unmatched Entries
Unmatched entries are given new names, as follows:
Entries are identified in the dev_info tree.
If a match is not located in user space, an arbitrary name is created.
Disk Access Records

Use the vxdisk list command to display all user-space disk access
records. The following example shows typical output from the vxdisk
list command.
DEVICE
TYPE
c1t0d0s2
sliced
c1t1d0s2
sliced
c1t2d0s2
sliced
c1t3d0s2
sliced
c1t4d0s2
sliced
c1t5d0s2
sliced
c1t6d0s2
sliced
c1t16d0s2
sliced
c1t17d0s2
sliced
c1t18d0s2
sliced
c1t19d0s2
sliced
5-8
DISK
rootdisk
rootmirror
disk01
-
GROUP
rootdg
rootdg
rootdg
-
STATUS
online
online
online spare
error
online
online
error
online
online
online
online


c1t20d0s2
c1t21d0s2
c1t22d0s2
sliced
sliced
sliced
error
error
error
Note Only disks that are members of imported disk groups have entries
in the DISK and GROUP columns.
Disk Ownership
The /etc/vx/volboot file is used to delineate disk ownership. The
vxdctl process uses this file to manage the state of vxconfigd and to
bootstrap rootdg during the VxVM software initialization. The following
example illustrates a basic volboot file.
volboot 3.1 0.2 30
hostid lowtide
end
###############################################################
###############################################################
###############################################################
###############################################################
###############################################################
###############################################################
###############################################################
#########################
If simple disks are part of a servers VxVM software configuration, these

disks must be listed in the volboot file, or the VxVM software does not
recognize them. The following example illustrates a /etc/vx/volboot
file that includes information on simple disks.
volboot 3.1 0.2 30
hostid lowtide
disk c1t6d0s4 simple privoffset=1
end
###############################################################
###############################################################
###############################################################
###############################################################
###############################################################
######################################################
This file must be 512 bytes in length, including padding characters. Do not
edit this file using vi or other text editor. If this file is corrupted, the
VxVM software initialization fails.

5-9

To update the /etc/vx/volboot file, use the following commands:
vxdctl hostid newname
vxdctl init newname
Running the vxdctl init newname command recreates the

/etc/vx/volboot file.
Caution If simple disks are part of a servers VxVM disk configuration,
these entries are deleted if a vxdctl init newname command is run.
Record this information prior to running this command to allow
restoration of the simple disk configuration using the vxdctl add disk
c#t#d#s# command.
Boot Mode
During this part of the VxVM software initialization, the software is
started in boot mode using the vxconfigd -m boot command. The
following initialization operations occur at this time:
The rootdg disk group is imported.
If the boot disk is under the VxVM software control, rootvol and
user volumes are started.
Note If the /etc/rcS.d/S25vxvm-sysboot file is modified for

vxconfigd debugging using the vxconfigd -x 1-9 command, it starts in
debug mode at this time. The 1-9 option increases the level of debugging
information which vxconfigd displays during initialization. Other debug
sub-options are available for use with the -x 1-9 option and are listed in
the vxconfigd man page.
Boot Disk Stale Plex

During /etc/rcS.d/S25vxvm-sysboot processing, booting from a stale
plex is checked. If a stale boot disk plex is detected, the following actions
occur:
5-10
1.
The boot process stops.
2.
An alternate mirror is suggested, if one exists.


An original equipment manufacturer (OEM)-supported script can be
created to automatically suggest the alternate mirror. The
/etc/vx/sbin/vxstale file is no longer packaged with the VxVM
software distribution.
Flag Files Are Checked

The following flag files are checked. The files can affect the way in which
the VxVM software is initialized:
/etc/vx/reconfig.d/state.d/install-db
This file is created during installation of the VxVM software

packages.
This file is also created if an installation of the VxVM software
is incomplete. In that case, if the boot disk is under the VxVM
software control, the system does not boot past single-user
mode.
This file prevents vxconfigd from starting when the system

boots. The VxVM software starts if the boot disk is under the
VxVM software control, but it is crippled.
Note Although vxconfigd may not be started, the vxdmp and vxspec
modules are started.
/VXVM#.#.#-UPGRADE/.start_runed
The value #.#.# is the software level to which the system is

being upgraded, such as 3.1.1.
This is a hidden file, created by the upgrade_start script. It is

removed when the upgrade is finished.
This file prevents vxconfigd from starting even if the boot disk
is under the VxVM software control.

5-11
The S35vxvm-startup1 Script

The processes started by execution of the S35vxvm-startup1 script are
illustrated in Figure 5-3.
Start special volumes.
/etc/rcS.d/
S35vxvm-startup1
Setup dump devices.
Figure 5-3
The /etc/rcS,d/S35vxvm-startup1 Script
This script executes after the / and /usr volumes are available and makes
other volumes available that are needed early in the Solaris OE boot
sequence.
Special Volumes
The following special volumes, if configured, are started:
swap
/var
Note These volumes must be in rootdg directory.

Recovery operations are not performed, including:
5-12
Mirror re-synchronization
Parity re-synchronization

Dump Device
The dump device is used to store core information when the system
panics. Dump device configuration is as follows:
A swap device must be listed in the /etc/vfstab file.
The swap device must be in rootdg.
A physical partition must be available underneath the swap volume.

Swap files cannot be used as a dump device.
The dump device is registered by adding and then removing the swap
device. The VxVM software does not have hooks for dumping, so the
swap device must be created prior to the creation of the dump
device.
The dump device must be the first swap device listed in the
/etc/vfstab file.
Core file creation and recovery are performed outside of the VxVM
software operations.
The S40standardmounts Script

The S40standardmounts script sets the swap device. This script is not
part of the VxVM software Swap devices have the following
characteristics:
The VxVM software treats all file systems with file type swap in the
/etc/vfstab file as swap volumes.
The primary swap volume must be:
Usage type swap
A real partition
In rootdg
Swap device size is limited to 2000 megabytes in the early releases of

the Solaris OE.

5-13
The S50devfsadm Script

Figure 5-4 illustrates the processes started by the S50devfsadm script. This
script is not part of the VxVM software but is used to scan the device tree
and build new devices.
Scan dev_info tree for new
devices.
/etc/rcS.d/S50devfsadm
New entries are created in

/devices.
Create new /dev/dsk

and
/dev/rdsk entries.
Figure 5-4
The /etc/rcS.d/S50devfsadm Script
The /etc/rcS.d/S50devfsadm script has the following characteristics:
The script executes during all boots.
When the script executes as part of a reconfiguration reboot, new

devices are discovered and configured. To perform a reconfiguration
reboot, do the following:
5-14
1.
Use the boot -r command.
2.
Set the /reconfigure flag.
Solaris OE releases prior to release 8 used the

/etc/rcS.d/S50drvconfig and S60devlinks scripts.


Discovery of new devices can be performed without a system reboot
using the following commands:
In the Solaris 8 OE:
devfsadm
vxdctl enable
In Solaris 7 OE and earlier:
drvconfig
disks
devlinks
vxdctl enable
The S85vxvm-startup2 Script

The processes started by the S85vxvm-startup2 script are illustrated in
Figure 5-5.
The vxconfigd is switched from Boot
mode to Enable mode.
The dev_info
tree is checked for new devices.
All auto-import disk groups are
imported.
All volumes are started.
Failed disk media records are attached

to Disk Access records.
Figure 5-5
The /etc/rcS.d/S85vxvm-startup2 Script

5-15

The /etc/rcS.d/S85vxvm-startup2 script must be run after the /, /usr,
and /var volumes are mounted. This script:
Starts some I/O daemons
Rebuilds the /dev/vx/dsk and /dev/vx/rdsk directories
Imports all disk groups
Starts all volumes that were not started earlier in the boot sequence
Other activities which affect the /etc/rcS.d/S85vxvm-startup2 script

are:
Flag files may affect this process.
Volume recovery is not performed.
New device entries are added, and invalid entries are kept.
The S86vxvm-reconfig Script

The processes started by the S86vxvm-reconfig script are shown in
Figure 5-6.
Flag files are queried to determine

action.
/etc/rcS.d/S86vxvm-reconfig
New disks are added.
Encapsulations are performed.
Figure 5-6
5-16
The /etc/rcS.d/S86vxvm-reconfig Script

Flag Files
Flag files are queried for reconfiguration and reboot procedures. The
/etc/vx/reconfig.d/state.d file contains flag files set by prior
operations, as follows:
Pre-recovery events that might have occurred
Encapsulation procedures Encapsulation requires a reboot and

creates flag files to delineate actions that need to be taken. If
encapsulation fails or is incomplete, the flag files must be removed
manually.
Note The root_done flag tells the VxVM software that the boot disk is
under VxVM software control and that this startup script can exit without
any action.
Multi-User Startup Files

This section describes the system startup files executed during multi-user
boot processing.
The S20syssetup Script

The S20syssetup script is not a VxVM software script. This script
discovers and processes any crash dumps. When the script processes
finish, it moves the dumps to /var/crash.
The S94vxnm-host_infod Daemon

The S94vxnm-host_infod daemon starts the host_infod daemon
This script requires a VVR license. The license is located in the
/etc/vx/elm file.
Note The host_infod daemon is new. It is used by VVR to support
remote operations between nodes without the use of rshell protocols.

5-17
The S94vxnm-vxnetd Daemon

The S94vxnm-vxnetd daemon starts vxnetd, but only if the
/etc/vx/reconfig.d/state.d/install-db flag is not present.
This daemon is required for replication on the secondary replication
volume group (RVG). Also, a valid VVR license must be present to enable
the S94vxnm-vxnetd daemon
The S95vxvm-recover Daemon

The S95vxvm-recover daemon performs the following:
Volume recovery and re-synchronization is started on all volumes.
Relocation daemons are started on all volumes.

If a disk failure is detected, subdisks are relocated to another disk.
During relocation operations, disks marked as spare are used to
relocate data.
The S96-vmsa-server Daemon

The S96-vmsa-server daemon starts the Volume Manager Storage
Administrator (VMSA) software.
Other Boot Process Failures

If the system does not boot and all boot-related VxVM software recovery
options fail, it may be necessary to reinstall the VxVM software. This
procedure can be performed without the loss of system or application
data.
To recover the VxVM software by reinstallation
5-18
1.
Install the VxVM software.
2.
Recover the VxVM software configuration.


3.
4.
Clean up the system configuration, following these steps:

a.
Clean up rootability.
b.
Clean up volumes.
c.
Clean up disk configuration.
d.
Reconfigure rootability.
e.
Perform the final volume reconfiguration.
Start the hot-relocation process.
Note A detailed and extensive reinstallation recovery procedure is

found in the VERITAS Volume Manager 3.2 Troubleshooting Guide under
the heading Recovery from Boot Disk Failure.

5-19
Troubleshooting Boot Process Failures

During system boot processing, the VxVM software daemons and devices
must be started or the VxVM software is unavailable to the system.
Successful startup of the VxVM software requires the following:
Bootable boot disk
Valid /etc/system file
Valid /etc/vfstab file
Valid rootdg disk group
Startable volumes, including no stale plexes
Non-corrupted driver and daemon binaries
Appropriate VxVM software binaries loaded on the system for the

specific release of the Solaris OE installed
Access to library files needed by the VxVM software for initialization

of devices and system volumes
Valid /etc/vx/volboot file
Existing /var/vxvm/tempdb directory and supporting tempdb files:

This includes the temporary storage area for the VxVM software
configuration copies. The vxconfigd process needs these files to
transition to the enabled state.
Execution of the appropriate VxVM software startup scripts
Bootable Boot Disk

A bootable boot disk must be selected from the OpenBoot PROM as the
system boot device. This boot disk should have an appropriate device
alias configured by the system administrator or the full path inserted in
the boot-device NVRAM environment variable by the Solaris OE
installation program. If the boot disk is under VxVM software control and
under the VxVM software management, the following device aliases are
built by the VxVM software:
vx-rootmirror /sbus@3,0/SUNW,socal@0,0/sf@1,0/ssd@w220000203713f579,0:a
vx-rootdisk /sbus@3,0/SUNW,socal@0,0/sf@1,0/ssd@w220000203713f643,0:a
5-20

Note The boot disk device address depends on the system and on which
storage device is configured as the boot disk.
If the boot disk is under VxVM software control, check that the
boot-device is set to the proper VxVM software-generated device alias,
as follows:
boot-file: data not available
boot-device=vx-rootdisk
local-mac-address?=false
Power user If the primary boot device fails and the system must be
rebooted, use the vx-rootmirror device alias to boot the system.
If the primary boot disk fails and a spare was configured in rootdg, after
the spare disk replaces the failed boot disk and the failed volumes are
being hot-relocated, the VxVM software builds an additional device alias
to enable booting from the spare disk, as follows:
vx-rdgspare01
/sbus@3,0/SUNW,socal@0,0/sf@1,0/ssd@w220000203713f96d,0:a
Use this device alias if the system must be booted from the spare disk.
Note The spare disk device address depends on the system and on
which storage device is configured as the spare disk. Also, the spare disk
name is arbitrary and is reflected in the device alias name.
Valid /etc/system File

The /etc/system file must exist and have the following line inserted at
the end of the file:
* vxvm_START (do not remove)
forceload: drv/vxdmp
forceload: drv/vxio
forceload: drv/vxspec
forceload: drv/ssd
forceload: drv/sf
forceload: drv/SUNW,socal
forceload: drv/sbus
forceload: drv/sd
* vxvm_END (do not remove)

5-21

The last two lines of the /etc/system file are only present if the boot disk
is under VxVM software control. If these lines are not present and the
boot disk is under VxVM software control, the system will not boot.
Also, the forceload statements for drv/sd and drv/ssd must be present
in the /etc/system file if the boot device is one of these class of disks. If
these forceload statements are not present or corrupted, the system
rebootsrecursively.
Power user Recovery of a corrupted /etc/system file requires booting
with the -a option and selecting a backup /etc/system file that has the
previously mentioned entries. If a backup /etc/system file is unavailable
or does not contain these statements, perform the following:
1.
Boot from CD-ROM. Type the following:

cdrom -s
2.
Mount the boot disk to /a.
3.
Set the terminal type to vt100.
4.
Use a text editor such as vi to edit /etc/system and fix the

problem, or copy the default from CD-ROM and modify it to match
the system configuration.
5.
Save the new /etc/system file.
6.
Unmount /a.
7.
Reboot the system.
Note This process is presented in greater detail in the VERITAS Volume

Manager Troubleshooting Guide, in the section Recovery from Boot Disk
Failure.
Valid /etc/vfstab File

The /etc/vfstab file must have valid entries for all system file systems
(/, swap, /usr, and /var), or the system drops into the single-user mode
and does not proceed further. If the boot disk is encapsulated, the
following entries must be present:
/dev/vx/dsk/swapvol
/dev/vx/dsk/rootvol
5-22
swap
no
ufs
no logging


/dev/vx/dsk/usr /dev/vx/rdsk/usr
/dev/vx/dsk/var /dev/vx/rdsk/var
/usr
/var
ufs
ufs
1
1
no
no
logging
logging
Recovery of this file requires editing the file to correct problems and, in
severe cases, can require booting from the CD-ROM.
Note Additional information on how to recover from problems in the
/etc/vfstab file are found in the VERITAS Volume Manager
Troubleshooting Guide in the section Recovery from Boot Disk Failure.
Valid rootdg Disk Group

The VxVM software requires that a valid rootdg exist, or it does not
initialize. If the boot disk is not encapsulated and rootdg does not exist,
the recovery process creates a new rootdg and reboots or restarts the
VxVM software. If vxinstall is run twice and two rootdg disk groups
exist, then remove the new rootdg and import the original rootdg. When
the original rootdg is the only one on the system, reboot or restart the
VxVM software manually.
If the boot disk is encapsulated, the procedure is more complex. Consult
the SunSolve SRDB 24657, which addresses the problem, to properly
correct failures with importing rootdg.
Startable Volumes
If the boot disk is under the VxVM software control, all system volumes
(/, swap, /usr, and /var) must start. One reason a volume might not start
is the presence of a stale or unusable plex.
A stale plex is defined as a plex that has data which is inconsistent with
other mirrors of that volume. During the boot process, only the plexes on
the boot disk are accessed until the VxVM software is fully initialized and
a complete configuration for the volumes on the boot disk can be
obtained. If the data on the boot disk is stale, then the system must be
rebooted from an alternate boot disk which does not contain stale plexes.

5-23

The vxconfigd command displays a bootable disk that is suitable for
booting and which does not contain stale or unusable plexes. If stale
plexes are present, vxconfigd displays the following message:
vxvm:vxconfigd: Warning Ples rootvol-01 for root volume is stale or unusable.
vxvm:vxconfigd: Error: System boot disk does not have a valid root plex
Please boot from one of the following disks:
Disk rootmirror
Device: c1t21d0s2
vxvm:vxconfigd:
Error: System startup failed
The system is down
Note Additional information on how to recover from stale or unusable

plexes is found in the VERITAS Volume Manager Troubleshooting Guide
under the heading Recovery from Boot Disk Failure.
If the boot-device nvram parameter is set, and aliases are set for both the
rootdisk and rootmirror, the system reboots use the rootmirror disk.
Non-Corrupted and Appropriate Binaries and Libraries

The VxVM software initialization requires that the appropriate daemon
and driver binaries and system libraries are available during boot
processing.
Binary Files
Each version of the Solaris OE requires different VxVM software daemon
and driver binaries. These binaries are found in /kernel/drv and
/kernel/drv/sparcv9 (for the 64-bit Solaris OE). The binaries are:
32-Bit Solaris OE:
bash-2.03# ls -las /kernel/drv/vx*

640 -rw-r--r-1 root
sys
608 -rw-r--r-1 root
sys
608 -rw-r--r-1 root
sys
640 -rw-r--r-1 root
sys
4 -rw-r--r-1 root
sys
3776 -rw-r--r-1 root
sys
sys
sys
sys
2 -rw-r--r-1 root
sys
30 -rw-r--r-1 root
other
28 -rw-r--r-1 root
sys
30 -rw-r--r-1 root
sys
5-24
314156 Nov 21
297920 Aug 15
300980 Aug 15
314156 Nov 21
1026 Aug 15
1920316 Nov 21
1860500 Aug 15
1878432 Aug 15
1920316 Nov 21
991 Aug 15
14968 May 5
14060 Aug 15
14388 Aug 15
23:03 ./vxdmp
2001 ./vxdmp.SunOS_5.6
23:03 ./vxdmp.SunOS_5.8
2001 ./vxdmp.conf
23:03 ./vxio
2001 ./vxio.SunOS_5.6
23:03 ./vxio.SunOS_5.8
2001 ./vxio.conf
08:32 ./vxspec
2001 ./vxspec.SunOS_5.6


30 -rw-r--r-4 -rw-r--r--
1 root
1 root
sys
sys
14968 Aug 15 2001 ./vxspec.SunOS_5.8

1315 Aug 15 2001 ./vxspec.conf
64-Bit Solaris OE:
bash-2.03# ls -las /kernel/drv/sparcv9/vx*

800 -rw-r--r-1 root
sys
393968 Nov 21
768 -rw-r--r-1 root
sys
380840 Aug 15
800 -rw-r--r-1 root
sys
393968 Nov 21
sys
2714688 Nov 21
sys
2666256 Aug 15
sys
2714688 Nov 21
38 -rw-r--r-1 root
other
18504 May 5
36 -rw-r--r-1 root
sys
17928 Aug 15
38 -rw-r--r-1 root
sys
18504 Aug 15
23:03 ./vxdmp
23:03 ./vxdmp.SunOS_5.8
23:03 ./vxio
23:03 ./vxio.SunOS_5.8
08:32 ./vxspec
Power user If a system message lists an incorrect version of the VxVM

software installed for the Solaris OE, the most probable cause is that the
incorrect version of the Solaris OE was selected during the VRTSvxvm
package addition. The solution to this is to copy the correct binary file
(vxio.xxxxx, vxdmp.xxxx, or vxspec.xxx) to either the vxio, vxspec or
xvdmp files. Additionally, if these files become corrupt, copy over these
files using the correct binary file for the Solaris OE installed.
If the boot disk is under the VxVM software control and a driver file is
corrupt, the system boot fails. The system administrator must perform a
basic or functional unencapsulation (refer to Performing a Basic or
Functional Unencapsulation on page 2-68 for details).
Library Files
The following library files must be in the /etc/vx/slib directory:
bash-2.03# ls -las /etc/vx/slib
total 5346
2 drwxr-xr-x
2 root
other
2 drwxr-xr-x
9 root
other
186 -rwxr-xr-x
1 root
other
10 -rwxr-xr-x
1 root
other
2256 -rwxr-xr-x
1 root
other
10 -rwxr-xr-x
1 root
other
46 -rwxr-xr-x
1 root
other
26 -rwxr-xr-x
1 root
other
26 -rwxr-xr-x
1 root
other
172 -rwxr-xr-x
1 root
other
256 -rwxr-xr-x
1 root
other
12 -rwxr-xr-x
1 root
other
50 -rwxr-xr-x
1 root
other
1776 -rwxr-xr-x
1 root
other
48 -rwxr-xr-x
1 root
other
140 -rwxr-xr-x
1 root
other
512
512
95052
4392
1146284
4848
23348
12432
12432
87108
119244
5456
24968
898600
24360
70864
May 11 13:18 .
May 20 07:20 ..
May 5 08:33 liba5k.so.2
May 5 08:33 liba5k_stub.so.2
May 5 08:32 libc.so.1
May 5 08:34 libc_psr.so.1
May 5 08:33 libdevice.so.1
May 5 09:56 libdevid.so
May 5 09:56 libdevid.so.1
May 5 08:33 libdevinfo.so.1
May 5 08:33 libg_fc.so.2
May 5 08:33 libg_fc_stub.so.2
May 5 08:33 libmp.so
May 5 08:33 libnsl.so.1
May 5 08:33 libnvpair.so.1
May 5 08:32 libsocket.so.1

5-25

288 -rwxr-xr-x
40 -rwxrwxr-x
1 root
1 root
other
sys
139224 May 5 09:55 libthread.so.1

20096 Nov 21 17:18 libvxdiscovery.so
If any of these library files are not accessible and the boot disk is
encapsulated, a message similar to the following is displayed:
Starting VxVM restore daemon...
VxVM starting in boot mode...
ld.so.1: vxconfigd: fatal: <missing library file name is displayed>: open failed: No
such file or directory
Killed
Errors were encountered in starting the root disk group, as a result
VxVM is unable to configure the root and/or /usr volumes. If you have
mirrored the root disk, you can try booting from that disk. Please
refer to Appendix C of the Installation Guide for more details.
If you cannot boot from the root disk, you can try to repair the problem
using a network-mounted root file system or some other alternate root
file system. Again, see the Installation Guide for more details.
Would you like a shell prompt right now? [no]
If this problem exists, perform a basic or functional unencapsulation and

copy the missing libraries from /usr/lib to /etc/vx/slib. Refer to
Performing a Basic or Functional Unencapsulation on page 2-68 for
details.
5-26

Valid /etc/vx/volboot File

The /etc/vx/volboot file is the VxVM software bootstrap file. This file
must be valid or the VxVM software does not initialize. The contents of
this file includes the following:
The /etc/vx/volboot file is an ASCII file that adheres to a strict

format and should not be edited. It has the following characteristics:
512 bytes in length, including padding
Updated using the vxdctl command
The /etc/vx/volboot file holds the VxVM software host identifier

hostid. This is usually the Solaris OE nodename, not the hardware
hostid. The VxVM software hostid does not have to match the
servers nodename, which can be very confusing.
This file has the following characteristics
It establishes disk and diskgroup ownership.
If two or more servers access the same disks using the same
bus, the VxVM software hostid ensures that the two hosts do
not interfere with each other when they are accessing VxVM
software disks.
If this file is corrupted or deleted, recover the file from backups of the
affected system. If backups are not available, a reinstall may be necessary.
VxVM Software Startup Scripts

There are five startup scripts located in /etc/rcX.d which are executed
during system boot processing:
/etc/rcS.d/S25vxvm-sysboot Checks system and vfstab files,

including install-db, to determine whether the VxVM software
should start. (For VxVM software version 2.3, this script also checks
for the /VXVM2.3 UPGRADE/.start_runed file.) If the software
should start, the script prints the message VxVM starting in boot
mode... and executes a vxconfigd -m boot command.
/etc/rcS.d/S35vxvm-startup1 If needed, special volumes such as

/ and swap are started.

5-27
/etc/rcS.d/S85vxvm-startup2 Prints the message VxVM

general startup... , and starts the vxiod daemons. This script
also enables vxconfigd or starts it if it is not already started. This
script is where the /dev/vx directory is made or updated, if needed,
and a vxrecover command starts all volumes.
/etc/rcS.d/S86vxvm-reconfig Checks for the existence of the

/etc/vx/reconfig.d/state.d/reconfig file. If there are
reconfigurations needed (for example, for encapsulated disks), the
work is done here. The system can be rebooted from this script if
needed.
/etc/rc2.d/S95vxvm-recover Runs vxrecover, which fixes any

remaining problems or starts any unstarted volumes. Also starts
vxrelocd, which starts another vxrelocd and a vxnotify deamon.
/etc/rc2.d/S96vmsa-server [VxVM software version 3.x only]

Starts the vmsa GUI server, which is needed for the vmsa GUI to
work.
Power user The recovery from errors resulting from the execution of
these scripts requires reading each individual script and determining
where the failure is within the script. This requires shell programming
expertise and a detailed understanding of the VxVM software commands
and support files. All of the VxVM software startup scripts are hard
linked from the /etc/init.d file.
5-28

Troubleshooting VxVM Software Failures
Troubleshooting VxVM Software Failures

Failures of the VxVM software other than boot-process failures include,
but are not limited to, the following:
Corrupt or missing commands and utilities The VxVM software

commands and utilities are found in the following directories:
/etc/vx/bin
/usr/sbin
/sbin
Corrupt or missing configuration files Most VxVM software

configuration files are in the /etc/vx directory. Driver configuration
files are in the /kernel/drv directory.
To correct these problems, restore a valid backup of the missing or

corrupted information, or copy non-system-specific files from another
system on the network.
Caution Be careful when copying files from other systems on the
network. If a system-specific file, such as /etc/vx/volboot, is copied
from another server, the system to which it is copied can be placed into a
non-bootable state. Use backups from the system missing the needed file
whenever possible.

5-29
Exercise: Determining the VxVM Software Problem

In this exercise, you will troubleshoot the VxVM software functional and
initialization problems injected into your lab system, using a supplied
break-and-fix script.
Preparation
The VxVM software must be installed and operational.
There must be one additional disk group other than rootdg with at
least one configured, started and mounted volume.
The instructor must give you the location and name of the breakand-fix script.
The break-and-fix script is a menu-driven script used to break a system

based on bugs submitted by Sun Support. This script performs the
following tasks:
Provides a list of bugs for use by students to inject real world bugs
into the lab systems.
Keeps track of bugs completed and those skipped or not completed

by lab teams.
Provides pointers to SunSolve SRDB records and INFODOCs.
Provides hints to help lab teams troubleshoot bugs.
Fixes the injected bug if the lab team desires to proceed to another
bug without resolving the current bug.
To run the break-and-fix script:

1.
Enter the following:

# breakfix
The following is displayed:
breakfix Main Menu
1.
2.
3.
4.
5-30
Problem
Problem
Problem
Problem
1
2
3
4
11.
12.
13.
14.
Problem
Problem
Problem
Problem
11
12
13
14


5.
6.
7.
8.
9.
10.
Problem
Problem
Problem
Problem
Problem
Problem
5
6
7
8
9
10
15.
16.
17.
18.
19.
20.
Problem
Problem
Problem
Problem
Problem
Problem
15
16
17
18
19
20
x. Exit
Enter option [x]:
2.
Select a bug by entering the bug number from the menu.

For example, if you select Bug 1, the following is displayed:
Problem #1, Invalid disk.exclude file

b)
s)
h)
m)
x)
Break it
Solution
Hint
Return to Main Menu
Exit
Enter option [x]:
3.
Enter b to break, f to fix and m to return to the main menu.

If you select the b or break option, a system reboot is necessary to
activate the bug. After the symptom is observed by your lab team,
access SunSolve Online, and search for the proper SRDB to help
diagnose the problem.
If you need or want a hint, re-execute the break-and-fix script and
select the h option for the bug currently being worked on.
This is SRDB ID: 14736.

The /etc/vx/disks.exclude file requires the following format: c#t#d#
for each disk you wish to exclude in /etc/vx/disks.exclude.
Press enter to return to the menu.
Notice that the referring SRDB is listed along with hints to help
resolve the problem.

5-31
Tasks
Execute the break-and-fix script, and select bugs 1 through 10. List
problem resolution steps for each bug in the space listed below.
Problem 1:
__________________________________________________________________
__________________________________________________________________
__________________________________________________________________
__________________________________________________________________
__________________________________________________________________
Problem 2:
__________________________________________________________________
__________________________________________________________________
__________________________________________________________________
__________________________________________________________________
__________________________________________________________________
Problem 3:
__________________________________________________________________
__________________________________________________________________
__________________________________________________________________
__________________________________________________________________
__________________________________________________________________
5-32


Problem 4:
__________________________________________________________________
__________________________________________________________________
__________________________________________________________________
__________________________________________________________________
__________________________________________________________________
Problem 5:
__________________________________________________________________
__________________________________________________________________
__________________________________________________________________
__________________________________________________________________
__________________________________________________________________
Problem 6:
__________________________________________________________________
__________________________________________________________________
__________________________________________________________________
__________________________________________________________________
__________________________________________________________________
Problem 7:
__________________________________________________________________
__________________________________________________________________
__________________________________________________________________

5-33

__________________________________________________________________
__________________________________________________________________
Problem 8:
__________________________________________________________________
__________________________________________________________________
__________________________________________________________________
__________________________________________________________________
__________________________________________________________________
Problem 9:
__________________________________________________________________
__________________________________________________________________
__________________________________________________________________
__________________________________________________________________
__________________________________________________________________
Problem 10:
__________________________________________________________________
__________________________________________________________________
__________________________________________________________________
__________________________________________________________________
__________________________________________________________________
The lab is complete when all 10 bugs are fixed, your lab system with the
VxVM software is operational, and all volumes started and accessible.
5-34

Exercise Summary
Exercise Summary
!
?

Experiences
Interpretations
Conclusions
Applications

5-35

Following is information about the solutions to this lab exercise.
Task Solutions
The solutions for the tasks in the lab exercise are found in the break-andfix script and on SunSolve in the appropriate SRDB and INFODOC files.
5-36

Module 6
Recovering Disk, Disk Group, and

Volume Failures
Objectives
Describe the VxVM software utilities, commands, and virtual devices

to help with recovery processes
Identify and repair failed disks
Perform disk recovery processes and procedures
Identify and repair volume errors
Identify and repair disk group failures

6-1
Relevance
Relevance
!
?
6-2
Discussion The following questions are relevant to understanding the

VxVM software disk recovery processes:
What tools, utilities, commands, or files are available to help identify

failed VxVM software disks and other objects used to access
managed storage devices?
What are the indications that a disk failed?
How are damaged disk groups repaired?
How are new disks made visible to the VxVM software?
What are the procedures to replace a failed VM disk?
How are duplicate record entries recognized and repaired?






SunSolveSM Online SRDBs and INFODOCs 13364, 14820, 21626 and

26367,
Man pages for the following commands:
vxmend
vxvol
vxplex
vxreattach
vxstat
vxrecover

Recovering Disk, Disk Group, and Volume Failures
6-3
Introducing Recovery Processes
Introducing Recovery Processes

The VxVM software provides utilities, commands, and virtual devices to
improve the fault resiliency of storage subsystems brought under VxVM
software management. When a disk experiences a failure, the VxVM
software runs processes that ensure continued availability of the data
hosted on the failed disk. This is only true if the data on that disk was
made fault resilient using RAID 1 or RAID 5 data resiliency protocols.
Recovery of failed disks requires the following:
The disk must be a VxVM software disk.
The disk group of which the disk is a member must be imported.
The disk must have subdisks that belong to an active volume that is
in use by the system.
Depending on the architecture of the volume, the loss of a single disk

should not effect the availability of the volume.
Disk groups configuration problems can prevent the import or creation of
the disk group. The result of this problem is that the resources managed
by the affected disk group are not available to the system. Disk group
import problems are generally due to ownership issues, such as the
appearance that a disk group is imported when it is not.
This module describes how to identify and recover from disk, disk group,
and volume failures using the VxVM software.
6-4

Identifying Disk Errors

When disk failures occur, the system and VxVM software display error
information which describes the failure. If properly configured, error
information is logged in the following locations:
/etc/vx/vxconfigd.log
/var/adm/messages
root mail
Use the following commands and utilities to display the operational state
of managed disks:
vxprint
vxdisk
vxdiskadm
vxstat
The vmsa GUI administrative console
Note The vmsa GUI administrative console is not discussed in this

module because it is unavailable when you are diagnosing problems
remotely. Configuration of vxconfigd logging, interpreting error
messages and using root mail is discussed in Module 4, Troubleshooting
Tools and Utilities.
This section discusses these commands and their use in recovering from
full or partial disk failures.
The vxprint Command

The vxprint command displays the status of the VxVM software objects.
It is a useful tool to determine the operational status of those objects.
This example from a vxprint command was collected after the
rootmirror disk failed.
DG NAME
NCONFIG
NLOG
DM NAME
DEVICE
TYPE
UBLEN
STATE
RV NAME
RLINK_CNT
KSTATE
MINORS
PRIVLE
GROUP-ID
STATE
PRIMARY
DATAVOLS
SRL

6-5

RL
V
PL
SD
SV
DC
SP
NAME
NAME
NAME
NAME
NAME
NAME
NAME
RVG
RVG
VOLUME
PLEX
PLEX
PARENTVOL
SNAPVOL
KSTATE
KSTATE
KSTATE
DISK
VOLNAME
LOGVOL
DCO
STATE
STATE
STATE
DISKOFFS
NVOLLAYR
REM_HOST
LENGTH
LENGTH
LENGTH
LENGTH
REM_DG
READPOL
LAYOUT
[COL/]OFF
[COL/]OFF
REM_RLNK
PREFPLEX
NCOL/WID
DEVICE
AM/NM
dg rootdg
default
default
1022020746.1025.lowtide
dm rootdisk
dm rootmirror
c1t0d0s2
-
sliced
-
3590
-
17678493 -
UTYPE
MODE
MODE
MODE
NODEVICE
v
pl
sd
pl
sd
rootvol
ENABLED ACTIVE
15581349 ROUND
root
rootvol-01
rootvol
ENABLED ACTIVE
15581349 CONCAT
RW
rootdisk 7181
15581349 0
c1t0d0
ENA
rootvol-02
rootvol
DISABLED NODEVICE 15581349 CONCAT
- RW
rootmirror-01 rootvol-02 rootmirror 0
15581349 0
- RLOC
v
pl
sd
pl
sd
swapvol
ENABLED ACTIVE
1052163 ROUND
swap
swapvol-01
swapvol
ENABLED ACTIVE
1052163 CONCAT
RW
rootdisk 15588530 1052163 0
c1t0d0
ENA
swapvol-02
swapvol
DISABLED NODEVICE 1052163 CONCAT
- RW
rootmirror-02 swapvol-02 rootmirror 15581349 1052163 0
- NDEV
Note in this example that the failed plex for volume rootvol is
rootvol-0. This volume has a failed subdisk named rootmirror-01. The
volume is unaffected because it is enabled and active.
The vxdisk Command

Use the vxdisk command to display the status of the VxVM software
disks. An example of this command shows the rootmirror from The
vxprint Command on page 6-5 in a failed state.
DEVICE
TYPE
c1t0d0s2
sliced
c1t3d0s2
sliced
c1t4d0s2
sliced
c1t5d0s2
sliced
c1t6d0s2
sliced
c1t16d0s2
sliced
c1t18d0s2
sliced
c1t19d0s2
sliced
c1t22d0s2
sliced
-
6-6
DISK
rootdisk
rootmirror
GROUP
rootdg
rootdg
STATUS
online
error
online
online
error
error
error
error
online
failed was:c1t22d0s2


The STATUS column displays the status of disks visible to the VxVM
software. A status of error does not always indicate a disk error; it
usually indicates that the disk is not under the VxVM software control.
The vxdiskadm list Option

The list option from the vxdiskadm utility is another source of disk
operational status information. The following example shows the list
option output for the rootmirror disk.
Menu: VolumeManager/Disk/ListDisk
Use this menu operation to display a list of disks. You can
also choose to list detailed information about the disk at
a specific disk device address.
Enter disk device or "all" [<address>,all,q,?] (default: all) c1t22d0
Device:
c1t22d0s2
devicetag: c1t22d0
type:
sliced
flags:
online error private autoconfig
errno:
Device path not valid
numpaths:
2
c1t22d0s2
state=disabled
c2t22d0s2
state=disabled
The vxstat Command

Use the vxstat command to determine the underlying subdisk of a failed
plex. This is useful when a partial disk failure occurs that only affects a
sub-set of the subdisks on a failed or failing disk. An instance of the
command when used for this purpose is shown in the following example.
The example assumes that there are two failing plexes, opt-01 and var01. The -s option displays information about individual subdisks, and the
-ff option displays failed read and write operations.
bash-2.03# vxstat -s ff opt-01 var-01
FAILED
TYP
NAME
READS
WRITES
sd rootmirror-05
0
0
sd rootmirror-06
0
0
sd rootdisk-04
1
0
sd rootdisk-05
1
0

6-7

This clearly shows that the rootdisk disk experienced a read failure
when accessing subdisks rootdisk-04 and -05.
Disk Failure Categories and root Mail

If hot-relocation is enabled, mail is sent to root to notify administrators of
either a disk or plex failure. There are two types of disk errors:
Full disk failure
Partial disk failure
Full Disk Failure

Full disk failures are caused by a total loss of a disk, often due to a cabling
problem or a catastrophic failure of the disks physical or logical
components. Mail is sent to root describing the failed disk and the
affected plexes.
This example of a root mail message is typical of a complete disk failure.
To: root
Subject: Volume Manager failure on lowtide
Failures have been detected by the VERITAS Volume Manager:
failed disks:
rootdisk
failed plexes:
opt-01
var-01
failing disks:
rootdisk
This root mail message shows that disk03 failed. Plexes home-02,
fin-02, and hr-01 also failed; if a spare disk with sufficient space is
available for hot-relocation, these disk are relocated.
Note For a problem of this type, check cabling or disk seating, or replace
the disk.
6-8

Partial Disk Failure

Partial disk failures are those errors that cause a region of a disk to fail,
affecting some but not all of the subdisks stored on the failed disk. Errors
of this type are usually caused by media errors, but do not ignore the
possibility of a cabling problem. Mail is sent to root describing the failed
plexes but not the failed disks or subdisks.
This example root mail message is typical of a partial disk failure.
To: root
Subject: Volume Manager failures on host lowtide
Failures have been detected by the VERITAS Volume Manager
failed plexes:
opt-01
var-01
Use the vxstat command to determine the failed disk. This procedure is
described in the The vxstat Command on page 6-7.

6-9
Replacing Disks
Replacing Disks
If a disk fails, it must be replaced. Use one of the following two methods
to replace a failed disk:
The vxdiskadm utility
The command line
Replacing Disks Using the vxdiskadm Utility

To replace a disk in the VxVM software, use the menu-driven vxdiskadm
utility option 5, Replace a failed or removed disk. This procedure
is described in SunSolveSM INFODOCs 14820 and 21626. Refer to these
documents if replacing a disk due to failure.
The vxdiskadm utility uses the vxreattach command to perform the
reattachment task. Prior to running the vxdiskadm utility, issue a
vxreattach -c command to show if reattachment is possible and to
determine whether to attempt the task. For more information on the
vxreattach command, consult the man page.
Replacing Disks Using the Command Line

Discrete command line utilities, such as vxdg and vxrecover, are also
available for use in replacing disks. Use the following commands to
replace a failed disk at the command line:
# vxdg -g <disk_group> rmdisk <failed_disk_VxVM_name>
# vxdg -g <disk_group> adddisk <failed_disk_VxVM_name>=cxtxdxs2
# vxrecover -bs.
Caution The vxrecover -bs command attempts to perform recovery

operations on all volumes with failed plexes, including starting any nonstarting volumes, prior to performing the plex recovery operations. This
might not be the correct operation to perform, depending on the current
volume configuration of the system being recovered. See the vxrecover
man page for more information.
6-10

Replacing Disks
Recovering a Disk Which VxVM Software Cannot See

When you are replacing a disk due to a failure, the disk must be visible to
the VxVM software or the replacement procedure fails. Figure 6-1 is a
flow diagram for troubleshooting disk visibility problems.
Start
See disk in
vxdisk list?
Yes
The VxVM Software

can see the disk.
No
The VxVM Software cannot

see the disk. Call support.
No
First time
through flow?
Yes
Can the Solaris OE

see the disk?
(format, prtvtoc)
No
Run devfsadm or
drvconfig; disks
Yes
Run vxdctl initdmp
and vxdctl enable
Yes
Can the Solaris OE

see the disk?
(format, prtvtoc)
No
Hardware or
Solaris OE Problem
Figure 6-1
Recovery Process When VxVM Software Cannot See a Disk

6-11
Replacing Disks
Perform the following steps to troubleshoot a disk visibility problem:
1.
Does the VxVM software see the disk? If not, check the Solaris OE. If
the Solaris OE cannot see a disk device, then the VxVM software
cannot see it. Use the format or prtvtoc command to verify that the
Solaris OE is able to detect the disk.
2.
If the Solaris OE cannot see the disk, run the devfsadm or

drvconfig;disks command to make the Solaris OE re-scan the
devices. If the Solaris OE can see the disk, then the visibility problem
is with the VxVM software.
3.
If the Solaris OE still cannot see the disk, check again to see if the
Solaris OE can see the disk.
If the Solaris OE can see the disk, proceed to the next step in the flow
diagram and execute the necessary commands that allow the VxVM
software to see the disk.
If the Solaris OE still cannot see the disk, there is a hardware,
software or driver problem that must be corrected before this disk
can be used by the VxVM software.
4.
It might be necessary to recreate the DMP user-level nodes for all

nodes in the system. In addition, it might be necessary to build (or
rebuild) the VxVM software user-space devices. To force the VxVM
software to re-scan the Solaris OE device tree and build the necessary
VxVM device nodes, run the vxdctl initdmp or the
vxdctl enable command.
When these procedures are complete the disk device should be available
for use, unless there is a hardware or functional VxVM software problem.
6-12

Troubleshooting Volume Errors

Volumes are the high-order VxVM software objects that are mounted by
the Solaris OE. If a volume has a configuration error or other condition
that prevents it from starting, it is in an unstartable state. If the volume
has an error that was caused by a disk failure and the data managed by
the volume is corrupted, the volume is in a disabled state. Volumes that
are in an unstartable or disabled state cannot be mounted and used by the
system until the error is resolved.
This section describes how to identify and work with volume-related
errors.
Listing Unstartable Volumes

To display unstartable volumes, use the following syntax:
# vxinfo -g disk_group_name volume
This example shows the results of the vxinfo -g command.
bash-2.03# vxinfo -g rootdg
usr
fsgen
Started
var
fsgen
Started
rootvol
root
Started
swapvol
swap
Started
bash-2.03# vxinfo -g rootdg rootvol
rootvol
root
Started
An unstartable volume is listed as Unstartable in the third column of

the command output.
After the problem with the volume is corrected, use the vxvol command
to restart the volume.

6-13
Restarting a Disabled Volume

Disabled volumes are caused by disk failures that breech the fault
resiliency of a mirrored or RAID 5 volume. Restore the backup data for a
disabled volume after the failed disk or disks are repaired. Prior to
restoring the volumes data from tape, restart the volume. Use the
following version of the vxvol command to restart the volume:
# vxvol -o bg -f start volume_name
The -f option forcibly restarts the volume, while the -o bg option
resynchronizes the volumes plexes in the background.
Recovering a Mirrored Volume

Some errors, such as a system crash or I/O error, can leave a mirrored
volume with multiple plexes in neither a clean or active state. This requires
administrator intervention to tell the VxVM software which plex has the
most current copy of the data, and then to use that plex to resynchronize
the remaining plexes of the volume.
Use the following procedure to recover a mirrored volume:
1.
Place the desired plex (using an administrative best guess) in the

clean state.
# vxmend fix clean plex_name
2.
Mark all other plexes as stale if they are not already in the stale state.
# vxmend fix stale plex_name
3.
Recover the other plexes from the plex just placed in the clean state.
# vxvol start volume_name
Note For more information on this procedure, refer to the VERITAS

Volume Manager Troubleshooting Guide for the version of the VxVM
software installed on the system, and to the man pages for the vxmend and
vxvol commands.
6-14

Recovering a RAID 5 Volume

RAID 5 volume failures can be more complex and therefore more difficult
and time consuming to recover than mirrored volumes. This is due to the
parity calculations and a RAID 5 volumes ability to reconstruct data from
a failed subdisk, while leaving the volumes data available to the
Solaris OE.
RAID 5 failures are of two varieties:
System failures The system stopped operating abruptly due to a

power outage or kernel panic. Errors of this type can leave the
RAID 5 volume with stale parity.
Stale parity must be reconstructed by reading all the data on the
volume and rebuilding the parity blocks. This can take considerable
time and impacts system performance. To mitigate errors of this
type, attach a RAID 5 log.
Disk failures Data on some number of disks can become

unavailable due to controller, disk or media failure. Errors of this
type have a subdisk detached and can leave the RAID 5 volume in
degraded mode.
Once the failure that forced the RAID 5 volume into degraded mode
is repaired, the data resident on the failed subdisk is reconstructed
from data on other stripe units in the stripe. This reconstruction of
the missing data, while the data managed by the volume remains
available to the Solaris OE, is called a reconstructing-read. This process
results in degraded performance and, if another subdisk fails during
the reconstruction, the volume experiences total failure and moves
into the disabled state.
Recovery of RAID 5 volumes is categorized into three groups, and is

accomplished as follows:
Parity resynchronization uses the vxvol command. Type the

following:
# vxvol resync RAID_5_volume_name
Log plex recovery uses the vxplex command. Type the following:
# vxplex att RAID_5_volume_name Log_Plex_name

6-15
Stale subdisk recovery uses two versions of the vxvol command.

To recover one stale subdisk, type the following:
# vxvol recover RAID_5_volume_name stale_plex_name
To recover multiple stale subdisks, type the following:
# vxvol recover RAID_5_volume_name
Note RAID 5 recovery is complex. For more information, reference the

VERITAS Volume Manager Troubleshooting Guide for the version of the
VxVM software installed on the system, and to the man pages for vxplex
and vxvol commands. Additional information is found in SunSolve
INFODOCs 13364 and 26367.
Forcibly Starting RAID 5 Volumes

Sometimes it is necessary to forcibly start a RAID 5 volume. This could be
due to maintenance activities which mark a plex as stale even though the
data is good.
Use the vxvol command with the -f option to forcibly start a RAID 5
volume. Use the following syntax:
# vxvol -f start RAID_5_volume _name
Caution Do not forcibly start a RAID 5 volume that is known to have
stale subdisks with out-of-date or corrupted data. This can cause
corruption of volume data and affect the operation of applications or the
Solaris OE.
6-16

Troubleshooting Disk Group Errors
Troubleshooting Disk Group Errors

Disk groups can move into an inconsistent state and become
unimportable. The usually occurs if a disk group move was in process at
the time of a system crash. After the system is restarted, the VxVM
software attempts to reverse or complete the operation. If it is unable to
do so, administrative intervention is required to recover the disk group.
Use the vxdg command to repair a disk group, as follows:
1.
To determine if a disk group is stuck in a move state, use the

vxprint command to print the disk group and look for move status
in the TUTIL0 field for the disk group.
Note The TUTIL filed temporarily locks objects during configuration

changes and recoveries. This field should be cleared after a system reboot.
The TUTIL and PUTIL fields are discussed in Changing Disk Group
Configurations in Appendix E, Configuring the VxVM Software.
2.
Try to use the vxdg command to complete the move. Type the
following:
# vxdg recover disk_group_name
3.
If the previous step does not work, try to reset the move flag as
follows:
# vxdg -o clean recover disk_group_name
4.
If the previous step does not work, try to remove the move flag. Type
the following:
# vxdg -o remove recover disk_group_name
Power user If these commands cannot be run on the system performing

the move and the disk group is still accessible from a second system, run
these commands on the second system. To make sure that any disk group
name conflicts are resolved, use a different name for the disk group being
repaired.

6-17
Exercise: Determining the VxVM Software Disk Problem

In this exercise, you troubleshoot the VxVM software disk problems
injected into your lab system using a supplied break-and-fix script.
Preparation
Make sure that the VxVM software is installed and operational.
There must be one additional disk group other than rootdg with at
least one configured, started, and mounted volume.
You must have access to another systems VxVM software-managed

storage devices, either through a storage area network (SAN) or by
reconfiguring shared storage attached to the two systems.
The instructor must give you the location and name of the breakand-fix script.
The break-and-fix script is a menu driven script that is used to break a

system based on bugs submitted by Sun Support. This script performs the
following tasks:
6-18
Provides a list of bugs for use by students to inject real world bugs
into lab systems.
Keeps track of bugs completed and those skipped or not completed

by lab teams.
Provides pointers to SunSolve SRDB records and INFODOCs.
Provide hints to help lab teams troubleshoot bugs.
Fixes the injected bug if the lab team desires to proceed to another
bug without resolving the current bug.


To run the break-and-fix script:
1.
Enter the following:

# breakfix
breakfix Main Menu
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
Problem
Problem
Problem
Problem
Problem
Problem
Problem
Problem
Problem
Problem
1
2
3
4
5
6
7
8
9
10
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
Problem
Problem
Problem
Problem
Problem
Problem
Problem
Problem
Problem
Problem
11
12
13
14
15
16
17
18
19
20
x. Exit
Enter option [x]:
2.
Select a bug by entering the bug number from the menu.

For example, if you select Bug 1, the following is displayed:
Problem #1, Invalid disk.exclude file

b)
f)
h)
m)
x)
Break it
Solution
Hint
Return to Main Menu
Exit
Enter option [x]:

6-19

3.
Enter b to break, f to fix and m to return to the main menu.

If you select the b or break option, a system reboot is necessary to
activate the bug. After the symptom is observed by your lab team,
access SunSolve Online and search for the proper SRDB to help
diagnose the problem.
If you need or want a hint, re-execute the break-and-fix script and
select the h option for the bug currently being worked on.
This is SRDB ID: 14736.

The /etc/vx/disks.exclude file requires the following format: c#t#d#
for each disk you wish to exclude in /etc/vx/disks.exclude.
Press enter to return to the menu.
Notice that the referring SRDB is listed along with hints to help
resolve the problem.
Tasks
Execute the break-and-fix script, and select bugs 11 through 20. List
problem resolution steps for each bug in the space provided.
Problem 11:
__________________________________________________________________
__________________________________________________________________
__________________________________________________________________
__________________________________________________________________
Problem 12:
__________________________________________________________________
__________________________________________________________________
__________________________________________________________________
__________________________________________________________________
6-20


Problem 13:
__________________________________________________________________
__________________________________________________________________
__________________________________________________________________
__________________________________________________________________
Problem 14:
__________________________________________________________________
__________________________________________________________________
__________________________________________________________________
Problem 15:
__________________________________________________________________
__________________________________________________________________
__________________________________________________________________
__________________________________________________________________
Problem 16:
__________________________________________________________________
__________________________________________________________________
__________________________________________________________________
__________________________________________________________________

6-21

Problem 17:
__________________________________________________________________
__________________________________________________________________
__________________________________________________________________
__________________________________________________________________
Problem 18:
__________________________________________________________________
__________________________________________________________________
__________________________________________________________________
__________________________________________________________________
Problem 19:
__________________________________________________________________
__________________________________________________________________
__________________________________________________________________
__________________________________________________________________
Problem 20:
__________________________________________________________________
__________________________________________________________________
__________________________________________________________________
__________________________________________________________________
The lab is complete when all 10 bugs are fixed and your lab system is
operational, with the VxVM software and all volumes started and
accessible.
6-22

Exercise Summary
Exercise Summary
!
?

Experiences
Interpretations
Conclusions
Applications

6-23

Following is information about the solutions to this lab exercise.
Task Solutions
The solutions for the tasks in the lab exercise are found in the break-andfix script and on SunSolve in the appropriate SRDB and INFODOC files.
6-24

Module 7
Upgrading the VxVM Software

Objectives
Describe the processes used to upgrade the VxVM software from

release 3.1 to 3.2 and from 3.2 to 3.5
Perform an upgrade of the VxVM software to release 3.5
Read, review, and interpret release notes
Identify the top three installation problems
Identify bugs, find patches, and apply patches for the release of the
VxVM software installed
Identify and resolve licensing issues
Upgrade the Solaris OE with VxVM is installed on the system

7-1
Relevance
Relevance
!
?
7-2
Discussion The following questions are relevant to understanding the

VxVM software upgrade processes:
What is the upgrade process for the VxVM software when the boot
disk is encapsulated?
What is the upgrade process for the VxVM software when the boot
disk is not encapsulated?
What is the resolution process for licensing problems when

upgrading the VxVM software?
How is the Solaris OE upgraded when the VxVM software is

installed?
Do release notes provide useful information when upgrading the

VxVM software?






SunSolveSM Online INFODOCS 14189, 17714, and 19492,

VERITAS Software Corporation support knowledge base

TechNote IDs 230184, 240004, and 240006,
[http://seer.support.veritas.com/nav_bar/index.asp?]
[content_sURL=%2Fsearch%5Fforms%2Ftechsearch%2Easp].

7-3
Surveying the Upgrade Processes and Procedures

Performing the VxVM software and Solaris OE upgrades requires
thoughtful planning and exact execution of the upgrade procedures to be
successful. Upgrades to the VxVM software are performed using either
scripted or manual procedures. Additionally, with the release of version
3.5 of the VxVM software, upgrade procedures may be performed using
the Solaris OE pkgadd utility.
The VxVM software upgrade procedures are affected by the presence of
the following:
Encapsulated boot disk
Non-encapsulated boot disk
The VxVM software licensing
Patches
The scripted, manual, and pkgadd upgrade processes make provisions for
the encapsulated state of the boot disk. These procedures, as applied to
the boot disk, must be executed precisely, or the upgrade fails.
Upgrades to the Solaris OE are also affected if the VxVM software is
installed. Some of the issues faced with this upgrade are:
Encapsulated boot disk
Volume configuration information preservation
Manual upgrade as opposed to a JumpStart process or flash

upgrade
Concurrent upgrades of both the Solaris OE and the VxVM software
The VxVM software licensing
Patches
Upgrading With a Script (VxVM 3.1/3.2/3.5)

The VxVM software provides upgrade scripts which can be used to
perform the VxVM software upgrades. These scripts are found in:
7-4
/CD_Path/scripts
/Package_Distribution_Path/scripts


The upgrade scripts include:
upgrade_start Performs first-phase checks and preliminary

operations for the VxVM software upgrade, including the following:
Check for encapsulated boot disk
Check for supported level of alternate pathing (AP) installed
Check for required patches
Check for problems that prevent VxVM software upgrades
Operational activities include the following:
Handles encapsulated boot disk issues
Performs necessary reboots
Saves system configuration files such as the /etc/vfstab and

/etc/system files
Saves pre-upgrade VxVM software configuration files such as

the /kernel/drv/dmp.conf file
Unmounts volumes
Upgrades the VxVM software eeprom entries
upgrade_finish Completes the VxVM software upgrade by

restoring saved files and restarting the VxVM software.
There are extensive sections of code in each script that is dedicated to

managing the upgrade process if AP is installed.
Although this is called a scripted upgrade, there are manual steps needed
to finish the upgrade process. A general description of the manual steps
needed is as follows:
1.
Obtain and install any new license keys needed for the new release
of the VxVM software.
2.
Make sure that any system-level file systems that are under VxVM
software control have at least one plex where they begin on a
cylinder boundary.
3.
If installing any documentation or man page packages, /opt must

exist, be writable and not be symbolically linked.
4.
Boot the system to single-user mode.
5.
Load and mount the VxVM software upgrade CD-ROM.
6.
Execute the upgrade_start script.
7.
Reboot to single-user mode.

7-5

8.
Remove the old VxVM software packages.
9.
Reboot the system to single-user mode.
10. Load and mount the VxVM software upgrade CD-ROM.

11. Install the VRTSlic package.
12. Install the remaining VxVM software upgrade packages.
13. Run the upgrade_finish script.
14. Perform a reconfiguration reboot.
A complete description of the upgrade process is found in VERITAS
Software Corporation TechNote ID 230184 and the VERITAS Volume
Manager 3.2 Installation Guide.
Caution Be sure to backup the boot disk prior to performing any
upgrade of the VxVM software or the Solaris OE.
Upgrading Manually (VxVM 3.1/3.2/3.5)

The manual upgrade process is performed according to the operations
handled by the upgrade scripts, as discussed in the previous section. In
the manual upgrade process, however, the scripts are not used, and all
operations are performed by administrators.
There are two types of manual upgrades.
Upgrading when the boot disk is encapsulated Detailed

instructions for this procedure are found in VERITAS Software
Corporation TechNote ID 240006.
Upgrading when the boot disk is unencapsulated Detailed

instructions for this procedure are found in VERITAS Software
Corporation TechNote ID 240004.
Caution Be sure to back up the boot disk prior to performing any

7-6

Manually Upgrading With an Encapsulated Boot Disk

Detailed instructions on manually upgrading the VxVM software with an
encapsulated boot disk are found in VERITAS Software Corporation
TechNote ID 240006. Refer to the procedures in this document when
upgrading the VxVM software.
The general steps to manually upgrade the VxVM software with an
encapsulated boot disk are:
1.
Back up the system partitions.
2.
If the boot disk is under VxVM control and it is mirrored, use the
vxplex command to break off the mirrors.
3.
Unencapsulate the boot disk using the vxunroot script.
4.
Completely remove the VMSA software package. Make sure that the
/opt/VRTSvmsa directory is removed.
5.
Remove the remainder of the VxVM software packages.
6.
Install the required Solaris OE patches.
7.
Install the new VxVM and VMSA software packages.
8.
Apply the latest VxVM and VMSA software patches for the release
installed
9.
Run the vxinstall command
10. Re-encapsulate the boot disk using the recommended Sun best
practices.
11. Re-mirror the boot disk.
12. Import the original disk groups. Make sure that they are upgraded, if
necessary, to use any new features provided by the new release of
the VxVM software.

7-7
Manually Upgrading With a Non-Encapsulated Boot Disk

Detailed instructions on manually upgrading the VxVM software with a
non-encapsulated boot disk are found in VERITAS Software Corporation
TechNote ID 240004. Refer to the procedures in this document when
upgrading the VxVM software.
The general steps to upgrade the VxVM software manually with a nonencapsulated boot disk are:
7-8
1.
Back up the system partitions.
2.
Completely remove the VMSA software package. Make sure that the
/opt/VRTSvmsa directory is removed.
3.
Install required Solaris OE patches.
4.
Install the new VxVM and VMSA software packages.
5.
Apply the latest VxVM and VMSA software patches for the release
installed.
6.
Run the vxinstall command.
7.
Import the original disk groups. Make sure that they are upgraded, if
necessary, to use any new features provided by the new release of
the VxVM software.

Upgrade Using pkgadd (VxVM 3.5)

Release 3.5 of the Volume Manager software provides a full upgrade from
previous releases using scripts included in the packages supplied with the
upgrade. These scripts perform all tasks needed to address boot disk
encapsulation and any other operations necessary to perform the
upgrade. The full upgrade procedure using pkgadd is found in the
VERITAS Volume Manager 3.5 Installation Guide.
Upgrading to release 3.5 using pkgadd has its advantages over using the
supplied upgrade scripts, and there are some disadvantages. Table 7-1
compares the two processes.
Table 7-1 VxVM 3.5 Upgrade Comparison
Advantage
Using pkgadd
Using Upgrade Scripts
Usually requires one

reboot.
1.
There will only be

one instance of the
VRTSvxvm package
displayed by the
pkginfo command.
2.
VxVM configuration
data is backed up.
3.
The root disk is

unencapsulated, so
there is no risk of
not being able to
boot the system.
4.
The VRTSlic
package can be
deinstalled if there
are no other
packages using it.

7-9

Table 7-1 VxVM 3.5 Upgrade Comparison (Continued)
Disadvantage
Using pkgadd
Using Upgrade Scripts
1.
If a system failure
occurs during the
pkgadd process, the
boot disk may
become unbootable.
Usually requires three

reboots.
2.
The new package is

displayed as
VRTSvxvm2 by
pkginfo unless
instance=
overwrite is
inserted in the
default packge
admin file.
Note The procedure for

modifying the default
packge admin file to
support the upgrade
without having multiple
instances of VxVM
packages is listed in
VERITAS TechNoteID:
248394
3.
The VRTSlic
package might not
be able to be
removed if a second
VRTSvxvm package
is using it.
Following is a high level overview of the VxVM 3.5 upgrade process

using pkgadd.
1.
Make a copy of the default packge admin file.
2.
Modify the copy so instance=overwrite.
3.
Upgrade to VxVM 3.5 using the following pkgadd command syntax:
# pkgadd -a <location_of_the_modified_package_admin_file> -d . VRTSvxvm
7-10


Caution It is important that the correct procedure is used to upgrade to
release 3.5. If the procedures listed in the VERITAS Volume Manager 3.5
Installation Guide are not followed precisely, the system may become
unusable.
Upgrading a Disk Group

When a new version of the VxVM software is installed, the versions of
configured disk groups might need to be upgraded to enable new
functionality provided by the upgrade.
To upgrade a disk group, use the vxdg command with the following
syntax:
# vxdg upgrade disk_group_name

7-11
Release Notes
All upgrades and patches include release notes. Be sure to read the release
notes prior to performing any upgrade or patch. Release notes contain
valuable installation and operation information specific to the patch or
release of the VxVM software being installed.
Release notes are normally found in the base directory of the patch or
upgrade distribution and contain either release or notes in the file
name.
Licensing
When upgrading the VxVM software, licensing must be taken into
consideration. If upgrading from the VxVM software version 2.x to 3.x, a
version 3.x license key must be installed prior to the upgrade, or the
VxVM software does not start.
Licensing issues are discusses in the following SunSolve INFODOCs:
17714 VxVM and SEVM Licensing Explained
19492 How to Move a VxVM License between Systems
In addition, the upgrade procedures referred to in this module have a

section on pre-upgrade licensing.
7-12

Upgrading the Solaris OE
Upgrading the Solaris OE

Upgrading the Solaris OE with the VxVM software installed requires
execution of additional steps to account for encapsulation of the boot disk.
Additionally, the VxVM software can be upgraded along with the
Solaris OE. This is a complex and risky upgrade.
The following documents provide procedures to upgrade the Solaris OE
and the VxVM software on a system with the VxVM software installed.
The instructions provided in these documents must be followed exactly to
ensure success:
VERITAS Volume Manager 3.2 Installation Guide
SunSolve INFODOC 14189
VERITAS Software Corporation TechNote ID 230184
Caution Be sure to back up the boot disk prior to performing any

A Solaris OE upgrade using the JumpStart process or flash install requires
extensive scripting to successfully complete the upgrade. It is beyond the
scope of this course to provide the knowledge necessary to build these
scripts.

7-13
Exercise: Upgrading the VxVM Software

In this exercise, you complete the following tasks:
Upgrade the VxVM software to a later release using either scripted

or manual methods
Upgrade disk groups
Preparation
All lab systems must have Solaris 8 OE flash or JumpStart installed,

with all supporting packages and patches.
At a minimum, the VxVM software version 3.2 must be installed and

configured.
The boot disk must be encapsulated.
The VxVM software packages and patches for the new version must
be available either through network file system (NFS) mounts, ftp,
or local access.
You must have access either to all SunSolve and VERITAS Software
Corporation documents referenced in this module.
Tasks
1.
Upgrade the VxVM software to a newer release using one of the

following methods:
Upgrade using the upgrade scripts located in the

/Package_Distribution_Path/scritps directory.
Reference the VERITAS Volume Manager 3.x Installation Guide
section Upgrading VxVM on an Encapsulated Root Disk. This
section provides a detailed, step-by-step procedure for using
the upgrade scripts.
Upgrade using manual procedures.

Reference VERITAS Software Corporation TechNote ID 240006
for the procedures to perform the upgrade without using the
upgrade scripts.
7-14


The method used by your lab group is not as important as
completing the procedure successfully. Both procedures work. Pick
the procedure that best fits the experience level of your lab group
Do not perform the backup of the boot disk as recommended by the
procedures. Tape drives are not available in the lab.
Upgrade using pkgadd procedures.

Reference VERITAS Software Corporation TechNote ID 248394
for the procedures to perform the upgrade without generating
additional instances of the VxVM software packages.
2.
Re-encapsulate the boot disk.
3.
Upgrade all configured disk groups if not specified by the upgrade

procedures.
4.
Mount any volumes not mounted as part of the upgrade procedures.
5.
Verify full functionality and fix any problems encountered.

7-15
Exercise Summary
Exercise Summary
!
?
7-16

Experiences
Interpretations
Conclusions
Applications

Appendix A
SunSolve INFODOCs
This appendix contains the following SunSolveSM Online Free Info Docs
online sources, available at:
http://sunsolve.Sun.COM/pub-cgi/search.pl?mode=advanced
INFODOC 16051 How to Encapsulate Disks With No Free Space

Using Volume Manager. 22 March 2002.
INFODOC 24663 Full and Basic/Functional Unencapsulation of a

Volume Manager Encapsulated Root Disk While Booted CDROM.
22 March 2002.

A-1
INFODOC 16051
INFODOC 16051
INFODOC ID: 16051
SYNOPSIS: How to 'Encapsulate' Disks With No Free Space Using Volume Manager
DETAIL DESCRIPTION:
To encapsulate a disk into Veritas Volume Manager or Sun Enterprise Volume Manager[TM],
you must have some free space on the disk in order for Volume Manager to write a private
region to the disk. The private region is generally smaller than 2mb.
However, if you absolutely do not have any free space on the disk, and you can't free up
any, and you really want to get this data under Volume Manager control, you can work
around this by TEMPORARILY "encapsulating" one or more slices from a disk into Volume
Manager so that the data may be mirrored to another disk. Once the data is mirrored to
a "real" Volume Manager disk with a private and public region, you can then break the
mirror, leaving the data on the "real" Volume Manager disk.
SOLUTION SUMMARY:
Here is how to do it:
For each slice on the disk (excluding slice 2), run the following command. In this
example, only slices 5 and 6 have data on them.
Then add each of these "slices" as a disk in a disk group and give them a name. This
example names them NPdisk05 and NPdisk06.
Next we create a simple volume (not a file system, just a volume) on each of these new
"disks" that spans the entire "disk". To do this we first check to see what the max size
is for the volumes we are about to create. We're looking for the len value to then use
with the vxassist command to create the volumes.
vxdisk list NPdisk05 | grep public
public:
vxdisk list NPdisk06 | grep public
public:
With this info we create the volumes, naming them NPdisk05vol and NPdisk06vol:
Next step is to mirror the volumes, assuming that we are mirroring the volumes to a disk
named disk01 that has enough space to mirror both volumes to it:
Once that is complete we then remove the original side of the mirror.
The final step is to remove the old disks from the disk group and return them to
their original state.
vxdisk rm c0t5d10s5
vxdisk rm c0t5d10s6
This leaves us with two concat volumes named NPdisk05vol and NPdisk06vol.
These volumes will contain the data that was orignally located on
c#t#d#s5 and c#t#d#s6.
A-2

INFODOC 16051
Keywords: SEVM, VxVM, Volume Manager, recover, recovery, configure, configured,
configuration
APPLIES TO: Storage/Veritas, Storage/Volume Manager, AFO Vertical Team Docs/Storage
ATTACHMENTS:

SunSolve INFODOCs
A-3
INFODOC 24663
INFODOC 24663
INFODOC ID: 24663
SYNOPSIS: Full and Basic/Functional Unencapsulation of a Volume Manager Encapsulated
Root Disk While Booted CDROM
DETAIL DESCRIPTION:
Overview:
This document explains the steps necessary to unencapsulate the root disk from Volume
Manager control. This document applies to both Sun Enterprise Volume Manager[TM] (SEVM)
2.x and Veritas Volume Manager (VxVM) 3.x.
This document is divided into two distinct sections. The first section describes full
unencapsulation while booted from a Solaris CDROM. This procedure should be used any
time it is necessary to completely remove the root disk from Volume Manager control and
bring the disk back to a pre-encapsulation state including all partitions such as
/export, and /opt.
The second section explains the steps to perform a Basic/Functional (BF) unencapsulation
while booted from a Solaris CDROM. Basic/Functional unencapsulation temporarily
unencapsulates the root disk so that troubleshooting of booting issues or other issues
can be done. BF unencapsulation gives you access to an unencapsulated /, swap, /usr, and
/var but no access to non "big-4" partitions.
SOLUTION SUMMARY:
Notes for Full Unencapsulation:
Under normal circumstances, if the system can be booted to at least single user mode, it
is recommended that the vxunroot command be used to unencapsulate root. A full
unencapsulation should be performed if the vxunroot command is not working for some
reason, or if the system cannot be booted and we want to completely remove Volume
Manager from having any control over the root disk.
You cannot perform a full unencapsulation and still maintain Volume Manager
functionality if the root disk is the ONLY disk in the rootdg diskgroup. If the root
disk is the only disk in rootdg, you can still unencapsulate, but Volume Manager will
not work until another disk is initialized into rootdg using vxinstall after the system
has been fully unencapsulated. Normally if root it is encapsulated, it is also mirrored
which gives us another disk in rootdg, however it should always be verified that there
is at least one other disk in rootdg before following this procedure so that we know
what to expect once root is unencapsulated.
Also note that Volume Manager will allow you to create volumes using free space on the
root disk, after the root disk has been encapsulated. Volumes created post-encapsulation
like this do not have underlying hard partitions and therefore are not recoverable with
this procedure. If at all possible, make backups of any volumes created on the rootdisk
post-encapsulation before following this procedure. Once you are unencapsulated, if you
have free space and a free partition, you could newfs that partition and restore to it
from your backup.
Steps for Full Unencapsulation:
Bring the system to the OK prompt and insert a Solaris CD into the CDROM drive.
Then issue:
boot cdrom -s
Once booted to the cdrom, set your terminal type so that vi will work correctly.
If TERM=sun doesn't work, often times TERM=vt100 will.
TERM=sun;export TERM
Fsck your root filesystem:
fsck -y /dev/rdsk/c#t#d#s0
A-4

INFODOC 24663
If fsck comes back cleanly, mount slice 0 to /a. If fsck cannot repair the root file
system, there are obviously a number of possibilities. This procedure does not attempt
to explain file system corruption or how to repair it beyond fsck. Fsck must come back
cleanly to continue and mount root.
mount /dev/dsk/c#t#d#s0 /a
Make a backup of /a/etc/system and then edit it:
cp /a/etc/system /a/etc/system.orig
vi /a/etc/system
Completely remove the following lines from the system file. If you re-encapsulate in the
future, these lines will be added back correctly so there is nothing to be lost by
removing them.
Make a backup of /a/etc/vfstab and then edit it:
cp /a/etc/vfstab /a/etc/vfstab.orig
vi /a/etc/vfstab
Edit the vfstab file back to it's original state, pointing /, swap, /usr, and /var to
hard partitions on the disk like /dev/dsk and /dev/rdsk rather than /dev/vx/ entries.
Temporarily comment out all other /dev/vx volumes from the /a/etc/vfstab file using the
# character. This includes filesystems like /opt and /export, if they exist.
The original /etc/vfstab will look something like this, assuming root is c0t0d0:
Note: Columns have been aligned and spaces added for clarity.
swap
ufs
1
no /dev/vx/dsk/usr
/dev/vx/rdsk/usr
/usr
ufs
1
no /dev/vx/dsk/var
/dev/vx/rdsk/var
/var
ufs
1
/dev/vx/rdsk/export
/export ufs
2
yes swap
/tmp
tmpfs
/somevol
ufs
2 yes -

--------------------------------------------------------------------------Once edited, the vfstab should look something like this:
--------------------------------------------------------------------------/dev/dsk/c1t0d0s1
swap
/dev/rdsk/c0t0d0s0
/
ufs
1
/dev/rdsk/c0t0d0s5
/usr
ufs
1
/dev/rdsk/c0t0d0s6
/var
ufs
1
/dev/rdsk/c0t0d0s7
/export ufs
2
yes swap
/tmp
tmpfs
#NOTE:
#NOTE:
#NOTE:
#NOTE:
#NOTE:
volume
volume
volume
volume
volume
/somevol
ufs
2 yes -
rootvol (/) encapsulated partition c0t0d0s0

swapvol (swap) encapsulated partition c0t0d0s1
usr (/usr) encapsulated partition c0t0d0s5
var (/var) encapsulated partition c0t0d0s6
export (/export) encapsulated partition c0t0d0s7

SunSolve INFODOCs
A-5
INFODOC 24663
--------------------------------------------------------------------------Now make sure Volume Manager does not start on the next boot:
touch /a/etc/vx/reconfig.d/state.d/install-db
This is important because IF the root disk contains mirrors, and the system boots up,
the mirrors will get resynced, corrupting the changes we just made.
Remove the flag that tells Volume Manager that the root disk is encapsulated:
rm /a/etc/vx/reconfig.d/state.d/root-done
Reboot the system for changes to take effect:
reboot
When we reboot we will come up in an partially unencapsulated state with /, /usr, /var,
and swap mounted. Volume Manager will not start but we can start it manually once we
are booted.
To start Volume Manager, run the following commands:
vxiod set 10
vxdctl enable
Now we can remove the volumes that existed on the encapsulated boot disk. They will
generally be rootvol, swapvol, usr, and var. They might also include home, opt, or
other non-standard root partitions. Use the command 'vxprint -htg rootdg' to list the
volumes in rootdg before removing them. Then, for each volume, run the command:
/usr/sbin/vxedit -rf rm <volume name>
Remove the rootdisk from rootdg now that it has no volumes, plexes or subdisks: The disk
name is usually 'rootdisk'
/usr/sbin/vxdg rmdisk <disk name>
The final step is to re-write the vtoc of the disk so that hard partitions are again
defined for the root file systems. There are several ways to put the hard partitions
back, including using fmthard on a modified /etc/vx/reconfig.d/disk.d/c#t#d#/vtoc file,
using format to manually repartition the disk, or using the vxmksdpart command. The
simplest method however is to use the vxedvtoc command as explained below.
When Volume Manager encapsulates a disk, it makes a record of the old vtoc of the disk.
This file is stored for each disk in /etc/vx/reconfig.d/disk.d/c#t#d#. This file is
stored in a Volume Manager specific format, so it can't be used as an argument to
fmthard unless it is modified. The 'vxedvtoc' command is similar to fmthard but knows
how to read this vtoc file and write that vtoc to a disk. The command takes the form:
vxedvtoc -f <filename> <devicename>
Assuming that the boot disk is c0t0d0 we would now run the command
/etc/vx/bin/vxedvtoc -f /etc/vx/reconfig.d/disk.d/c0t0d0/vtoc /dev/rdsk/c0t0d0s2
# THE ORIGINAL PARTITIONING IS AS FOLLOWS :
#SLICE
TAG FLAGS
START
SIZE
0
0x0 0x200
0
0
1
0x0 0x200
0
0
2
0x5 0x201
0 8794112
3
0x0 0x200
0
0
4
0x0 0x200
0
0
5
0x0 0x200
0
0
6
0xe 0x201
0 8794112
7
0xf 0x201 8790016
4096
# THE NEW PARTITIONING WILL BE AS FOLLOWS :
#SLICE
TAG FLAGS
START
SIZE
0
0x0 0x200
0 2048000
1
0x0 0x200 2048000 2048000
2
0x5 0x201
0 8794112
3
0x0 0x201 4096000 2048000
A-6

INFODOC 24663
4
0x0 0x201 6144000 2048000
5
0x0 0x200
0
0
6
0x0 0x200
0
0
7
0x0 0x200
0
0
DO YOU WANT TO WRITE THIS TO THE DISK ? [Y/N] :y
WRITING THE NEW VTOC TO THE DISK
This will partition the disk back to a pre-encapsulation state. Now we can uncomment
the entries for any of the non Big-4 root partitions from /etc/vfstab, as well as any
data volumes. In this example we removed comments from /export and the data volume
/somevol.
vi /etc/vfstab
/dev/dsk/c0t0d0s1
/dev/dsk/c0t0d0s0
/dev/dsk/c0t0d0s5
/dev/dsk/c0t0d0s6
/dev/dsk/c0t0d0s7
swap
/dev/rdsk/c0t0d0s0
/dev/rdsk/c0t0d0s5
/dev/rdsk/c0t0d0s6
/dev/rdsk/c0t0d0s7
-
/
/usr
/var
/export
/tmp
swap
ufs
ufs
ufs
ufs
tmpfs
1
1
1
2
-
no
no
no
no
yes
yes
/dev/vx/dsk/datadg/somevol /dev/vx/rdsk/datadg/somevol /somevol ufs 2 yes Just to make sure, start all volumes
/usr/sbin/vxvol startall
Now issue a mountall to mount the now uncommented volumes
mountall
At this point the root disk is completely free of Volume Manager control, Volume Manager
daemons are started, and all file systems / volumes should be mounted.
Notes for Basic/Functional Unencapsulation:
This section explains the steps necessary to temporarily unencapsulate the root disk
from Volume Manager control. BF unencapsulation allows the system to be booted from the
raw Solaris partitions while still leaving the root disk under Volume Manager control.
This is a good method for troubleshooting boot issues that appear to be due to the disk
being encapsulated because it can be undone by reversing the steps. This procedure can
be used even if the root disk is the only disk in the rootdg diskgroup because,
throughout the procedure, root will keep it's private and public region. This procedure
will only allow you to mount /, /usr, /var, and swap. Any non "Big-4" partitions will
not be mounted. If you must have non "Big-4" partitions available, you should perform a
full unencapsulation as outlined above.
Steps for Basic/Functional Unencapsulation:
Bring the system to the OK prompt and insert a Solaris CD into the cdrom drive. Then
issue:
boot cdrom -s
Once booted to the cdrom, set your terminal type so that vi will work correctly. If
TERM=sun doesn't work, often times TERM=vt100 will.
TERM=sun;export TERM
Fsck your root filesystem:
fsck -y /dev/rdsk/c#t#d#s0
If fsck comes back cleanly, mount slice 0 to /a. If fsck cannot repair the root file
system, there are obviously a number of possibilities. This procedure does not attempt
to explain file system corruption or how to repair it. fsck must come back cleanly to
continue and mount root.
mount /dev/dsk/c#t#d#s0 /a
Make a backup of /a/etc/system and then edit it:
vi /a/etc/system

SunSolve INFODOCs
A-7
INFODOC 24663
Comment out the following lines using double asterisks **.
Make a backup of /a/etc/vfstab and then edit it:
vi /a/etc/vfstab
Edit the vfstab file back to it's original state, pointing /, swap, /usr, and /var to
hard partitions on the disk like /dev/dsk and /dev/rdsk rather than /dev/vx/ entries.
Temporarily comment out all other /dev/vx volumes from the /a/etc/vfstab file using the
# character. This includes filesystems like /opt and /export, if they exist.
The original /etc/vfstab will look something like this, assuming root is c0t0d0: Note:
Columns have been aligned and spaces added for clarity.
swap
ufs
1
no /dev/vx/dsk/usr
/dev/vx/rdsk/usr
/usr
ufs
1
no /dev/vx/dsk/var
/dev/vx/rdsk/var
/var
ufs
1
/dev/vx/rdsk/export
/export ufs
2
yes swap
/tmp
tmpfs
/somevol
ufs
2 yes -

--------------------------------------------------------------------------Once edited, it should look something like this:
--------------------------------------------------------------------------/dev/dsk/c0t0d0s1
swap
/dev/rdsk/c0t0d0s0
/
ufs
1
/dev/rdsk/c0t0d0s5
/usr
ufs
1
/dev/rdsk/c0t0d0s6
/var
ufs
1
/dev/rdsk/c0t0d0s7
/export ufs
2
yes swap
/tmp
tmpfs
yes #/dev/vx/dsk/datadg/somevol /dev/vx/rdsk/datadg/somevol /somevol ufs 2 yes #NOTE: volume rootvol (/) encapsulated partition c0t0d0s0
--------------------------------------------------------------------------Now make sure Volume Manager does not start on the next boot:
touch /a/etc/vx/reconfig.d/state.d/install-db
The presence of install-db tells Volume Manager not to start at boot. This is important
because IF the root disk contains mirrors, and the system boots up, the mirrors will get
resynced, corrupting the changes we just made.
Remove the flag that tells Volume Manager that the root disk is encapsulated:
rm /a/etc/vx/reconfig.d/state.d/root-done
Reboot the system for changes to take effect:
reboot
A-8

INFODOC 24663
When we reboot we will come up in an unencapsulated state with /, /usr, /var, and swap
mounted.
At this point we have performed a Basic/Functional unencapsulation. This is a not a
state that they system should be left in permanently. It is a state that is useful for
troubleshooting and system maintenance.
If problems with the system are resolved and you are ready to re-encapsulate, perform
the following:
touch /etc/vx/reconfig.d/state.d/root-done
cp /a/etc/vfstab.orig /a/etc/vfstab
cp /a/etc/system.orig /a/etc/system
reboot
Keywords: SEVM, VxVM, Volume Manager, encapsulation
APPLIES TO: Operating Systems/Solaris/Solaris 8, Operating Systems/Solaris/Solaris 7,
Operating Systems/Solaris/Solaris 2.6, Operating Systems/Solaris/Solaris 2.5.1,
Storage/Veritas, Storage/Volume Manager, AFO Vertical Team Docs/Storage
ATTACHMENTS:

SunSolve INFODOCs
A-9
Appendix B
Disk Encapsulation Processes

This appendix contains flow charts which outline the processes used to
perform the following tasks:
Encapsulate non-root or data disks
Unencapsulate non-root or data disks
Encapsulated boot disks
Unencapsulate boot disks by using the vxunroot command
Unencapsulate boot disks by not using the vxunroot command
Unencapsulate boot disks when the system is booted from a CDROM
Perform a basic boot disk unencapsulation
Perform the Sun Enterprise Servicess best practice procedures for

managing boot disks with the VxVM software

B-1
Start
Encapsulate?
Yes
Boot Disk?
Yes
Page 10 of 26
No
No
Unencapsulate?
No
Yes
Page 2 of 26
Boot Disk?
Yes
Page 11 of 26
No
Page 7 of 26
End
Disk Encapsulation Flowchart

Rev-B
Page 1 of 26
B-2

07/18/2002
This flow has been selected because a
disk with existing data must be brought

under VxVM software control.
Selected disk must

have sufficient space to
Select disk to
hold the private region
encapsulate.
and have two unused

slices.
Answer all prompts
Enter the disk

group name that
this disk shall
appropriate for this

encapsulation procedure
join
If encapsulating the boot
disk, do not use default
disk name.
Selected disk has

sufficient free space and
No
partitions for
If a default disk
name was not
to 31 characters in
length.
previously
encapsulation?
Disk names can be up
selected, enter
If encapsulating the boot
an arbitrary
disk, use the name
name now
rootdisk.
Page 3
Yes
;AI
of 26
is the normal
selection for this

Open
vxdiskadm
# vxdiskadm
question. If
changing default
Use default
No
private region
utility.
size, all disks in

this disk group
size?
must have the

same private
Select
encapsulation
region size.
Yes
utility.
Exit
Encapsulate
additional disks?
Select disk to
encapsulate
No
vxdiskadm
utility and reboot
c#t#d#
the system.
End
Answer yes
continue
to
operation
Rev-B
07/18/2002
Page 2 of 26
Yes

B-3
This flow has been selected because a disk with existing data
(non root) that does not met minimum encapsulation

requirements, must be brought under VxVM software control.
Select the disk to

be encapsulated
For each slice on

the disk (excluding
slice 2) use
vxdisk
to bring
under VxVM
vxdisk define c#t#d#s# type=nopriv

Example
softwar control
#vxdisk define c1t3d0s5 type=nopriv

#vxdisk define c1t3d0s6 type=nopriv
Yes
Configure
additional slices?
No
Add each of these
"slices" as a disk to
an existing disk
group
vxdg -g <diskgroup> adddisk <diskname>=c#t#d#s#

Example
#vxdg -g datadg adddisk NPdisk05=c1t3d0s5

#vxdg -g datadg adddisk NPdisk06=c1t3d0s6
Yes
Configure
additional slices?
No

Page 4 of 26
Rev-B
Page 3 of 26
B-4

07/18/2002
This flow has been selected because a process

executing in flow E was directed here.
Document the
maximum size of
the "disks" just
vxdisk list <diskname> | grep public
created. This
information shall
be used to create
Example
#vxdisk list NPdisk05 | grep

public:
slice=0 offset=0
#vxdisk list NPdisk06 | grep
public:
slice=0 offset=0
volumes
Yes
public
len=8196096
public
len=9400320
Document
additional slices?
Use the information

from the previous
step to create
vxassist -g <diskgroup> make <volumename>

layout=nostripe alloc="<diskname>"
<size> \
volumes.
Example
#vxassist -g datadg make NPdisk05vol 8196096

layout=nostripe \ alloc="NPdisk05"
#vxassist -g datadg make NPdisk06vol 9400320
layout=nostripe \ alloc="NPdisk06"
Yes
Create additional
volumes?
No
Page 5 of 26
Rev-B
07/18/2002
Page 4 of 26

B-5

executing in flow F was directed here.
Mirror the volumes
Yes
Example
Mirror additional
volumes?
No
vxassist -g <diskgroup> mirror <volumename>

layout=nostripe \
alloc="<mirrordiskname>"
#vxassist -g datadg mirror NPdisk05vol layout=nostripe \

alloc="disk01"
#vxassist -g datadg mirror NPdisk06vol layout=nostripe \
alloc="disk01"
No
Mirroring process
complete?
Yes
Remove the original

side of the mirror.
vxplex -g <diskgroup> -o rm dis <originalplexname>

Example
#vxplex -g datadg -o rm dis NPdisk05vol-01

#vxplex -g dtatdg -o rm dis NPdisk06vol-01
Yes
Remove additional
mirrors?
No

Page 6 of 26
Rev-B
Page 5 of 26
B-6

07/18/2002
executing in flow G was directed here.
Remove the old

disks from the disk
group
vxplex -g <diskgroup> rmdisk <diskname>

vxdisk rm c#t#d#s#
Example
Yes
Remove additional
#vxdg -g datadg rmdisk NPdisk05

#vxdg -g datadg rmdisk NPdisk06
#vxdisk rm c1t3d0s5
#vxdisk rm c1t3d0s6
disks?
No
End
Note: To return these volumes to full redundancy, re-mirror the volumes

using another initialized disk within the disk group.

Rev-B
07/18/2002
Page 6 of 26

B-7
This flow has been selected because an encapsulated data disk
(non root) must be unencapsulated. Applications using the volumes

encapsulated on this disk must be shutdown.
Any grown
Disk is unable to be
Yes
volumes?
unencapsulated.
No
Any volumes
structurally
Yes
modified?
unencapsulated.
No
Original
disk
Yes
replaced?
unencapsulated.
No
Encapsulated disks use slices six and
seven for public and private regions.
Identify disk to be
unencapsulated.
Mirror disks use slices three and four.
Backup data and restore to

non VxVM managed disk.
Use the following commands:
vxprint
format / prtvtoc
No
End
Volumes
Review content of
/etc/vfstab file
backed-up?
to define
pre-encapsulation
Yes
configuration
B-8
Page 8 of 26
Rev-B
07/18/2002
Page 7 of 26

executing in flow C was directed here.
Unmirror all volumes on the

disk to be unencapsulated
vxplex -g <diskgroup> -o rm dis <plexname>

Example
#vxplex -g datadg -o rm dis fs1-02

#vxplex -g dtatdg -o rm dis fs2-02
Yes
Remove additional
mirrors?
No
Verify that all mirrors have

been detached.
vxprint -qhtg <diskgroup>

The information needed to execute this step is found in the
/etc/vfstab file and output from the vxprint -qhtg

<diskgroup> command.
Recreate original partitions.
vxmksdpart -g <diskgroup> <plexname> <partition#>

\tags flags
Example
#/etc/vx/bin/vxmksdpart -g datadg datadg01-03 0
0x00 0x00
#/etc/vx/bin/vxmksdpart -g datadg datadg01-02 1
0x00 0x00
Yes
Additional
partitions to
restore?
No
The disk is unable to be

unencapsulated, restore
Partititions
No
Restored?
the data from backups to
End
non VxVM software

managed disk.
Yes

Rev-B
Page 9 of 26
07/18/2002
Page 8 of 26

B-9

executing in flow I was directed here.
Unmount the volumes
Yes
Unmount
additional
volumes?
No
Edit /etc/vfstab file and

restore pre encapsulation
mount statements.
Re mount filesystems
vxedit -r -f rm <volume>
Example
Recursively remove each

volume.
#vxedit -r -f rm fs1
#vxedit -r -f rm fs2
Yes
Remove additional
volumes?
No
Remove disk from

diskgroup.
vxdg rmdisk <diskname>

vxdisk rm c#t#d#
Example
Remove Public and Private
#vxdg rmdisk datadg01

#vxdisk rm c1t3d0
Regions using format utility.
End

Rev-B
07/18/2002
Page 9 of 26
B-10

This flow has been selected because the
boot disk must be brought under Volume

Manager control.
Encapsulate
and mirror using
Yes
Sun ES
Page 24 of 26
Best Practice for

boot disks?
No
This option is used during the initial

installation of the Volum e Manager
Encapsulate using
vxinstall?
s o f t wa r e . T o e n c a p s u l a t e t h e b o o t
Yes
disk, answer
OAI
Boot Disk"
to the
prompt
"Encapsulate
and
answer
presented questions as appropriate for

the installation
No
End
Page 2 of 26

Rev-B
07/18/2002
Page 10 of 26

B-11
This flow has been selected because the

boot disk must be removed from Volume
Manager control.
vxunroot?
Yes
Yes
Page 12 of 26
No
Manual?
Page 15 of 26
No
Booted
from
Yes
Page 19 of 26
Page 22 of 26
CDROM?
No
Basic
Functionality?
No
Yes
This unencapsulation procedure, is designed to help troubleshoot

system problems that require the VxVM software to be shutdown.
It is also a good technique to use for system maintenance tasks
End
that require the VxVM software to be shutdown.

Rev-B
Page 11 of 26
B-12

07/18/2002

executing in flow D was directed here.
No
The following procedure requires a reboot of the

system. Terminate all production applications
and logoff all non administrative users prior to
Current boot
starting.
disk backup?
Yes
Unm ount all file system s

under the control of VxVM
/, /usr,
software except (
/var, /opt) if resident on
Properly configured boot disks only contain

system partitions. This should not be an issue.
the boot disk.
Record the current

configuration of
rootdg
Record the current disk

configuration.
vxprint -qhtg rootdg > /rootdg.print
vxdisk -o alldgs list > /vxdisk_alldgs.list

vxprint -qhtg rootdg -s | grep -i rootmirror | \
awk '{print $3}' > /rmsub.plex
Detach all plexes associated
rootmirror" disk.
with the "
Veify all "
rootmirror "
plexes are detached and

No
for i in `cat ./rmsub.plex`

>do
>done
vxprint -qhtg rootdg
removed.
All
rootmirror"
"
plexes detached
and removed?
This is a critical step.

must be removed for
All "rootmirror" plexes

all volumes residing on the
boot disk. If this step does not complete

successfully,
Yes
vxunroot
will fail.

Rev-B
07/18/2002
Page 12 of 26

B-13

Remove rootability using the
vxunroot
/etc/vx/bin/vxunroot
command.
Successful?
No
Page 14 of 26
Yes
Reboot system
Reboot
successful?
No
Page 14 of 26
No
Page 14 of 26
Yes
Verify system
file system mounts.
Using non
VxVM software
objects
Yes
End

Rev-B
07/18/2002
Page 13 of 26
B-14


executing in flow O was directed here.
vxunroot
1st
time through
Go to manual
still
No
flow?
fails?
process.
No
Yes
vxunroot
command
Go to CDROM
Reboot
No
unencapsulation
Yes
failed?
failed?
unencapsulation
Yes
process.
No
Execute
Yes
vxunroot
Still using
Verify all non
unencapsulation
VxVM software
system volumes
process again.
Yes
objects?
on this disk are
Follow
instructions
unmounted.
exactly as written
No
Verify all volumes
on this disk have
Other
not been grown, or

structually
modified.
Yes
problem?
No
Call support.
End
Verify all
rootmirror"
"
mirrors have been

detached and
System
bootable?
Still using
Yes
VxVM software
objects?
No
Other
No
problem?
removed.
End
No
Execute
vxunroot
unencapsulation
process again.
Yes
Go to CDROM
Go to manual
unencapsulation
unencapsulation
process.
process.
Yes
Call support.
Rev-B
07/18/2002
Page 14 of 26

B-15

No
Current boot
disk backup?
starting.
Yes
Unmount all file systems
under the control of VxVM
/ /usr, /
software except ( ,
var, /opt) if resident on the
Properly configured boot disks only contain

system partitions. This should not be an issue.
boot disk.
Record the current

configuration of
rootdg
Record the current disk
vxprint -qhtg rootdg > /rootdg.print
vxdisk -o alldgs list > /vxdisk_alldgs.list
configuration.
vxprint -qhtg rootdg -s | grep -i rootmirror | \

awk '{print $3}' > /rmsub.plex
Detach all plexes associated
rootmirror" disk.
with the "
rootmirror"
Veify all "
plexes are detached and

No
removed.
All
rootmirror"
"
plexes detached
and removed?
for i in `cat ./rmsub.plex`

>do
>done
All "rootmirror" plexes must be

all volumes residing on the boot disk. If this step
This is a critical step.

removed for
does not complete successfully, this procedure shall fail.
Yes
B-16

Page 16 of 26
Rev-B
07/18/2002
Page 15 of 26


executing in flow N was directed here.
Remove rootability
using the following
processes:
rm /etc/vx/reconfig.d/state.d/
root-done
Remove the
root-done file.
Edit the
system
cp /etc/system /etc/system.orig
vi /etc/system
/etc/
file and
Remove (do not comment) the following two
rootdev"
remove "
lines:
statements.
/
swap, /usr, /var) partitions
If there are non system ( ,
Edit the
/etc/
Use either the comments at the end of the
on this disk, restore those
vfstab
mount statements at this time.
restore to it's
These non system partitions
pre boot disk
could include
encapsulation state.
/opt, /home or
/etc/
vfstab.prevm to restore the original
file and
file or the contents of the

mount statements.
The
other named filesystems.
/etc/vfstab.prevm file can be

/etc/vfstab file providing
copied over the
there have been no changes to this systems

storage configuration after the boot disk was
If there are any non
encapsulated.
"system" partitions on
vxmksdpart -g <diskgroup> \
<plexname> <partition#> tags
flags
this disk, use
vxmksdpart
command to restore.
Example
#/etc/vx/bin/vxmksdpart -g
datadg \
datadg01-04 5 0x00 0x00
reboot the system
Reboot
No
successful?
Yes
Page 18 of 26

Rev-B
Page 17 of 26
07/18/2002
Page 16 of 26

B-17

executing in flow O was directed here.
All boot
disk file systems
mounted?
Fix problem. Check

No
/etc/vfstab
entries. Make sure
that all non system partitions were properly restored and

match
vfstab
entries.
Yes
Recursively remove
boot volumes from
rootdg
Verify volume
No
removal
vxedit -r -f rm rootvol
vxedit -r -f swapvol
Volumes
removed?
Yes
Remove the boot disk

from VxVM software
vxdg -g rootdg rmdisk rootdisk
control.
Verify disk removal.
No
Disk
removed?
Yes
Update the VxVM
Delete public and

private region
software's view of
vxdctl enable
installed disks.
partitions using the

format utility.
End
Rev-B
Page 17 of 26
B-18

07/18/2002

executing in flow P was directed here.
System
bootable?
End
Yes
No
Manual
Go to CDROM
unencapsulation
failed. First time
unencapsulation
Yes
process.
through flow?
No
CDROM
unencapsulation failed
No
leaving an unbootable
Other
Yes
problem?
Call support.
system?
No
End
Yes
Boot disk
backup
No
available?
Reinstall Solaris
Operating Environment
End
Yes
Rebuild boot disk from
backups. Once restored, boot
Pre
encapsulation
backup?
No
from CDROM, disable boot

disk encapsulation using
CDROM unencapsulation
procedure.
Yes
Restore boot disk from

backups
End

Rev-B
07/18/2002
Page 18 of 26

B-19

No

Current boot
starting.
disk backup?
Yes
Bring system to boot
prom and boot to single
user mode from
CDROM.
Once booted, set TERM

type to
vt100 so vi will
work.
fsck root filesystem.

Mount root partition
(slice 0) to
/a

type to
vt100 so vi will
work.
Edit the
system
vi /a/etc/system
/etc/
file and
rootdev"
remove "
statements.
Remove (do not comment) the following two lines:
Make a backup copy of
/a/etc/vfstab
B-20
Rev-B
Page 20 of 26
07/18/2002
Page 19 of 26

If there are non system (/,
swap, /usr, /var) partitions on

executing in flow S was directed here.
this disk, do not restore those

mount statements at this time.
These non system partitions
could include /opt, /home or
other named filesystem.
Edit the /etc/vfstab file
These mount statements shall
and restore to it's
be restored later in this
pre boot disk
process after their partitions
encapsulation state.
Use either the comments at the end of the

file or the contents of the /etc/vfstab.prevm
to restore the original mount statements.
have been recovered. It is
The /etc/vfstab.prevm file can be copied
acceptable to restore the
over the /etc/vfstab file providing there
mount statements at this time
have been no changes to this systems
if they are commented out so
storage configuration after the boot disk
they will not be executed
was encapsulated.
during the next system reboot.
touch /a/etc/vx/reconfig.d/state.d/ \
install-db
Prevent Volume
Manager from starting on
the next boot.
Remove
rm /a/etc/vx/reconfig.d/state.d/ \
root-done
boot disk
encapsulation flag.
reboot the system
Page 18 of 26
No
Reboot
successfull?
The system will reboot in a partially unencapsulated
state with /, swap, /usr and /var mounted.
Yes
Manually start the

Volume Manager
software.
rm /etc/vx/reconfig.d/state.d/ \
install-db
vxiod set 10
vxdctl enable
List all volumes

encapsulated on the boot
disk. Note these for use
later.
Rev-B
07/18/2002
Page 21 of 26
Page 20 of 26

B-21

executing in flow S was directed here.
Remove the volumes

that existed on the
encapsulated disk.
Example
/usr/sbin/vxedit
/usr/sbin/vxedit
/usr/sbin/vxedit
/usr/sbin/vxedit
/usr/sbin/vxedit
Remove the boot disk
from
rootdg
-rf
-rf
-rf
-rf
-rf
rm
rm
rm
rm
rm
rootvol
swapvol
usr
var
opt
/usr/sbin/vxdg -k rmdisk <disk name>

Example
Restore the boot disk's
vtoc
/usr/sbin/vxdg -k rmdisk rootdisk
to it's pre
encapsulation state. This

defines hard partitions
for the file systems
resident on the boot disk.
Use the
vxedvtoc
command and answer Y

when prompted to write
new
vtoc
/etc/vx/bin/vxedvtoc -f /etc/vx/reconfig.d/disk.d/ \
<bootdisk c#t#d#>/vtoc /dev/rdsk/<bootdisk c#t#d#s2
Example
/etc/vx/bin/vxedvtoc -f /etc/vx/reconfig.d/disk.d/ \
c0t0d0/vtoc /dev/rdsk/c0t0d0s2
CAUTION:
c#t#d#
Be absolutely sure that the correct
addresses are used in this step or loss
of data could occur. Additionally, the boot disk

Edit the /etc/vfstab file
and restore the mount
could be corrupted causing the system to

become un-bootable.
statements for any non

system filesystems
resident on the boot disk.
Non system mount statements could include
home /export
,
Start all volumes.
/opt /
,
and others.
Mount all non system file

systems that may have
been resident on the
boot disk.
Rev-B
End
B-22
Page 21 of 26

07/18/2002

No
Current boot
disk backup?
starting.
Yes
Bring system to boot
prom and boot to single
user mode from
CDROM.

type to vt100 so vi will
work.
fsck root filesystem.

Mount root partition
(slice 0) to
/a

type to
vt100
so vi will
work.
Edit the
/etc/
system file and

comment "rootdev"
statements.
vi /a/etc/system
Do not skip this step. This copy is used to
restore the system back to it's original state
once system maintenance or troublrshooting
has been completed.
Make a backup copy of
/a/etc/vfstab
Page 23 of 26
Rev-B
07/18/2002
Page 22 of 26

B-23

executing in flow V was directed here.
/etc/vfsta
/, swap, /
usr and /var
Edit the
b file
and restore
to their pre
/etc/vfstab
/dev/vx
/ swap /usr
/var
W hen completed, the
encapsulation physical
should have all

device mount statements.
out and
file
devices commented
and
should be
mounting from their pre-encapsulation physical

Comment out all VxVM
managed volumes
vx...
/export
(/dev/
/opt
devices.
). This includes
and
if they exist.
Prevent VxVM software

from starting on the next
boot.
Remove
boot disk
encapsulation flag.
touch /a/etc/vx/reconfig.d/state.d/ \
install-db
rm /a/etc/vx/reconfig.d/state.d/ \
root-done
Reboot the system
The system is now in an
The system is now ready for troubleshooting or
unencapsulated
maintenance.
with /,
/var
state
swap /usr
,
m ounted.
software
and
VxVM
daem ons are
stopped.
All VxVM software devices are unavailable, this could

include
/opt /home /export

,
or other non-system file
system resident on the boot disk. If these filesystems are

needed, a full unencapsulation of the boot is necessary.
No
Maintenance
finished?
Yes
Restore the system to it's

encapsulated and VxVM
touch /etc/vx/revconfig.d/state.d/root-done
cp /etc/vfstab.orig /etc/vfstab
cp /etc/system.orig /etc/system
reboot
software enabled state.

Rev-B
End
B-24
Page 23 of 26

07/18/2002

executing in flow B was directed here.
Recommended Guidelines:
1. Use a maximum of three slices 2. Never configure
/usr
/, swap
and
/var
(optional).
as a separate slice.
3. Attach mirrors in geographical order not in alphabetical order.
vxdiskadm
attaches mirrors in alphabetical order. Do not use
to mirror the boot disk if
slices other than
/, swap and /var
vxdiskadm
builds and
are present.
4. Replace the encapsulated boot disk with an initialized disk. Replacing the
encapsulated disk with an initialized disk insures that the boot disk and the mirror disk
are identical. This reduces the complexity of the mirrored boot disk configuration.
/,
5. Map the core system volumes to slices/partitions. This should not be necessary if
swap
and
/var
are the only slices on the boot disk and you are using VxVM 3.x or
higher.
6. Ensure that all boot disk mirrors are bootable and have a devalias built in the
OpenBoot Prom.
7. Create a clone disk. This disk must be able to be booted from slices (not under VxVM
control, non encapsulated copy of the boot disk) and be able to run VxVM software
utilities (VxVM installed). This disk is used if there is a complete failure of the VxVM
managed boot disk making the system un-bootable.
Save the boot

disk
vtoc
# mkdir /var/tmp/sysdocs
# prtvtoc /dev/dsk/c#t#d#s2 > \
/var/tmp/sysdocs/rootdisk_prevm.vtoc
for
reference if
needed.
Encapsulate the boot

disk using
vxdiskadm
vxinstall
or
Initialize and add the

disk to be used as the
rootmirror
rootdg.
for
# /etc/vx/bin/vxdisksetup -i c#t#d#
# vxdg -g rootdg adddisk rootmirror=c#t#d#
Page 25 of 26
Rev-B
07/18/2002
Page 24 of 26

B-25
executing in flow X was directed here.
Attach the mirrors in

the geographic order
of the encapsulated
disk.
# /etc/vx/bin/vxrootmir rootmir
# vxassist -g rootdg mirror swapvol rootmirror&
# vxassist -g rootdg mirror var rootmirror&
Note: If the original boot disk was configured only using
and
/var
slices; mirroring using
vxdiskadm
/, swap
is acceptable.
Note: The following hand loop can be executed to display the

progress of the mirroring process:
while true
> do
> vxtask list
> sleep 15
> echo "#####################"
> done
W ait for the mirroring
process to complete
No
Mirroring
Process
Complete?
Yes
Dissociate the
rootdisk plexes
vxplex
vxples
vxplex
vxedit
Page 26 of
-g rootdg dis rootvol-01

-g rootdg dis swapvol-01
-g rootdg dis var-01
-fr rm rootvol-01 swapvol-01 var-01
26

Rev-B
07/18/2002
Page 25 of 26
B-26

executing in flow Y was directed here.
Remove the
rootdisk
rootdg
from
vxdg -g rootdg rootdisk
Initialize the
/etc/vx/bin/vxdisksetup -i c#t#d#
vxdg -g rootdg adddisk rootdisk=c#t#d#
Attach the mirrors in
/etc/vx/bin/cxrootmir rootmir
vxassist -g rootdg mirror swapvol rootmirror&
vxassist -g rootdg mirror var rootmirror&
rootdisk and add

it back to rootdg
the geographic order

of the encapsulated
disk.
If needed, create the

underlying system
partitions using
vxmksdpart.
# /usr/vx/bin/vxmksdpart -g rootdg rootdisk-03 \

6 0x07 0x00
(This example creates the overlay partition for /var if configured.)
Note: If
/, swap
and
/var
are the only slices configured on the
boot disk, vxdiskadm can be used to create the new mirror and
vxdiskadm will build all the overlay partitions making this step
unnecessary.
vxprint -qhtg
rootdg prior to executing the
vxmksdpart command to get the subdisk names for the overlay
Note: Run
partitions needing to be built.
Set the dump device

to a non VM disk if
# dumpadm -d /dev/dsk/c#t#d#s#
available.
Create the
OBP
device aliases if
necessary.

End
Rev-B
07/18/2002
Page 26 of 26

B-27
Appendix C
Example Five-Slice Boot Disk Encapsulation

Encapsulation of a boot disk that has only the /, swap, /usr, and /var
partitions is a straightforward process. The encapsulation process creates
the public and private regions needed by the VxVM software and an
overlap partition for each of the system partitions needed to boot the
system. Unencapsulation is also a straightforward process as well, as long
as the vxunroot utility is successful.
When a boot disk has other slices in addition to /, swap, /usr, and /var,
encapsulation is more complex. The main system partitions (/, swap,
/usr, and /var) are still visible, but the additional partitions, such as /opr
or /home, disappear. A similar result occurs during the data disk
encapsulation process. The data is not moved or overwritten. It is still
accessible by the VxVM software and mounted as a VxVM software
volume.

C-1
Figure C-1 illustrates an example partitioning scheme of a complex, fiveslice boot disk.
Boot Disk
Encapsulated Data Disk

rootvol
Slice 2
Slice 0 - /
Overlay Partition - / (0)
Slice 1 - swap
Slice 3 - /usr
Slice 4 - /var
Slice 5 - /opt
Slice7
Private Region
Free Space
swapvol
Slice 2 and
Slice 6
(Public Region)
Overlaps
Full Disk.
/opt is
Encapsulated
in Public Region
Mirror Disk
rootvol
Preserved
/opt
Overlay Partition - /opt (5)

swapvol
Overlay Partition - / (0)
Slice 4
(Public Region)
Slice 2
Figure C-1
C-2
Five-Slice Boot Disk Encapsulation


The following are example configuration data from a five-slice boot disk
prior to encapsulation.

Output from the following example of the prtvtoc command clearly
shows five slices and sufficient free space for encapsulation.
*
* Dimensions:
*
512 bytes/sector
*
133 sectors/track
*
27 tracks/cylinder
*
*
4926 cylinders
*
*
* Flags:
*
1: unmountable
* 10: read-only
*
*
First
Sector
Last
*
Sector
Count
Sector
*
16795107
886977 17682083
*
*
First
Sector
Last
Sector
Count
Sector
0
2
00
0
2100735
2100734
1
3
01
2100735
2100735
4201469
2
5
00
0 17682084 17682083
3
4
00
4201470
4197879
8399348
4
7
00
8399349
4197879 12597227
5
0
00
12597228
4197879 16795106
Mount Directory
/
/usr
/var
/opt

C-3
Pre-Encapsulation format Print Utility

A description of the example disk from the format utility delineates each
partition on a cylinder boundary. Notice that there are two free partitions
for use by the encapsulation process to assign to the public and private
regions.
partition> print
Part
Tag
0
root
1
swap
2
backup
3
usr
4
var
5 unassigned
6 unassigned
7 unassigned
Flag
wm
wu
wm
wm
wm
wm
wm
wm
Cylinders
0 - 584
585 - 1169
0 - 4923
1170 - 2338
2339 - 3507
3508 - 4676
0
0
Size
1.00GB
1.00GB
8.43GB
2.00GB
2.00GB
2.00GB
0
0
Blocks
(585/0/0)
2100735
(585/0/0)
2100735
(4924/0/0) 17682084
(1169/0/0) 4197879
(1169/0/0) 4197879
(1169/0/0) 4197879
(0/0/0)
0
(0/0/0)
0
Example output from the df -k command delineates the physical devices
supporting currently mounted file systems.
bash-2.03# df -k
Filesystem
/dev/dsk/c1t2d0s0
/dev/dsk/c1t2d0s3
/proc
fd
mnttab
/dev/dsk/c1t2d0s4
swap
swap
/dev/dsk/c1t2d0s5
kbytes
1018382
2055705
0
0
0
2055705
1222104
1222104
2055705
used
avail capacity
47206 910074
5%
772430 1221604
39%
0
0
0%
0
0
0%
0
0
0%
956248 1037786
48%
16 1222088
1%
16 1222088
1%
2133 1991901
1%
Mounted on
/
/usr
/proc
/dev/fd
/etc/mnttab
/var
/var/run
/tmp
/opt
Pre-Encapsulation swap -l Command

Output from the swap -l command for this example shows that slice one
of the boot disk is used as a swap device.
bash-2.03# swap -l
swapfile
/dev/dsk/c1t2d0s1
C-4
dev swaplo blocks

free
118,145
16 2100704 2100704


The following output was captured after encapsulation of a five-slice boot
disk.
Post-Encapsulation prtvtoc /dev/rdsk/c1t2d0s2

Command
The following example displays post-encapsulation partitioning for the
boot disk. The /, /usr, /var, and swap partitions inhabit their original
locations on the boot disk and are represented as overlay partitions.
Partition 5, which holds the /opt file system, is encapsulated into the
public region (slice 6) and does not have an overlay partition assigned.
Since /opt is not needed to boot the system, it is encapsulated just like
any other data partition.
*
* Dimensions:
*
512 bytes/sector
*
133 sectors/track
*
27 tracks/cylinder
*
*
4926 cylinders
*
*
* Flags:
*
1: unmountable
* 10: read-only
*
*
First
Sector
Last
*
Sector
Count
Sector
*
2100735 4292866561 4294967295
*
17682084 4279385947
2100734
*
12597228
5081265 17678492
*
*
First
Sector
Sector
Count
0
2
00
0
2100735
1
3
01
2100735
2100735
2
5
00
0 17682084
3
4
00
4201470
4197879
4
7
00
8399349
4197879
6
14
01
0 17682084
7
15
01
17678493
3591
Last
Sector
2100734
4201469
17682083
8399348
12597227
17682083
17682083
Mount Directory

C-5
Post-Encapsulation format print Utility

An example of output from the format utility shows the partitioning
scheme on cylinder boundaries and mount points for the overlay
partitions. Notice that partition 5, /opt, is unassigned with a zero block
size. It was encapsulated into the disks public region.
partition> print
Part
Tag
0
root
1
swap
2
backup
3
usr
4
var
5 unassigned
6
7
-
Flag
wm
wu
wm
wm
wm
wm
wu
wu
Cylinders
0 - 584
585 - 1169
0 - 4923
1170 - 2338
2339 - 3507
0
0 - 4923
4923 - 4923
Size
1.00GB
1.00GB
8.43GB
2.00GB
2.00GB
0
8.43GB
1.75MB
Blocks
(585/0/0)
2100735
(585/0/0)
2100735
(4924/0/0) 17682084
(1169/0/0) 4197879
(1169/0/0) 4197879
(0/0/0)
0
(4924/0/0) 17682084
(1/0/0)
3591
The following example of output from the df -k command shows that
the VxVM software volumes are used for the file systems provided by the
boot disk.
bash-2.03# df -k
Filesystem
/dev/vx/dsk/rootvol
/dev/vx/dsk/usr
/proc
fd
mnttab
/dev/vx/dsk/var
swap
swap
/dev/vx/dsk/opt
kbytes
1018382
2055705
0
0
0
2055705
1180368
1180424
2055705
used
75398
805992
0
0
0
974276
16
72
53686
avail capacity
881882
8%
1188042
41%
0
0%
0
0%
0
0%
1019758
49%
1180352
1%
1180352
1%
1940348
3%
Mounted on
/
/usr
/proc
/dev/fd
/etc/mnttab
/var
/var/run
/tmp
/opt
Post-Encapsulation vxprint Command

An example of the vxprint output displays all the relevant VxVM
software information about this encapsulated boot disk, its volumes, and
its mirrors. Information from this command is important if this disk needs
to be manually unencapsulated.
C-6


dg rootdg
default
default
1020469848.1025.lowtide5
dm rootdisk
dm rootmirror
3590
3590
17678493 17678493 -
c1t2d0s2
c1t20d0s2
sliced
sliced
v
pl
sd
pl
sd
opt
opt-01
opt
rootdisk-03 opt-01
opt-02
opt
rootmirror-01 opt-02
ENABLED ACTIVE
ENABLED ACTIVE
rootdisk 12597227
ENABLED ACTIVE
rootmirror 0
v
pl
sd
sd
pl
sd
rootvol
rootvol-01
rootvol
rootvol-02
rootvol
v
pl
sd
pl
sd
4197879
4197879
4197879
4197879
4197879
ROUND
CONCAT
0
CONCAT
0
c1t2d0
c1t20d0
gen
RW
ENA
RW
ENA
ENABLED ACTIVE
2100735
ENABLED ACTIVE
2100735
rootdisk 17678492 1
rootdisk 0
2100734
ENABLED ACTIVE
2100735
rootmirror 4197879 2100735
ROUND
CONCAT
0
1
CONCAT
0
c1t2d0
c1t2d0
c1t20d0
root
RW
ENA
ENA
RW
ENA
swapvol
swapvol-01
swapvol
swapvol-02
swapvol
ENABLED ACTIVE
2100735
ENABLED ACTIVE
2100735
rootdisk 2100734 2100735
ENABLED ACTIVE
2100735
rootmirror 6298614 2100735
ROUND
CONCAT
0
CONCAT
0
c1t2d0
c1t20d0
swap
RW
ENA
RW
ENA
v
pl
sd
pl
sd
usr
usr-01
usr
rootdisk-05 usr-01
usr-02
usr
ENABLED ACTIVE
4197879
ENABLED ACTIVE
4197879
rootdisk 4201469 4197879
ENABLED ACTIVE
4197879
rootmirror 8399349 4197879
ROUND
CONCAT
0
CONCAT
0
c1t2d0
c1t20d0
gen
RW
ENA
RW
ENA
v
pl
sd
pl
sd
var
var-01
var
rootdisk-04 var-01
var-02
var
ENABLED ACTIVE
4197879 ROUND
ENABLED ACTIVE
4197879 CONCAT
rootdisk 8399348 4197879 0
ENABLED ACTIVE
4197879 CONCAT
rootmirror 12597228 4197879 0
c1t2d0
c1t20d0
gen
RW
ENA
RW
ENA

The /etc/vfstab file in the following example was modified to reflect
the encapsulated volumes. A copy of the original was saved to the
/etc/vfstab.prevm directory.
bash-2.03# more /etc/vfstab
#device
device
mount
#to mount
to fsck
point
#
fd
/dev/fd fd
/proc
/proc
proc
/dev/vx/dsk/swapvol
-
no
no
swap
FS
type
fsck
pass
mount
mount
at boot options
ufs
-
yes
no

C-7

/dev/vx/dsk/rootvol
/
ufs
/dev/vx/dsk/usr /dev/vx/rdsk/usr
/usr
ufs
1
/dev/vx/dsk/var /dev/vx/rdsk/var
/var
ufs
1
/dev/vx/dsk/opt /dev/vx/rdsk/opt
/opt
ufs
2
swap
/tmp
tmpfs
yes
#NOTE: volume opt (/opt) encapsulated partition c1t2d0s5
1
no
no
yes
no
-
Post-Encapsulation Mirror Disk prtvtoc Command

The partitioning scheme in the following example of the mirror disk for a
five-slice boot disk is very different from that of a two-slice boot disk.
Notice that there is a relocation of overlay partitions. Partition 5, which is
/opt, was moved to the front of the disk after the private region. This
preserves the contents of that encapsulated slice and allows recovery of
that partitions data if the original disk fails and is replaced.
*
* Dimensions:
*
512 bytes/sector
*
133 sectors/track
*
27 tracks/cylinder
*
*
4926 cylinders
*
*
* Flags:
*
1: unmountable
* 10: read-only
*
*
First
Sector
Last
*
Sector
Count
Sector
*
17682084 4277288803
3590
*
16798698
883386 17682083
*
*
First
Sector
Sector
Count
0
2
00
4201470
2100735
1
3
01
6302205
2100735
2
5
01
0 17682084
3
15
01
0
3591
4
14
01
3591 17678493
5
0
00
3591
4197879
6
4
00
8402940
4197879
7
7
00
12600819
4197879
C-8
Last
6302204 <-- /
8402939 <-- swap
17682083 <-- overlap/backup
3590 <-- Private
17682083 <-- Public
4201469 <-- /opt
12600818 <-- /usr
16798697 <-- /var

Post-Encapsulation Mirror Disk format Print Utility

The following example of output from the format utility displays the
partitioning scheme for this disk on cylinder boundaries.
partition> print
Part
Tag
0
root
1
swap
2
backup
3
4
5 unassigned
6
usr
7
var
Flag
wm
wu
wu
wu
wu
wm
wm
wm
Cylinders
1170 - 1754
1755 - 2339
0 - 4923
0 0
1 - 4923
1 - 1169
2340 - 3508
3509 - 4677
Size
1.00GB
1.00GB
8.43GB
1.75MB
8.43GB
2.00GB
2.00GB
2.00GB
Blocks
(585/0/0)
2100735
(585/0/0)
2100735
(4924/0/0) 17682084
(1/0/0)
3591
(4923/0/0) 17678493
(1169/0/0) 4197879 <-- /opt
(1169/0/0) 4197879
(1169/0/0) 4197879

C-9
Example Five-Slice Boot Disk Unencapsulation
Example Five-Slice Boot Disk Unencapsulation

Unencapsulating a five-slice boot disk is similar to unencapsulating a twoor four-slice disk. The most direct and easiest method is to use the
vxunroot command. If that fails, use one of the three remaining manual
methods to successfully complete the unencapsulation.
Unencapsulating Using the vxunroot Command

The unencapsulation process for a five-slice disk is identical to the
vxunroot procedure described in Unencapsulating a Boot Disk Using
the vxunroot Utility on page 2-57.
Manually Unencapsulating
The manual unencapsulation process for a five-slice disk is similar to the
manual procedure described in Manually Unencapsulating a Boot Disk
on page 2-60. The one difference in the process of unencapsulating a fiveslice boot disk is the recovery of the /opt partition; this procedure is
covered as part of the manual unencapsulation procedure.
Additionally, the procedures in Unencapsulating When Booted From the
CD-ROM on page 2-64, and Performing a Basic or Functional
Unencapsulation on page 2-68 are similar for a five-slice boot disk.
Execute these procedures as written in the referenced sections. Be sure to
recover the /opt partition using the instructions for one of the following
commands:
C-10
The vxmksdpart command See step 6.c on page 2-62.
The vxedvtoc command See step 19.b on page 2-67.

Appendix D
The Boot Process

This appendix contains an annotated boot process code list that identifies
when each of the VxVM software start-up scripts execute during the
system boot process.
{0} ok boot -v
Rebooting with command: boot -v
Boot device: /sbus@3,0/SUNW,socal@0,0/sf@0,0/ssd@w220000203713fc9f,0:a File and args: v
Size: 339096+90034+75930 Bytes
SunOS Release 5.8 Version Generic_108528-14 64-bit
Copyright 1983-2001 Sun Microsystems, Inc. All rights reserved.
Ethernet address = 8:0:20:7d:5d:60
mem = 262144K (0x10000000)
avail mem = 232955904
root nexus = 8-slot Sun Enterprise 4000/5000
sbus0 at root: UPA 0x2 0x0 ...
sbus0 is /sbus@2,0
sbus1 at root: UPA 0x3 0x0 ...
sbus1 is /sbus@3,0
socal0 at sbus1: SBus1 slot 0x0 offset 0x0 and slot 0x0 offset 0x10000 and slot 0x0
offset 0x20000 SBus level 3 sparc9 ipl 5
socal0 is /sbus@3,0/SUNW,socal@0,0
sf0 at socal0: socal_port 0
sf0 is /sbus@3,0/SUNW,socal@0,0/sf@0,0
sf1 at socal0: socal_port 1
sf1 is /sbus@3,0/SUNW,socal@0,0/sf@1,0
soc0 at sbus0: SBus0 slot 0xd offset 0x10000 Onboard device sparc9 ipl 5
soc0 is /sbus@2,0/SUNW,soc@d,10000
ssd0 at sf1: name w210000203713f582,0, bus address c7
ssd0 is /sbus@3,0/SUNW,socal@0,0/sf@1,0/ssd@w210000203713f582,0
<SUN9.0G cyl 4924 alt 2 hd 27 sec 133>
/sbus@3,0/SUNW,socal@0,0/sf@1,0/ssd@w210000203713f582,0 (ssd0) online
ssd2 at sf1: name w210000203713e0b9,0, bus address e2
ssd2 is /sbus@3,0/SUNW,socal@0,0/sf@1,0/ssd@w210000203713e0b9,0
/sbus@3,0/SUNW,socal@0,0/sf@1,0/ssd@w210000203713e0b9,0 (ssd2) online
ssd1 at sf1: name w210000203713f643,0, bus address cc

D-1
ssd3 at sf1: name w210000203713f565,0, bus address e1

ssd4 at sf1: name w210000203713fc9f,0, bus address ef
ssd4 is /sbus@3,0/SUNW,socal@0,0/sf@1,0/ssd@w210000203713fc9f,0
/sbus@3,0/SUNW,socal@0,0/sf@1,0/ssd@w210000203713fc9f,0 (ssd4) online
ssd5 at sf1: name w210000203713df01,0, bus address dc
ssd5 is /sbus@3,0/SUNW,socal@0,0/sf@1,0/ssd@w210000203713df01,0
/sbus@3,0/SUNW,socal@0,0/sf@1,0/ssd@w210000203713df01,0 (ssd5) online
ssd6 at sf1: name w210000203713f96d,0, bus address e8
ssd6 is /sbus@3,0/SUNW,socal@0,0/sf@1,0/ssd@w210000203713f96d,0
/sbus@3,0/SUNW,socal@0,0/sf@1,0/ssd@w210000203713f96d,0 (ssd6) online
ssd7 at sf1: name w210000203713f7f0,0, bus address cb
ssd7 is /sbus@3,0/SUNW,socal@0,0/sf@1,0/ssd@w210000203713f7f0,0
/sbus@3,0/SUNW,socal@0,0/sf@1,0/ssd@w210000203713f7f0,0 (ssd7) online
ssd8 at sf1: name w21000020372d5917,0, bus address c9
ssd8 is /sbus@3,0/SUNW,socal@0,0/sf@1,0/ssd@w21000020372d5917,0
/sbus@3,0/SUNW,socal@0,0/sf@1,0/ssd@w21000020372d5917,0 (ssd8) online
ssd9 at sf1: name w210000203713fc0f,0, bus address c6
ssd10 at sf1: name w2100002037140269,0, bus address cd
ssd10 is /sbus@3,0/SUNW,socal@0,0/sf@1,0/ssd@w2100002037140269,0
/sbus@3,0/SUNW,socal@0,0/sf@1,0/ssd@w2100002037140269,0 (ssd10) online
ssd12 at sf1: name w210000203716cf3d,0, bus address e0
ssd12 is /sbus@3,0/SUNW,socal@0,0/sf@1,0/ssd@w210000203716cf3d,0
/sbus@3,0/SUNW,socal@0,0/sf@1,0/ssd@w210000203716cf3d,0 (ssd12) online
ssd13 at sf1: name w210000203713f49f,0, bus address ca
ssd13 is /sbus@3,0/SUNW,socal@0,0/sf@1,0/ssd@w210000203713f49f,0
/sbus@3,0/SUNW,socal@0,0/sf@1,0/ssd@w210000203713f49f,0 (ssd13) online
ssd14 at sf0: name w220000203713f582,0, bus address c7
ssd15 at sf0: name w220000203713f643,0, bus address cc
ssd16 at sf0: name w220000203713e0b9,0, bus address e2
ssd16 is /sbus@3,0/SUNW,socal@0,0/sf@0,0/ssd@w220000203713e0b9,0
D-2


/sbus@3,0/SUNW,socal@0,0/sf@0,0/ssd@w220000203713e0b9,0 (ssd16) online
ssd18 at sf0: name w220000203713fc9f,0, bus address ef
ssd19 at sf0: name w220000203713df01,0, bus address dc
ssd19 is /sbus@3,0/SUNW,socal@0,0/sf@0,0/ssd@w220000203713df01,0
/sbus@3,0/SUNW,socal@0,0/sf@0,0/ssd@w220000203713df01,0 (ssd19) online
ssd20 at sf0: name w220000203713f96d,0, bus address e8
ssd20 is /sbus@3,0/SUNW,socal@0,0/sf@0,0/ssd@w220000203713f96d,0
/sbus@3,0/SUNW,socal@0,0/sf@0,0/ssd@w220000203713f96d,0 (ssd20) online
ssd21 at sf0: name w220000203713f7f0,0, bus address cb
ssd21 is /sbus@3,0/SUNW,socal@0,0/sf@0,0/ssd@w220000203713f7f0,0
/sbus@3,0/SUNW,socal@0,0/sf@0,0/ssd@w220000203713f7f0,0 (ssd21) online
ssd22 at sf0: name w22000020372d5917,0, bus address c9
ssd22 is /sbus@3,0/SUNW,socal@0,0/sf@0,0/ssd@w22000020372d5917,0
/sbus@3,0/SUNW,socal@0,0/sf@0,0/ssd@w22000020372d5917,0 (ssd22) online
ssd23 at sf0: name w220000203713fc0f,0, bus address c6
ssd24 at sf0: name w2200002037140269,0, bus address cd
ssd24 is /sbus@3,0/SUNW,socal@0,0/sf@0,0/ssd@w2200002037140269,0
/sbus@3,0/SUNW,socal@0,0/sf@0,0/ssd@w2200002037140269,0 (ssd24) online
ssd26 at sf0: name w220000203716cf3d,0, bus address e0
ssd26 is /sbus@3,0/SUNW,socal@0,0/sf@0,0/ssd@w220000203716cf3d,0
/sbus@3,0/SUNW,socal@0,0/sf@0,0/ssd@w220000203716cf3d,0 (ssd26) online
ssd27 at sf0: name w220000203713f49f,0, bus address ca
ssd27 is /sbus@3,0/SUNW,socal@0,0/sf@0,0/ssd@w220000203713f49f,0
/sbus@3,0/SUNW,socal@0,0/sf@0,0/ssd@w220000203713f49f,0 (ssd27) online
/sbus@3,0/SUNW,fas@3,8800000 (fas0):
rev 2.2 FEPS chip
fas0 at sbus1: SBus1 slot 0x3 offset 0x8800000 and slot 0x3 offset 0x8810000 SBus level
3 sparc9 ipl 5
fas0 is /sbus@3,0/SUNW,fas@3,8800000
sd6 at fas0: target 6 lun 0
sd6 is /sbus@3,0/SUNW,fas@3,8800000/sd@6,0
root on /pseudo/vxio@0:0 fstype ufs

The Boot Process
D-3
WARNING: forceload of drv/SUNW,socal failed

fhc0 at root: UPA 0x0 0xf8800000
fhc0 is /fhc@0,f8800000
ac0 board 0 bank 0: base 0x0 size 256mb rstate 2 ostate 1 condition 1
ac0 is /fhc@0,f8800000/ac@0,1000000
fhc1 is /fhc@2,f8800000
ac1 is /fhc@2,f8800000/ac@0,1000000
central0 at root: UPA 0x1f 0x0 ...
fhc2 is /central@1f,0/fhc@0,f8800000
WARNING: Core Power Supply 3 Failing
sysctrl0 is /central@1f,0/fhc@0,f8800000/clock-board@0,900000
sysctrl0: Key switch is not in the secure position
environ0 is /fhc@0,f8800000/environment@0,400000
environ1 is /fhc@2,f8800000/environment@0,400000
simmstat0 is /fhc@0,f8800000/simm-status@0,600000
sram0 is /fhc@0,f8800000/sram@0,200000
zs0 is /central@1f,0/fhc@0,f8800000/zs@0,902000
zs1 is /central@1f,0/fhc@0,f8800000/zs@0,904000
cpu0: SUNW,UltraSPARC (upaid 0 impl 0x10 ver 0x40 clock 168 MHz)
cpu1: SUNW,UltraSPARC (upaid 1 impl 0x10 ver 0x40 clock 168 MHz)
Starting /etc/rcS.d/S25vxvm-sysboot script
Starting VxVM restore daemon...
VxVM starting in boot mode...
ses16 at sf1: name w5080020000034909,0, bus address d2
ses16 is /sbus@3,0/SUNW,socal@0,0/sf@1,0/ses@w5080020000034909,0
ses17 at sf1: name w508002000003490a,0, bus address b5
ses17 is /sbus@3,0/SUNW,socal@0,0/sf@1,0/ses@w508002000003490a,0
ses18 at sf0: name w508002000003490b,0, bus address d2
ses18 is /sbus@3,0/SUNW,socal@0,0/sf@0,0/ses@w508002000003490b,0
ses19 at sf0: name w508002000003490c,0, bus address b5
ses19 is /sbus@3,0/SUNW,socal@0,0/sf@0,0/ses@w508002000003490c,0
NOTICE: vxvm:vxdmp: enabled path 118/0x20 belonging to the dmpnode 68/0x0
NOTICE: vxvm:vxio: Cannot open disk c1t3d0s2: kernel error 6
SUNW,hme0 : Sbus (Rev Id = 22) Found
hme0 at sbus0: SBus0 slot 0x1 offset 0x8c00000 and slot 0x1 offset 0x8c02000 and
0x1 offset 0x8c04000 and slot 0x1 offset 0x8c06000 and slot 0x1 offset 0x8c07000
level 4 sparc9 ipl 7
hme0 is /sbus@2,0/SUNW,hme@1,8c00000
SUNW,hme1 : Sbus (Rev Id = 22) Found
hme1 at sbus1: SBus1 slot 0x3 offset 0x8c00000 and slot 0x3 offset 0x8c02000 and
0x3 offset 0x8c04000 and slot 0x3 offset 0x8c06000 and slot 0x3 offset 0x8c07000
level 4 sparc9 ipl 7
hme1 is /sbus@3,0/SUNW,hme@3,8c00000
configuring IPv4 interfaces: hme0.
configuring IPv6 interfaces: hme0.
Hostname: lowtide
Executing /etc/rcS.d/S35vxvm-startup1 Script
D-4

slot
SBus
slot
SBus
VxVM starting special volumes ( swapvol var )...

Executing /etc/rcS.d/standardmounts Script
SUNW,hme0 : Internal Transceiver Selected.
SUNW,hme0 :
100 Mbps Full-Duplex Link Up
Executing /etc/rcS.d/S50devfsadm Script
Executing /etc/rcS.d/S85vxvm-startup2 Script
pseudo-device: devinfo0
devinfo0 is /pseudo/devinfo@0
VxVM general startup...
dump on /dev/dsk/c1t0d0s1 size 513 MB
Executing /etc/rcS.d/S86vxvm-reconfig Script
The system is coming up.
Please wait.
Executing /etc/rc2.d/S20sysetup Script

Starting IPv6 neighbor discovery.
Setting default IPv6 interface for multicast: add net ff00::/8: gateway
fe80::a00:20ff:fe7d:5d60
starting rpc services: rpcbind done.
Setting netmask of hme0 to 255.255.255.0
Setting default IPv4 interface for multicast: add net 224.0/4: gateway lowtide
syslog service starting.
Print services started.
Jun 24 14:39:05 lowtide pseudo: pseudo-device: tod0
Jun 24 14:39:05 lowtide genunix: tod0 is /pseudo/tod@0
Jun 24 14:39:05 lowtide pseudo: pseudo-device: pm0
Jun 24 14:39:05 lowtide genunix: pm0 is /pseudo/pm@0
volume management starting.
Executing /etc/rc2.d/S94vxnm-host_infod Script
Executing /etc/rc2.d/S94vxnm-vxnetd Script
Jun 24 14:39:07 lowtide pseudo: pseudo-device: vol0
Jun 24 14:39:07 lowtide genunix: vol0 is /pseudo/vol@0
Executing /etc/rc2.d/S95vxvm-recover Script
Executing /etc/rc2.d/S96vmsa-server Script

The Boot Process
D-5
Starting RMI Registry

Starting VERITAS VM Storage Administrator Command Server
Starting VERITAS VM Storage Administrator Server
The system is ready.
lowtide console login:
D-6

Appendix E
Configuring the VxVM Software

Objectives
This appendix contains information related to VxVM software device
configuration tasks.

E-1
Relevance
Relevance
!
?
Discussion Modules in this course discussed the basic elements and

architecture of the VxVM software. This appendix provides detailed
information on how to build and assess VxVM software configurations in
an environment, and how to determine the effectiveness of those
configurations.
The following questions are relevant to the material presented in this
appendix:
E-2
What are important factors to look at when formulating a VxVM

software solution?
Can you identify the standards for RAID installations?

details on the topics discussed in this module:

VERITAS Volume Manager 3.2 SRT Instructor Guide. Mountain View,

California: VERITAS Software Corporation.
VERITAS Volume Manager 3.2 Release Notes. Mountain View,

SunSolveSM Online INFODOC 40143,

Man pages for the vxassist and vxdg commands.

E-3
Supported RAID Levels
Supported RAID Levels

The VxVM software supports the following levels of RAID:
RAID 0 Uses striping
RAID 1 Uses mirroring
RAID 0 + RAID 1 Uses striping and mirroring
RAID 1 + RAID 0 Uses mirroring and stripingreferred to as

RAID 10
RAID 5 Uses parity striping
This module describes the differences among these RAID configurations.
Layered Volumes
Prior to version 3.2, the VxVM software applied default rules to assign
disks using the vxassist command. The default rules essentially created
the RAID stripe and then the mirror. The VxVM software version 3.0
introduced layered volumes, allowing for RAID 1 + RAID 0 (mirror and
striping) configurations. This feature is now fully implemented in VxVM
software version 3.2 through the vxassist command -o ordered option.
Ordered Allocation of Storage

When the -o ordered option of the vxassist command is used, the
default configuration function creates volumes in the following order:
1.
Concatenate disks
2.
Form columns
3.
Form mirrors
The VxVm software version 3.2 provides configuration control, allowing

the selection of either the default configuration (RAID 0 + 1) or a layered
volumes configuration (RAID 1 + 0), also referred to as a stripe
professional or concat professional configuration.
E-4

VxVM Software States

State information is critical to understanding the health of a configuration.
This section defines the various state information for plexes, volumes, and
disks.
Plex States
Plex state information reflects consistent or inconsistent configurations
and the state of those configurations.
Valid plex states are described in detail in Module 1, Introducing the
VERITAS Volume Manager Software Architecture, in the section Plex
State Descriptions on page 1-34.
Plex and Volume Kernel States

The plex and volume kernel states indicate the accessibility of the plex.
Plex kernel states are described in detail in Module 1, Introducing the
VERITAS Volume Manager Software Architecture, in the section Plex
Kernel States on page 1-36. Volume kernel states are described in the
same manner as plex kernel state.
Volume States
Volume states consist of the following:
Clean Volume is not started; kernel state is disabled, but plexes are
synchronized.
Active Volume is started; kernel state is enabled.
Empty Volume is not initialized; kernel state is disabled.
Sycn Volume is in recovery mode; kernel state is enabled. Also

indicates volume is recovered after boot; kernel state is disabled, and
plexes need to be resynchronized.
Needsync Volume requires resynchronization.
Replay Volume is in transient state as part of log replay (only valid

for RAID 5).

E-5
Disk States
The following example shows output of the vxdisk list command:
DEVICE
c0t0d0s2
c0t2d0s2
c1t0d0s2
c2t0d0s2
c3t0d1s2
c3t0d4s2
c3t1d1s2
c3t2d0s2
c3t2d1s2
c3t2d2s2
c3t3d0s2
c3t3d2s2
c3t3d3s2
c3t4d0s2
c3t4d3s2
c3t5d2s2
-
TYPE
DISK
GROUP
STATUS
sliced disk06
online
sliced error
sliced rootdisk rootdg online
sliced error
sliced online
sliced rootmir rootdg online
sliced newdg01 newdg
online spare
online spare
sliced disk01
rootdg online
online reserved
online nohotuse
online invalid
sliced error
online failing
online altused
online
newdg03 newdg removed was:c3t2d1s2
newdg08 newdg removed was:c3t3d3s2
newdg11 newdg failed was:c3t4d2s2
As shown in the previous example, disk states are the following:
E-6
Invalid A private area exists, but the information in it is not a valid

configuration.
Altused The alternate configuration copy in the private area is in

use.
Failed was Indicates full disk failure.
Removed was The vxdiskadm #4 command was executed.
Error No private area on this disk.
Reserved This disk is not used by the vxassist command to make

new volumes or for relocation.
Failing (VxVM software version 2.x and 3.x) This disk incurred a
hardware error in the past.
Nohotuse (VxVM software version 3.x) This disk is not used for
relocated data.
Dgname (VxVM software version 3.x) The disk group name, shown
only when using the -o alldgs option.

The vxprint Command
The vxprint Command

Execution of the vxprint command displays the state of a RAID
configuration, providing an overview and the current state of the
configuration.
Output Header Information

The command output header contains important information. An
example of output from the vxprint command is shown:
DG NAME
NCONFIG
NLOG
DM NAME
DEVICE
TYPE
RV NAME
RLINK_CNT
KSTATE
RL NAME
RVG
KSTATE
V NAME
RVG
KSTATE
PL NAME
VOLUME
KSTATE
SD NAME
PLEX
DISK
SV NAME
PLEX
VOLNAME
DC NAME
PARENTVOL
LOGVOL
SP NAME
SNAPVOL
DCO
MINORS
PRIVLEN
STATE
STATE
STATE
STATE
DISKOFFS
NVOLLAYR
GROUP-ID
PUBLEN
PRIMARY
REM_HOST
LENGTH
LENGTH
LENGTH
LENGTH
STATE
DATAVOLS
REM_DG
READPOL
LAYOUT
[COL/]OFF
[COL/]OFF
SRL
REM_RLNK
PREFPLEX
NCOL/WID
DEVICE
AM/NM
dg rootdg
default
default
1023554924.1025.lowtide
dm rootdisk
dm rootmirror
c1t0d0s2
c1t1d0s2
sliced
sliced
3590
3590
17678493 17674902 -
v
pl
sd
sd
pl
sd
rootvol
rootvol-01
rootvol
rootvol-02
rootvol
ENABLED ACTIVE
ENABLED ACTIVE
rootdisk 17678492
rootdisk 0
ENABLED ACTIVE
rootmirror 0
4197879
4197879
1
4197878
4197879
4197879
ROUND
CONCAT
0
1
CONCAT
0
c1t0d0
c1t0d0
c1t1d0

UTYPE
MODE
MODE
MODE
root
RW
ENA
ENA
RW
ENA
E-7
The vxprint Command
Field Descriptions
This section contains an explanation of the fields in the vxprint output.
For additional information about the vxprint command, see the vxprint
man page.
E-8
Record type dg:
Disk group name
Disk ID
Record type dm:
Record name
Underlying disk access record
Disk access record type (sliced, simple, or nopriv)
Length of the disks private region
Length of the disks public region
Record type sd:
Record name
Associated plex, or dash (-) if the subdisk is dissociated
Name of the disk media record used by the subdisk
Device offset in sectors
Subdisk length in sectors
Plex association offset Optionally, this value is preceded by

subdisk column number for subdisks associated to striped
plexes, LOG for log subdisks, or the putil[0] field if the
subdisk is dissociated. The putil[0] field can be non-empty to
reserve the subdisk space for non-volume uses. If the putil[0]
field is empty, it is a dissociated subdisk.
Subdisk state string:
ENA The subdisk is usable.
DIS The subdisk is disabled.
RCOV The subdisk is part of a RAID-5 plex and has stale

content.
DET The subdisk is detached.
KDET The subdisk is detached in the kernel due to an

error.

The vxprint Command
RMOV The media record on which the subdisk is defined

was removed from its disk access record by a utility.
RLOC The subdisk has failed and is waiting to be

relocated.
NDEV The media record on which the subdisk is defined

has no associated access record.
Record type sv:
Record name.
Associated plex, or dash (-) if the subvolume is dissociated.
Name of the underlying (layered) volume record used by the

subvolume.
Number of layers used in the subvolume.
Subvolume length in sectors.
Plex association offset This value is optionally preceded by

subvolume column number for subvolumes associated with
striped plexes.
Number of active plexes, followed by the number of plexes in

the underlying (layered) volume.
Subvolume state string:
ENA The subvolume is usable.
DIS The subvolume is disabled.
KDET The subvolume was detached in the kernel due to

an error.
Record type pl:
Record name.
Associated volume, or dash (-) if the plex is dissociated.
Plex kernel state.
Plex utility state If an exception condition is recognized on the

plex (such as an I/O failure, a removed or inaccessible disk, or
an unrecovered stale data condition), that condition is listed
instead of the value of the plex records state field.
Plex length in sectors.
Plex layout type.

E-9
The vxprint Command
Number of columns and plex stripe width, or if the plex is not

striped
Plex I/O mode:
RW Read-Write
WO Write only
RO Read only
For volumes, the output consists of the following fields, from left to right:
E-10
Record type v:
Record name
Associated usage type
Volume kernel state
Volume utility state
Volume length in sectors
Volume read policy
The preferred plex, if the read policy uses a preferred plex
Record type dc:
Record name
Associated volume, or dash (-) if the data change object (DCO)

is dissociated
Name of the DCO log volume, or dash (-) if no DCO log

volume is associated with the DCO object
Record type sp:
Record name
Name of the volume which this snap record describes
Name of the DCO with which this snap record is associated
Record type rv:
Record name
Associated remote link (RLINK) object count
Remote volume group (RVG) kernel state (derived from various

flags)
RVG utility state
RVG primary flag (primary or secondary)

The vxprint Command
Associated data volume count
The srl volume
Record type rl:
Record name
Associated RVG, or dash (-) if the RLINK is dissociated
RLINK kernel state (derived from various flags)
RLINK utility state
The remote host
The remote disk group
The remote RLINK
Stripe Width and Stripe Units

Data is allocated in equal-sized units, called stripe units, that are
interleaved between the columns. Each stripe unit is a set of contiguous
blocks on a disk. The total number of blocks defined by the value stripe
units determine the per-column stripe width. When used during volume
creation, the terms stripe unit and stripe width are synonymous.
Differentiation between the two terms occurs at the command level.
The vxassist command option stripeunit defines stripe size. The
vxmake command option stwidth (for stripe width) defines stripe size.
The vxassist command builds the plex and volume in a single
command. The vxmake command builds individual plexes that are then
associated to volumes using additional iterations of the vxmake command.
The following example commands create a stripe of four columns with a
stripe size of 32 kilobytes:
# vxassist make stripevol 80g layout=stripe stripeunit=32k ncols=4
# vxmake plex pl-01 80g layout=stripe stwidth=32k ncols=4 \
sd=disk01-01,disk02-01,disk03-01,disk04-01

E-11
Complex RAID Levels
Complex RAID Levels

This section describes the concepts and commands used to build the
concat-pro and stripe-pro volumes, also known as RAID 1 + 0, or RAID 10.
In previous releases, when mirroring was used, the mirroring had to occur
above striping. In the VxVM software version 3.2, mirroring can occur
both above and below striping. Mirroring put below striping mirrors each
column of the stripe. If the stripe is large enough to have multiple
subdisks per column, each subdisk can be individually mirrored. Instead
of forming subdisks into plexes and then mirroring the plexes, RAID 1 + 0
provides a separate plex for each disk that is then mirrored individually.
Figure E-1 shows a comparison between RAID 0 + 1 and RAID 1 + 0
configurations.
RAID 0+1
Volume
RAID1+0
Plex
Stripe Volumes
Mirror
Plex
Volume
Plex
Mirror
Subplex
sd
Stripe
Stripe
Figure E-1
Subplex
sd
Stripe Volumes
mirror
Mirror
subplex
subplex
sd
sd
sd
sd
RAID 0 + 1 and RAID 1 + 0 Comparison
The layout in Figure E-1 is referred to as layer volumes or a VERITAS stripepro or VERITAS concat-pro layout. This layout enhances redundancy and
reduces recovery time in case of an error. In a mirror-stripe layout, if a
disk fails, the entire plex is detached, thereby losing redundancy on the
entire volume. When the disk is replaced, the entire plex must be brought
up to date. Recovering the entire plex can take a substantial amount of
time.
E-12

Complex RAID Levels

If a disk fails in a stripe-mirror layout, only the failing subdisk must be
detached, and only that portion of the volume loses redundancy. When
the disk is replaced, only a portion of the volume needs to be recovered.
Compared to mirroring plus striping, striping plus mirroring offers a
volume more tolerant to disk failure and quicker recovery time.
The following example demonstrates RAID 1 + 0 using the -o ordered
option:
# vxassist -g newdg3dg -o ordered make vol01 1g layout=stripe-mirror ncol=2 disk04
disk02 disk01 disk03
# vxprint -htr
.
.
.
The previous example shows stripes of RAID 0 combined with two

subvolumes of a RAID 1 mirror, as shown in Figure E-2.
Volume - vol01
Plex - vol01-03
vol01-S01
Stripe
vol01-S02
Figure E-2
Top Layers of a RAID 1 + 0 Volume

E-13
Complex RAID Levels

Matching the diagram in Figure E-2 on page E-13 against the output of the
example vxprint command shows the following objects:
Volume (v) vol01
Plex (pl) vol01-03
Subvolume vol01-S01
Subvolume vol01-S02
These objects provide the stripe. Notice that even though mirroring is
implemented, it is not visible to the single upper layer plex. In essence, this
configuration is a single stripe that is mirrored.
There are two subvolumes with a total of four subdisks in the plex. From
the perspective of the plex, the capacity is equal to two subdisks. The
other subdisks provide the data redundancy. Underneath the plex, logical
subvolumes provide the mirroring as shown in Figure E-3.
Plex
vol01-S01
vol01-L01
vol01-P01 vol01-P02
Disk 04
Disk 01
Mirror
vol01-S02
vol01-L02
vol01-P03 vol01-P04
Disk 02
Disk 03
Mirror
Figure E-3
Lower Layers of a RAID 1 + 0 Volume
The subvolumes each contain two subplexes which are concatenated as

part of a mirror. Each subplex can be detached or resynchronized
individually, allowing for faster recovery times.
E-14

Complex RAID Levels

The following examples show a RAID 1 + 0 configuration without using
the -o ordered option:
# vxassist -g testdg make vol01 1g layout=stripe-mirror ncol=2
# vxassist -g testdg make vol01 1g layout=stripe-mirror ncol=2 disk04 disk02 disk01
disk03
Both operations provide the following result:

# vxprint -htr (partial output)
v
vol01
ENABLED
pl
vol01-03
vol01
ENABLED
sv
vol01-S01
vol01-03 vol01-L01
v2
vol01-L01
ENABLED
p2
vol01-P01
vol01-L01 ENABLED
s2
disk01-02
vol01-P01 disk01
p2
vol01-P02
vol01-L01 ENABLED
s2
disk03-02
vol01-P02 disk03
sv
vol01-S02
vol01-03 vol01-L02
v2
vol01-L02
ENABLED
p2
vol01-P03
vol01-L02 ENABLED
s2
disk02-02
vol01-P03 disk02
p2
vol01-P04
vol01-L02 ENABLED
s2
disk04-02
vol01-P04 disk04
ACTIVE 2097152
ACTIVE 2097152
1
1048576
ACTIVE 1048576
ACTIVE 1048576
0
1048576
ACTIVE 1048576
0
1048576
1
1048576
ACTIVE 1048576
ACTIVE 1048576
0
1048576
ACTIVE 1048576
0
1048576
SELECT
STRIPE
0/0
SELECT
CONCAT
0
CONCAT
0
1/0
SELECT
CONCAT
0
CONCAT
0
vol01-03
2/128
2/2
c0t1d0
c1t9d0
2/2
c1t8d0
c1t10d0
fsgen
RW
ENA
fsgen
RW
ENA
RW
ENA
ENA
fsgen
RW
ENA
RW
ENA
Additional Layout Options

Additional attributes are used with the vxassist command to control the
data layout in the volume.
One attribute is the col_switch attribute, which specifies how to
concatenate space on the disks into columns. For example, the following
command creates a mirrored-stripe volume with two columns:
# vxassist -o ordered make new 10g layout=mirror-stripe ncol=2
col_switch=3g,2g disk01 disk02 disk03 disk04 disk05 disk06 disk07 disk08

E-15
Complex RAID Levels

This command allocates spacing, as shown in Figure E-4.
Mirrored Stripe Volume
Striped Plex
Column 1
Column 2
Column 3
disk01-01
disk03-01
disk05-01
disk02-01
disk04-01
disk06-01
Mirror
Mirror
Mirror
Column 1
Column 2
Column 3
disk07-01
disk09-01
disk11-01
disk08-01
disk10-01
disk12-01
Striped Plex
Figure E-4
Mirrored Volume with Multiple Subdisks
Storage is also allocated based on controllers and enclosures with ordered

allocation. For example, the following command creates a three column,
mirrored-stripe volume across controllers:
# vxassist -o ordered make mirvol 80g layout=mirror-stripe ncol=3 \
ctlr:c1 ctlr:c2 ctlr:c3 ctlr:c4 ctlr:c5 ctlr:c6
This command allocates space for column 1 from disks on controllers c1,
for column 2 from disks on controller c2, and so on.
E-16

Complex RAID Levels
Additional Mirror Attributes

The VxVM software provides additional attributes for controlling
mirroring, using the vxassist command:
# vxassist make volume length layout=layout mirror=[attribute]
Mirror attributes include the following:
The attribute mirror=ctlr specifies placement of disks in one

mirror on a different controller than disks in other mirrors within the
same volume.
The attribute mirror=target specifies mirroring of volumes

between identical target IDs on different controllers.
The attribute mirror=enclr specifies placement of disks in one

mirror in a different enclosure than disks in other mirrors within the
same volume.
The following command creates a mirrored volume with two plexes on

different controllers:
# vxassist make volmnt 10g layout=mirror nmirror=2 mirror=ctlr ctlr:c1 ctlr:c2
In this example, the disks in one plex are all attached to controller c1, and
the disks in the other plex are all attached to controller c2. If a controller
fails, only one side of the mirror is lost.

E-17
Disk Layout Practices

Consider the following recommended guidelines for file system layouts:
Start simple.
Start with a simple disk layout; then add complexity. Learn the
layout needed to identify the tool set requirements, and start further
tuning.
More spindles are better.

The ability to get top performance from a system depends on
avoiding contention and evenly spreading throughput.
Bigger is not always better.

Do not confuse capacity and performance. Often one feature must be
compromised to achieve the other. Doubling the capacity of storage
does not increase the performance speed.
Capacity is measured in gigabytes while performance is measured in
I/O per second.
Look at the big picture.

Learn the system environment before focusing too much on where a
problem is located. Look at the whole environment, and apply efforts
across the entire environment.
Trade-offs must be made.

There is no perfect solution, and there is always more than one
solution. This usually results in a mutually-exclusive choice between
things which cannot be obtained at the same time.
Performance is never the only consideration.

Availability, reliability, and manageability are very high on the list of
concerns in system layout, as are price of ownership (not to be
confused with purchase price alone) and scalability (that is, room to
grow).
E-18

Guidelines for RAID Array Layouts

This section contains recommendations for configuring arrays.
RAID 0 Mirroring Production file systems are mirrored. Staging

and development file systems are not.
Proper backup procedures must be implemented. When possible,
disks are mirrored across controllers. Experience shows that when
this is done correctly, either a disk or a controller can fail and the
system remains fully operational.
Mirror across arrays and label the arrays A and B. Doing this allows
a system to lose an entire array and still stay on line. A typical
response to this situation is use the VxVM software and rename the
disks to include the array identifier. For example, rename disk22 to
diskA22 and the mirror to diskB22.
RAID 1 Striping Striping is highly effective when used correctly to

solve a known and understood problem. Experience shows that
when this is used incorrectly, however, it compounds problems and
makes problem resolution difficult.
Implement striping in a standardized manner, where used, across all
platforms. This reduces administration overhead.
RAID 5 Parity disks are used at the hardware level only or on readonly oriented file systems.
Guidelines for Complex File System Layouts

To lay out a complex file system, a general work practice is to first create
simple, single-volume, single-disk file system. When the simple system is
in place, take performance measurements and use them to create the
required I/O RAID 1 file system.
Use the following guidelines for layout strategies:
Small file systems (under 2 gigabytes) are generally mapped to a

single disk.
Medium file systems (between 2 and 8 gigabytes) are usually striped

over four disks.
Large file systems (8 gigabytes and above) are usually striped over
four to eight disks.

E-19
Striping Considerations
The question of how to perform striping is a subject of hot debate with no
correct answer. This is partly due to the fact that striping can either help
or hurt performance, depending on the workload, the number of stripes
per disk, concurrent access, and how well the application uses I/O.
Striping Characteristics
The objective of striping is to reduce head contention. Potentially,
especially with high capacity disks, striping can increase head contention
by increasing concurrent access, resulting in inconsistent allocation of the
stripe or poor data layouts (multiple heavy I/O stripes on the same
disks).
After striping is set up, it can be more difficult to reconfigure than
unstriped environments. This can result in additional time the system
administrator needs to set up and maintain the striped environment.
Another consideration is the human overhead associated with configuring
and managing a striped I/O subsystem. This impact is far higher on a
busy system than on an idle one. Operators and administrators must be
trained on the striping setup and the impact each disk has on the system
performance.
Guidelines for Striping

Use the following guidelines to determine striping layout need:
The primary goal for striping is to identify large file systems which
require striping. Large file systems with high-intensity sequential
reads or writes require striping.
Place those file systems which do not require striping in a single disk
volume.
When striping, you should use increments of disks available on the

system. Usually, that is a four-column or six-column stripe. Using
symmetrical striping avoids creating roving hot spots.
A good starting point is a four-way stripe set with a 64 kilobyte

stripe unit. The stripe should become a standard implementation.
RAID 1 + 0 stripes can be potentially larger because recovery times
are not as dramatically impacted.
E-20

Avoid a stripe width of greater than eight columns. Risk of

contention goes up and the performance gains level off in systems
with greater than eight-column stripes.
The stripe size should always be a multiple of the database block

size. It is grossly inefficient to split an I/O over multiple I/O
requests.
When performing I/Os that require multiple blocks, it is best to
ensure that each I/O can be satisfied from one disk. Examples of this
type of I/O requests are table scans, backups, and disaster recovery
restore operations. Chose a stripe size that is a multiple of the file
system block (for example, in Oracle this would be a multiple of
db_block_multiblock_read_count init.ora parameter).
Random access, non-table scanning applications tolerate a smaller
block size. Do not set the stripe size so small that backups and
restores are inefficient.
Table scanning applications benefit from large stripe sizes. Good
stripe sizes vary from 128 kilobytes for online transaction processing
(OLTP) applications to several megabytes for data warehouse
applications.

E-21
Manipulating Disk Layouts

This section describes procedures for working with disk layouts.
Online Re-Layout
Online re-layout allows reconfiguration of RAID layouts on currently
configured volumes. A volume can be re-laid out or converted to another
layout.
Online re-layout is supported for the following operations:
RAID 5 to mirror
Mirror to RAID 5
Mirror to Concat
Concat to RAID 5
RAID 5 to Concat
Adding or removing parity
Adding or removing columns
Changing stripe width
Re-Laying Out Volumes

To re-layout volumes, use the following format for the vxassist
command with the relayout option:
# vxassist relayout vol01 layout=stripe,log nmirror=2 ncolumn=5 stwidth=256K
Only one plex is used if you are re-laying out to RAID 5. See the
vxassist man pages for additional information.
E-22

Converting RAID Configurations

The following commands convert a RAID configuration to a new layout:
# vxassist -g new-dg convert vol01 layout=stripe-mirror
# vxassist -g new-dg convert vol01 layout=mirror-stripe
# vxassist -g new-dg convert vol01 layout=concat-mirror
Growing File Systems

To grow a volume that has a file system on it, you must grow the volume
first, and then the file system.
Using the vmsa GUI

The vmsa GUI allows a user to grow either a volume or a file system.
Depending on whether the user selects a volume or file system, the
selected item is either grown or shrunk. For example, selecting the volume
enables a resize menu entry to grow only the volume. Selecting the file
system enables a resize menu entry to grow both the volume and the file
system.
Using the Command Line

To grow an existing volume, perform the following procedures:
1.
Display the current size of the volumes. Type the following:
# vxprint -g test01 -t vol01

V NAME
USETYPE
KSTATE
v vol01
fsgen
ENABLED
2.
STATE
ACTIVE
LENGTH
40960
READPOL
SELECT
PREFPLEX
-
Determine the current size of the file system.

a.
For UFS, type the following:
# fstyp -v /dev/vx/dsk/test01/vol01 | head -10 | grep ncg

ncg 5
size
20480
blocks 19183
The number after size is the size of the file system, in kilobytes. To
translate to sectors, multiply by 2.
b.
For VxFS, type the following:
# fstyp -v /dev/vx/dsk/test01/vol01 | grep nau

bsize 1024 size 20480 dsize 91392 ninode 0
nau 3

E-23

Multiply the number after bsize by the number after size. That
determines the size of the file system, in bytes. To translate to sectors,
divide that number by 512.
1024 (bsize) x 20480 (size) / 512 = 40960 sectors
This file system size is 40960 sectors, which matches the volume size.
3.
Determine the largest size an existing volume can be grown.

Type the following:
# vxassist -g rootdg maxgrow vol01 disk01 disk02 disk03

Volume vol01 can be extended by 12244992 to 12285952 (5999Mb)
If you do not specify the disks, the vxassist command uses the
disks in the diskgroup.
4.
Grow the volume to the required size. Type the following:
# vxassist -g rootdg growto vol01 10639360 disk01 disk02 disk03
This command only grows the volume. Remember to grow the file
system as well, if needed.
Note The vxassist command also has a growby option. See the
vxassist man pages for more information.
Manually Growing the File System

Determine to which size to grow the file system. Usually, this is the size of
the volume. Proceed as follows:
1.
Determine the size of volume with the following command:
# vxprint -g rootdg -t vol01

V
NAME USETYPE
KSTATE
v
vol01 fsgen
ENABLED
2.
STATE LENGTH
READPOL
ACTIVE 10639360 SELECT
PREFPLEX
-
Grow the file system. Type
# /usr/lib/fs/ufs/mkfs -F ufs -M /mnt /dev/vx/rdsk/rootdg/vol01 10639360
The mkfs command has an option to grow a file system instead of making
one. To use this option, the file system must be mounted. You must also
use the full path to the mkfs program; do not use /usr/sbin/mkfs.
Note The above process is described in detail by Sun Solve INFODOC
14881.
E-24


To grow a file system, execute the following command for VxFS:
# /usr/lib/fs/vxfs/fsadm -b 10639360 -r /dev/vx/rdsk/rootdg/vol01 /mnt
Examples of Grown File System

Below are examples of grown file systems. Notice the change in the offset
positions in the first example and the change in the disk offset in the
second example.
An example of a stripe before layout is shown.
# vxprint -ht -g newdg vol04
V
PL
SD
v
pl
sd
sd
sd
sd
NAME
USETYPE
NAME
VOLUME
NAME
PLEX
vol04
fsgen
vol04-01 vol04
newdg01-01vol04-01
newdg03-01vol04-01
newdg05-01vol04-01
newdg07-01vol04-01
KSTATE
KSTATE
DISK
ENABLED
ENABLED
newdg01
newdg03
newdg05
newdg07
STATE
STATE
DISKOFFS
ACTIVE
ACTIVE
0
0
0
0
LENGTH
LENGTH
LENGTH
154224
154224
51408
51408
51408
51408
READPOL
LAYOUT
[COL/]OFF
SELECT
STRIPE
0/0
1/0
2/0
3/0
PREFPLEX
NCOL/WID
DEVICE
vol04-01
4/128
c4t0d0
c4t2d0
c4t4d0
c4t0d1
MODE
MODE
RW
ENA
ENA
ENA
ENA
An example of a stripe after layout is provided.

V
PL
SD
NAME
NAME
NAME
USETYPE
VOLUME
PLEX
v
pl
sd
sd
sd
sd
sd
sd
sd
sd
vol04
fsgen
vol04-01 vol04
newdg01-01vol04-01
newdg02-01vol04-01
newdg03-01vol04-01
newdg04-01vol04-01
newdg05-01vol04-01
newdg06-01vol04-01
newdg07-01vol04-01
newdg08-01vol04-01
KSTATE
KSTATE
DISK
STATE
LENGTH
STATE
LENGTH
DISKOFFS LENGTH
READPOL
PREFPLEX
LAYOUT
NCOL/WID
[COL/]OFF DEVICE
MODE
MODE
ENABLED
ENABLED
newdg01
newdg02
newdg03
newdg04
newdg05
newdg06
newdg07
newdg08
ACTIVE
ACTIVE
0
0
0
0
0
0
0
0
SELECT
STRIPE
0/0
0/51408
1/0
1/51408
2/0
2/51408
3/0
3/51408
RW
ENA
ENA
ENA
ENA
ENA
ENA
ENA
ENA
614400
616944
51408
102816
51408
102816
51408
102816
51408
102816
vol04-01
4/128
c4t0d0
c4t1d0
c4t2d0
c4t3d0
c4t4d0
c4t5d0
c4t0d1
c4t1d1
A second example of a stripe before layout is shown.

V
NAME
USETYPE
KSTATE
PL
NAME
VOLUME
KSTATE
SD
NAME
PLEX
DISK
STATE
LENGTH
STATE
LENGTH
DISKOFFS LENGTH
READPOL
PREFPLEX
LAYOUT
NCOL/WID
[COL/]OFF DEVICE
MODE
MODE
v
pl
ACTIVE
ACTIVE
SELECT
STRIPE
RW
vol04
fsgen
vol04-01 vol04
ENABLED
ENABLED
205632
205632
vol04-01
4/128

E-25

sd
sd
sd
sd
newdg12-01vol04-01
newdg13-01vol04-01
newdg08-01vol04-01
newdg02-01vol04-01
newdg12
newdg13
newdg08
newdg02
0
0
0
0
51408
51408
51408
51408
1/0
2/0
3/0
0/0
c4t0d1
c4t2d4
c4t3d1
c4t0d0
ENA
ENA
ENA
ENA
Another stripe after layout example is shown.

V
NAME
USETYPE
KSTATE
PL
NAME
VOLUME
KSTATE
SD
NAME
PLEX
DISK
STATE
LENGTH
STATE
LENGTH
DISKOFFS LENGTH
READPOL
PREFPLEX
LAYOUT
NCOL/WID
[COL/]OFF DEVICE
v
pl
sd
sd
sd
sd
sd
sd
ACTIVE
ACTIVE
0
358848
0
0
358848
0
SELECT
STRIPE
0/0
0/51408
1/0
2/0
2/51408
3/0
E-26
vol04
fsgen
vol04-01 vol04
newdg12-01vol04-01
newdg12-01vol04-01
newdg13-01vol04-01
newdg08-01vol04-01
newdg08-03vol04-01
newdg02-01vol04-01
ENABLED
ENABLED
newdg12
newdg12
newdg13
newdg08
newdg08
newdg02
409600
411552
51408
51408
102816
51408
51408
102816
vol04-01
4/128
c4t0d0
c4t0d0
c4t0d1
c4t2d4
c4t2d4
c4t3d1
MODE
MODE
RW
ENA
ENA
ENA
ENA
ENA
ENA

Changing Disk Group Configurations

To reorganize disk groups, move disks and objects between the disk
groups. Generally this is driven by organizational change, online
maintenance requirements, or when the capacity of the private region is
reached. Changing the disk group has less impact than growing the
private region.
To change configurations, you must ensure that the vxdg command
options interact with vxconfigd to validate, identify, and populate new
configurations. These commands are described in the following
paragraphs.
Disk Group Reconfiguration Commands

Use the following syntax to move objects from one disk group to a new
disk group:
vxdg [-o expand] split sourcedg targetdg [object ...]
Caution Using the same target name as an existing deported disk group
destroys that group.
To join disk groups, use the following syntax:
vxdg [-o expand] join sourcedg targetdg [object ...]
To preview a move, use the following syntax:

vxdg [-o expand] listmove sourcedg targetdg [object ...]
To move objects from one imported disk group to another, use the
following syntax:
vxdg [-o expand] move sourcedg targetdg [object ...]
For each of these vxdg commands, the option -o expand includes all
disks from volumes sharing subdisks.
For a complete list of options for the vxdg command, see the man pages.

E-27
Example Disk Group Reconfiguration Commands

The following examples demonstrate the use of commands to modify disk
groups.
The following example commands each split a disk group and then
restart the volumes in the new disk group:
# vxdg split old-dg new-dg disk01 disk02 disk03
# vxrecover -g new-dg -m vol01
# vxvol -g new-dg start vol01
The following commands join a disk group:

# vxdg join new-dg old-dg
# vxrecover -g old-dg -m vol01
This command displays potential objects to move, including shared

subdisks
# vxdg -o expand listmove old-dg new-dg
This command moves objects from one disk group to another:

# vxdg -o expand move old-dg new-dg vol04
Disk Group Reconfiguration Recovery

If a problem occurs during the reconfiguration of a disk group, such as a
system crash, the VxVM software release 3.2 provides recovery
capabilities.
First, the VxVM software release 3.2 attempts to recover automatically by
either completing the operation or reversing the operation, depending on
the progress of the reconfiguration. If the recovery is successful, the
VxVM software imports the affected disk group and starts the volumes.
If the VxVM software recovery is not successful, perform the following
procedures to recover the disk group manually:
1.
Use the vxprint command to examine the configuration of both

disk groups. Objects in disk groups for which the move operation is
incomplete have a TUTIL0 field set to MOVE. For example:
# vxprint -h new-dg
Disk group: newdg
TY
dg
E-28
NAME ASSOC KSTATE LENGTH PLOFFS STATE TUTIL0 PUTIL0

new-dgnewdg - - - MOVE


The VxVM software uses the TUTIL0 and PUTIL0 fields to lock the
affected objects during transition.
2.
Enter the following command to complete the move:

# vxdg recover new-dg
vxvm: vxdg: ERROR: diskgroup: Disk group does not exist
3.
If the recovery fails, check to see if the disk group was imported onto
another host. If it was imported, deport it from that host, and import
it onto the current host.
4.
If all the required objects already exist in either the source or target
disk group, use the following command to reset the MOVE flags in
that disk group:
# vxdg -o clean recover new-dg
5.
Use the following command on the other disk group to remove the
objects that have TUTIL0 fields marked as MOVE :
# vxdg -o remove recover old-dg
6.
If only one disk group is available to be imported, use the following

command to reset the MOVE flags on this disk group:
# vxdg -o clean recover old-dg
Listing the PUTIL and TUTIL Fields

The PUTIL and TUTIL fields are used as locks on volumes, plexes,
subdisks, and disks during most types of configuration changes and
recoveries. For example, if attaching a plex to a volume, the plex TUTIL0
field is set to ATT. Once the attach is complete, the TUTIL0 field is cleared
automatically.
If a user attempts to perform a function on a entity with a set PUTIL or
TUTIL flag, the following message is received:
vxvm:vxplex: ERROR: Volume volname is locked by another utility
The PUTIL fields are permanent and stay set even after a reboot. The
TUTIL fields are temporary and do not survive a reboot.

E-29

To list the PUTIL and TUTIL fields, use the following command:
# vxprint -h vol01
TY
NAME
ASSOC
v
vol01
fsgen
pl
vol04-01 vol01
sd
newdg02-02vol04-01
pl
vol01-01 vol01
sd
newdg11-01vol01-01
KSTATE
ENABLED
ENABLED
ENABLED
ENABLED
ENABLED
LENGTH
2050272
2050272
2050272
2050272
2050272
PLOFFS
0
0
STATE
ACTIVE
ACTIVE
TEMPRMSD
-
TUTIL0
ATT1
ATT
-
PUTIL0
-
Clearing the PUTIL and TUTIL Fields

To clear the PUTIL and TUTIL lock flags:
1.
Ensure that no recover command is in process. Execute one of the

following commands:
# ps -ef | grep vx
# vxtask -l list
If there is a vxvol, vxplex or vxrecover operation in process, the
PUTIL and TUTIL fields are probably set for a reason.
2.
Use the following syntax to clear the flags:

# vxmend -g dgname clear all objectname
Replace objectname with the volume, plex, subdisk, or disk which
has a PUTIL or TUTIL flag set.
Reconfiguration Considerations
The disk group move, split, and join features have the following
limitations:
E-30
Disk groups involved in a move, split, or join must be version 90 or

greater.
The reconfiguration must involve physical disks.
Objects to be moved must not contain open volumes.
Moved volumes are initially disabled following a disk group move,

split or join. Use either vxrecover -m or vxvol startall to restart the
volumes.
Data change objects (DCOs) and snap objects that are dissociated by
persistent fast resynchronization cannot be moved between disk
groups.
Sun StorEdge Volume Replicator (VR) objects cannot be moved

between disk groups.

For a disk group reconfiguration to succeed, the source and target

disk group must be able to store copies of the configuration database
after the move.
The configuration database in the target disk group must also be able
to store all the object information in the enlarged disk group.
Splitting or moving a volume into a different disk group changes the

volume record ID.

E-31
Hot-Relocation
Hot-Relocation
Hot-relocation allows a system to relocate data automatically in a
redundant configuration, in the event of a subdisk failure. There are,
however, a number of restrictions for hot-relocation use:
Subdisks must be in a redundant configuration (mirror or RAID 5).
Space must be available to contain the recovered data.
Hot-relocation fails if:
Space is only available in the same plex of the mirror as the

failed sub-disk.
Space is only available on a plex that contains a RAID 5 log.
The failed subdisk is in the same plex as a DRL log.
RAID 5 logs or DRL logs are created, not relocated.
This section discusses the hot-relocation process and how to perform it.
Hot-Relocation Process
To execute relocation, the hot-relocation deamon vxrelocd handles four
distinct operations, in the following order:
1.
Failure detection
2.
Notification
3.
Relocation
4.
Recovery
The vxrelocd daemon monitors events that signify a subdisk or plex

failure. When a failure occurs, data space is selected from hot-relocationreserved disks or from available free space in the disk group, and a
resynchronization begins. The failed disk is marked as failing. The
deamon notifies designated users and then reconstructs the objects in the
new location; the new subdisks retain the existing names. The last
operation is to recover the data.
Figure E-5 on page E-33 shows the interaction of various processes during
these procedures.
E-32

Hot-Relocation
.
vxrelocd
vxnotify
failure
detected
vxrelocd
vxrelocd
correct,
action
mailx
vxconfigd
determine
available
space
updates
configuration
vxassist
notify
access
disk
builds
objects
vxrecover
recovers data
Figure E-5
Hot-Relocation Process Interaction
Failure Detection
The vxrelocd deamon is a script running in the background, interacting
with vxnotify, to monitor three types of failure:
Disk failures using the failing flag in DM records
Plex failures due to uncorrectable I/O errors
Subdisk failures in a RAID 5 volume

E-33
Hot-Relocation
After a failure is detected, the vxrelocd deamon takes object-specific
actions and can include a kernel-initiated change that alters the
configuration. If a configuration change is required, the vxconfigd
deamon is notified with the request update. The vxconfigd deamon, in
turn, writes out the configuration change to the database copies.
Notification
The vxrelocd deamon notifies users (by default, the root user) using the
mailx command, providing information about the failure and the status
of relocation and recovery. The file /etc/rc2.d/s95-vxvm-recover
contains a list of users to notify.
To add users to the notification list, modify the
/etc/rc2.d/s95-vxvm-recover file as follows:
vxrelocd root username1 username2 &
Changes to this file take effect at the next reboot or when the vxrelocd
command is executed from the command line. To execute this command
from the command line, kill the running vxrelocd deamon first, but be
careful not to kill the deamon in the middle of a relocation process.
Error notification takes the following form:
Date: Wed, 6 Feb 2002 12:04:27 GMT
From: root
Subject: Volume Manager failures on host fred
To: root
Volume vol02 Subdisk disk04-02 relocated to disk06-03, but not yet recovered.
E-34

Hot-Relocation
Relocation
The vxrelocd daemon determines available space by looking for the
spare flag in the DM record.
To display the spare flag use either the vxdisk or vxprint command, as
shown in the following example:
# vxdisk list
DEVICE TYPE DISK GROUP STATUS
c0t0d0s2 sliced rootdisk rootdg online
c1t12d0s2 sliced altboot rootdg online
c1t13d0s2 sliced newdg01 newdg online
c1t14d0s2 sliced newdg02 newdg online
c1t15d0s2 sliced newdg03 newdg online spare
If a spare flag is not set then the vxrelocd deamon uses available space in
the disk group to build the VxVM object. To exclude a disk from use in
hot-relocation, set the nohotuse flag as follows:
# /etc/vx/bin/vxedit -g disk_group set nohotuse=on disk_name
If a disk fails, then remove it from space allocation as follows:
# vxedit set failing=on fail_disk
To rejoin space allocation:
# vxedit set failing=off fail_disk
After space is determined, the vxrelocd deamon uses the vxassist
command with the move option to create the new objects, which retain the
existing names.
Recovery
Recovery is the last step in the hot-relocation process. After new objects
are moved, the vxrelocd deamon calls the vxrecovery command to
recover the data. Two fields are added to the DM record to identify the
original location of the object: the orig_dmname= and orig_dmoffset=
fields. These assist in manually restructuring the original object.

E-35
Hot-Relocation
Hot-Relocation Configuration
The vxdiskadm utility has new options which support hot-relocation,
available as of the VxVM software version 3.1.
A sample of the vxdiskadm menu display is as follows:
1 Add or initialize one or more disks
2 Encapsulate one or more disks
3 Remove a disk
4 Remove a disk for replacement
5 Replace a failed or removed disk
6 Mirror volumes on a disk
7 Move volumes from a disk
8 Enable access to (import) a disk group
9 Remove access to (deport) a disk group
10 Enable (online) a disk device
11 Disable (offline) a disk device
12 Mark a disk as a spare for a disk group
13 Turn off the spare flag on a disk
14 Unrelocate subdisks back to a disk
15 Exclude a disk from hot-relocation use
17 Prevent multipathing/Suppress devices from VxVMs view
18 Allow multipathing/Unsuppress devices from VxVMs view
list List disk information
q Exit from menus
Alternatively, use the vxedit command to set the spare flag from the
command line. Use the following command:
# vxedit set spare=on disk_name
E-36

Hot-Relocation
Unrelocating
If you try to manually move a relocated subdisk using the vxsd
command, the following message is displayed:
vxvm:vxsd: ERROR: Relocate trace information in subdisk disk04-01 not empty. Use -r to
retain or -d to discard it
You can then decide whether to retain the orig_dmname information or

discard it. The VxVM software version 3.1 provides the vxunrelocate
command for use to restore a relocated subdisk to its original location.

E-37
Hot-Spares
Hot-Spares
The hot-spare process is similar to hot-relocation, but there are significant
differences between the two. The primary functional difference is disk
selection in the event of a failure; hot-relocation is able to relocate
subdisks, whereas the hot-spare process relocates entire disks. With hotrelocation, the event of a failure to a subdisk no longer impacts all
volumes on the physical disk, unless it is the disk that fails. Hot-relocation
is the recovery process available in the VxVM software, starting with
version 2.3.
In the hot-spare process, the /etc/vol/sparelist file is maintained to
determine locations of available spare disks. In the hot-relocation process,
inherent policies determine where subdisks are relocated. The primary
policy for both the hot-spare and hot-relocation processes is to locate
recovered data based on the spare flag and drive locality. The closer the
drive is to the failed drive, the greater the preference.
Comparison of Hot-Relocation and Hot-Spares

While hot-relocation is more flexible than the hot-spare function, it also
has the potential to disrupt carefully designed layouts.
For example, a mirror configuration can be set up across two arrays. In
the event of a failure, if sufficient disk space is not available, then the hotrelocation process can potentially recover the failed plex across multiple
disks over both arrays in small chunks. If a 1 gigabyte subdisk fails and
the only free space available in the disk group are 100 megabyte chunks
on 10 separate disks, the VxVM software concatenates 10 subdisks
together to form the 1 gigabyte failed subdisk, thus spreading the data out
onto 10 separate disks.
E-38

Hot-Spares
Activating the Hot-Spare Function

Hot-relocation can be unconfigured in the
/etc/rc2.d/S95-vxvm-recover file and the hot-spare function
implemented. To implement the hot-spare function, change
vxrelocd root & to vxsparecheck root & in this file.
Hot-relocation can be configured to use only disks marked as spare. Back
up the /etc/vx/bin/vxrelocd script; then edit the file and change
spare=yes to spare=only in this script.
Note Be aware that any upgrade removes these modifications.

E-39

Sun - VERITAS Volume Manager - Student Guide

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Sun - VERITAS Volume Manager - Student Guide

Uploaded by

Copyright:

Available Formats

VERITAS Volume Manager

Sun Microsystems, Inc.

January 13, 2003 5:15 pm

THIS MANUAL IS DESIGNED TO SUPPORT AN INSTRUCTOR-LED TRAINING

CE MANUEL DE RFRENCE DOIT TRE UTILIS DANS LE CADRE DUN COURS

Resynchronizing Volumes.............................................................. 1-29

Sun Proprietary: Internal Use Only

Examining Sun Enterprise Services Best Practices for VxVM

Sun Proprietary: Internal Use Only

VxVM Software Architecture and DMP................................ 3-4

Sun Proprietary: Internal Use Only

The vxexplorer Utility ......................................................... 4-18

Sun Proprietary: Internal Use Only

Recovering Disk, Disk Group, and Volume Failures .................... 6-1

Sun Proprietary: Internal Use Only

SunSolve INFODOCs....................................................................... A-1

Guidelines for RAID Array Layouts ................................... E-19

Sun Proprietary: Internal Use Only

About This Course

Define information technology (IT) storage management

Improve the availability of the VxVM software

Identify and repair common disk and disk group management

Upgrade the VxVM software

Resolve VxVM software version and licensing problems

Sun Proprietary: Internal Use Only

Sun Proprietary: Internal Use Only

Topics Not Covered

Topics Not Covered

Storage area networks Covered in ES-475: Design and Administration

Solaris Operating Environment (OE) administration Covered in

Basic VxVM software administration Covered in IES-310:

Hitachi Lightning SE9900 administration Covered in HDS-335:

Sun StorEdge T3 administration Covered in ES-255: Sun

Sun Proprietary: Internal Use Only

How Prepared Are You?

How Prepared Are You?

Can you install the VxVM software?

Can you use the vxprint utility to print VxVM software

Can you mirror the boot disk on a server?

Sun Proprietary: Internal Use Only

Title, function, and job responsibility

Experience related to topics presented in this course

Reasons for enrolling in this course

Expectations for this course

Sun Proprietary: Internal Use Only

How to Use Course Materials

How to Use Course Materials

Goals You should be able to accomplish the goals after finishing

Objectives You should be able to accomplish the objectives after

Lecture The instructor will present information specific to the

Activities The activities take on various forms, such as an exercise,

Sun Proprietary: Internal Use Only

Discussion Indicates a small-group or class discussion on the current

Power user Indicates additional supportive topics, ideas, or other

Sun Proprietary: Internal Use Only

Courier italic is used for variables and command-line place-holders

Courier italic bold is used to represent variables whose values are to

Sun Proprietary: Internal Use Only

Introducing the VERITAS Volume Manager

Describe the two storage management methodologies

Describe the relationship between the VxVM software and the

Identify and describe the major components of the VxVM software

Identify and define all the VxVM software objects

Describe the resynchronization process