You are on page 1of 406

VERITAS Volume Manager

4.0 Administration
ES-310
Student Guide With Instructor Notes

Sun Microsystems, Inc.


UBRM05-104
500 Eldorado Blvd.
Broomfield, CO 80021
U.S.A.
Revision D

December 19, 2003 10:15 am

Copyright 2004 Sun Microsystems Inc. 4150 Network Circle, Santa Clara, California 95054, U.S.A. All rights reserved.
This product or document is protected by copyright and distributed under licenses restricting its use, copying, distribution, and
decompilation. No part of this product or document may be reproduced in any form by any means without prior written authorization of
Sun and its licensors, if any.
Third-party software, including font technology, is copyrighted and licensed from Sun suppliers.
Sun, Sun Microsystems, the Sun logo, Solaris, OpenBoot, Ultra, Sun Blade, Sun StorEdge, Solstice DiskSuite, RSM, SunPlex, Sun Fire, Java,
Sun BluePrints, Sun Enterprise, SunOS, and SunSolve are trademarks or registered trademarks of Sun Microsystems, Inc. in the U.S. and
other countries.
All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International, Inc. in the U.S. and
other countries. Products bearing SPARC trademarks are based upon an architecture developed by Sun Microsystems, Inc.
UNIX is a registered trademark in the U.S. and other countries, exclusively licensed through X/Open Company, Ltd.
Adobe is a registered trademark of Adobe Systems, Incorporated. PostScript is a trademark or a registered trademark of Adobe Systems,
Incorporated, which may be registered in certain jurisdictions.
ORACLE is a registered trademark of Oracle Corporation.
The OPEN LOOK and Sun Graphical User Interface was developed by Sun Microsystems, Inc. for its users and licensees. Sun acknowledges
the pioneering efforts of Xerox in researching and developing the concept of visual or graphical user interfaces for the computer industry.
Sun holds a non-exclusive license from Xerox to the Xerox Graphical User Interface, which license also covers Suns licensees who
implement OPEN LOOK GUIs and otherwise comply with Suns written license agreements.
U.S. Government approval might be required when exporting the product.
RESTRICTED RIGHTS: Use, duplication, or disclosure by the U.S. Government is subject to restrictions of FAR 52.227-14(g)(2)(6/87) and
FAR 52.227-19(6/87), or DFAR 252.227-7015 (b)(6/95) and DFAR 227.7202-3(a).
DOCUMENTATION IS PROVIDED AS IS AND ALL EXPRESS OR IMPLIED CONDITIONS, REPRESENTATIONS, AND
WARRANTIES, INCLUDING ANY IMPLIED WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE
OR NON-INFRINGEMENT, ARE DISCLAIMED, EXCEPT TO THE EXTENT THAT SUCH DISCLAIMERS ARE HELD TO BE
LEGALLY INVALID.
THIS MANUAL IS DESIGNED TO SUPPORT AN INSTRUCTOR-LED TRAINING (ILT) COURSE AND IS INTENDED TO BE
USED FOR REFERENCE PURPOSES IN CONJUNCTION WITH THE ILT COURSE. THE MANUAL IS NOT A STANDALONE
TRAINING TOOL. USE OF THE MANUAL FOR SELF-STUDY WITHOUT CLASS ATTENDANCE IS NOT RECOMMENDED.
Export Control Classification Number (ECCN): 19 December 2001

Please
Recycle

Copyright 2004 Sun Microsystems Inc. 4150 Network Circle, Santa Clara, California 95054, Etats-Unis. Tous droits rservs.
Ce produit ou document est protg par un copyright et distribu avec des licences qui en restreignent lutilisation, la copie, la distribution,
et la dcompilation. Aucune partie de ce produit ou document ne peut tre reproduite sous aucune forme, par quelque moyen que ce soit,
sans lautorisation pralable et crite de Sun et de ses bailleurs de licence, sil y en a.
Le logiciel dtenu par des tiers, et qui comprend la technologie relative aux polices de caractres, est protg par un copyright et licenci
par des fournisseurs de Sun.
Sun, Sun Microsystems, le logo Sun, Solaris, OpenBoot, Ultra, Sun Blade, Sun StorEdge, Solstice DiskSuite, RSM, SunPlex, Sun Fire, Java,
Sun BluePrints, Sun Enterprise, SunOS, et SunSolve sont des marques de fabrique ou des marques dposes de Sun Microsystems, Inc. aux
Etats-Unis et dans dautres pays.
Toutes les marques SPARC sont utilises sous licence sont des marques de fabrique ou des marques dposes de SPARC International, Inc.
aux Etats-Unis et dans dautres pays. Les produits portant les marques SPARC sont bass sur une architecture dveloppe par Sun
Microsystems, Inc.
UNIX est une marques dpose aux Etats-Unis et dans dautres pays et licencie exclusivement par X/Open Company, Ltd.
Adobe est une marque enregistree de Adobe Systems, Incorporated. PostScript est une marque de fabrique dAdobe Systems, Incorporated,
laquelle pourrait tre dpose dans certaines juridictions aux Etats-Unis et dans dautres pays.
ORACLE est une marque dpose registre de Oracle Corporation.
Linterfaces dutilisation graphique OPEN LOOK et Sun a t dveloppe par Sun Microsystems, Inc. pour ses utilisateurs et licencis.
Sun reconnat les efforts de pionniers de Xerox pour larecherche et le dveloppement du concept des interfaces dutilisation visuelle ou
graphique pour lindustrie de linformatique. Sun dtient une licence non exclusive de Xerox sur linterface dutilisation graphique Xerox,
cette licence couvrant galement les licencis de Sun qui mettent en place linterface dutilisation graphique OPEN LOOK et qui en outre
se conforment aux licences crites de Sun.
Laccord du gouvernement amricain est requis avant lexportation du produit.
LA DOCUMENTATION EST FOURNIE EN LETAT ET TOUTES AUTRES CONDITIONS, DECLARATIONS ET GARANTIES
EXPRESSES OU TACITES SONT FORMELLEMENT EXCLUES, DANS LA MESURE AUTORISEE PAR LA LOI APPLICABLE, Y
COMPRIS NOTAMMENT TOUTE GARANTIE IMPLICITE RELATIVE A LA QUALITE MARCHANDE, A LAPTITUDE A UNE
UTILISATION PARTICULIERE OU A LABSENCE DE CONTREFAON.
CE MANUEL DE RFRENCE DOIT TRE UTILIS DANS LE CADRE DUN COURS DE FORMATION DIRIG PAR UN
INSTRUCTEUR (ILT). IL NE SAGIT PAS DUN OUTIL DE FORMATION INDPENDANT. NOUS VOUS DCONSEILLONS DE
LUTILISER DANS LE CADRE DUNE AUTO-FORMATION.

Please
Recycle

Table of Contents
About This Course .............................................................Preface-xiv
Course Goals........................................................................ Preface-xiv
Course Map............................................................................ Preface-xv
Topics Not Covered.............................................................Preface-xvi
How Prepared Are You?....................................................Preface-xvii
Introductions ..................................................................... Preface-xviii
How to Use Course Materials ............................................ Preface-xix
Conventions ........................................................................... Preface-xx
Icons ............................................................................... Preface-xx
Typographical Conventions ..................................... Preface-xxi
Notes to the Instructor........................................................Preface-xxii
Sun Storage Concepts .....................................................................1-1
Objectives ........................................................................................... 1-1
Disk Storage Administration Introduction .................................... 1-2
VxVM Software Installation .................................................... 1-2
VxVM Initialization .................................................................. 1-2
RAID Volume Design.............................................................. 1-3
RAID Volume Creation............................................................ 1-3
RAID Volume Administration................................................ 1-4
Interfaces for Sun Storage Devices .................................................. 1-5
SCSI Overview........................................................................... 1-5
SCSI Interface Implementation .............................................. 1-6
SCSI Interface Standards.......................................................... 1-7
SCSI Priority.............................................................................. 1-9
SCSI Phases and the Move to Fibre Channel ........................ 1-9
Fibre Channel Technology....................................................... 1-9
Fibre Channel-Arbitrated Loop ........................................... 1-10
Advantages of FC-AL............................................................. 1-10
Fibre Channel Compared to SCSI........................................ 1-11
RAID Technology ............................................................................ 1-12
Host-Based RAID (Software RAID Technology)................ 1-12
Controller-Based RAID (Hardware RAID Technology) ... 1-13

iv
Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Disk Storage Concepts..................................................................... 1-14


Hot Swapping.......................................................................... 1-14
Storage Area Networking ..................................................... 1-16
Multihost Storage Access...................................................... 1-21
Multipath Storage Access ..................................................... 1-23
Storage Configuration Identification ............................................ 1-28
Conducting Physical Inventory ............................................ 1-28
Displaying Storage Configurations ...................................... 1-28
Identifying Controller Addressing...................................... 1-30
Identifying Device Path Components.................................. 1-31
Identifying DMP Devices...................................................... 1-34
Storage Array Firmware ................................................................. 1-35
Fibre Channel HBA Cards ..................................................... 1-35
Verifying Fibre Channel HBA Firmware ........................... 1-36
Verifying SPARCstorage Array 100 Firmware.................. 1-37
Verifying Sun StorEdge A5x00 Array Firmware............... 1-38
Verifying Sun StorEdge T3 Array Firmware ..................... 1-39
Verifying Sun StorEdge A5x00 Disk Drive Firmware ...... 1-40
Firmware Upgrade Best Practices........................................ 1-41
Exercise: Recording Your Storage Configuration ....................... 1-42
Preparation............................................................................... 1-42
Task 1 Reviewing Sun Storage Features ........................... 1-42
Task 2 Identifying Host Adapter Configurations ........... 1-44
Task 3 Identifying Storage Array Configurations.......... 1-45
Task 4 Verifying Storage Interface Firmware
Revisions ............................................................................... 1-46
Task 5 Verifying Array Disk Drive Firmware
Revisions ............................................................................... 1-47
Exercise Summary............................................................................ 1-48
Managing Data ................................................................................. 2-1
Objectives ........................................................................................... 2-1
Virtual Disk Management ................................................................ 2-2
Availability................................................................................. 2-2
Performance ............................................................................... 2-2
Scalability .................................................................................. 2-3
Maintainability .......................................................................... 2-3
RAID Technology Introduction ....................................................... 2-4
Supported RAID Standards..................................................... 2-4
RAID Terminology .................................................................. 2-5
RAID Level Common Features ........................................................ 2-6
Concatenation RAID 0........................................................... 2-6
Striping RAID 0 ...................................................................... 2-8
Mirroring RAID 1................................................................ 2-10
Mirrored Stripe RAID 0+1 ................................................. 2-12
Mirrored Concatenation RAID 0+1 .................................. 2-14
Striped Mirror RAID 1+0 ................................................... 2-15

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Concatenated Mirror RAID 1+0........................................ 2-17


Striping With Distributed Parity RAID 5 ........................ 2-19
Exercise: Optimizing System Configurations .............................. 2-21
Preparation............................................................................... 2-21
Task 1 Reviewing Software RAID Features .................... 2-22
Task 2 Identifying Availability and Performance
Cabling.................................................................................. 2-25
Task 3 Optimizing RAID-0 Volumes ............................... 2-26
Task 4 Optimizing RAID-1 Volumes ............................... 2-27
Task 5 Optimizing RAID-5 Volumes ............................... 2-28
Task 6 Optimizing RAID 0+1 Volumes ........................... 2-29
Task 7 Optimizing RAID 1+0 Volumes ........................... 2-30
Task 8 Identifying Effective Storage Utilization............. 2-31
Task 9 Selecting Disk Drives for Use................................. 2-32
Exercise Summary............................................................................ 2-33
VERITAS Volume Manager Installation ..........................................3-1
Objectives ........................................................................................... 3-1
Installation Planning.......................................................................... 3-2
System Downtime..................................................................... 3-2
Storage Configuration Assessment ........................................ 3-2
Upgrade Resources ................................................................... 3-3
Licensing .................................................................................... 3-3
Current System Checkpoint .................................................... 3-3
Backups....................................................................................... 3-3
Testing the New Configuration .............................................. 3-3
Researching Software Patches.......................................................... 3-4
Researching Current Patch Information................................ 3-4
Installing Patches ..................................................................... 3-6
Vendor Software Patches ......................................................... 3-6
Installing VxVM Software ................................................................ 3-7
Software Distribution Overview............................................. 3-7
Software Package Installation ................................................ 3-9
Software Installation User Interaction ................................ 3-10
Initializing VxVM Using the vxinstall Utility ......................... 3-13
The vxinstall Utility Dialogue .......................................... 3-13
Licensing Requirements......................................................... 3-14
Verifying Licensed Features ................................................. 3-15
VxVM Post-Installation Environment........................................... 3-16
VxVM System Files ................................................................. 3-16
System Startup Messages...................................................... 3-18
System Startup Processes...................................................... 3-19
System and User Executable Files ....................................... 3-21
Verifying Initial Disk Drive Status ....................................... 3-23
Preparing for VxVM Disk Drive Management............................ 3-24
Disk Drive Initialization Process .......................................... 3-24
Disk Drive Encapsulation Process........................................ 3-27

vi
Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Protecting Storage Devices From Usage............................. 3-29


Global Exclusion .................................................................... 3-32
Installing the VEA............................................................................ 3-35
VEA Software Initialization.................................................. 3-36
VEA Client Software Startup ............................................... 3-37
Host Connection Window .................................................... 3-38
Resolving Low-Bandwidth Access Problems .................... 3-39
Using Basic VEA Features .............................................................. 3-40
Main Window Functional Areas........................................... 3-40
Resizing Display Panes .......................................................... 3-45
Modifying Preferences .......................................................... 3-46
Customizing the Grid Display ............................................. 3-47
Examining VEA Command Logs ......................................... 3-48
Using the VEA Search Tool .................................................. 3-49
Decoding VxVM Error Messages .................................................. 3-50
Exercise: Configuring VxVM.......................................................... 3-51
Preparation............................................................................... 3-51
Task 1 Reviewing Key Lecture Points.............................. 3-52
Task 2 Installing the VxVM Software .............................. 3-55
Task 3 Verifying the VxVM System Files ........................ 3-57
Task 4 Evaluating the Storage Configuration ................. 3-58
Task 5 Installing the VEA Client Software ...................... 3-59
Task 6 Starting the VEA Client Software......................... 3-60
Task 7 Customizing the VEA GUI Appearance ............. 3-61
Task 8 Navigating the VxVM Technical Manuals........... 3-61
Task 9 Using the VxVM Error Numbering System........ 3-63
Exercise Summary............................................................................ 3-64
VERITAS Volume Manager Basic Operations ............................... 4-1
Objectives ........................................................................................... 4-1
VxVM Disk Group Functions........................................................... 4-2
Primary Functions of a Disk Group ....................................... 4-2
VxVM Disk Drives .................................................................... 4-3
Standard VxVM Disk Groups ................................................ 4-4
Shared VxVM Disk Groups .................................................... 4-5
Cross-Platform Data Sharing Disk Groups .......................... 4-6
VxVM Disk Group Operations ........................................................ 4-7
Verifying Disk Group Status ............................................................ 4-8
Using the vxdisk Command to Verify Disk Group
Status........................................................................................ 4-8
Using the vxdg Command to Verify Disk Group Status..... 4-8
Administering Disk Groups Using the vxdiskadm Utility.......... 4-9
Functional Overview ............................................................. 4-10
Creating a New Disk Group................................................. 4-11
Removing a Disk Drive From a Disk Group...................... 4-12

vii

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Administering Disk Groups Using Command-Line


Programs ........................................................................................... 4-13
Using the vxdiskunsetup Command ................................. 4-13
Initializing Disk Drives ......................................................... 4-14
Using the vxdg Command..................................................... 4-14
Adding and Removing Disk Drives.................................... 4-16
Importing and Deporting Disk Groups .............................. 4-17
Destroying a Disk Group ...................................................... 4-19
Renaming VxVM Disk Drives............................................... 4-19
Administering Disk Groups Using the VEA GUI ....................... 4-20
Creating a New Disk Group.................................................. 4-20
Adding and Removing Disk Drives.................................... 4-22
Deporting Disk Groups......................................................... 4-24
Importing Disk Groups ......................................................... 4-25
Destroying a Disk Group ...................................................... 4-26
Renaming VxVM Disk Drives.............................................. 4-27
Displaying VEA Object Properties ...................................... 4-28
Exercise: Performing VxVM Basic Operations ............................ 4-29
Preparation.............................................................................. 4-30
Task 1 Reviewing Key Lecture Points.............................. 4-31
Task 2 Verifying the VxVM Environment....................... 4-32
Task 3 Verifying the Initial Disk Drive Status ................ 4-33
Task 4 Setting the Default Disk Drive Format ................ 4-34
Task 5 Initializing Disk Drives.......................................... 4-35
Task 6 Creating New Disk Groups.................................... 4-36
Task 7 Viewing Command Logs....................................... 4-38
Task 8 Importing and Deporting Disk Groups................ 4-38
Task 9 Destroying a Disk Group....................................... 4-40
Task 10 Renaming Disk Drives .......................................... 4-40
Task 11 Using the vxdiskadm Utility to Perform Basic
Operations (Optional) ......................................................... 4-41
Task 12 Verifying Ending Lab Status ............................... 4-42
Exercise Summary............................................................................ 4-43
VERITAS Volume Manager Volume Operations ............................5-1
Objectives ........................................................................................... 5-1
Interpreting Volume Structure Listings.......................................... 5-2
Subdisks...................................................................................... 5-2
Plexes .......................................................................................... 5-3
Volumes...................................................................................... 5-4
Volume Structure Examples................................................... 5-5
Volume Planning ............................................................................... 5-6
Volume Distribution................................................................. 5-6
Volume Naming Conventions ............................................... 5-8
Space Allocation Planning ....................................................... 5-8
Selecting Volume Types......................................................... 5-11

viii
Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Creating Volumes Using the VEA GUI ........................................ 5-13


Disk Selection Method............................................................ 5-13
Using the Disk Selection Form............................................. 5-14
Using the Volume Attributes Form..................................... 5-15
Using the Create File System Form ..................................... 5-16
Creating Volumes Using the vxassist Command.................... 5-17
The vxassist Command ...................................................... 5-17
Specifying Volume Size.......................................................... 5-17
Using vxassist Command Options .................................. 5-18
Modifying Volume Access Attributes........................................... 5-20
Verifying Volume Ownership............................................... 5-20
Modifying Volume Ownership and Permissions............... 5-20
Adding a UFS File System to Existing Volumes ......................... 5-21
Using the VEA GUI to Add a File System........................... 5-21
Adding a File System From the Command Line............... 5-23
Enabling the Solaris OS UFS Logging Feature ................... 5-24
Administering Volume Logs.......................................................... 5-25
Using DRLs .............................................................................. 5-25
Using RAID-5 Logs................................................................ 5-26
Planning Log Placement ....................................................... 5-27
Adding a Volume Log From the VEA GUI......................... 5-28
Adding a Volume Log From the Command Line ............. 5-29
Removing Volume Logs Using the VEA GUI.................... 5-30
Removing Volume Logs From the Command Line.......... 5-31
Using the VEA GUI to Analyze Volume Structures ................... 5-32
Displaying Volume Layout Details ...................................... 5-32
Viewing Disk Volume Mapping and Performance........... 5-33
Exercise: Creating Volumes and File Systems ............................. 5-34
Preparation............................................................................... 5-34
Task 1 Reviewing Key Lecture Points.............................. 5-35
Task 2 Creating a Volume.................................................. 5-37
Task 3 Adding a Volume Mirror ...................................... 5-39
Task 4 Adding a File System to a Volume........................ 5-41
Task 5 Adding a DRL ......................................................... 5-43
Task 6 Resizing a Volume and File System..................... 5-45
Task 7 Creating a RAID-5 Volume ................................... 5-47
Task 8 Analyzing Volumes Using the VEA GUI............ 5-49
Task 9 Verifying Ending Lab Status ................................. 5-54
Exercise Summary............................................................................ 5-55
VERITAS Volume Manager Advanced Operations ....................... 6-1
Objectives ........................................................................................... 6-1
Boot Disk Encapsulation and Mirroring......................................... 6-2
Optimizing the Boot Disk Hardware Configuration ........... 6-2
Boot Disk Encapsulation Prerequisites ................................. 6-3
Encapsulating the System Boot Disk..................................... 6-4

ix

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Mirroring the System Boot Disk ............................................ 6-5


Verifying the Completed Boot Disk Configuration ............ 6-6
Creating a Best Practice Boot Disk Configuration......................... 6-8
Establishing Boot Disk Configuration Policies .................. 6-10
Modifying an Existing Boot Disk Configuration............... 6-11
Administering Hot Devices ............................................................ 6-13
Selecting Hot Device Operational Mode ............................. 6-13
Evaluating Hot-Device Configurations .............................. 6-16
Administering Hot Devices Using Command-Line
Programs .............................................................................. 6-17
Administering Hot Devices Using the VEA GUI.............. 6-18
Controlling Relocation Recovery Time............................... 6-19
Evacuating a Disk Drive ................................................................. 6-20
Identifying Evacuation Conflicts .......................................... 6-20
Preparing for Evacuation ....................................................... 6-20
Identifying Suitable Evacuation Disk Drives..................... 6-21
Evacuation Using the vxevac Command .......................... 6-22
Evacuation Using the VEA GUI........................................... 6-23
Evacuation Using the vxdiskadm Utility ........................... 6-24
Moving Disk Drives Without Preserving Data ........................... 6-25
Moving a Disk Drive Using the Command Line................ 6-25
Moving a Disk Drive Using the VEA GUI ......................... 6-26
Moving Populated Disk Drives to a New Disk Group............... 6-27
Evaluating Disk Drive Involvement .................................... 6-27
Saving the Configuration....................................................... 6-28
Moving the Disk Drives to a New Disk Group .................. 6-29
Reloading the Volume Configuration.................................. 6-30
Backing Up and Restoring Disk Group Configurations ............ 6-31
Automatic Configuration Backup ........................................ 6-31
Manual Configuration Backup and Restore ...................... 6-33
Importing Disk Groups After a System Crash ............................ 6-36
Importing the rootdg Disk Group After a Crash ............. 6-37
Volume Snapshot Operations ........................................................ 6-38
Snapshot Process..................................................................... 6-38
Using the VEA GUI to Create a Snapshot .......................... 6-39
Using the Command Line to Create a Snapshot ............... 6-43
Online Volume Relayout ................................................................ 6-44
Volume Relayout Prerequisites............................................. 6-44
Volume Relayout Using the Command Line..................... 6-45
Volume Relayout Using the VEA GUI................................ 6-47
Creating Layered Volumes............................................................. 6-49
Layered Volume Disk Requirements ................................... 6-49
Evaluating Available Disk Space......................................... 6-50
Creating Layered Volumes From the Command Line ..... 6-51
Creating Layered Volumes Using the VEA GUI............... 6-52
Identifying Layered Volume Subcomponents................... 6-53

x
Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Basic Intelligent Storage Provisioning Administration .............. 6-54


Primary ISP Components ...................................................... 6-54
Using Storage Pool Set Templates ....................................... 6-56
Using Storage Pool Templates ............................................. 6-58
Using Application Volume Templates ............................... 6-60
Creating Application Volumes Using the vxvoladm
Command............................................................................. 6-62
Creating Application Volumes Using the VEA GUI ........ 6-63
Interpreting Application Volume Configurations ............ 6-65
Replacing Failed Disk Drives ......................................................... 6-66
Failure Behavior ...................................................................... 6-66
Evaluating Failure Severity ................................................... 6-67
General Disk Drive Replacement Process .......................... 6-70
Exercise: Performing Advanced Operations................................ 6-72
Preparation............................................................................... 6-72
Task 1 Reviewing Key Lecture Points.............................. 6-73
Task 2 Encapsulating the System Boot Disk ................... 6-76
Task 3 Mirroring the System Boot Disk ............................ 6-78
Task 4 Performing an Online Volume Relayout............. 6-80
Task 5 Evacuating a Disk Drive ........................................ 6-82
Task 6 Moving a Populated Volume................................. 6-82
Task 7 Performing a Snapshot Backup ............................ 6-84
Task 8 Creating a Layered Volume .................................. 6-85
Task 9 Replacing a Failed Disk Drive .............................. 6-86
Task 10 Using Intelligent Storage Provisioning .............. 6-88
Task 11 Configuring a Best Practice Boot Disk............... 6-91
Exercise Summary............................................................................ 6-93
VERITAS File System Basic Operations........................................ 7-1
Objectives ........................................................................................... 7-1
Basic VxFS Features ........................................................................... 7-2
Extent-Based Space Allocation................................................ 7-2
File System Intent Logging ..................................................... 7-3
Installing the VxFS Software ............................................................ 7-4
Creating VxFS File Systems .............................................................. 7-5
Extended VxFS Mount Options ....................................................... 7-6
Intent Log Behavior .................................................................. 7-6
Error Handling Behavior ......................................................... 7-7
Other VxFS Mount Options..................................................... 7-7
Online File System Administration ................................................. 7-8
Online Defragmentation .......................................................... 7-8
Online Resizing ......................................................................... 7-8
Online Backup and Restore ..................................................... 7-8

xi

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Configuring VxFS.............................................................. 7-9


Preparation................................................................................. 7-9
Task 1 Reviewing Key Lecture Points.............................. 7-10
Task 2 Installing the VxFS Software................................. 7-11
Task 3 Creating a VxFS File System ................................. 7-13
Task 4 Resizing a VxFS File System .................................. 7-14
Task 5 Defragmenting a VxFS File System....................... 7-15
Task 6 Backing Up and Restoring a VxFS File System ... 7-16
Task 7 Using VxFS Extended Mount Options ................. 7-17
Exercise Summary............................................................................ 7-19
VERITAS Volume Manager Performance Management ................8-1
Objectives ........................................................................................... 8-1
Performance Improvement Techniques ......................................... 8-2
Data Assignment Strategies..................................................... 8-2
Volume Structure Strategies.................................................... 8-4
Read Policy Strategies ............................................................. 8-6
Hardware Configuration Strategies ....................................... 8-7
Using Performance Analysis Tools ................................................. 8-8
Gathering Volume Performance Statistics ............................ 8-8
Gathering Application Performance Statistics................... 8-11
RAID-5 Write Performance ............................................................ 8-12
Read-Modify-Write Operations ............................................ 8-12
Reconstruct-Write Operations............................................... 8-13
Full-Stripe Write Operations ................................................. 8-14
Exercise: Demonstrating Performance Differences..................... 8-15
Preparation............................................................................... 8-15
Task 1 Reviewing Key Lecture Points.............................. 8-16
Task 2 Performing a RAID-5 Write Performance Test .. 8-18
Task 3 Performing a Striped Volume Write Performance
Test ........................................................................................ 8-21
Exercise Summary............................................................................ 8-22

xii
Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Preface

About This Course


Course Goals
Upon completion of this course, you should be able to:

Install and initialize VERITAS Volume Manager (VxVM) software

Define VxVM objects

Describe public and private regions

Start and customize the Volume Manager Storage Administrator


VERITAS Enterprise Administrator (VEA) graphical user interface
(GUI)

Perform operations using the command-line interface

Perform disk and volume operations

Create redundant array of independent disk (RAID) volumes

Set up dirty-region logs (DRLs)

Perform common file system operations using the VEA GUI

Create new disk groups, remove disks from group, move disks
between disk groups, and deport and import disk groups between
servers

Simulate disk failure and complete a disk recovery

Create and mange hot-spare pools

Manage and disable the hot-relocation feature

Perform basic performance analysis

Preface-xiv
Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Course Map

Course Map
The following course map enables you to see what you have
accomplished and where you are going in reference to the course goals.

Overview
Sun Storage
Concepts

Managing Data

VERITAS Volume Manager Introduction


VERITAS Volume
Manager
Installation

VERITAS Volume
Manager Basic
Operations

VERITAS Volume Manager Operations


VERITAS
Volume
Manager
Volume
Operations

VERITAS
Volume Manager
Advanced
Operations

VERITAS
File System
Basic
Operations

VERITAS File System Operations


VERITAS
Volume Manager
Performance
Management

Preface-xv

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Topics Not Covered

Topics Not Covered


This course does not cover the following topics. Many of these topics are
covered in other courses offered by Sun Educational Services:

Solaris Operating System (Solaris OS) administration Covered in


SA-239: Intermediate System Administration for the Solaris 9 Operating
Environment and SA-299: Advanced System Administration for the
Solaris 9 Operating Environment

Storage system maintenance Covered in SM-250: Sun Software


RAID Storage Systems Maintenance

Refer to the Sun Educational Services catalog for specific information and
registration.

About This Course


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Preface-xvi

How Prepared Are You?

How Prepared Are You?


To be sure you are prepared to take this course, can you answer yes to the
following questions?

Preface-xvii

Can you perform the following Solaris 9 OS administration


operations?

Configure environmental variables

Control system run levels

Use basic OpenBoot commands

Can you physically configure the following Sun systems and


peripherals?

Desktop or server-class systems

Sun storage arrays

Can you use the following UNIX administrative commands?

chmod, chown, and chgrp

format

man

mkdir

mkfs/newfs

mount

patchadd

pkgadd and pkgrm

vi

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Introductions

Introductions
Now that you have been introduced to the course, introduce yourself to
the other students and the instructor, addressing the following items:

Name

Company affiliation

Title, function, and job responsibility

Experience related to topics presented in this course

Reasons for enrolling in this course

Expectations for this course

About This Course


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Preface-xviii

How to Use Course Materials

How to Use Course Materials


To enable you to succeed in this course, these course materials employ a
learning module that is composed of the following components:

Preface-xix

Goals You should be able to accomplish the course goals after


finishing this course and meeting all of its objectives.

Objectives You should be able to accomplish the objectives after


completing a portion of instructional content. Objectives support
goals and can support other higher-level objectives.

Lecture The instructor presents information specific to the


objectives of the module. This information helps you learn the
knowledge and skills necessary to succeed with the activities.

Activities The activities take on various forms, such as exercises,


self-checks, discussions, and demonstrations. Activities are used to
facilitate mastery of an objective.

Visual aids The instructor might use several visual aids to convey a
concept, such as a process, in a visual form. Visual aids commonly
contain graphics, animation, and video.

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Conventions

Conventions
The following conventions are used in this course to represent various
training elements and alternative learning resources.

Icons

Discussion Indicates a small-group or class discussion on the current


topic is recommended at this time.

Note Indicates additional information that can help students but is not
crucial to their understanding of the concept being described. Students
should be able to understand the concept or complete the task without
this information. Examples of notational information include keyword
shortcuts and minor system adjustments.
Caution Indicates that there is a risk of personal injury from a
nonelectrical hazard, or risk of irreversible damage to data, software, or
the operating system. A caution indicates that the possibility of a hazard
(as opposed to certainty) might happen, depending on the action of the
user.

About This Course


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Preface-xx

Conventions

Typographical Conventions
Courier is used for the names of commands, files, directories,
programming code, and on-screen computer output; for example:
Use ls -al to list all files.
system% You have mail.
Courier bold is used for characters and numbers that you type; for
example:
To list the files in this directory, type:
# ls

Courier italics is used for variables and command-line placeholders


that are replaced with a real name or value; for example:
To delete a file, use the rm filename command.

Courier italics bold is used to represent variables whose values are


to be entered by the student as part of an activity; for example:
Type chmod a+rwx filename to grant read, write, and execute
rights for filename to world, group, and users.
Palatino italics is used for book titles, new words or terms, or words that
you want to emphasize; for example:
Read Chapter 6 in the Users Guide.
These are called class options.

Preface-xxi

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Notes to the Instructor

Notes to the Instructor


For classroom setup suggestions, refer to the ES310_revC_setup.txt file.
The ES310_revC_setup.txt file contains specific setup instructions about:

General equipment configuration for this course

Patch download instructions

VxVM requirements and download instructions

Supported hardware

Required lab files

About This Course


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Preface-xxii

Module 1

Sun Storage Concepts


Objectives
Upon completion of this module, you should be able to:

Describe the major disk storage administration tasks

Describe Sun storage interface types

Describe available RAID technologies including:

Host-based RAID technology

Controller-based RAID technology

Describe disk storage concepts that are common to many storage


installations including:

Hot swapping

Storage area networking

Multihost access

Multipath access

Identify storage configurations including:

Conducting physical inventory

Displaying storage configurations

Identifying controller addresses

Decoding logical device paths

Verify storage array firmware revisions

1-1
Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Disk Storage Administration Introduction

Disk Storage Administration Introduction


Installing, configuring, and managing a VxVM installation requires both
software and hardware knowledge.

VxVM Software Installation


Installing the VxVM software is essentially the same as installing any
Solaris OS application. You can use either the standard command-line or
GUI installation tools.

VxVM Initialization
When you install VxVM, at least one disk drive must be brought under
VxVM control using the vxinstall utility. You can either encapsulate a
disk, which preserves existing data on the disk, or you can initialize a disk,
which effectively destroys existing data.

Required Hardware Knowledge


The VxVM installation process is always the same, regardless of the
system platform or storage technology used. However, you must be able
to identify storage array device addresses and differentiate them from
other types of disk storage addresses.
If you are not familiar with the device address strategy in your particular
installation, you might accidentally initialize the wrong disk drives. This
error could destroy valuable data, including the operating system.

1-2

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Disk Storage Administration Introduction

RAID Volume Design


Generally, RAID volume structures are designed with one or more of the
following goals in mind:

Cost savings

Performance

Availability

Maintainability

In most cases, compromises are made when choosing among cost savings,
performance, availability, and maintainability.

Required Hardware Knowledge


You must have a thorough understanding of interface types, addressing
schemes, and internal hardware structure to achieve design goals. It is
possible to design virtual volume structures without this background
knowledge, but the result might perform poorly and might not have the
reliability that is required for your application.

RAID Volume Creation


You can create RAID volume structures using the VxVM GUI or
command-line programs. Command-line programs are sometimes a better
choice to use, especially when volume creation must be automated using
script files.
You can configure the GUI to display command-line equivalents for each
operation.

Required Hardware Knowledge


Even though you might not be responsible for the design of your VxVM
volume structures, you must still be familiar with most aspects of your
particular storage devices.

Sun Storage Concepts


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

1-3

Disk Storage Administration Introduction

RAID Volume Administration


In larger installations, the most common VxVM administrative task is
identifying and replacing failed disk drives. At the most basic level, this
task involves the use of a single VxVM utility, the vxdiskadm utility. For
certain storage platforms, you must use another utility, luxadm, during
the disk replacement process.
Most VxVM administrative tasks require analyzing error messages using
VxVM utilities, such as vxprint and vxdisk, along with some basic
Solaris OS commands.

Required Hardware Knowledge


Administering RAID volumes requires a number of hardware-related
skills including:

1-4

Decoding device error messages

Relating device addresses to physical devices

Following hardware removal procedures that are appropriate for


each particular disk storage technology

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Interfaces for Sun Storage Devices

Interfaces for Sun Storage Devices


Sun storage devices are accessed through one of two basic interface types:

Small Computer System Interface (SCSI)

Fiber-optic

Each of the basic interface types has two or more variations, which have
evolved over a period of several years. The interfaces have improved in
the following areas:

Data transfer speed

Data transfer latency

Interface cable lengths

SCSI Overview
SCSI was initially implemented in the 1980s as a way of making the
interface between the host computer system and the disks independent of
the computer manufacturer. Prior to the introduction of SCSI, all the
computer manufacturers had their own way of connecting the host
computer system to the disk drives.
SCSI introduced the idea of intelligent disk drives where the host
computer system requested the transfer of a block of data from the disk.
The host system had no need to know the underlying disk geometry. It
issued a request to the disk for the transfer of a block of data. The shift of
intelligence from the host computer system to the disk allowed the same
disk to be used by different manufacturers, which ultimately led to
cheaper, faster, and larger disk drives.
The connection between the host system was by the SCSI bus, for which a
set of standards was agreed upon. The speed and data capacity of the
SCSI bus has been increased to allow for the higher demands of todays
servers. One of the earliest problems faced with SCSI was the differing
cable lengths from the host system to the disk drives themselves. For the
SCSI bus to reliably operate over differing cable lengths, two electrical
connections methods were defined: Single-ended (for short connection
lengths) and differential (for connection over longer cables).

Sun Storage Concepts


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

1-5

Interfaces for Sun Storage Devices

SCSI Interface Implementation


Both single-ended SCSI and differential SCSI are currently used with Sun
storage products.

Single-Ended SCSI
As shown in Figure 1-1, each bit is transmitted using one signal,
referenced to 0 volts. This allows cable lengths up to six meters with
standard SCSI-1 devices.

Figure 1-1

Single-Ended SCSI

Differential SCSI
As shown in Figure 1-2, the data bits are sent using two equal and
opposite voltages. These allow the signal to travel farther without
degradation. Differential SCSI allows cable lengths up to 25 meters.

inv

Figure 1-2

1-6

Differential SCSI

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

add

Interfaces for Sun Storage Devices

SCSI Interface Standards


A partial list of the SCSI-1, SCSI-2, and SCSI-3 standards currently used
by Sun are shown in Table 1-1. Other SCSI standards are used by different
manufacturers.
Table 1-1 Sun SCSI Standards
Name
(Type)

Clock
Speed

Data
Width

Data Rate
(Command
Rate)

Number
of
Devices

SCSI
(SCSI-1)

5 MHz

8 bits

5 Mbytes/sec

Fast SCSI
(SCSI-2)

10 MHz

8 bits

10 Mbytes/sec

Wide SCSI
(SCSI-2)

5 MHz

16 bits

10 Mbytes/sec

16

Fast/Wide
SCSI
(SCSI-2)

10 MHz

16 bits

20 Mbytes/sec

16

Ultra
Differential
Wide SCSI
(SCSI-3)

20 MHz

16 bits

40 Mbytes/sec

16

Sun Storage Concepts


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

1-7

Interfaces for Sun Storage Devices

SCSI-3 Standards
Currently, only recent Sun systems such as the Ultra 60 workstation,
Ultra 80 workstation, and the Sun Blade 1000 workstations, have SCSI-3
compatible interfaces. The related specifications are:

Ultra 60 workstation 68-pin connector, 40-Mbyte/sec Ultra SCSI


(SCSI-3), two channels (synchronous)

Ultra 80 workstation 68-pin connector, 40-Mbyte/sec Ultra SCSI


(SCSI-3), two channels (synchronous)

Sun Blade 1000 workstation One single-ended 68-pin SCSI


connector (16-bit Ultra/Fast/Wide, 40-Mbyte/sec), internal disks are
Fibre Channel

Table 1-2 shows some of the current SCSI-3 standards.


Table 1-2

SCSI-3 Standards
High
Voltage
Differential
Length

Low
Voltage
Differential
Length

Targets

Type

Speed

Width

SingleEnded
Length

SCSI 3U

20 MB/sec

8 bits

1.5 m

25 m

SCSI 3 U

20 MB/sec

8 bits

3m

SCSI 3 FW

20 MB/sec

16 bits

6m

25 m

16

SCSI 3 UW

40 MB/sec

16 bits

25 m

16

SCSI 3 UW

40 MB/sec

16 bits

1.5 m

SCSI 3 UW

40 MB/sec

16 bits

3m

SCSI 3 U2

40 MB/sec

8 bits

12 m

12 m

SCSI 3 U2

80 MB/sec

16 bits

25 m

25 m

SCSI 3 U2W

80 MB/sec

16 bits

12 m

12 m

16

SCSI 3 U2W

80 MB/sec

16 bits

25 m

25 m

1-8

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Interfaces for Sun Storage Devices

SCSI Priority
The bus arbitration mechanism for SCSI uses the SCSI target ID to
determine priority. Narrow SCSI has target addresses 07. Target 7 has
highest priority (usually the ID of the controller), and target 8 has the
lowest. Performance can be affected through injudicious use of SCSI target
addresses.
Wide SCSI uses target addresses 015 in the following priority: 7, 6, 5, 4, 3,
2, 1, 0, 15, 14, 13, 12, 11, 10, 9, 8. Address 7 is the highest priority.

SCSI Phases and the Move to Fibre Channel


Although it appears as if SCSI has improved performance eight-fold, this
is an exaggeration. SCSI has six phases, one of which is data transfer to or
from the disk. Other phases include sense (status), command transfer, and
bus arbitration. SCSI supports backward compatibility, so a narrow, slow
disk can run off a fast-wide controller. This capability is achieved by
having the administration of SCSI run at 5 megahertz (MHz) on narrow
cabling. Each device is allowed to negotiate upwards. That is, to be fast,
ultra fast, wide, and so on. The data transfer phase benefits from the
improved clock speed, but command transfer does not. It takes about
5 milliseconds (ms) on each form of SCSI.
The last 10 years have witnessed a 10,000-fold increase in computer
performance. At the same time, requirements are increasing for more
robust, highly available, disaster-tolerant computing resources. Computer
resources continue to be pushed to their limits.

Fibre Channel Technology


Fibre Channel technology is the answer to the growing problems of
SCSI-based peripherals. Fibre Channel is a high-performance serial
interconnect standard designed for bidirectional, point-to-point
communications between servers, storage systems, workstations,
switches, and hubs. It offers a variety of benefits over other link-level
protocols, including efficiency, high performance, scalability, simplicity,
ease of use and installation, and support for popular high-level protocols.
Fibre Channel also offers a higher level of security and reliability through
electrical isolation. There is no electromagnetic (EM) influence.

Sun Storage Concepts


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

1-9

Interfaces for Sun Storage Devices

Fibre Channel-Arbitrated Loop


An important enhancement to Fibre Channel technology has been the
development of Fibre Channel-Arbitrated Loop (FC-AL), which was
developed specifically to meet the needs of storage interconnects. Using a
loop topology, FC-AL can support basic configurations and sophisticated
arrangements of hubs, switches, servers, and storage systems.
Furthermore, by using SCSI protocols over the much faster, more robust
Fibre Channel link, FC-AL provides higher levels of performance without
requiring expensive and complex changes to existing device drivers and
firmware.

Advantages of FC-AL
The FC-AL development effort is part of the American National Standards
Institute/International Organization for Standardization (ANSI/ISO)
accredited SCSI-3 standard. This standard helps to prevent the creation of
non-conforming, incompatible implementations. Virtually all major
system vendors are implementing FC-AL, as are all major disk drive and
storage system vendors.
FC-AL operates on both fiber-optic cable and copper wire, and it can be
used for more than just disk input/output (I/O). The Fibre Channel
specification supports high-speed system and network interconnects
using a wide variety of popular protocols, including:

1-10

SCSI

Internet Protocol (IP)

Adaptation Layer for Computer Data (AAL5) (ATM)

Fibre Channel Link Encapsulation (FC-LE)

Institute of Electrical and Electronics Engineers specification for data


link layer transmission (IEEE 802.2)

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Interfaces for Sun Storage Devices

Fibre Channel Compared to SCSI


As shown in Table 1-3, Fibre Channel technology has many advantages
compared to SCSI.
Table 1-3 Fibre Channel Compared to SCSI
Fibre Channel

SCSI

Full duplex operation

Half-duplex operation

100 Mbytes/sec

40 Mbytes/sec

Performance not affected by


disk position.

Performance can be negatively


affected by the physical disk
position.

Multipath disk access is


supported.

A disk can connect to only one


SCSI channel.

Up to four hosts can connect to


an appropriate array.

A maximum of two hosts can


connect to a single SCSI
channel.

Serial data transmission

Parallel data transmission

Suitable for use in a SAN. It


allows the use of switches and
hubs.

Unsuitable for use in a SAN. It


allows no use of switches and
hubs.

Can connect 30 km (25 MHz)/


10 km (100 MHz) maximum
distances.

Maximum distance is 25 meters


with a differential SCSI.

Sun Storage Concepts


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

1-11

RAID Technology

RAID Technology
RAID virtual data structures can be created and managed by software
applications, or they can be a resident-hardware function of some storage
devices.

Host-Based RAID (Software RAID Technology)


VxVM is a good example of software RAID technology. All Sun storage
arrays can be used with VxVM. As shown in Figure 1-3, user applications
access a virtual structure through a single path that is composed of three
separate disk drives.

User or
Application
Access

VM
Software

3-Gbyte
virtual volume

Controller c4
1-Gbyte
physical
disks
T1

T2

T3

Storage Array

Figure 1-3

Host-Based RAID Technology

A typical virtual volume path name would be similar to the following:


/dev/vx/dsk/dga/volume-01
Although the physical paths to the three disk drives in Figure 1-3 still
exist, they are not accessed directly by users or applications. Only the
virtual volume paths are referenced by users.
Software that runs on the host system creates and manages the virtual
software.

1-12

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

RAID Technology

Controller-Based RAID (Hardware RAID Technology)


Controller-based RAID solutions use firmware that runs internally on
storage array logic boards to maintain virtual structures that are
composed of one or more physical disk drives. Sun hardware RAID
models include: Sun StorEdge A1000, Sun StorEdge A3000, Sun
StorEdge A3500, Sun StorEdge A3500FC, and Sun StorEdge T3 arrays.
As shown in Figure 1-4, RAID configuration software (Sun StorEdge
RAID Manager) running on the host system configures virtual structures
in the array controller board. After initial configuration, the controller
board firmware manages the virtual structures.
Storage Array

Host System

RAID
Firmware

Host Bus
Adapter

User
Access
RAID
Configuration
Software

Figure 1-4

Disk
Disk
Disk
Disk

Disk
Disk
Disk
Disk

Controller-Based RAID Technology

Note The Sun StorEdge T3 array RAID structures are configured using
either the Sun StorEdge Component Manager software or resident storage
array operating system commands.
A typical hardware RAID device appears to be the same as any logical
device path, such as /dev/dsk/c0t5d0s0. Applications do not recognize
the underlying RAID structures.
Hardware RAID solutions can have better performance than host-based
RAID solutions for some types of RAID structures. Hardware RAID
overhead calculations are performed at very high speeds by the
controller-resident hardware, instead of on the host system as in
host-based RAID. Hardware-based RAID-5 write performance can be
much better than host-based RAID 5.

Sun Storage Concepts


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

1-13

Disk Storage Concepts

Disk Storage Concepts


This section describes the following disk storage concepts that are
common to many storage installations:

Hot swapping

Storage area networking

Multihost access

Multipath access

Hot Swapping
Most Sun storage arrays are engineered so that a failed disk drive can be
replaced without interrupting customer applications. The
disk-replacement process includes one or more software operations that
can vary with each disk storage platform.

General VxVM Disk-Replacement Procedure


In its basic form, the process to replace a failed disk drive that is under
VxVM control is as follows:
1.

Use the VxVM vxdiskadm utility to logically remove the disk.

2.

Use the VxVM vxdiskadm utility to logically install the new disk.

Caution Never hot-swap a disk that is under control of VxVM. Always


use the vxdiskadm utility or command-line equivalents.

1-14

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Disk Storage Concepts

Alternative Disk-Replacement Procedure


The VxVM disk replacement process is more complex for some storage
arrays, such as the Sun StorEdge A5x00 array. The Sun StorEdge A5x00
array procedure is as follows:

1.

Use the VxVM vxdiskadm utility options 4 and 11 to logically


remove the disk and place it offline.

2.

Use the luxadm utilitys remove_device command.

3.

Use the luxadm utilitys insert_device command.

4.

Run the vxdctl enable command to read in the new configuration.

5.

Use the VxVM vxdiskadm utility option 5 to logically install the new
disk.

Note You must be familiar with the disk-replacement process for your
particular disk storage devices.

Sun Storage Concepts


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

1-15

Disk Storage Concepts

Storage Area Networking


The current Sun storage area network (SAN) implementation is composed
of the following hardware and software:

Supported arrays are Sun StorEdge A3500, Sun StorEdge, A5200, Sun
StorEdge T3, and Sun StorEdge T3+ arrays

Sun Fibre Channel switches (8 and 16 ports)

Management software (fabric and switch management)

SANs are usually based on the Fibre Channel data communications


standard.

Basic SAN Configuration


Traditionally, data is made available through Network File System (NFS)
mounts over Ethernet user networks. This configuration is referred to as
network-attached storage (NAS). User networks can be overloaded by heavy
data traffic. A SAN is a Fibre Channel network that off-loads data traffic
from the user network. As shown in Figure 1-5, a SAN is a highperformance network composed of servers, storage devices, and
interconnect devices, such as switches, hubs, cables, and converters.
Switches
Sun StorEdge T3 Array Partner Pair
TL

Host Adapter

Host Adapter
TL

Fiber-optic cables

Figure 1-5

SAN Components

A SAN can enable gigabit-speed data transfer with high-availability (no


single-point-of-failure). Redundant SAN switches can be configured to
have automatic failover capability. Data is automatically re-routed
through a redundant switch.

1-16

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Disk Storage Concepts

SAN Definitions
The SAN fabric. is composed of several elements. Without a basic
understanding, the terminology can be confusing or meaningless.

SAN Fabric
Networks that use Fibre Channel switches are referred to as fabrics. The
term fabric characterizes a network of multiple switches as opposed to a
network with a single switch. Each connection in a fabric can use the full
100-megabytes per second (Mbyte/sec) Fibre Channel bandwidth.
Switches in a fabric use a routing technique known as cut-through
switching. Cut-through switching refers to the process by which the Fibre
Channel switch can route the incoming data frame almost immediately by
reading its link-level destination ID (D_ID). It does not need to read the
entire frame prior to transmitting it to its target.

Fibre Channel Frames


The data elements referred to as data packets in Transmission Control
Protocol/Internet Protocol (TCP/IP) are referred to as frames in Fibre
Channel. The basic Fibre Channel frame structure contains a 24-byte
header followed by up to 2112 bytes of data. A key portion of the 24-byte
header is a 3-byte link-level D_ID that defines the port address to which
the data frame must be transmitted.

Fibre Channel Devices


There are two types of Fibre Channel devices, public and private.
A Fibre Channel private loop is a traditional storage configuration, such
as one or more Sun StorEdge A5200 arrays, connected to a host system
either singly, in a daisy-chain configuration, or through a Fibre Channel
hub. The devices are available on the loop only to the physically
connected host.
Private devices do not have full Fibre Channel addressing capability. They
have only the Arbitrated Loop Physical Address (ALPA) portion of the
Fibre Channel physical address. These devices exist only on loops, and,
unless the Switch offers extra support, these devices cannot communicate
outside their own loop.

Sun Storage Concepts


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

1-17

Disk Storage Concepts


Public devices have full Fibre Channel addressing capability and,
therefore, can communicate with any other public device on the fabric.
They can be connected directly to the switch (one device per port) or
arranged in an arbitrated loop with up to 127 nodes in the loop and the
loop connected to the switch.

Switch Port Functions


You can configure the Fibre Channel switch ports to function in several
ways using switch management software. The primary reason for
different port functionality is to allow selective access between Fibre
Channel devices and host systems. You should only use the following
port configurations:

Fabric port (F_Port) A fabric port connects a Fibre Channel switch


to a fabric-aware node port (or N_Port) on an end-device.

Segmented loop port (SL_Port) Segmented loop ports provide


support for private arbitrated loops on a Fibre Channel switch. All
segmented loop ports in the same SL zone behave as one private
arbitrated loop (and so they share the same ALPA space).

Translated loop port (TL_Port) Translated loop ports provide


support for public and private loop devices on a Fibre Channel
switch. Translated loop ports translate between private and public
addresses, allowing public devices and private devices to
communicate with one another.

Trunk port (T_Port) A trunk port connects a Fibre Channel switch


to another Fibre Channel switch (this is known as cascading).

Zones
Zoning refers to the deliberate segregation of SAN resources from other
SAN resources. Zones essentially create sub-networks that provide
different levels of connectivity or addressability between different hosts
and devices on the network. Routing tables control the access of hosts to
devices. You can individually configure device ports to be accessible or
inaccessible to other specific ports.

1-18

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Disk Storage Concepts

Sample SAN Configuration


The example in Figure 1-6 shows how a multihost SAN configuration
might be arranged to support both private and public use of storage
resources.
Sun StorEdgeTM L180
or Sun StorEdgeTM
L700 FC Tape Library

Sun EnterpriseTM
420 Server

Switch 0

1
Host
Adapter

2
Zone

5 Zone 6
Host
Adapter

Sun StorEdge T3
Array Partner Pairs

7 Zone 8

Sun EnterpriseTM
3500 Server
Switch 1
Host
Adapter

1
3

Host
Adapter

2
Zone

5 Zone 6
7 Zone 8
Sun StorEdge
A5200 Array

Figure 1-6

Sample SAN Multihost Configuration

Note To take advantage of the multipath connections from each host


system, you must use a multipath product, such as VERITAS DMP or the
Sun StorEdge Traffic Manager software.

Sun Storage Concepts


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

1-19

Disk Storage Concepts

SAN Device Addressing


Traditionally, the Solaris OS has named storage devices based on the
controller, target ID, and logical unit number of the device. When large
SANs with multiple paths to large storage arrays are constructed, this
naming convention could lead to thousands of targets per controller and
storage identified more than once by different controllers and targets. The
new method incorporates the World Wide Name (WWN) of the device
into the device name used by the host. The new names provide the benefit
of uniquely identifying storage devices to the host. When using
Sun StorEdge Traffic Manager software, a device with multiple
connections to a host is known to that host by one name.
Old device path:
/devices/pci@f,4000/pci@4/SUNW,qlc@4/fp@0,0/ssd@3,0
New device path:
/devices/pci@f,4000/pci@4/SUNW,qlc@4/fp@0,0/ssd@w50020f2000
00225,0
Old symbolic device name:
/dev/dsk/c4t3d0s2
New symbolic designation:
/dev/dsk/c4t50020f200000225d0s2
The number of storage devices that can be attached to a host can grow to
the thousands with the advent of SANs with native fabric connectivity.
Probing all these devices at boot time and creating device nodes can
increase the boot time greatly. In addition, a host might not need access to
all of the storage devices it can access. The Sun StorEdge Network FC
Switch-16, Version 3.0, no longer creates device nodes for every storage
device attached. Instead, the administrator creates device nodes on
demand by using the cfgadm utility. The device nodes, once created, are
persistent across reboots.

1-20

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Disk Storage Concepts

Multihost Storage Access


With the advent of technology, such as the Sun StorEdge A5200 array, as
many as four different hosts can be connected to the same storage device.
Multihost connections are also possible on some versions of the SCSI.

Multi-Initiated SCSI
Sun StorEdge MultiPack storage devices support physical SCSI interface
connections from two different host systems. The SCSI interface on each
of the systems must have a different initiator identifier (ID) setting, which
is a system firmware configuration known as the scsi-initiator-id.
As shown in Figure 1-7, you must change the scsi-initiator-id on one
of the host systems to eliminate the addressing conflict between the two
host systems.
Host System B

Host System A
Internal SCSI Bus

Internal SCSI Bus

scsi-initiator-id=7

scsi-initiator-id=7

Internal SCSI Bus

Internal SCSI Bus

scsi-initiator-id=6

scsi-initiator-id=7

SCSI
Card

SCSI
Card

In

t9

Out

t12

t10
t11

Figure 1-7

t13
t14

Multi-Initiated SCSI Configuration

The SCSI initiator values are changed using complex system firmware
commands. The process of changing these values varies with system
hardware platforms.
Do not change the external SCSI bus, scsi-initiator-id, globally, change it at the interface card level.
Read the documentation carefully. The procedures are hardware platform-specific.

Sun Storage Concepts


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

1-21

Disk Storage Concepts

Multihost Fiber-Optic Interface


Two different fiber-optic interface storage arrays support multiple host
connections. The SPARCstorage Array 100 unit allows up to two host
systems to connect to a single storage array. The Sun StorEdge A5000, Sun
StorEdge 5100, and Sun StorEdge 5200 (Sun StorEdge A5x00) arrays allow
up to four host system connections. The serial optical channel (SOC)
connections are shown in Figure 1-8.
SOC+
host adapter

Host 0

Host 1

Sun StorEdge
A5x00 Storage
Array

Interface
Board A

Host 2
Interface
Board B
Host 3

SOC
host adapter

SPARCstorage
Array 100

Host 1
Port A
Host 2
Port B

Figure 1-8

1-22

Multihost Fiber-Optic Configurations

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Interface
board

Disk Storage Concepts

Multipath Storage Access


Multipathing is defined as dual connections to a storage array from a
single host system. Multipathing can be used to provide redundant access
paths in case of hardware failures and, in some cases, to perform load
balancing between the two access paths. There are several different
hardware and software schemes that can be used to implement multipath
configurations.

Redundant Dual Active Controller Driver


Some Sun storage devices allow dual connections to a storage array from
a single host system. As shown in Figure 1-9, one host adapter can be
configured as a backup if the primary access path fails.
Storage Array
Drive

Drive

Drive

Drive

Drive

Drive

Controller

Drive

Ultra SCSI
Card

Drive

Host System

C1

Ultra SCSI
Card
C2

Controller

RDAC
Driver

RAID
Configuration
Software

Figure 1-9

Redundant Dual Active Controller Driver

The Redundant Dual Active Controller (RDAC) driver is a special


purpose driver that manages dual-interface connections. This driver is
available with some of the Sun storage arrays, which include the Sun
StorEdge A3500 and Sun StorEdge A3500FC array models.
Applications directly interface with the RDAC driver and are unaware of
interface failure. If one of the dual-controller paths fails, the RDAC driver
automatically directs I/O to the functioning path.

Sun Storage Concepts


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

1-23

Disk Storage Concepts

Note The Sun StorEdge A3500FC array uses a Fibre Channel interface
instead of the SCSI interface used on the other RDAC-controlled storage
arrays.

Alternate Path Driver for the Solaris OS


The Alternate Path (AP) software for the Solaris OS works with Dynamic
Reconfiguration (DR) to provide the ability to move all I/O off a system
board before removal for upgrade or repair. AP is not applicable to all
architectures.
As shown in Figure 1-10, the AP software package contains user-level
applications and kernel device drivers to allow the construction of
meta-I/O devices from multiple controllers to a single physical device
(disk or network). This function allows I/O on active disk and network
I/O adapters to be seamlessly redirected to a previously specified and
configured I/O device. Failed I/O operations to a device are
automatically retried on the alternate path.
System Interconnect
Board #1

Board #2

Dual-Ported Storage Device

Figure 1-10 Alternate Path Hardware Configuration

1-24

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Disk Storage Concepts

Dynamic Multipathing Driver


The Dynamic Multipathing (DMP) driver is unique to the VxVM product.
It is used only with fiber-optic interface storage arrays. As shown in
Figure 1-11, the DMP driver can access the same storage array through
more than one path. The DMP driver automatically manages multiple
paths to the storage array. Depending on the storage array model, the
paths are either used for load-balancing in a primary mode of operation
or in a backup mode of operation in the event of a failure.
Storage Array
Drive

Drive

Drive

Drive

Drive

Drive

HBA
Card

Interface

Drive

HBA
Card

Drive

Host System

C1

Interface

C2

DMP
Driver

HBA=Host Bus Adapter

Figure 1-11 Dynamic Multipathing Driver


Specific paths can be enabled and disabled with the VxVM vxdmpadm
command.
Note During a VxVM installation, you must take special steps to ensure
that the DMP feature is compatible with AP, SAN, and the Sun StorEdge
Traffic Manager software.

Sun Storage Concepts


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

1-25

Disk Storage Concepts

Sun StorEdge Traffic Manager Software


The Sun StorEdge Traffic Manager software is a new architecture for
representing and managing Fibre Channel devices that are accessible
through multiple host controller interfaces (HCI).
Sun StorEdge Traffic Manager software provides automatic load balancing
and automatic failover to an alternate controller. A typical Sun StorEdge
Traffic Manager software configuration is shown in Figure 1-12.
System Interconnect
Board #1

Board #2

Dual-Ported Storage Device

Figure 1-12 Sun StorEdge Traffic Manager Software Configuration


Sun StorEdge Traffic Manager is the official name for the MPxIO product.

The minimum software and hardware requirements for Sun StorEdge


Traffic Manager software are:

1-26

Solaris 8 OE Update 4

VxVM 3.2

Solstice DiskSuite 4.2.1 software

Sun StorEdge A5200 and Sun StorEdge T3 arrays

Peripheral Component Interconnect (PCI) single- and dual-Fibre


Channel network adapters

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Disk Storage Concepts

Multipath Software Summary


The RDAC, AP, DMP, and Sun StorEdge Traffic Manager software can
coexist in some configurations, but you might need to selectively
configure each interface for use by only one of the applications. Table 1-4
compares some of the multipathing software features.
Table 1-4 Multipath Software Comparison
Functionality

Sun StorEdge
Traffic Manager

Alternate
Pathing

Vendor

Sun

Maximum
paths

DMP

RDAC

Sun

VERITAS
Software
Corporation

LSI Logic/
Symbios

Two

Two

Four

Two

Active/active

Yes

No

Yes

No

Active/passive

Yes

Yes

Yes

Yes

Automatic
failover

Yes

Yes

Yes

Yes

Supported
devices

Sun StorEdge
A5200 array, Sun
StorEdge T3
array

SPARCstorage
Array 100, Sun
StorEdge A5x00
array, Sun StorEdge
T3 array

Any VxVM
supported
disk

LSI Logic
devices
only

Host bus
adapter type

Fibre Channel
only

Fibre Channel only

SCSI and
SCSI and
Fibre Channel Fibre
Channel

Comments

Cannot run
with DMP
at this time

Cannot run with


DMP, and not
supported on
Sun Fire servers

VxVMmanaged
disks only

LSI Logic/
Symbios
controllers
only

Naming

Special names
Special names
/dev/scsi_vhci /dev/ap

Special
names
/dev/vx/dmp

Native
names

Environment

Solaris 8 OE,
Update 4 or
higher

Multiple
operating
systems and
levels

Multiple
operating
systems
and levels

Solaris 2.5.1 or
higher

Sun Storage Concepts


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

1-27

Storage Configuration Identification

Storage Configuration Identification


To be an effective administrator, you must be able to identify storage
configurations and decode logical device addresses.

Conducting Physical Inventory


The first step in determining your storage configuration is to examine and
record the number and types of storage arrays. You can trace cables from
a system to attached storage devices.
It is helpful to create a configuration diagram showing all attached
storage devices and any unique identification information associated with
them, such as world wide numbers, IP addresses, or names.

Displaying Storage Configurations


The format utility and the luxadm utility are commonly used to display
all available storage devices.

Using the luxadm Command to Display Storage


Use the luxadm probe option as follows to locate several types of Sun
storage arrays including Sun StorEdge T3 array logical unit numbers
(LUNs). In the following example, two SENA type arrays were found
along with two single-LUN T3 storage arrays. The luxadm command does
not identify the exact model of storage.
# luxadm probe
Found Enclosure(s):
SENA Name:AA Node WWN:5080020000034ed8
Logical Path:/dev/es/ses0
Logical Path:/dev/es/ses1
Node WWN:50020f200000c193 Device Type:Disk device
Logical Path:/dev/rdsk/c2t1d0s2
Node WWN:50020f200000c367 Device Type:Disk device
Logical Path:/dev/rdsk/c3t1d0s2

1-28

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Storage Configuration Identification

Using the format Command to Display Storage


The Solaris OS format utility is a reliable program for gathering basic
storage configuration information. This utility reports all storage devices,
regardless of type or model. The following sample output shows a
configuration consisting of two internal disk drives and a single Sun
StorEdge T3B array that is divided into six LUNs.
# format
Searching for disks...done

AVAILABLE DISK SELECTIONS:


0. c0t0d0 <SUN4.2G cyl 3880 alt 2 hd 16 sec 135>
/pci@1f,4000/scsi@3/sd@0,0
1. c0t1d0 <SUN4.2G cyl 3880 alt 2 hd 16 sec 135>
/pci@1f,4000/scsi@3/sd@1,0
2. c2t1d0 <SUN-T300-0201 cyl 35113 alt 2 hd 14 sec 128>
/pci@6,4000/pci@2/SUNW,qlc@5/fp@0,0/ssd@w50020f230000c193,0
3. c2t1d1 <SUN-T300-0201 cyl 35113 alt 2 hd 14 sec 128>
/pci@6,4000/pci@2/SUNW,qlc@5/fp@0,0/ssd@w50020f230000c193,1
4. c2t1d2 <SUN-T300-0201 cyl 35113 alt 2 hd 14 sec 128>
/pci@6,4000/pci@2/SUNW,qlc@5/fp@0,0/ssd@w50020f230000c193,2
5. c2t1d3 <SUN-T300-0201 cyl 35113 alt 2 hd 14 sec 128>
/pci@6,4000/pci@2/SUNW,qlc@5/fp@0,0/ssd@w50020f230000c193,3
6. c2t1d4 <SUN-T300-0201 cyl 35113 alt 2 hd 14 sec 128>
/pci@6,4000/pci@2/SUNW,qlc@5/fp@0,0/ssd@w50020f230000c193,4
7. c2t1d5 <SUN-T300-0201 cyl 35113 alt 2 hd 14 sec 128>
/pci@6,4000/pci@2/SUNW,qlc@5/fp@0,0/ssd@w50020f230000c193,5
Specify disk (enter its number):

Sun Storage Concepts


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

1-29

Storage Configuration Identification

Identifying Controller Addressing


When the Solaris OS software is first configured, all disk storage interfaces
are assigned a unique controller number. As shown in Figure 1-13,
controller numbers are a single point of attachment to a single storage
device or hub.
Host System
Internal
SCSI c0

Sun StorEdge
D1000 Array
UDWS
c1
FC-AL
c2

Sun StorEdge
A5x00 Array

FC-AL
c3
0 1 2 3 4 5 6
FC-AL Hub
Sun StorEdge
A5x00 Array

Sun StorEdge
A5x00 Array

Figure 1-13 Controller Address Numbering

Using VxVM Commands to Display Controller Addresses


After the VxVM software is installed and licensed, you use the vxdmpadm
command to display the basic controller configuration. The following is
an example.
# vxdmpadm listctlr all
CTLR-NAME
ENCLR-TYPE
STATE
ENCLR-NAME
=====================================================
c0
Disk
ENABLED
Disk
c2
SENA
ENABLED
SENA0
c3
SENA
ENABLED
SENA1
c4
T3
ENABLED
T30
c5
T3
ENABLED
T31

1-30

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Storage Configuration Identification

Identifying Device Path Components


To configure and maintain storage devices, you must understand the
terminology that describes and locates storage interface boards, storage
arrays, and disk drives within a storage array.
Each disk drive in a non-hardware RAID Sun storage array is identified
by two unique paths:

Physical device path (under the /devices directory)

Logical device path (under the /dev/dsk and /dev/rdsk directories)

System drivers and applications use the device paths to access specific
disk drives.

Physical Device Path Components


Physical paths describe the addresses of physical hardware components
that connect to a particular device. A typical physical device path is:
# ls -l /dev/dsk/c0t0d0s2
lrwxrwxrwx
1 root root 41 Oct 21 21:01 /dev/dsk/c0t0d0s2 ->../..
/devices/pci@1f,4000/scsi@3/sd@0,0:c
# ls -l /dev/dsk/c2t1d0s2
lrwxrwxrwx
1 root root 74 Oct 21 21:01 /dev/dsk/c2t1d0s2 ->../..
/devices/pci@6,4000/pci@2/SUNW,qlc@5/fp@0,0/ssd@w50020f230000c193,0:c
# ls -l /dev/dsk/c3t1d0s2
lrwxrwxrwx
1 root root 74 Sep 24 22:46 /dev/dsk/c2t1d0s2 ->../..
/devices/pci@6,4000/pci@3/SUNW,qlc@4/fp@0,0/ssd@w21000020370c055a,0:c

Physical device paths describe internal hardware paths in a manner that


varies depending on system type, interface type, and storage device type.
The paths show the location and type of interface cards and also the type
of software drivers used.
Use the man pages to research physical path components as follows:
# man pci
# man qlc
# man ssd

Sun Storage Concepts


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

1-31

Storage Configuration Identification

Logical Device Path Components


Logical device paths to disk drives are found under the /dev/dsk
directory for block devices and under the /dev/rdsk directory for raw
devices.
A logical device path is a more basic representation of a physical path. A
typical logical device path is:
/dev/dsk/c2t3d0s7
Each logical device is linked to a physical device path:
# ls -l /dev/dsk/c1t3d0s2
lrwxrwxrwx
1 root
root
90 Dec 2 1998
/dev/dsk/c1t3d0s2 ->
../../devices/iounit@f,e1200000/sbi@0,0/SUNW,soc@3,0/
SUNW,pln@a0000800,201cad7e/ssd@3,0:c
The following path, /dev/dsk/c0t3d0s2, is an example of a typical
logical device path. Standard logical device paths are composed of four
components:

1-32

Controller address (c0)

Target address (t3)

Device address (d0)

Slice address (s2)

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Storage Configuration Identification

Typical Storage Device Addresses


The number of devices associated with each target depends on the type of
storage device. The relationship between target and device numbers for
software RAID Sun storage is as follows:

SPARCstorage Array 100:

Thirty disks

Six targets, t0t5

Five devices (d0d4) associated with each target

SPARCstorage RSM array:

Two selectable target ranges

Seven disks

Seven targets, t0t6 or t8t14

A single device (d0) associated with each target

Sun StorEdge D1000 array:

Two selectable target ranges

Eight disks, t0t3 and t8t11

Twelve disks, t0t5 and t8t13

A single device (d0) associated with each target

Sun StorEdge A5x00 array:

Four selectable target ranges

Fourteen disks, targets, t0t6 and t16t22

Twenty-two disks, targets t0t10 and t16t26

A single device (d0) associated with each target

Sun StorEdge MultiPack array:

Two selectable target ranges for a six-disk model

Six disks, targets, t1t6 or t9t14

Twelve disks, targets t2t5 and t8t15

A single device (d0) associated with each target

Sun StorEdge MultiPack-FC array:

Fifteen selectable target ranges

Six disks, targets, t8t13

A single device (d0) associated with each target

Sun Storage Concepts


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

1-33

Storage Configuration Identification

Identifying DMP Devices


DMP connections can be identified using the format utility as follows:
AVAILABLE DISK SELECTIONS:
0. c0t0d0 <SUN2.1G cyl 2733 alt 2 hd 19 sec 80>
/sbus@3,0/SUNW,fas@3,8800000/sd@0,0
1. c2t33d0 <SUN9.0G cyl 4924 alt 2 hd 27 sec 133>
/sbus@3,0/SUNW,socal@0,0/sf@0,0/ssd@w22000020370c0de8,0
2. c3t33d0 <SUN9.0G cyl 4924 alt 2 hd 27 sec 133>
/sbus@3,0/SUNW,socal@0,0/sf@1,0/ssd@w21000020370c0de8,0
Notice that the device paths for devices 1 and 2 have the same disk drive
identifier, 20370c0de8. Because the controller numbers are different,
devices 1 and 2 are connected to two different controller interfaces in the
same system.

1-34

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Storage Array Firmware

Storage Array Firmware


Several Fibre Channel hardware components have resident firmware that
can be at various revision levels. You must verify the minimum levels for
this firmware before starting an installation. If necessary, you need to
upgrade the firmware to an acceptable revision level. The related Fibre
Channel components are:

Fibre Channel HBA cards

Storage array controller/interface boards

Storage array disk drives

Fibre Channel HBA Cards


There are several generations of Fibre Channel HBA cards including:

FC/S (also known as FC25/S or SOC cards)


The FC/S HBAs are SBus-based cards that were used with the older
SPARCstorage arrays and SBus systems.

FC100/S (also known as SOC+ cards)


The FC100/S HBAs were faster versions of the FC/S cards and were
also used with older SBus-based systems.

FC100/P and FC100/2P (ISP 2100 or ISP 2200 cards)


The FC100/P HBAs are PCI bus cards. The FC100/P with ISP 2100
chips are single channel only. The FC100 cards with ISP 2200 chips
can be single channel (/P) or dual channel (/2P). There are different
firmware files for the ISP 2100 and ISP 2200 versions.

Compact PCI (cPCI) Dual Fibre Channel Adapter

Note The numbers ISP 2100 and ISP 2200 are model numbers of
integrated circuit chips on the interface cards.

Sun Storage Concepts


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

1-35

Storage Array Firmware

Verifying Fibre Channel HBA Firmware


Fibre Channel HBA card firmware revisions can be checked with the
luxadm utility, which is a standard Solaris OS command that resides in the
/usr/sbin directory.
There are different luxadm command options for each generation of Fibre
Channel HBA cards. However, the most current version of luxadm has a
single option (fcode_download) that can be used to verify and upgrade
firmware on most Fibre Channel HBA cards. An example of the command
output follows.
# /usr/sbin/luxadm fcode_download -p
Found Path to 0 FC/S Cards
Complete
Found Path to 0 FC100/S Cards
Complete
Found Path to 2 FC100/P, ISP2200 Devices
Opening Device: /devices/pci@6,4000/SUNW,ifp@2:devctl
Detected FCode Version:
FC100/P FC-AL Host Adapter
Driver: 1.9 00/03/10
Opening Device: /devices/pci@6,4000/SUNW,ifp@3:devctl
Detected FCode Version:
FC100/P FC-AL Host Adapter
Driver: 1.9 00/03/10
Complete
The firmware revision in the preceding example is 1.9.
Caution The same luxadm command option also downloads new
firmware into the HBA cards.

1-36

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Storage Array Firmware

Verifying SPARCstorage Array 100 Firmware


You use the luxadm display command to determine SPARCstorage
Array 100 controller board firmware revisions. You use the controller
address to identify the array you want to examine. An example follows.
# /usr/sbin/luxadm display c3
SPARCstorage Array 110 Configuration
(luxadm version: 1.42 SunOS 5.6)
Controller path:
/devices/sbus@6,0/SUNW,soc@d,10000/SUNW,pln@a0000000,78ccf9
:ctlr
DEVICE STATUS
TRAY 1
TRAY 2
TRAY 3
slot
1
RESERVED
Drive: 2,0
Drive: 4,0
2
Drive: 0,1
Drive: 2,1
Drive: 4,1
3
Drive: 0,2
Drive: 2,2
Drive: 4,2
4
Drive: 0,3
Drive: 2,3
Drive: 4,3
5
Drive: 0,4
Drive: 2,4
Drive: 4,4
6
Drive: 1,0
Drive: 3,0
Drive: 5,0
7
Drive: 1,1
Drive: 3,1
Drive: 5,1
8
Drive: 1,2
Drive: 3,2
Drive: 5,2
9
Drive: 1,3
Drive: 3,3
Drive: 5,3
10
Drive: 1,4
Drive: 3,4
Drive: 5,4
CONTROLLER STATUS
Vendor:
SUN
Product ID:
SSA110
Product Rev:
1.0
Firmware Rev: 3.6
Serial Num:
00000078CCF9
Accumulate Performance Statistics: Enabled

Sun Storage Concepts


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

1-37

Storage Array Firmware

Verifying Sun StorEdge A5x00 Array Firmware


You can use the luxadm command to display the firmware revision of
Sun StorEdge A5x00 array interface boards. An example follows.
# luxadm display AA
SENA DISK STATUS
SLOT
FRONT DISKS
(Node WWN)
REAR DISKS
(Node WWN)
0
Not Installed
On (O.K.)
200000203704f9fc
1
On (O.K.)
20000020370c055a
Not Installed
2
Not Installed
Not Installed
3
On (O.K.)
20000020370d5c28
On (O.K.)
20000020370d54a9
4
Not Installed
On (O.K.)
20000020370d528a
5
Not Installed
Not Installed
6
On (O.K.)
20000020370d5676
On (O.K.)
20000020370c0b8a
SUBSYSTEM STATUS
FW Revision:1.09
Box ID:0
Node WWN:5080020000034ed8
Enclosure
Name:AA
Power Supplies (0,2 in front, 1 in rear)
0 O.K.(rev.-02) 1 O.K.(rev.-02) 2 Not Installed
Fans (0 in front, 1 in rear)
0 O.K.(rev.-05) 1 O.K.(rev.-00)
ESI Interface board(IB) (A top, B bottom)
A: O.K.(rev.-04)
GBIC module (1 on left, 0 on right in IB)
0 O.K.(mod.-05)
1 Not Installed
B: O.K.(rev.-04)
GBIC module (1 on left, 0 on right in IB)
0 Not Installed
1 Not Installed
Disk backplane (0 in front, 1 in rear)
Front Backplane: O.K.(rev.-04)
Temperature sensors (on front backplane)
0:33C 1:34C 2:36C 3:36C 4:34C 5:33C
6:36C (All temperatures are NORMAL.)
Rear Backplane: O.K.(rev.-04)
Temperature sensors (on rear backplane)
0:36C 1:36C 2:37C 3:34C 4:33C 5:36C
6:37C (All temperatures are NORMAL.)
Interconnect assembly
O.K.(rev.-02)
Loop configuration
Loop A is configured as a single loop.
Loop B is configured as a single loop.

1-38

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Storage Array Firmware

Verifying Sun StorEdge T3 Array Firmware


The Sun StorEdge T3 arrays have internal configuration and control
software (pSOS) that you use to verify firmware revisions. You must
perform a telnet command to a particular array, log in as user root, and
use the ver (version) command to verify the current firmware revision.
An example of a typical session follows.
$ telnet t3
Trying 129.150.47.115...
Connected to purple15.
Escape character is ^].
pSOSystem (129.150.47.115)
Login: root
Password:
T300 Release 1.00 1999/12/15 16:55:46 (129.150.47.115)
t3:/:<1> ver
T300 Release 1.14 1999/12/15 16:55:46 (129.150.47.115)

Note In the preceding example, the firmware version is 1.14.

Sun Storage Concepts


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

1-39

Storage Array Firmware

Verifying Sun StorEdge A5x00 Disk Drive Firmware


You can use the luxadm command to verify the firmware revision of the
disk drives, as shown in the following example. The firmware is
upgraded using a special program that is contained in the related disk
firmware patch.
# luxadm disp AA,f1
DEVICE PROPERTIES for disk: AA,f1
Status(Port A):
O.K.
Vendor:
SEAGATE
Product ID:
ST19171FCSUN9.0G
WWN(Node):
20000020370d336f
WWN(Port_A):
21000020370d336f
Revision:
117E
Serial Num:
9812R69750
Unformatted Capacity: 8637.338 MByte
Read Cache:
Enabled
Minimum prefetch:
0x0
Maximum prefetch:
0xffff
Location:
In slot 1 in the Front of
the enclosure named: AAA
Path(s):
/dev/rdsk/c2t1d0s2
/devices/pci@6,4000/SUNW,ifp@3/ssd@w21000020370d336f,0:c,raw
In the preceding example, the disk drive firmware revision is 117E.
Note You can also use the format utilitys inquiry command option to
verify firmware revisions in selected disk drives.

1-40

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Storage Array Firmware

Firmware Upgrade Best Practices


Updating storage array firmware can be a complex task. There are strict
dependency rules and procedures that must be followed or permanent
damage to interface cards can result. Check with your Sun field
representative before attempting any firmware updates. The most current
versions of firmware are available through software patches.
You should read all firmware-related patch README notes carefully. The
README notes frequently have specific warnings and procedure
requirements that can help prevent extended system downtime.
In some cases, permanent HBA damage can result if you try to upgrade
firmware from an old version to a new version. Review the patch
README notes for instructions informing you to first perform an
upgrade to an intermediate revision level.
You should always idle your system when upgrading storage-related
firmware. Usually, the firmware patch README notes will instruct you to
place all systems attached to the storage in single-user mode.
Ask for help if you are not sure how to proceed.

Sun Storage Concepts


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

1-41

Exercise: Recording Your Storage Configuration

Exercise: Recording Your Storage Configuration


In this exercise, you complete the following tasks:

Review Sun storage features

Identify host adapter configurations

Identify storage array configurations

Verify storage interface firmware revisions

Verify array disk drive firmware revisions

Preparation
Ask your instructor to identify the system and storage that is assigned for
your use during this exercise.
If you want to simplify the task of documenting the training system configuration, you can precede this lab
with a short lecture describing your particular system configuration.

Task 1 Reviewing Sun Storage Features


Answer the following questions concerning the general features of Sun
storage devices and tools.
1.

List at least four major administration tasks associated with VxVM.


_____________________________________________________________
_____________________________________________________________
_____________________________________________________________
_____________________________________________________________

The answers are installation, initialization, volume design, volume creation, and volume administration.

2.

Which of the following are not related to multihost storage access?


a.

Multi-initiated SCSI

b.

Dynamic Multipathing

c.

Multihost fiber-optic

d.

Multi-level sequencing

The answers are b and d.

1-42

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Recording Your Storage Configuration


3.

List four software applications that can be used to configure


multipath storage.
_____________________________________________________________
_____________________________________________________________
_____________________________________________________________
_____________________________________________________________

The answers are Redundant Dual Active Controller Driver (RDAC), Solaris Alternate Path Driver (AP),
VERITAS Dynamic Multipathing driver (DMP), and Sun StorEdge Traffic Manager software.

4.

List two applications that are used to administer controller-based


RAID storage.
_____________________________________________________________
_____________________________________________________________

The answers are Sun StorEdge RAID Manager, pSOS, or Sun StorEdge Component Manager.

5.

Which of the following tools assist with swapping out a failed disk
drive?
a.

vxdiskunsetup

b.

vxdiskadm

c.

vxdisk

d.

vxdctl

e.

luxadm

The answers are b, d, and e.

6.

Which of the following apply to traditional SCSI interfaces?


a.

They are SAN compatible.

b.

They transfer data in parallel.

c.

You can attach up to four hosts to a single storage unit.

d.

Maximum data transfer speed is currently 40-Mbytes/sec.

The answers are b and d.

7.

What is the main advantage of differential SCSI interfaces?


_____________________________________________________________

The answer is longer maximum cable lengths.

Sun Storage Concepts


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

1-43

Exercise: Recording Your Storage Configuration


8.

What is the primary reason for using SAN?


_____________________________________________________________

The answer is to off-load data traffic from user networks.

9.

What SAN feature isolates data resource from general access?


a.

Access control lists (ACL)

b.

Partitioning

c.

Zoning

d.

Segmentation

e.

Isolation protocol

The correct answer is c.

10. What is the primary SAN data transfer mechanism?


a.

Packets

b.

Bitmaps

c.

Groups

d.

Frames

The correct answer is d.

Task 2 Identifying Host Adapter Configurations


Perform the following steps to determine the controller numbers of the
HBA cards in your classroom system.
1.

Record the model number of your classroom system. Determine this


by visual inspection and the uname -a command.
_____________________________________________________________

2.

Record the controller numbers of each HBA on your classroom


system. Either inspect the /dev/dsk directory contents or use the
format utility to examine your system.
_____________________________________________________________
_____________________________________________________________

1-44

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Recording Your Storage Configuration

Task 3 Identifying Storage Array Configurations


Each of the following sections is dedicated to a particular storage array
model and describes the most useful methods of gathering more specific
storage array information.
Before proceeding with this task, record the type of storage arrays
attached to your classroom system and how many of each type there are.
Determine this by visual inspection.
Type of Storage Arrays

Number of Each Type

_______________

_____

_______________

_____

_______________

_____

SPARCstorage Array 100 and Sun StorEdge A5x00 Array


Use the luxadm utility to determine very specific information about both
of these storage devices. The most commonly used commands are:
#
#
#
#
#
#

luxadm probe
luxadm display controller_number
luxadm display enclosure_name
luxadm display logical_path
luxadm display enclosure_name, disk_location
luxadm fcode_download -p

For each storage array, record the WWN, the enclosure name
(Sun StorEdge A5x00 array only), the number of disks present, and the
controller number.
WWN
(12 or 16 digits)

Enclosure
Name

Number of
Disks Installed

Controller
Number

_______________

_____

_____

_____

_______________

_____

_____

_____

_______________

_____

_____

_____

Note For the Sun StorEdge A5x00 array, the WWN of the enclosure is
not used in the physical path. You must use luxadm display command
to determine the WWN of the Sun StorEdge A5x00 units.

Sun Storage Concepts


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

1-45

Exercise: Recording Your Storage Configuration

Sun StorEdge D1000 Array


The luxadm utility does not recognize the Sun StorEdge D1000 array.
Specific array information must be gathered using other tools such as the
format utility and visual identification.
Use the format utility to determine the controller number of each Sun
StorEdge D1000 HBA card and the number of disks in each storage unit.
Controller Number

Array Type

Number of Disks

_______

__________

_________

_______

__________

_________

_______

__________

_________

_______

__________

_________

Task 4 Verifying Storage Interface Firmware


Revisions
Use the luxadm utility to determine the firmware revision levels of your
classroom system storage interface components.
1.

2.

1-46

Record the firmware revision of each HBA card.


Controller Number

Host Adapter Firmware Revision

_______

______________________

_______

______________________

_______

______________________

_______

______________________

Record the firmware revision of each SPARCstorage Array 100


controller board or Sun StorEdge A5x00 interface board.
Controller Number
Revision

Array Controller/IB Firmware

_______

______________________

_______

______________________

_______

______________________

_______

______________________

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Recording Your Storage Configuration

Task 5 Verifying Array Disk Drive Firmware Revisions


Use the luxadm utility or the format utility to determine the firmware
revisions of your classroom storage array disk drives.
Record the firmware revision of at least one Sun StorEdge A5x00 disk
drive if you have this array model.
Controller Number

Disk Drive Firmware Revision

_______

______________________

_______

______________________

_______

______________________

_______

______________________

Sun Storage Concepts


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

1-47

Exercise Summary

Exercise Summary

Discussion Take a few minutes to discuss what experiences, issues, or


discoveries you had during the lab exercises.

Manage the discussion here based on the time allowed for this module, which was given in the About This
Course module. If you find you do not have time to spend on discussion, then just highlight the key concepts
students should have learned from the lab exercise.

Experiences

Ask students what their overall experiences with this exercise have been. You might want to go over any
trouble spots or especially confusing areas at this time.

Interpretations

Ask students to interpret what they observed during any aspects of this exercise.

Conclusions

Have students articulate any conclusions they reached as a result of this exercise experience.

Applications

Explore with students how they might apply what they learned in this exercise to situations at their workplace.

1-48

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Module 2

Managing Data
Objectives
Upon completion of this module, you should be able to:

List the advantages of using virtual disk management

Describe standard RAID terminology

List the common features of each supported RAID level including:

Concatenation RAID 0

Striping RAID 0

Mirroring RAID 1

Mirrored Stripe RAID 0+1

Mirrored Concatenation RAID 0+1

Striped Mirror RAID 1+0

Concatenated Mirror RAID 1+0

Striping with distributed parity RAID 5

Describe the optimum hardware configuration for each supported


RAID level

2-1
Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Virtual Disk Management

Virtual Disk Management


Servers today must meet high standards in the areas of availability,
performance, scalability, and maintainability.

Availability
VxVM provides availability improvements in this area in the following
ways:

Preventing failed disk drives from making data unavailable


The probability of a single disk drive failure increases with the
number of disk drives on a system. Data redundancy techniques
prevent failed disk drives from making data unavailable.

Allowing file systems to grow while they are in use


Allowing file systems to grow while they are in use reduces the
system downtime and eases the system administration burden.

Allowing multiple-host configurations


In a dual-host configuration, one host can take over disk drive
management for another failed host. This configuration prevents a
failed host from making data unavailable.

Performance
Many applications today require high data throughput levels. The VxVM
products can assist in this area by more efficiently balancing the I/O load
across disk drives.

2-2

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Virtual Disk Management

Scalability
Traditionally, file system size has been limited to the size of a single disk
drive. Using VxVM techniques, you can create file systems that consist of
many disk drives. The fact that there are multiple disk drives is
transparent to all applications. The size limit of file systems is increased to
the UNIX limit of 1 terabyte (Tbyte).
Note The size of a single file is limited to approximately 800 gigabytes
(Gbytes) because of the space required for file system overhead.

Maintainability
Administering large installations can be much easier with the assistance
of well-designed tools. VxVM has both graphical and command-line tools
to assist administrators.
VxVMs graphical administration tool is called VERITAS Enterprise
Administrator (VEA). VEA offers complete management assistance for all
functions.
VxVM also has a number of command-line programs and utilities that are
useful and are preferred by many administrators. They can also be used in
shell programs to perform virtually all administration tasks.

Managing Data
Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

2-3

RAID Technology Introduction

RAID Technology Introduction


The RAID concept was introduced at the University of California at
Berkeley in 1987 by David Patterson, Garth Gibson, and Randy Katz.
Their goal was to show that RAID could be made to achieve performance
comparable to or higher than the available performance on single, large
expensive disk drives of the day.
During the development phase of the project, it was determined that it
was necessary to provide redundancy to avoid data loss due to frequent
disk drive failures. This aspect of the project became of great importance
to the future of RAID.

Supported RAID Standards


Many RAID levels are technologically possible but are not commonly
used. The complete list of RAID levels includes:

RAID 0 Striping or concatenation

RAID 1 Mirroring

RAID 0+1 Mirroring plus concatenation or striping

RAID 1+0 Concatenation or striping plus mirroring

RAID 2 Hamming code correction

RAID 3 Striping with dedicated parity

RAID 4 Independent reads and writes

RAID 5 Striping with distributed parity

RAID 6 RAID 5 with a second parity calculation

Note RAID levels 2, 3, 4, and 6 are not available with VxVM. They are
not commonly implemented in commercial applications. RAID 0+1 and
RAID 1+0 are not true RAID levels but are abstractions composed of more
than one RAID level.
RAID 6 is a proprietary controller-based RAID scheme owned by a single manufacturer. It is similar to
RAID 5, but has additional parity that is written to disk. It has a higher tolerance to disk drive failures than
RAID 5 structures.

2-4

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

RAID Technology Introduction

RAID Terminology
In the explanation of RAID levels in the following pages, a number of
technical terms are used to describe RAID structure components. The
following are some of the definitions:

Stripe unit refers to a sequential group of data blocks on a single disk


drive. The stripe unit size is configurable.

The terms disk drive and column are synonymous in RAID


discussions.

Stripe width is the stripe unit size times the number of columns.

Transfer rate and I/O per second (IOPS) are performance metrics:

Transfer rate is the speed (measured in Mbytes per second) at


which a system can move data through its controller. In RAID
systems, read and write transfer rates can vary considerably.
High transfer rates are particularly valuable for applications
that must move large amounts of data quickly, such as
document imaging, data mining, or digital video applications.

IOPS is a measure of the ability of a storage system to handle


multiple, independent I/O requests in a certain period of time.
RAID systems with high transfer rates do not always have good
IOPS performance. Database and transaction processing
systems are examples of applications that typically require high
I/O rate performance.

Managing Data
Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

2-5

RAID Level Common Features

RAID Level Common Features


The section reviews the common features of the following supported
RAID levels:

Concatenation RAID 0

Striping RAID 0

Mirroring RAID 1

Mirrored Stripe RAID 0+1

Mirrored Concatenation RAID 0+1

Striped Mirror RAID 1+0

Concatenated Mirror RAID 1+0

Striping with distributed parity RAID 5

Concatenation RAID 0
The primary reason for using concatenation is to create a virtual disk
drive that is larger than one physical disk drive. Concatenation obtains
more storage capacity by logically combining portions of two or more
physical disk drives. Concatenation also enables you to grow a virtual
disk drive by concatenating additional physical disk drive space to it. This
technique does not restrict the mix of different size drives and member
drives can be of any size. Therefore, no storage space is lost.
The example in Figure 2-1 on page 2-7 shows the concatenation of three
physical disk drives. Each portion of the concatenation is managed by
VxVM and is called a subdisk. A subdisk is the basic unit that VxVM uses
to assemble and control all data storage areas.

2-6

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

RAID Level Common Features


The array management software is responsible for taking the three
physical disk drives and combining them into one virtual disk drive. This
disk drive is presented to the application as a contiguous storage area.
Block 1

Physical
Disk 1

Block 1000
Block 1
Physical
Disk 2

Block 1001

Array Management
Software

Block 2000
Block 3000
Physical
Disk 3

Block 2001

Virtual Disk

Block 3000

Figure 2-1

RAID-0 Concatenated Structure

The term block represents a disk drive block or sector (512 bytes) of data.
Concatenation is the default vxassist configuration.

Advantages
The following advantages can be gained by using concatenated structures:

Concatenation can improve performance for random I/O because


the data is spread over multiple disk drives.

Ask students to discuss the impact of concatenation on performance and cost.

One hundred percent of the disk drive capacity is available for user
data.

Managing Data
Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

2-7

RAID Level Common Features

Limitations
The limitations of concatenated structures include:

If you use only concatenation, there is no redundancy. Concatenated


volumes can be mirrored to achieve redundancy.

Concatenation is less reliable because the loss of one disk drive


results in the loss of data on all disk drives.

When the disk drives are full, the data is spread throughout all the
members. However, when the disk drives are not full, the last disk
drives are unused, thereby lowering the use of all the drives.

Ask students to discuss the impact of concatenation on failure and recovery.

Striping RAID 0
The primary reason for using striping is to improve IOPS performance.
The performance increase comes from accessing the data in parallel.
Parallel access increases I/O throughput because all disk drives in the
virtual device are busy most of the time servicing I/O requests.
The array management software is responsible for making the array look
like a single virtual disk drive. Striping takes portions of multiple physical
disk drives and combines them into one virtual disk drive that is
presented to the application.
As shown in Figure 2-2 on page 2-9, the I/O stream is divided into
segments called stripe units (SUs), which are mapped across two or more
physical disk drives, forming one logical storage unit. The stripe units are
interleaved so that the combined space is made alternately from each
slice, and is, in effect, shuffled like a deck of cards. The stripe units are
analogous to the lanes of a freeway.

2-8

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

RAID Level Common Features


There is no data protection in this scheme and, because of the way that
striping is implemented, the loss of one disk drive results in loss of data
on all striped disk drives. Therefore, while this implementation improves
performance, it degrades reliability.

Physical
Disk 1

Physical
Disk 2

Physical
Disk 3

SU 1

SU 4

SU 2

SU 5

SU 3

SU 6

Array Management
Software

SU
SU
SU
SU
SU
SU

1
2
3
4
5
6

Virtual Disk
SU = Stripe Unit

Figure 2-2

RAID 0 Striped Structure

The layout specification to use for vxassist is layout=stripe.

Advantages
The following advantages can be gained by using a striped structure:

Performance is improved for large sequential I/O requests and for


random I/O. You can optimize stripe unit size for sequential or
random access.

One hundred percent of the disk drive capacity is available for data.

Ask students to discuss the impact of striping on performance and cost.

Limitations
Some of the limitations of striped structures are:

There is no redundancy.

Striping is less reliable, because the loss of one disk drive results in
the loss of data on all striped disk drives.

Ask students to discuss the impact of striping on failure and recovery.

Managing Data
Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

2-9

RAID Level Common Features

Mirroring RAID 1
The primary reason for using mirroring is to provide a high level of
availability or reliability.
Mirroring provides data redundancy by recording data multiple times on
independent spindles. The mirrored disk drives appear as one virtual disk
drive to the application. In the event of a physical disk drive failure, the
mirror on the failed disk drive becomes unavailable, but the system
continues to operate using the unaffected mirrors.
The array management software takes duplicate copies of the data located
on multiple physical disk drives and presents one virtual disk drive to the
application, as shown in Figure 2-3.
Block 1
Block 2
Block 3
Block 1

Block 4
Array Management
Software
Block 1

Block 2
Block 3
Block 4

Block 2

Virtual Disk

Block 3
Block 4

Figure 2-3

RAID-1 Mirror Structure

In VxVM, the mirror is seen as a single logical address, block 0 to n blocks


in length. Because of this, VxVM does not concern itself with the format of
each individual mirror. One side of the mirror can be striped, and the
other side of the mirror can be concatenated. This implementation can be
used if there is a lack of enough physical disk drives or for performance
reasons. VxVM can guarantee consistent data across both sides of the
mirror, no matter what the format, because it writes to a given logical
block address.
The layout specification to use for vxassist is layout=mirror.

2-10

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

RAID Level Common Features

Advantages
The following advantages can be gained by using a RAID-1 mirrored
structure:

There is a fully redundant copy of the data on one or more disk


drives.
If the mirror resides in a storage array that is attached to a different
interface board, a high level of availability can be achieved.

All drives can be used for reads to improve performance.


Mirroring improves read performance only in a multiuser or
multitasking situation where more than one disk drive member can
satisfy read requests. Conversely, if there is only a single thread
reading from the volume, performance is not improved.

You can set up three-way mirroring, but there is a performance


penalty.
Write performance can suffer up to 44 percent with a three-way
mirror.

Ask students to discuss the impact of mirroring on performance and cost.


Ask students to discuss the impact of mirroring on failure and recovery.

Limitations
Some of the limitations of using a RAID-1 mirrored structure are:

Mirroring uses twice as many disk drives, which essentially doubles


the cost per Mbyte of storage space.

Mirroring degrades write performance by about 15 percent. This is


substantially less than the typical RAID-5 write penalty (which can
degrade as much as 70 percent).

Managing Data
Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

2-11

RAID Level Common Features

Mirrored Stripe RAID 0+1


The primary reason for using mirrored stripe volumes is to gain the
performance offered by striping and the availability offered by mirroring.
Mirrored stripe configurations are a relatively high-cost installation, but
many customers consider it a worthwhile investment.
As shown in Figure 2-4, two drives are first striped and then mirrored.
The reliability is as high as with mirroring. Because the technique of
striping is also used, the performance is much better than when using just
mirroring.

SU
SU
SU
SU

1
3
5
7

SU
SU
SU
SU

2
4
6
8

Array
Management
Software
Striping

Physical Disk
SU
SU
SU
SU

1
3
5
7

SU
SU
SU
SU

2
4
6
8

SU
SU
SU
SU
SU
SU
SU
SU

1
2
3
4
5
6
7
8

Virtual Disks

Array
Management
Software
Mirroring

Array
Management
Software
Striping

Figure 2-4

SU
SU
SU
SU
SU
SU
SU
SU

1
2
3
4
5
6
7
8

Mirrored Stripe RAID Structure

The layout specification to use for vxassist is layout=mirror-stripe.

2-12

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

SU
SU
SU
SU
SU
SU
SU
SU

1
2
3
4
5
6
7
8

Virtual Disk

RAID Level Common Features

Advantages
The primary advantage of mirrored stripe volumes is spreading data
across multiple disk drives (improved I/O per second) while providing
added redundancy of the data.

Limitations
Some of the limitations of using a Mirrored Stripe RAID 0+1 are:

Mirrored stripe volumes suffer the high cost of mirroring, requiring


twice the disk drive space of non-redundant volumes.

The loss of a single disk drive disables a mirror and results in the
loss of redundancy.

Managing Data
Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

2-13

RAID Level Common Features

Mirrored Concatenation RAID 0+1


The primary reason for using mirrored concatenation volumes is to gain
the availability offered by mirroring while maximizing storage utilization.
Mirrored concatenation volumes can be a relatively high-cost installation,
but many customers consider it a worthwhile investment.
As shown in Figure 2-5, two drives are first concatenated and then
mirrored for increased reliability. Because the technique of concatenation
is used, varied storage segments of dissimilar size can be combined to
maximize storage utilization.

Blocks 0-499

Blocks 500-999

Array
Management
Software
Concatenation
Block 0
Array
Management
Software
Mirroring

Volume

Blocks 0-499

Blocks 500-999

Figure 2-5

Block 999

Array
Management
Software
Concatenation

Mirrored Concatenation RAID Structure

The layout specification to use for vxassist is layout=mirror-concat.

Advantages
The primary advantage of mirrored concatenations is increased reliability
combined with maximized storage utilization.

Limitations
Mirrored concatenated volumes suffer the high cost of mirroring,
requiring twice the disk drive space of non-redundant volumes.

2-14

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

RAID Level Common Features

Striped Mirror RAID 1+0


Striped mirror volumes have the performance and reliability advantages
of mirrored stripe volumes, but can tolerate a higher percentage of disk
drive failures without data loss.
Striped mirror volumes also have a quicker recovery time after a disk
drive failure because only a single stripe must be resynchronized instead
of an entire mirror. As a best practice, use striped mirror volumes for large
volumes where failure recovery time and performance are issues.
As shown in Figure 2-6, the concept of striped mirrors is fundamentally
different from mirrored stripes. In a striped mirror configuration, each
stripe is mirrored separately. Additional internal structures called
sub-volumes (SVs) are used to implement RAID 1+0 volumes.

SV 1
Array
Management
Software

SU 1

Mirroring
SV 2
Physical Disk
Sub-Volumes

Virtual Disks

Array
Management
Software

SU 1
SU 2

Striping

Virtual Disk

SV 2
Array
Management
Software

SU 2

Mirroring
SV 2

Figure 2-6

Striped Mirror RAID Structure

Note VxVM refers to the striped mirror configuration as a layered


volume or as a Pro volume structure.

The layout specification to use for vxassist is layout=stripe-mirror.

Managing Data
Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

2-15

RAID Level Common Features

Advantages
The following advantages can be gained by using a Striped Mirror
RAID 1+0 structure:

Because each stripe is mirrored separately, a larger number of disk


drive failures can be tolerated without disabling the volume.
Availability increases exponentially with disk drive (stripe
component) count.

Striped mirror configurations have the same performance benefits of


mirrored stripe volumes.

Limitations
Some of the limitations of using a Striped Mirror RAID 1+0 are:

Striped mirror volumes suffer the high cost of mirroring, requiring


twice the disk drive space of non-redundant volumes.

A mirror cannot be detached and used to perform an online backup.


With a RAID 0+1 volume, you can detach a mirror and back it up to
tape.

Note Do not detach a volume mirror to perform online data backups.


The active data is not protected by redundancy during the backup
operation.

2-16

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

RAID Level Common Features

Concatenated Mirror RAID 1+0


Concatenated mirror volumes generally have a much quicker recovery
time after a disk drive failure because only a portion of the concatenation
must be resynchronized (instead of an entire mirror).
As a best practice, use concatenated mirror volumes for large volumes
where failure recovery time is an issue.
As shown in Figure 2-6, the concept of a concatenated mirror volume is
fundamentally different from a mirrored concatenation volume. In a
concatenated mirror configuration, each part of a concatenation is
mirrored separately.

Block 1

Block 1

Mirror

Block 1000

Block 1000
Block 1

Block 1001

Block 2000

Block 2001

Block 3000

Block 1001

Mirror

Array
Management
Software

Block 2000

Block 2001

Mirror

Block 3000
Virtual Disk

Block 3000

Figure 2-7

Concatenated Mirror RAID Structure

Note VxVM refers to the concatenated mirror configuration as a layered


volume and as a Pro volume structure.

The layout specification to use for vxassist is layout=concat-mirror.

Managing Data
Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

2-17

RAID Level Common Features

Advantages
Because each portion of the concatenation is mirrored separately, recovery
time is much faster.

Limitations
Some of the limitations of using a Concatenated Mirror RAID 1+0 are:

Concatenated mirror volumes suffer the high cost of mirroring,


requiring twice the disk drive space of non-redundant volumes.

A mirror cannot be detached and used to perform an online backup.


With a mirrored concatenated volume, you can detach a mirror and
back it up to tape.

Note Do not detach a volume mirror when performing online data


backups. The active data is not protected by redundancy during the
backup operation.

2-18

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

RAID Level Common Features

Striping With Distributed Parity RAID 5


RAID-5 configurations can be an attractive choice for read-intensive
applications that require increased data protection.
Three of the RAID levels introduced by the Berkeley Group have been
referred to as parity RAID because they use a common data protection
mechanism. RAID 3, 4, and 5 all use the concept of bit-by-bit parity to
protect against data loss.
RAID 3, 4, and 5 all implement the Boolean Exclusive OR (XOR) function to
compute parity. Parity is implemented bit-by-bit to corresponding stripe
units of member drives, and the result is written to a corresponding parity
disk drive found in RAID 3 and 4. As shown in Figure 2-8, the parity (P)
is distributed throughout all the member drives in RAID 5.

Disk 1
SU 1 SU 5

SU 9 P(10-12)

Disk 2
SU 2 SU 6 P(7-9) SU 10
Array Management
Software

Disk 3
SU 3 P(4-6) SU 7

SU 11

P(1-3)SU 4

SU 12

Disk 4
SU 8

SU = Stripe Unit

SU 1
SU 2
SU 3
SU 4
SU 5
SU 6
SU 7
SU 8
SU 9
SU 10
SU 11
SU 12
Virtual Disk

Figure 2-8

RAID-5 Structure

Additional features include:

Independent access is available to individual drives.

Data and parity are both striped across spindles.

Reads per second can reach the disk drive rate multiplied by the
number of disk drives (minus the parity disk) in a column.

Ask students to discuss the impact of RAID 5 on performance, cost, failure, and recovery.
The layout specifications to use for vxassist is layout=raid5 (or raid5nolog), logging is the default.

Managing Data
Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

2-19

RAID Level Common Features

Advantages
Some advantages of RAID-5 volumes are:

Parity protects against single disk drive failure.

RAID 5 requires only one additional drive beyond those used for
data.

Read performance can be similar to a RAID 0 striped volume.

Limitations
Some limitations of RAID-5 volumes are:

A minimum of three disk drives are required to implement RAID 5


in VxVM.

RAID 5 cannot be mirrored using VxVM. Redundancy is provided


through the parity information.

Write-intensive performance is poor.


Overall random I/O performance is dependent on the percentage of
writes. If more than 20 percent of the I/O operations are writes,
consider an alternative RAID option, such as RAID 1+0.

There can be severe performance degradation with a failed disk


drive in a write-intensive environment.

Point out that this is the reason for hardware-based RAID, such as that used in the Sun StorEdge A3500 and
Sun StorEdge A1000 arrays.

2-20

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Optimizing System Configurations

Exercise: Optimizing System Configurations


In this exercise, you complete the following tasks:

Review software RAID features

Identify availability and performance cabling

Optimize RAID-0 volumes

Optimize RAID-1 volumes

Optimize RAID-5 volumes

Optimize RAID-0+1 volumes

Optimize RAID-1+0 volumes

Identify effective storage utilization

Select disk drives for use

Preparation
The first several tasks in this exercise are group discussions about
optimizing hardware configurations to suit particular volume structures.
The last task involves identifying and recording your VxVM server
configuration and selecting six disk drives for use during the remainder of
this course.
You should work on VxVM servers in small groups of two or three. Each
group has six disk drives with which to work. Ideally, three of the disk
drives are on one array, and three are on a different array.
You must examine your classroom setup and determine which disk drives
you are going to use.
Caution More than one group might be working on the same server. It is
essential that each group is using their own disk drives and does not
accidentally reconfigure disk drives that are being used by another group.

Managing Data
Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

2-21

Exercise: Optimizing System Configurations

Task 1 Reviewing Software RAID Features


Answer the following questions concerning the general features of
software RAID administration.
1.

List four major areas of concern when using software RAID


products.
_____________________________________________________________
_____________________________________________________________
_____________________________________________________________
_____________________________________________________________

The answers are availability, performance, scalability, and maintainability.

2.

Which of the following RAID levels are not supported by VxVM?


a.

RAID 0

b.

RAID 1

c.

RAID 2

d.

RAID 3

e.

RAID 4

f.

RAID 5

g.

RAID 6

The answers are c, d, e, and g.

3.

Which of the following are characteristics of a RAID-0 concatenated


structure?
a.

High tolerance to disk drive failures

b.

Maximized disk space utilization

c.

Enhanced write performance

d.

Good random read performance

e.

Data storage space is contiguous

The answers are b, d, and e.

2-22

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Optimizing System Configurations


4.

Which of the following are characteristics of a RAID-0 striped


structure?
a.

High tolerance to disk drive failures

b.

Maximized disk space availability

c.

Enhanced I/O performance

d.

Good random read performance

e.

Data storage space is contiguous

The answers are b, c, and d.

5.

Which of the following are characteristics of a RAID-1 mirrored


structure?
a.

Enhanced tolerance to disk drive failures

b.

Maximized disk space availability

c.

Enhanced read performance

d.

Enhanced write performance

e.

Data storage space is contiguous

The answers are a and c.

6.

Which of the following are characteristics of a RAID-0+1 mirrored


stripe structure?
a.

Enhanced tolerance to disk drive failures

b.

Maximized disk space availability

c.

Enhanced I/O performance

d.

Poor write performance

e.

Data storage space is contiguous

The answers are a and c.

7.

What is the size of disk drive block?


_____________________________________________________________

The answer is 512 bytes

Managing Data
Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

2-23

Exercise: Optimizing System Configurations


8.

Which of the following are characteristics of a RAID-1+0 striped


mirror structure?
a.

High tolerance to disk drive failures

b.

Maximized disk space availability

c.

Enhanced I/O performance

d.

Poor write performance

e.

Data storage space is contiguous

The answers are a and c.

9.

Which of the following are characteristics of a RAID-5 structure?


a.

Enhanced tolerance to disk drive failures

b.

Maximized disk space availability

c.

Enhanced I/O performance

d.

Potentially poor write performance

e.

Data storage space is contiguous

The answers are a and d.

10. What is another term for striping?


_____________________________________________________________
The answer is interleaving.

11. Which of the following RAID structures has the highest tolerance to
disk drive failures?
a.

RAID 0+1

b.

RAID 5

c.

RAID 1

d.

RAID 1+0

e.

RAID 0

The answer is d.

2-24

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Optimizing System Configurations

Task 2 Identifying Availability and Performance


Cabling
For increased performance, availability, or both, you can distribute virtual
volume components across system boards, host bus adapters (HBAs), and
storage array targets, as shown in Figure 2-9.
Host System
System Board

HBA

c3

System Board

HBA

c4

Array
t1

Array
t1

t2

t2

t3

Figure 2-9

t3

Availability and Performance Cabling

Discussion Consider the following:

!
?

What are the general advantages of the physical configuration shown in


Figure 2-9?

Multiple boards, HBAs, and arrays reduce the possibility of a catastrophic failure that disables an entire site.
Volume mirrors can be distributed for higher availability.
Volume stripes can be distributed for higher performance.
Distribution across array targets can increase general performance.

Managing Data
Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

2-25

Exercise: Optimizing System Configurations

Task 3 Optimizing RAID-0 Volumes


Each of the following volume structures represent some advantage. The
advantage might be cost reduction, simplicity, availability, or
performance.
Availability, performance, or both, can be improved for some RAID
structures by optimizing the physical system configuration. For some
RAID structures, such as the basic concatenation or stripe shown in
Figure 2-10, you can improve performance but not availability.
Volume

Volume

Subdisk 1

Subdisk 1 Subdisk 2 Subdisk 3 Subdisk 4

Subdisk 2

RAID 0 (Striped)

Subdisk 3
Subdisk 4
RAID 0 (Concatenated)

Figure 2-10 RAID-0 Volume Structures

Discussion Consider the following:

!
?

1.

What hardware configurations can improve the availability or


performance of either structure shown in Figure 2-10?

2.

What is the ideal hardware configuration assuming there are no


hardware budget limitations?

Hint Assume each subdisk or stripe is a different disk drive: losing one disk drive disables the volume.
Good One system board, one HBA, and one array is as good as the concatenation gets. The stripe
performance can be improved a lot.
Better One system board, four HBAs, and four arrays provide the best stripe performance.
Best Four system boards, four HBAs, and four arrays provide a slight performance gain.

2-26

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Optimizing System Configurations

Task 4 Optimizing RAID-1 Volumes


RAID-1 volumes consist of two mirrors, as shown in Figure 2-11. Each of
the mirrors can be a concatenation, but that does not affect the optimized
hardware configuration.

Volume

RAID 1 (Mirrored)

Subdisk 1

Subdisk 1

Subdisk 2

Subdisk 2

Subdisk 3

Subdisk 3

Subdisk 4

Subdisk 4

Mirror

Mirror

Figure 2-11 RAID-1 Volume Structures

Discussion Consider the following:

!
?

1.

What hardware configurations can improve the availability or


performance of the structure shown in Figure 2-11?

2.

What is the ideal hardware configuration assuming there are no


hardware budget limitations?

Hint Assume each subdisk is a different disk drive: losing two disk drives can disable the volume.
Good One system board, two HBAs, and two arrays provide near maximum availability, but not much
possibility of performance increase.
Better Two system boards, two HBAs, and two arrays provide a slight availability increase, but no
performance gains.
Best There is not much else to do except use a SunPlex platform configuration.

Managing Data
Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

2-27

Exercise: Optimizing System Configurations

Task 5 Optimizing RAID-5 Volumes


RAID-5 volumes are constructed in a fashion similar to a RAID-0 striped
volume, but they have the added complication of distributed parity. A
typical RAID-5 volume structure is shown in Figure 2-12.
RAID 5
Volume
Data
or
Parity

Data
or
Parity

Data
or
Parity

Data
or
Parity

Subdisk 1 Subdisk 2 Subdisk 3 Subdisk 4

Figure 2-12 RAID-5 Volume Structures

Discussion Consider the following:

!
?

1.

What hardware configurations can improve the availability or


performance of the structure shown in Figure 2-12?

2.

What is the ideal hardware configuration assuming there are no


hardware budget limitations?

Hint Assume each stripe or log is a different disk drive: losing two disk drives can disable the volume.
Good One system board, one HBA, and one array: a single HBA failure disables the volume.
Better Four system boards, four HBAs, and four arrays provide some availability increase and better
performance.
Best There is no best. It is generally all or nothing with RAID-5 structures.

2-28

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Optimizing System Configurations

Task 6 Optimizing RAID 0+1 Volumes


As shown in Figure 2-13, RAID 0+1 volumes combine mirroring and
striping with the mirror components above the stripes. The primary
shortcoming is that the loss of any stripe disables a mirror.
Volume

Mirror

Mirror

Subdisk 1 Subdisk 2 Subdisk 3 Subdisk 4

Subdisk 1 Subdisk 2 Subdisk 3 Subdisk 4

Figure 2-13 RAID 0+1 Volume Structure

Discussion Consider the following:

!
?

1.

What hardware configurations can improve the availability or


performance of the structure shown in Figure 2-13?

2.

What is the ideal hardware configuration assuming there are no


hardware budget limitations?

Hint Assume each stripe is a different disk drive: losing two disk drives can disable the volume.
Good Two system boards, two HBAs, and two arrays provide some availability and a performance gain from
striping.
Better Two system boards, two HBAs, two hubs, and eight arrays increase availability and improve
performance.
Best Eight system boards, eight HBAs, and eight arrays provide the best availability and performance.

Managing Data
Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

2-29

Exercise: Optimizing System Configurations

Task 7 Optimizing RAID 1+0 Volumes


As shown in Figure 2-14, RAID 1+0 volumes combine striping and
mirroring with the stripe components above the mirrors. If the hardware
is properly configured, a RAID 1+0 volume can tolerate a higher
percentage of hardware failures without disabling the volume.

Volume

Stripe
Unit

Stripe
Unit

Mirror
Subdisk

Mirror
Subdisk

Mirror
Subdisk

Stripe
Unit

Mirror
Subdisk

Mirror
Subdisk

Stripe
Unit

Mirror
Subdisk

Mirror
Subdisk

Mirror
Subdisk

Figure 2-14 RAID 1+0 Volume Structure

Discussion Consider the following:

!
?

What hardware configurations can improve the availability or


performance of the structure shown in Figure 2-14?

Hint Assume each mirror is a different disk drive: you can lose up to four disk drives without disabling the
volume.
Good Two system boards, two HBAs, and two arrays provide availability and some performance gains from
striping. Primary mirrors are on one array, secondaries are on the other.
Better Two system boards, two HBAs, two hubs, and eight arrays increase availability and improve
performance.
Best Eight system boards, eight HBAs, and eight arrays provide the best availability and performance.

2-30

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Optimizing System Configurations

Task 8 Identifying Effective Storage Utilization


The storage configuration shown in Figure 2-15 is sufficient to
simultaneously configure an example of every supported RAID structure.
HBA
HBA
HBA
HBA

c6

c5

c4

c3

t1

t32

t64

t96

t2

t33

t65

t97

t3

t34

t66

t98

t4

t35

t67

t99

t5

t36

t68

t100

Array

Array

Array

Array

Figure 2-15 Storage Utilization Study


1.

2.

Circle the disk drives in Figure 2-15 that you would use to build the
following RAID structures:

A three-disk RAID-0 striped volume

A two-disk RAID-0 concatenated volume

A two-disk RAID-1 mirrored volume

A four-disk RAID-5 volume (no log)

A four-disk RAID-0+1 mirror-stripe volume

A four-disk RAID-1+0 stripe-mirror volume

If all the disk drives in Figure 2-15 are 9 Gbytes in size, what is the
approximate data storage available for each of the structures?

Assume you are using entire disk drives. There is one disk drive left over for a spare.

Managing Data
Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

2-31

Exercise: Optimizing System Configurations

Task 9 Selecting Disk Drives for Use


Each workgroup must select six disk drives for use during the rest of this
course. The ideal configuration of disk drives is:

Three disk drives on one storage array

Three more disk drives on a different storage array

You create and destroy disk groups and several different volume
structures, so it is important that the disk drives you select are not being
used by another group.
Use the following commands to select and record the logical addresses of
the six disk drives that your group chooses:

format

luxadm probe

luxadm display

Record the logical paths to the six disk drives for your workgroup in the
form c2t3d4.
Disk: _______________

Disk: _______________

Disk: _______________

Disk: _______________

Disk: _______________

Disk: _______________

Caution If there are other workgroups using the same VxVM server,
you must check with them to ensure that you are not using some of their
disk drives.

2-32

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise Summary

Exercise Summary

Discussion Take a few minutes to discuss what experiences, issues, or


discoveries you had during the lab exercises.

Manage the discussion here based on the time allowed for this module, which was given in the About This
Course module. If you find you do not have time to spend on discussion, then just highlight the key concepts
students should have learned from the lab exercise.

Experiences

Ask students what their overall experiences with this exercise have been. You might want to go over any
trouble spots or especially confusing areas at this time.

Interpretations

Ask students to interpret what they observed during any aspects of this exercise.

Conclusions

Have students articulate any conclusions they reached as a result of this exercise experience.

Applications

Explore with students how they might apply what they learned in this exercise to situations at their workplace.

Managing Data
Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

2-33

Module 3

VERITAS Volume Manager Installation


Objectives
Upon completion of this module, you should be able to:

List the key elements of pre-installation planning

Research VxVM software patch requirements

Install the VxVM software

Initialize the VxVM software

Verify the post-installation environment

Prepare for virtual disk drive management

Install the VEA client software

Use the basic VEA features

Use the VxVM error numbering system

3-1
Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Installation Planning

Installation Planning
VxVM installations vary in size from small desktop systems to large
servers with Tbytes of data storage. Regardless of the system size, the
installation should be carefully planned in advance.

System Downtime
During a new installation or an upgrade, some system downtime is
always required. Usually you should schedule system downtime so that it
occurs during off-peak system-usage time. Thorough pre-installation
planning usually minimizes the system downtime.

Storage Configuration Assessment


You must decide which disks should be managed by VxVM. As you
further configure VxVM, you must determine the logical organization or
grouping of the disk drives. For example, 10 of the disk drives should be
allocated for accounting applications, and 20 disk drives should be
allocated for marketing applications.
You have the option of not placing certain disk drives under VxVM
control. This option is useful if you have applications that are currently
using file systems or partitions, and you do not want to update the
applications references to these file systems or partitions.
In contrast, you might want to put your system disk under VxVM control
so that it can be mirrored.
Existing data on disk drives can be encapsulated. When data is
encapsulated, the partition is added to VxVMs control, leaving the
partition intact and maintaining the integrity of the data. This action can
be done at installation or at a later time.
You might also need to plan for new disk storage devices. In addition, you
might need to add more memory and larger backup tape systems to
compensate for the increased storage load.

3-2

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Installation Planning

Upgrade Resources
Some of the most frustrating installation issues can be discovering that
you are missing a CD-ROM, discovering that you do not have the needed
patches, or discovering that you have misplaced the installation
documents. Having all the required CD-ROMs, patches on the
appropriate media, and documentation minimizes your frustration. Not
only should you have the documentation (for example, release notes and
installation procedures), but you should read it. Reading the installation
documentation is the only way to ensure that you have all of the required
patches.

Licensing
VxVM uses license keys to control access. If you have a
SPARCstorage Array 100 or a Sun StorEdge A5x00 array attached to your
system, VxVM automatically configures a basic-use license. You can also
configure non-array drives connected to the same host. Other storage
arrays might require manual license installations.

Current System Checkpoint


When installing a new and complex application, such as VxVM, you must
always be prepared to return your system to its original state.

Backups
Not only must you have backups, but you must verify them. If there is a
hardware failure or not enough space to facilitate the upgrade, you must
be able to recover or back out the software. Perform a complete backup
immediately prior to the installation process.

Testing the New Configuration


If this is a new installation, test it prior to going into production mode. It
would be ideal to test all the components, including the storage
subsystem. During this time, any issues related to patches and firmware
can be resolved.

VERITAS Volume Manager Installation


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

3-3

Researching Software Patches

Researching Software Patches


Before installing the VxVM software, you should research any patches
that might be required. Your Sun field representative can assist you.
Ensure that you read all of the README notes in all of the patches.
Storage-related patches can require careful attention and knowledge
because they frequently must be installed in groups and in an exact
sequence.
Array firmware patches usually install new software drivers that are
sometimes designed to work only with a small range of array firmware
revisions.
Caution If the mismatch between the system software drivers and the
array-resident firmware is too great, the storage arrays can become
unavailable. Correcting the problem can be difficult and might require
Sun support and hours of downtime.

Researching Current Patch Information


Current patch information can be obtained from the external Sun Web site
at http://patchpro.sun.com. From the Web site, you can use the
PatchPro Interactive tool to display a customized patch listing that reflects
your system configuration. You can configure a PatchPro query to include
the following information:

3-4

Solaris OS release

Platform type (system model)

Disk arrays and disk drives

Tape libraries and tape drives

Switches and HBA models

SAN products

Software versions (common applications, such as VxVM)

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Researching Software Patches


Currently, the information is organized and retrieved in separate
categories, such as Enterprise Systems Products and Network Storage
Products.
Figure 3-1 shows a typical SunSolveSM PatchPro Network Storage
Products listing for the following configuration:

Solaris 9 12/02 Operating System

Sun Fire V480 server

Sun StorEdge A5200 arrays

Sun StorEdge T3B arrays

X6799A (FC100/2P) Fibre Channel interface cards

Figure 3-1

SunSolve Program PatchPro Information

The configuration used to produce the preceding output purposely uses some out-of-date components.
A configuration using all of the most recent components produces little or no output.
At the time of writing, PatchPro has still not been updated to reflect the current VxVM version.

Additional patches might be required or recommended by the application


vendor.

VERITAS Volume Manager Installation


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

3-5

Researching Software Patches

Installing Patches
The following is a typical patch installation scenario for the configuration
shown in Figure 3-1 on page 3-5:
1.

Pay close attention to the PatchPro listing Legend section and the
Patch Fixes column.
The order of the patches can be critical. Firmware patches must be
installed with care. You must carefully study all firmware patch
README notes before taking any action, especially in the following
areas:

2.

Keyword and Synopsis section

Patches Required With This Patch section

WARNING and Patch Installation Instructions sections

Examine the /var/sadm/patch directory to check for patches that


were installed after the operating system installation.
You can also use the patchadd -p command, but it displays many
screens of patches that are incorporated into the currently installed
operating system.

3.

Verify all firmware levels before attempting to install firmware


patches.
Verifying firmware levels varies according to system and storage
types. Older products are checked using the luxadm command.
Newer products, such as the Sun StorEdge T3 array, require you to
use array-resident firmware programs to verify revision levels.

Vendor Software Patches


Patches that are distributed on vendor CD-ROMs or listed in vendor
documentation are usually out of date. Do your research before starting
patch installation.

3-6

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Installing VxVM Software

Installing VxVM Software


With every release of VxVM, there is an installation guide that
accompanies the software. This guide is a comprehensive document that
defines the various scenarios for installing and upgrading current
VERITAS software products, as well as the Solaris OS.

Software Distribution Overview


The VxVM software distribution CD-ROM contains the following
top-level directories:

pkgs
The pkgs directory contains all the software files and documentation.

scripts
The scripts directory contains script files to assist you in
performing VxVM upgrades. These scripts (upgrade_start and
upgrade_finish) are referenced in the upgrade procedures in the
VxVM documentation.
There are other scripts (fixsetup, fixstartup, fixmountroot, and
fixunroot) that can be used to attempt recovery from difficult boot
failure situations. The use of these scripts is not documented, and
they should not be used without express instructions from technical
support personnel.

win32
The win32 directory contains a single disk operating system (DOS)
executable, SETUP.EXE, for installing the VEA client software on
Microsoft Windows systems.

patches
The patches directory contains any patches that were critical at the
time of the product release.

release_notes
The release_notes directory contains several small Portable
Document Format (PDF) and Adobe PostScript release documents.

VERITAS Volume Manager Installation


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

3-7

Installing VxVM Software

support
The support directory contains a group of support tools used to
gather configuration information. Use these tools only under the
direction of technical support personnel. Sun technical support
personnel use a different information gathering tool called Explorer.

veritas_enabled
The veritas_enabled directory contains many library files to
support a wide range of Sun and third-party storage arrays.

Software Package Summary


The VxVM software distribution CD-ROM contains the software packages
listed in Table 3-1. Installation should be performed in the order shown.
Table 3-1 VxVM Software Packages
Package

Title

VRTSvlic

VERITAS licensing utilities

VRTSvxvm

VxVM software

VRTSvmdoc

VxVM user documentation

VRTSvmman

VxVM manual pages

VRTSob

VEA Service

VRTSobgui

VEA

VRTSalloc

Intelligent Storage Provisioning

VRTSddlpr

Device Discovery Layer

VRTSvmpro

VERITAS Volume Manager Management Services


Provider

VRTSfspro

VERITAS File System Management Services Provider

The VERITAS File System (VxFS) package, VRTSfspro, is required even if


you are not going to install the VxFS software. There are several other
VxFS packages on the distribution CD-ROM that are not discussed in this
module.

3-8

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Installing VxVM Software

Software Package Installation


You should use the pkgadd command to install the VxVM software
packages. To avoid package dependency messages, install the packages in
the order shown in the following example.
1.

Install the license package and the basic VxVM packages.


# pkgadd -d . VRTSvlic VRTSvxvm VRTSvmdoc VRTSvmman

2.

Install the VEA software packages and reference the custom


administration file, VRTSobadmin.
# pkgadd -a ../scripts/VRTSobadmin -d . VRTSob
VRTSobgui

3.

Install the remaining VxVM software packages.


# pkgadd -d . VRTSalloc VRTSddlpr VRTSvmpro VRTSfspro

Note There is an additional localization package, VRTSmuob, that adds


some localized VEA information in French, Japanese, or Chinese. The
VRTSmuob package can be added at any time if needed.
The example does not show the installation of the VxFS software packages.

VERITAS Volume Manager Installation


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

3-9

Installing VxVM Software

Software Installation User Interaction


Throughout the VxVM software installation, you are asked questions, and
some warning messages appear. The following sections summarize
required interactions by software package.

The VRTSvlic Package Installation Questions


This package contains scripts which will be executed
with super-user permission during the process of
installing this package. Do you want to continue with
the installation of <VRTSvlic> [y,n,?] y

The VRTSvxvm Package Installation Questions


Install for which version of Solaris? [9, 8, 7]
(default: 9): 9
This package contains scripts which will be executed
with super-user permission during the process of
installing this package. Do you want to continue with
the installation of <VRTSvxvm> [y,n,?] y

The VRTSvmdoc Package Installation Questions (None)


The VRTSvmdoc package does not prompt for any user-input during
its installation.

The VRTSvmman Package Installation Questions


The following files are already installed on the system
and are being used by another package:
/opt/VRTS <attribute change only>
/opt/VRTS/man <attribute change only>
Do you want to install these conflicting files
[y,n,?,q] y

3-10

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Installing VxVM Software

The VRTSob Package Installation Questions


The selected base directory </opt/VRTSob> must exist
before installation is attempted.
Do you want this directory created now [y,n,?,q] y

The VRTSobgui Package Installation Questions


The VRTSobgui package does not prompt for any user-input during
its installation.

The VRTSalloc Package Installation Questions


This package contains scripts which will be executed
with super-user permission during the process of
installing this package. Do you want to continue with
the installation of <VRTSalloc> [y,n,?] y

The VRTSddlpr Package Installation Questions


This package contains scripts which will be executed
with super-user permission during the process of
installing this package. Do you want to continue with
the installation of <VRTSddlpr> [y,n,?] y

The VRTSvmpro Package Installation Questions


This package contains scripts which will be executed
with super-user permission during the process of
installing this package. Do you want to continue with
the installation of <VRTSvmpro> [y,n,?] y

The VRTSfspro Package Installation Questions


The following files are already installed on the system
and are being used by another package:
* /opt/VRTSob <attribute change only>
* - conflict with a file which does not belong to any
package.

VERITAS Volume Manager Installation


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

3-11

Installing VxVM Software


Do you want to install these conflicting files
[y,n,?,q] y
This package contains scripts which will be executed
with super-user permission during the process of
installing this package. Do you want to continue with
the installation of <VRTSfspro> [y,n,?] y
At this point, the VxVM software is installed but not operational. If you
reboot the system, you see at least two VxVM error messages similar to
the following:
VxVM NOTICE V-5-2-3347 Volume Manager not started
VxVM NOTICE V-5-2-3365 VxVM not started
VxVM Provider initialization warning: Configuration
daemon is not accessible
Only the vxsvc daemon is running. VxVM must be initialized using the
vxinstall utility before it can start successfully.

3-12

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Initializing VxVM Using the vxinstall Utility

Initializing VxVM Using the vxinstall Utility


The VxVM software cannot start correctly until it has been initialized
using the vxinstall utility. In previous VxVM versions, a disk group
named rootdg containing at least one disk drive was a mandatory
requirement for VxVM operation. The VERITAS Volume Manager 4.0
release has eliminated that requirement.

The vxinstall Utility Dialogue


As of this release, the vxinstall utility no longer calls the vxdiskadm
utility to configure an initial disk group. As a result of the vxinstall
changes and the removal of the rootdg disk group requirement, the
VxVM initialization process is now greatly simplified. The following is an
example of the entire dialogue.
# vxinstall
VxVM uses license keys to control access. If you have a SPARCstorage
Array (SSA) controller or a Sun Enterprise Network Array (SENA)
controller attached to your system, then VxVM will grant you a limited
use license automatically. The SSA and/or SENA license grants you
unrestricted use of disks attached to an SSA or SENA controller, but
disallows striping, RAID-5, and DMP on non-SSA and non-SENA disks. If
you are not running an SSA or SENA controller, then you must obtain a
license key to operate.
Licensing information:
System host ID: 80960386
Host type: SUNW,Ultra-4
SPARCstorage Array or Sun Enterprise Network Array: found
Do you want to use enclosure based names for all disks ?
[y,n,q,?] (default: n) n
Sep 3 11:38:17 ns-east-104 vxdmp: NOTICE: VxVM vxdmp V-5-0-34 added disk
array DISKS, datype = Disk
Sep 3 11:38:17 ns-east-104 vxdmp: NOTICE: VxVM vxdmp V-5-0-34 added disk
array 5080020000034ed8, datype = SENA
Sep 3 11:38:17 ns-east-104 vxdmp: NOTICE: VxVM vxdmp V-5-0-34 added disk
array 5080020000029e70, datype = SENA
Sep 3 11:38:17 ns-east-104 vxdmp: NOTICE: VxVM vxdmp V-5-0-34 added disk
array 60020f200000c3670000000000000000, datype = T3

VERITAS Volume Manager Installation


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

3-13

Initializing VxVM Using the vxinstall Utility


Sep 3 11:38:17 ns-east-104 vxdmp: NOTICE: VxVM vxdmp V-5-0-34 added disk
array 60020f200000c1930000000000000000, datype = T3
Sep 3 11:38:18 ns-east-104 vxdmp: WARNING: VxVM vxdmp V-5-0-336
Unlicensed array S/N 60020f200000c3670000000000000000 installed
Sep 3 11:38:18 ns-east-104 vxdmp: WARNING: VxVM vxdmp V-5-0-336
Unlicensed array S/N 60020f200000c1930000000000000000 installed
Do you want to setup a system wide default disk group?
[y,n,q,?] (default: y) n
The installation is successfully completed.

Licensing Requirements
The configuration used in the following example features two
Sun StorEdge T3B arrays. According to the restrictions outlined in the
vxinstall output, an additional license must be installed.
If there are no automatically licensed storage arrays attached, the
vxinstall utility prompts you for a license key.
SPARCstorage Array or Sun Enterprise Network Array: No
arrays found. No valid licenses found.
VM lite/full license(s) is needed to proceed with
vxinstall. See VERTIAS Volume Manager Installation Guide
for more details on obtaining and installing the license
keys.
Are you prepared to enter a license key [y,n,q] (default:
y) y
Enter your license key : 8ZYE-XJ4R-UPQ3-DOBL-PPO3-PNT6-PRVP

Note You can also use the vxlicinst utility to manually install a license
key at any time.

3-14

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Initializing VxVM Using the vxinstall Utility

Verifying Licensed Features


After the system is initialized using the vxinstall command, you verify
the licensed features using the vxdctl license command. Following are
examples of several licensed configurations.

Example 1 The following is a system with only Sun StorEdge


A5200 arrays attached and an automatically generated license.

# vxdctl license
Licensed features:
Mirroring
Root Mirroring
Concatenation
Disk-spanning
DMP (multipath enabled)
Striping (restricted to Sun Enterprise Network Array)
RAID-5 (restricted to Sun Enterprise Network Array)

Example 2 The following is a system with only Sun StorEdge


A5200 arrays attached and with an additional manually installed
license.

# vxdctl license
All features are available:
Mirroring
Root Mirroring
Concatenation
Disk-spanning
Striping
RAID-5
VxSmartSync
DMP (multipath enabled)

Example 3 The following is a system with only Sun StorEdge T3B


arrays attached and with a manually installed license.

# vxdctl license
All features are available:
Mirroring
Root Mirroring
Concatenation
Disk-spanning
Striping
RAID-5
VxSmartSync
DMP (multipath enabled)

VERITAS Volume Manager Installation


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

3-15

VxVM Post-Installation Environment

VxVM Post-Installation Environment


After the VxVM software is installed and initialized, you must be familiar
with the general environment to be an effective administrator.

VxVM System Files


During VxVM installation, the following changes are made to the
/etc/system file, and VxVM startup files are added to several of the
/etc run-level directories.

Changes to the /etc/system File


Entries are appended to the end of the /etc/system file to force-load
various VxVM drivers, as shown in the example:
* vxvm_START (do not remove)
forceload: drv/vxdmp
forceload: drv/vxio
forceload: drv/vxspec
* vxvm_END (do not remove)

Run-Level File Additions


A number of script files are added to the /etc run-level directories to
control the VxVM software when the system is booted or shut down.

/etc/rc0.d/K50isisd
This script file stops the ISIS service daemon (vxsvc) associated with
the VEA graphical interface when transitioning to run-level 0 during
a system shutdown.

/etc/rcS.d/K29vxvm-vsshutdown
This script file sends a shutdown message to VEA clients

/etc/rcS.d/K50isisd
This script file stops the ISIS service daemon (vxsvc) associated with
the VEA graphical interface when transitioning to the single-user
run-level.

3-16

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

VxVM Post-Installation Environment

/etc/rcS.d/S25vxvm-sysboot
This script file runs early in the boot sequence to configure the / and
/usr volumes. This file also contains configurable debugging
parameters.

/etc/rcS.d/S35vxvm-startup1
This script file runs after the / and /usr volumes are available. It
also makes other volumes available that are needed by the Solaris OS
early in the Solaris OS boot sequence, such as swap and /var.

/etc/rcS.d/S85vxvm-startup2
This script file starts I/O daemons, rebuilds the /dev/vx/dsk and
/dev/vx/rdsk directories, imports all disk groups, and starts all
volumes that were not started earlier in the boot sequence.

/etc/rcS.d/S86vxvm-reconfig
This script file contains commands to execute the fsck utility on the
root partition before anything else on the system executes.

/etc/rc2.d/S50isisd
This script file starts the ISIS service daemon (vxsvc) associated with
the VEA graphical interface during the system boot sequence.

/rc2.d/S94vxnm-vxnetd
This script file starts the vxnetd daemon if the VVR software option
is installed and licensed.

/etc/rc2.d/S95vxvm-recover
This script file attaches and resynchronizes plexes and starts several
VxVM watch daemons, including: vxrelocd, vxcached, and
vxconfigbackupd. You can also modify this file to change the
default VxVM disk drive failure response from hot relocation to hot
sparing.

/etc/rc2.d/S96vradmind
This script file starts the vradmind daemon if the VERITAS Volume
Replicator (VVR) software option is installed and licensed.

/etc/rc2.d/S96vxrsyncd
This script file starts the vxrsyncd daemon if VVR is installed and
licensed.

VERITAS Volume Manager Installation


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

3-17

VxVM Post-Installation Environment

System Startup Messages


When the Solaris OS is booted and the VxVM startup files execute, several
important boot messages appear. In the following example, the key
messages are in bold type.
Rebooting with command: boot
Boot device: disk:a File and args:
SunOS Release 5.9 Version Generic_112233-03 32-bit
Copyright 1983-2002 Sun Microsystems, Inc. All rights reserved.
Use is subject to license terms.
VxVM sysboot INFO V-5-2-3245 Starting restore daemon...
VxVM sysboot INFO V-5-2-3242 starting in boot mode...
NOTICE: VxVM vxdmp V-5-0-34
added disk array DISKS, datype = Disk
NOTICE: VxVM vxdmp V-5-0-34
added disk array 5080020000034ed8, datype = SENA
NOTICE: VxVM vxdmp V-5-0-34
added disk array 5080020000029e70, datype = SENA
NOTICE: VxVM vxdmp V-5-0-34
added disk array 60020f200000c3670000000000000000, datype = T3
NOTICE: VxVM vxdmp V-5-0-34
added disk array 60020f200000c1930000000000000000, datype = T3
Hostname: ns-east-104
VxVM vxvm-startup2 INFO V-5-2-503 VxVM general startup...
The system is coming up. Please wait.
NIS domain name is Ecd.East.Sun.COM
starting rpc services: rpcbind keyserv ypbind done.
Setting netmask of lo0 to 255.0.0.0
Setting netmask of hme0 to 255.255.255.0
Setting default IPv4 interface for multicast: add net 224.0/4: gateway
ns-east-104
syslog service starting.
volume management starting.
The system is ready.
ns-east-104 console login:

3-18

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

VxVM Post-Installation Environment

System Startup Processes


The following is an example of typical VxVM processes to help you
understand how they are started during a system boot operation.
# ps -e |grep vx
PID TTY
13 ?
296 ?

TIME CMD
0:04 vxconfig
0:05 vxsvc

473 ?
507 ?

0:00 vxrelocd
0:00 vxrelocd

495 ?
509 ?
513 ?

0:00 vxnotify
0:00 vxnotify
0:00 vxnotify

474 ?
512 ?

0:00 vxcached
0:00 vxcached

475 ?
494 ?

0:00 vxconfig
0:00 vxconfig

(vxconfigd)

(vxconfigbackupd)
(vxconfigbackupd)

The vxconfigd Daemon


The volume configuration daemon (vxconfigd) is started by the
/etc/rcS.d/S25vxvm-sysboot script early in the boot process. This
daemon must be running for the VxVM software to function.

The vxsvc Daemon


The vxsvc daemon is started by the /etc/rc2.d/S50isisd script. The
vxsvc daemon services requests from the VEA graphical interface.

The vxrelocd or vxsparecheck Daemons


One of these daemons is started by the S95vxvm-recover script during
the boot process. The vxrelocd daemons monitor for VxVM failure
events and relocate failed subdisks. By default, the
/etc/rc2.d/S95vxvm-recover script file starts the newer vxrelocd
processes, which provide hot relocation. You can edit the file so that it
starts the older vxsparecheck process that provides only hot sparing.

VERITAS Volume Manager Installation


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

3-19

VxVM Post-Installation Environment

The vxnotify Daemons


These daemons are started by either the vxrelocd or the vxsparecheck
daemons. The vxnotify daemons display events related to disk and
configuration changes, as managed by the VxVM configuration daemon,
vxconfigd.

The vxcached Daemons


The vxcached daemons manage cache volumes associated with
space-optimized snapshots. Cache space is added when needed.

The vxconfigbackupd Daemons


The vxconfigbackupd daemons are started by the
etc/rc2.d/S95vxvm-recover script. These daemons record any VxVM
disk group configuration changes in the /etc/vx/cbr/bk directory.

The vxrecover Daemon


This daemon can be run by the /etc/rcS.d/S35vxvm-startup1,
/etc/rcS.d/S85vxvm-startup2, or the /etc/rc2.d/S95vxvm-recover
script files during a system boot, depending on the need for volume
repair. The daemon terminates when the repair process is completed.
Volume-repair needs differ depending on several variables, such as a
system crash. The repair typically involves attaching plexes and
resynchronizing mirrors.

3-20

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

VxVM Post-Installation Environment

System and User Executable Files


VxVM uses many different script and binary program files to perform
management and administration functions. Many programs call other
programs. Many of the script files and programs should not be directly
run by a user.

VxVM Software in the /opt Directory


The following directories are present in the /opt directory after a
standard VxVM software installation:

/opt/VRTS
This directory contains the VxVM man pages.

/opt/VRTSalloc
This directory contains the VxVM storage allocation software known
as Intelligent Storage Provisioning.

/opt/VRTSddlpr
This directory contains software to enable and disable the device
discovery layer feature associated with the VEA application.

/opt/VRTSfspro
This directory contains VxFS registry information used by the vxsvc
daemon.

/opt/VRTSob
This directory contains the VEA client and server software.

/opt/VRTSlic
This directory contains VxVM license installation and verification
programs

/opt/VRTSvmpro
This directory contains VxVM registry information used by the
vxsvc daemon.

/opt/VRTSvxms
This directory contains library files for a DMP plug-in.

/opt/VRTSvxvm
This directory contains the VERITAS technical manuals.

VERITAS Volume Manager Installation


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

3-21

VxVM Post-Installation Environment

VxVM Software in the /usr/sbin Directory


Although there are many VxVM program files in the /usr/sbin directory,
only a few are commonly used. They include vxassist, vxstat, vxinfo,
vxprint, vxtask, vxinstall, vxdg, vxdisk, and vxdiskadm.
# ls /usr/sbin/vx*
vxadm
vxdiskpr
vxassist
vxdmpadm
vxcache
vxedit
vxclust
vxexport
vxcmdlog
vxibc
vxconfigd
vxinfo
vxdco
vxinstall
vxdctl
vxiod
vxddladm
vxmake
vxdg
vxmemstat
vxdisk
vxmend
vxdiskadd
vxnetd
vxdiskadm
vxnotify
vxdiskconfig
vxplex

vxpool
vxprint
vxrecover
vxrecover.wrap
vxrelayout
vxrlink
vxrsync
vxrvg
vxscriptlog
vxsd
vxsnap
vxsp
vxspcshow
vxstart_vvr

vxstat
vxtask
vxtemplate
vxtrace
vxtranslog
vxtune
vxusertemplate
vxvol
vxvoladm
vxvoladmtask
vxvset

VxVM Software in the /etc/vx/bin Directory


The script and program files in the /etc/vx/bin directory are called by
higher-level user commands and are not commonly used directly.
# ls /etc/vx/bin
egettxt
vxckdiskrm
strtovoff
vxclustadm
ugettxt
vxclustipc
vsshutdown
vxcntrllist
vxa5kchk
vxconfigbackup
vxapslice
vxconfigbackupd
vxbadcxcld
vxconfigrestore
vxbaddxcld
vxconvarrayinfo
vxbootsetup
vxcxcld
vxcached
vxdarestore
vxcap-part
vxdevlist
vxcap-vol
vxdevpromnm
vxcdsconvert
vxdisksetup
vxcheckda
vxdiskunsetup
vxchksundev
vxdxcld

3-22

vxedvtoc
vxeeprom
vxencap
vxevac
vxldiskcmd
vxmirror
vxmksdpart
vxnewdmname
vxparms
vxpartadd
vxpartinfo
vxpartrm
vxpartrmall
vxprtvtoc
vxr5check

vxr5vrfy
vxreattach
vxrelocd
vxresize
vxroot
vxrootmir
vxslicer
vxspare
vxsparecheck
vxsplitlines
vxswapctl
vxtaginfo
vxunreloc
vxunroot

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

VxVM Post-Installation Environment

Verifying Initial Disk Drive Status


After a fresh VxVM installation, you should immediately assess the
current disk storage status. You should carefully research any existing file
systems before initiating encapsulation or initialization operations.
The vxdisk utility shows the current VxVM status of all disks drives
attached to the system.
# vxdisk list
DEVICE
TYPE
c0t0d0s2
auto:none
c0t1d0s2
auto:none
c2t1d0s2
auto:none
c2t3d0s2
auto:none
c2t5d0s2
auto:none
c2t16d0s2
auto:none
c2t18d0s2
auto:none
c2t20d0s2
auto:none
c2t22d0s2
auto:none
c3t32d0s2
auto:none
c3t33d0s2
auto:sliced
c3t35d0s2
auto:none
c3t37d0s2
auto:none
c3t50d0s2
auto:none
c3t52d0s2
auto:none
c4t1d0s2
auto:none
c5t1d0s2
auto:none

DISK
testdg01
-

GROUP
testdg
-

STATUS
online
online
online
online
online
online
online
online
online
online
online
online
online
online
online
online
online

invalid
invalid
invalid
invalid
invalid
invalid
invalid
invalid
invalid
invalid
invalid
invalid
invalid
invalid
invalid

Disks that show a status of online invalid are not under VxVM control.
Disks that show a status of online have been initialized, but are not
assigned to a disk group. When online disks are added to a disk group,
they are assigned a name which appears in the DISK column. By default,
the disk name is derived from the name of the disk group.
What is not evident is that slice 7 of the disk c2t16d0 is mounted with a
file system. You must plan for all existing data before proceeding with
disk drive encapsulation or initialization.

VERITAS Volume Manager Installation


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

3-23

Preparing for VxVM Disk Drive Management

Preparing for VxVM Disk Drive Management


Disk drives can be placed under VxVM software control in two
fundamentally different ways. If a disk drive has existing file systems,
VxVM will prompt you to encapsulate the disk, thereby preserving any
existing data. An alternate method of placing disks under VxVM control
is to initialize the disk drive, which destroys existing data.
You configure special files to protect specific storage devices from
inadvertent use by VxVM command-line programs.

Disk Drive Initialization Process


When any disk drive is initialized by VxVM, the disk drive is
repartitioned into a standard VxVM configuration. The initialized disk
drive has only two slices, slices 3 and 4. One slice is very small and stores
VxVM configuration records. The other slice is the rest of the disk drive.
When a disk drive is initialized by VxVM, all existing data is lost.
Note When you use VxVM software such as the vxdiskadm utility to
manage disk drives, the software takes extensive steps to detect any
existing data structures.

VxVM Initialized Disk Format


As shown in Figure 3-2 on page 3-25, a physical disk drive that has been
initialized by VxVM is divided into two sections called the private region
and the public region:

3-24

The private region is used for configuration information.

The public region is used for data storage.

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Preparing for VxVM Disk Drive Management


By default, VxVM uses partitions 3 and 4 for the private and public
regions.

VxVM configuration and


management information
Private Region

VxVM User Data Storage


Public Region

Figure 3-2

VxVM Physical Disk Drive Layout

VxVM requires a single cylinder for the private region. On larger drives,
one cylinder can store more than an Mbyte.
The public region is configured to be the rest of the physical disk drive.
The volume table of contents (VTOC) listing for a freshly initialized
VxVM disk drive is shown in the following example. Some output is
omitted for clarity.
# prtvtoc /dev/dsk/c2t22d0s2
...
...
*
First
* Partition Tag Flags
Sector
2
3
4

5
15
14

01
01
01

0
0
3591

Sector
Count
17682084
3591
17678493

Last
Sector
17682083
3590
17682083

VERITAS Volume Manager Installation


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

3-25

Preparing for VxVM Disk Drive Management

Private Region Disk Header


The disk header is a block stored in the private region of a disk drive that
defines the following import properties of the disk drive:

Current host ownership of the disk drive


When a disk drive is part of a disk group that is in active use by a
particular host, the disk drive is stamped with that hosts host ID
(host name). If another VxVM system attempts to access the disk
drive, its VxVM daemons detect that the disk drive has a
nonmatching host ID (host name) and disallows access until the first
system releases the disk drive.

Disk identifier
A 64-byte unique identifier is assigned to a physical disk drive when
its private region is initialized.

Private Region Configuration Database


The configuration database (sometimes called configdb) contains
information about the configuration of a particular disk group. By default,
VxVM keeps four copies of the configuration database per disk group to
avoid any possibility of losing the disk group information. Each copy of
the configuration database contains the following information:

dgname The disk group name is assigned by the administrator.

dgid The disk group ID is a 64-byte universally unique identifier


that is assigned by VxVM to a disk group when the disk group is
created. This identifier is in addition to the disk group name.

Records VxVM virtual structure information for the disk group.

Kernel Log
The kernel log is kept in the private region on the disk drive and is
written by the VxVM kernel. The kernel log contains records describing
certain types of actions, such as transaction commits, plex detaches
resulting from I/O failures, dirty-region log failures, the first write to a
volume, and volume close information. The kernel log is used after a
crash or clean reboot to recover the state of the disk group just prior to the
crash or reboot.

3-26

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Preparing for VxVM Disk Drive Management

Disk Drive Encapsulation Process


When any disk drive is encapsulated by VxVM, existing file systems are
preserved and a small portion of the disk drive is used for VxVM
overhead. If the disk drive is the system boot disk, VxVM is aware of this
distinction and takes special care to preserve all data.
Generally, you do not encapsulate disk drives with existing data unless
you want to increase availability or performance of the data through the
use of software RAID structures.
When a disk with existing data structures (such as a mounted file system)
is encapsulated, VxVM analyzes the disk structure and takes measures to
preserve all existing data and the disk partition map found on block zero.

Disk Drive Before Encapsulation


The following shows the partition map and mount information of a
Solaris OS disk drive with a mounted file system. For clarity, some output
is omitted.
# prtvtoc /dev/rdsk/c2t16d0s2
First
Partition Tag Flags Sector
2
7

5
0

01
00

0
0

Sector
Count

Last
Sector

Mount
Directory

17682084 17682083
2100735 2100734 /Test

# more /etc/vfstab |grep Test


/dev/dsk/c2t16d0s7/dev/rdsk/c2t16d0s7 /Test ufs 2 yes -

VERITAS Volume Manager Installation


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

3-27

Preparing for VxVM Disk Drive Management

Disk Drive After Encapsulation


After encapsulation, using a VxVM utility such as vxdiskadm, the disks
partition map and mount information are modified as shown in the
following example.
# prtvtoc /dev/rdsk/c2t16d0s2
First
Partition Tag Flags Sector
2
3
4

5
14
15

01
01
01

Sector
Count

Last
Sector

Mount
Directory

0
17682084 17682083
0
17682084 17682083
17674902 7182
17682083

# more /etc/vfstab |grep Test


/dev/vx/dsk/testdg/Test /dev/vx/rdsk/testdg/Test
/Test
ufs
2
yes #NOTE: volume Test (/Test) encapsulated partition c2t16d0s7

3-28

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Preparing for VxVM Disk Drive Management

Protecting Storage Devices From Usage


Before initializing or encapsulating storage devices, you can automatically
exclude storage devices from VxVM use using two different methods.
Depending on which method you use, the effects range from limited
exclusion to total exclusion. The exclusion files prevent the vxdiskadm
utility and the VEA GUI from using the excluded devices.

Determining the VERITAS Storage Types


Although the format utility displays all known storage devices, you must
use the vxdmpadm command as shown to display the storage types and
enclosure names for use in device exclusion.
# vxdmpadm listctlr all
CTLR-NAME
ENCLR-TYPE
STATE
ENCLR-NAME
=====================================================
c0
Disk
ENABLED
Disk
c2
SENA
ENABLED
SENA0
c3
SENA
ENABLED
SENA1
c4
T3
ENABLED
T30
c5
T3
ENABLED
T31
# vxdmpadm listenclosure all
ENCLR_NAME ENCLR_TYPE ENCLR_SNO
STATUS
ARRAY_TYPE
=========================================================================
Disk
Disk
DISKS
CONNECTED
Disk
SENA0
SENA
5080020000034ed8
CONNECTED
A/A
SENA1
SENA
5080020000029e70
CONNECTED
A/A
T30
T3
60020f200000c3670000 CONNECTED
T31
T3
60020f200000c1930000 CONNECTED
The controller names (CTLR-NAME) and enclosure names (ENCLR-NAME) are
used in the appropriate exclude files.
The disk array type field (datype) displayed during the system boot
process equates to the ENCLR_TYPE field displayed in the output of the
vxdmpadm command.

VERITAS Volume Manager Installation


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

3-29

Preparing for VxVM Disk Drive Management

Limited Exclusion
You can exclude enclosure, controllers, or devices from use by the
vxinstall and vxdiskadm utilities by manually modifying three special
files in the /etc/vx directory. The following describes the three files and
an explanation of their intent.

enclr.exclude
Each line of the enclr.exclude file specifies an enclosure to be
excluded, for example, sena1.

cntrls.exclude
Each line of the cntrls.exclude file specifies the address of a
controller to exclude, for example, c2.

disks.exclude
Each line of the disks.exclude file specifies a disk drive to exclude,
for example, c0t3d0.

Caution The three manual exclusion files do not prevent other VxVM
commands from seeing and operating on the storage devices. You can still
see and perform operations on all the devices using VxVM commands,
such as vxdg, vxdisk, vxdisksetup, and vxassist.
The manual exclusion files are used to protect specific storage devices
from being initialized or encapsulated after an initial software installation.
The exclusion files are also useful to protect specific SAN storage devices.
You can remove or rename the manual exclusion files after you complete
the initialization or encapsulation process.

3-30

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Preparing for VxVM Disk Drive Management

Manual Exclusion File Formats


The following example shows the format using each of the different
manual exclusion files.
# more /etc/vx/enclr.exclude
SENA0
# more /etc/vx/cntrls.exclude
c4
# more /etc/vx/disks.exclude
c0t0d0
c0t1d0
In the previous example, a Sun StorEdge A5000 array (SENA0) is excluded,
a Sun StorEdge T3 array connected to controller c4 is excluded, and two
internal system disk drives are excluded.
If you try to initialize all attached storage using the vxdiskadm utility, you
see exclusion messages similar to the following:
Select disk devices to add:[<pattern-list>,all,list,q,?]
all
VxVM INFO V-5-2-428
This disk that you specified has been excluded by the
/etc/vx/cntrls.exclude file:
/dev/vx/rdmp/c4t1d0s2
These disks that you specified have been excluded by the
/etc/vx/disks.exclude file:
/dev/vx/rdmp/c0t0d0s2 /dev/vx/rdmp/c0t1d0s2
These disks that you specified have been excluded by the
/etc/vx/enclr.exclude file:
/dev/vx/rdmp/SENA0_0s2
/dev/vx/rdmp/SENA0_2s2
/dev/vx/rdmp/SENA0_4s2
/dev/vx/rdmp/SENA0_6s2

/dev/vx/rdmp/SENA0_1s2
/dev/vx/rdmp/SENA0_3s2
/dev/vx/rdmp/SENA0_5s2

VERITAS Volume Manager Installation


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

3-31

Preparing for VxVM Disk Drive Management

Global Exclusion
There are two additional files, /etc/vx/vxvm.exclude and
/etc/vx/vxdmp.exclude, that should not be manually edited. They are
modified indirectly using the vxdiskadm utility option 17, Prevent
multipathing/Suppress devices from VxVMs view.
Storage that is suppressed using the vxdiskadm utility, is removed from
the view of all VxVM software. This feature is intended for complex
configurations that require the coexistence of AP, DMP, and SAN devices.
The following is an abbreviated example of excluding a controller from
VxVMs view using the vxdiskadm utility.
# vxdisk list
DEVICE
TYPE
DISK
c0t0d0s2
auto:none c0t1d0s2
auto:none c2t1d0s2
auto:none c2t3d0s2
auto:none c2t5d0s2
auto:none c2t16d0s2
auto:none c2t18d0s2
auto:none c2t20d0s2
auto:none c2t22d0s2
auto:none c3t32d0s2
auto:none c3t33d0s2
auto:none c3t35d0s2
auto:none c3t37d0s2
auto:none c3t50d0s2
auto:none c3t52d0s2
auto:none c4t1d0s2
auto:none c5t1d0s2
auto:none -

GROUP
-

STATUS
online
online
online
online
online
online
online
online
online
online
online
online
online
online
online
online
online

# vxdiskadm
....
....
Select an operation to perform: 17
Exclude Devices
Menu: VolumeManager/Disk/ExcludeDevices
VxVM INFO V-5-2-1239

3-32

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

invalid
invalid
invalid
invalid
invalid
invalid
invalid
invalid
invalid
invalid
invalid
invalid
invalid
invalid
invalid
invalid
invalid

Preparing for VxVM Disk Drive Management


This operation might lead to some devices being suppressed
from VxVMs view or prevent them from being multipathed by
vxdmp (This operation can be reversed using the vxdiskadm
command).
Do you want to continue ? [y,n,q,?] (default: y) y
Volume Manager Device Operations
Menu: VolumeManager/Disk/ExcludeDevices
1 Suppress all paths through a controller from VxVMs view
2 Suppress a path from VxVMs view
3 Suppress disks from VxVMs view by specifying a VID:PID
4 Suppress all but one paths to a disk
5 Prevent multipathing of all disks on a controller by VxVM
6 Prevent multipathing of a disk by VxVM
7 Prevent multipathing of disks by specifying a VID:PID
8 List currently suppressed/non-multipathed devices
....
....
Select an operation to perform: 1
Exclude controllers from VxVM
Menu: VolumeManager/Disk/ExcludeDevices/CTLR-VXVM
Use this operation to exclude all paths through a
controller from VxVM.
Enter a controller name [<ctlr-name>,all,list,listexclude,q,?] c4
All paths to the following enclosures through controller c4
will be hidden from VxVM as a result of this operation: T30
Continue operation? [y,n,q,?] (default: y) y
Sep 11 14:21:57 ns-east-104 vxdmp: NOTICE: VxVM vxdmp V-50-110 disabled controller
/pci@6,4000/pci@4/SUNW,qlc@4/fp@0,0 connected to disk array
60020f200000c36700000000
00000000
VxVM NOTICE V-5-2-1323
The controller c4 will be disabled. The entries for paths
on controller c4 will still be visible through VxVM
utilities in a disabled state till a reboot. After a reboot
these entries will not be seen.
....

VERITAS Volume Manager Installation


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

3-33

Preparing for VxVM Disk Drive Management


....
# reboot
....
....
# vxdisk list
DEVICE
TYPE
DISK
c0t0d0s2
auto:none c0t1d0s2
auto:none c2t1d0s2
auto:none c2t3d0s2
auto:none c2t5d0s2
auto:none c2t16d0s2
auto:none c2t18d0s2
auto:none c2t20d0s2
auto:none c2t22d0s2
auto:none c3t32d0s2
auto:none c3t33d0s2
auto:none c3t35d0s2
auto:none c3t37d0s2
auto:none c3t50d0s2
auto:none c3t52d0s2
auto:none c5t1d0s2
auto:none -

GROUP
-

STATUS
online
online
online
online
online
online
online
online
online
online
online
online
online
online
online
online

invalid
invalid
invalid
invalid
invalid
invalid
invalid
invalid
invalid
invalid
invalid
invalid
invalid
invalid
invalid
invalid

# more /etc/vx/vxvm.exclude
exclude_all 0
paths
#
controllers
c4 /pci@6,4000/pci@4/SUNW,qlc@4/fp@0,0
#
product
#
pathgroups
#

Note The format utility still sees the c4 controller and can use it
normally.

3-34

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Installing the VEA

Installing the VEA


The VEA software is composed of server and client software. The server
software must reside on the VxVM server. The client software can be
installed on the VxVM server, but it can also be installed on one or more
remote administration systems.
The VEA software is a graphical VxVM administrative interface.
As shown in Figure 3-3, you can use the VEA client software to:

Run the client software remotely on an administration system

Run the client software on the VxVM server and display the VEA
GUI locally

Run the client software on the VxVM server and display the VEA
GUI remotely
Remote System
VEA Client
Software

Network
VM Server

Local Display
Monitor

VEA Client
Software

VEA Server
Software

Disk
Disk Groups
Volumes

Figure 3-3

VEA Server/Client Relationship

To remotely display an application, you set the local DISPLAY variable to


point to a remote system: setenv DISPLAY 129.148.152.93:0.0.

VERITAS Volume Manager Installation


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

3-35

Installing the VEA

VEA Software Initialization


The VEA GUI is a Java technology application that can be run locally on
the VxVM server or remotely on any networked system. The VEA client
software can run in any Java runtime environment.

VEA Software Package Installation


The VRTSob and VRTSobgui packages are both installed on the VxVM
server, which includes the VEA client interface and the VEA server
software. You can start the client software on the server. However, the
VEA client software package, VRTSobgui, is more commonly installed on
a remote administration workstation.

VEA Server Software Startup


The VEA server software daemon, vxsvc, is automatically started at boot
time by the /etc/rc2.d/S50isisd script.
You can manually stop and start the VEA server software on the VxVM
server using the /etc/init.d/isisd stop (or start) command options.
Table 3-2 shows the options you can use to control the
/opt/VRTSob/bin/vxsvc program directly:
Table 3-2 The /opt/VRTSob/bin/vxsvc Program Options

3-36

Option

Function

-k

Shuts down the vxsvc daemon.

-m

Returns the current VEA service state.

-v

Prints the version of the VEA service.

-r

Specifies the registry file to be used.

-n

Disables client connection authentication.

-f

Starts the vxsvc daemon in the foreground for debugging.

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Installing the VEA

VEA Client Software Startup


You start the VEA client software by manually running the
/opt/VRTSvmsa/bin/vmsa script file. The VEA client software can be
started and displayed on the server, started on the server and remotely
displayed on another system, or loaded and started on a remote system.
Start the VEA client software by running the /opt/VRTSob/bin/vea
script file:
# /opt/VRTSob/bin/vea &
You can also use the startup options shown in Table 3-3 as needed.
Table 3-3 Startup Options
Option

Function

-v

Prints the version of the VEA client software.

-d

Starts the client software in debug mode.

-s

Specifies the fully qualified class name for the skin.

-c

Specifies the absolute path of the configuration file for the


current skin.

-p

Specifies the frequency (in minutes) for the VEA GUI to


check the status of the VEA server processes.

-cp

Specifies the users class path (for libraries) to be appended


to the applications class path.

-DVAR

Specifies the environment variable to be set pointing to the


Java software directory. This option works only on
Microsoft Windows.

VERITAS Volume Manager Installation


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

3-37

Installing the VEA

Host Connection Window


When the client software starts, a Connection window, as shown in
Figure 3-4, is displayed. Enter the name of the VxVM server and the
appropriate authentication information.

Figure 3-4

VEA Host Connection Window

If you enable the Remember password feature, the next time you connect
you select the hostname from the pull-down menu. The Username and
Password fields are automatically configured.

3-38

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Installing the VEA

Resolving Low-Bandwidth Access Problems


If you need to access a VxVM server using a low-bandwidth connection,
such as a dial-up modem, you must take special measures to prevent fatal
connection errors.

Low-Bandwidth Download Failures


During the initial VEA client-to-server connection cycle, approximately
7 Mbytes of VEA plug-ins are downloaded from the VxVM server into the
client $HOME/.VRTSob directory. In a low-bandwidth environment, the
download process commonly fails approximately 15 minutes into the
process with cryptic and misleading errors such as File System Full and
Java exception errors.

Low-Bandwidth Corrective Measures


Perform the following steps to permanently eliminate low-bandwidth
download failures:
1.

The first time you start the VEA client software, increase the host
discovery frequency from the default value of 3 minutes to
approximately 45 minutes.
% vea -p 45 &

2.

In the VEA, on the Tools menu, perform the following steps:


a.

Select Preferences, and then select General.

b.

Disable the Delete temporary files on exit feature.


After the first-time download, the plug-in download files
remain in the $HOME/.VRTSob directory and do not need to be
downloaded again.

3.

Initiate the connection to the VxVM server.

The first connection might take up to 20 minutes to download the plug-in


files. Subsequent connections take less than 2 minutes. Administrative
operations response times are acceptable even through a 56-Kbyte
modem.
If the connection fails while performing administrative tasks, reconnect
using the toolbar Connect icon.

VERITAS Volume Manager Installation


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

3-39

Using Basic VEA Features

Using Basic VEA Features


Before using the VEA GUI for administrative duties, you must be familiar
with the VEA components and options.

Main Window Functional Areas


The VEA GUI has distinct functional areas, as shown in Figure 3-5.
Menu bar

Object tree

Figure 3-5

Grid area

Tool bar

Message area

VEA GUI

Note The Actions menu entries change according to the type of objects
being displayed in the grid area. Some of the toolbar icons functions also
change as different objects are displayed in the grid area.

3-40

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Using Basic VEA Features

Menu Bar Functions


The menu bar in the VEA GUI has the functions shown in Figure 3-6. The
Actions menu entries change according to the type of objects currently
displayed in the grid area.
The tear off menu opens a separate window relating to the tabs in the
current grid area display. The tear off feature is useful when analyzing
multiple aspects of a grid area display.

Figure 3-6

Menu Bar Functions

VERITAS Volume Manager Installation


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

3-41

Using Basic VEA Features

Toolbar Buttons
The toolbar, shown in Figure 3-7, provides direct access to general VEA
functions. Some of the toolbar selections change according to the type of
objects being displayed in the grid area.

Figure 3-7

Toolbar Icons

All the toolbar functions are available elsewhere in menus, but the toolbar
offers a convenient way to access commonly used functions.
The toolbar buttons perform the following functions:

3-42

Connect to a VEA server

Disconnect from the current VEA server

Create a new volume

Create a new disk group

Search for virtual objects by criteria

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Using Basic VEA Features

Object Tree Pane


The object tree window, shown in Figure 3-8, has an icon for every type of
VEA object that is referenced during VxVM administration. The objects
are arranged in a hierarchy starting with VxVM servers at the top.
You can expand small nodes on the object tree branches to display
detailed information about the nodes subject.

Figure 3-8

Object Tree Expansion

VERITAS Volume Manager Installation


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

3-43

Using Basic VEA Features

Grid Area Pane


When you select an object tree icon with the first mouse button, expanded
configuration information about that object appears in the grid area.
The grid area display, shown in Figure 3-9, results from selecting
Enclosure in the object tree.

Figure 3-9

3-44

Grid Area

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Using Basic VEA Features

Resizing Display Panes


The object tree and message areas, shown in Figure 3-10, have two types
of pane resizing controls: arrows for fixed size changes and a slider for
variable size changes.

Pane resizing arrows

Pane resizing bar

Figure 3-10 Pane Resizing Controls


In the previous example, the object tree pane has been widened using its
resizing bar, and the message area has been fully collapsed using its
resizing arrow so that it is no longer visible.

VERITAS Volume Manager Installation


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

3-45

Using Basic VEA Features

Modifying Preferences
The Preferences window, accessed by selecting Preferences from the Tools
menu, contains three tabs.

Appearance tab (shown in Figure 3-11) Used to modify the general


look and feel of the VEA GUI.

Figure 3-11 Preferences Window Appearance Tab

3-46

General tab Used to modify settings so that any downloaded


plug-in files are retained locally when you exit VEA. This can save 45
minutes of download time if you are running the VEA client
software from a dial-up modem.

Volume Manager General tab Present only when you are connected
to a VxVM server and modifies several minor features.

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Using Basic VEA Features

Customizing the Grid Display


Using drag-and-drop and the display tabs, you can modify the type of
data displayed in the grid area, and you can customize the location of the
data status columns.

Rearranging Grid Display Status Columns


You can temporarily rearrange display columns in the grid using
drag-and-drop on the column headers. The effect is temporary.

Using the Grid Display Data Tabs


Each type of grid display has data tabs associated with it. You can use the
tabs to display different information related to the current grid display.
By default, when a disk group is displayed in the grid area, the Disks tab
is active. In the example shown in Figure 3-12, the Volumes tab is selected
so that all volumes associated with the sdgA disk group are displayed.

Figure 3-12 Grid Display Data Tabs

VERITAS Volume Manager Installation


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

3-47

Using Basic VEA Features

Examining VEA Command Logs


The VxVM server logs commands resulting from VEA client software
operations in the /var/vx/isis/command.log file. The log file is a useful
tool for learning VxVM command-line program syntax.
Examples of typical command log entries:
Description: Add Disk
Date: Tue Dec 11 20:09:15 2003
Command: /usr/sbin/vxdg init sdgA sdgA00=c3t32d0s2
sdgA01=c3t33d0s2 sdgA02=c3t35d0s2 sdgA03=c3t37d0s2
Output:
Exit Code: 0
Description:
Date:
Command:
sdisk01
Output:
Exit Code:

Rename Disk
Tue Dec 11 20:26:15 2003
/usr/sbin/vxedit -g sdgA -v rename sdgA01

Description: Create Volume


Date: Wed Dec 12 11:51:41 2003
Command: /usr/sbin/vxassist
-g sdgA -b make newvol_01
1638400s layout=nostripe
Output:
Exit Code: 0

Note An exit code of 0 means the command executed successfully. See


the vxintro man page for a list of standard exit codes.

3-48

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Using Basic VEA Features

Using the VEA Search Tool


You can use the VEA Search form, as shown in Figure 3-13, to compile
configuration summaries for supported VxVM objecting including
controllers, enclosures, disks, disk groups, volumes, and file systems.
In the example shown in Figure 3-13, a search was made for all volumes
at least 1 Gbyte in size.

Figure 3-13 VEA Search Form


To use the search tool, perform the following steps:
1.

Use the pull-down lists and type-in fields to define the criteria to use
in your search.

2.

Click Add to add the criteria to the criteria list.

3.

Repeat the previous two steps for each criteria to use.

4.

Click the Search Now button to perform the search and display the
results.

VERITAS Volume Manager Installation


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

3-49

Decoding VxVM Error Messages

Decoding VxVM Error Messages


The VxVM 4.0 software error message format is organized differently than
in previous VxVM releases. All VxVM 4.0 error messages contain unique
error numbers that are listed in the VERITAS Volume Manager 4.0
Troubleshooting Guide.
The new error message numbers are grouped into two sections as follows:

Errors V-5-0-2 through error V-5-0-386

Errors V-5-1-90 through error V-5-1-5929

Error message are composed of the following 5 fields:

Product name

Product component

Severity level

Error message number

Message text

An example of a typical error message follows.


# vxdg deport dgSP
VxVM vxdg ERROR V-5-1-584 Disk group dgSP: Some volumes in
the disk group are in use
Most of the error listings in the VERITAS Volume Manager 4.0
Troubleshooting Guide contain a description of the error and, if appropriate,
suggested administrative actions.

3-50

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Configuring VxVM

Exercise: Configuring VxVM


In this exercise, you complete the following tasks:

Review key lecture points

Install the VxVM software

Verify the VxVM system files

Evaluate the storage configuration

Install the VEA client software

Start the VEA client software

Customize the VEA client appearance

Navigate the VxVM technical manuals

Use the VxVM error numbering system

Preparation
If you are installing the VxVM software on a central server, your
instructor must perform the installation as a demonstration.
1.

Ask your instructor for the location of the VxVM software.


VxVM location: ______________________________

2.

If the lab system does not have certain Sun storage array models
attached, you are asked for a license string during the VxVM
VRTSvlic package installation. Ask your instructor for a
demonstration license string.
Demo license: _____________________________

The Adobe Acrobat Reader program must be available to the students to examine the PDF versions of the
VERITAS documents.
Discuss the classroom configuration and the process students should follow until everyone is clear about how
to proceed.
Divide students into groups depending on how many systems you have available for VxVM installation.
You can install and initialize the VxVM software yourself while students watch. It is up to you to determine
which method works best in your classroom configuration.

VERITAS Volume Manager Installation


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

3-51

Exercise: Configuring VxVM

Task 1 Reviewing Key Lecture Points


Answer the following questions about the major areas of concern during
the VxVM installation and initialization process.
1.

Which of the following are not part of planning a VxVM installation?


a.

Estimating system downtime

b.

Backing up your system

c.

Running a system performance check

d.

Verifying correct boot disk configuration

e.

Determining licensing requirements

The answer is c.

2.

Which of the following are useful for researching patch


requirements?
a.

The patch README notes

b.

The prtvtoc command

c.

The patchadd command

d.

The PatchPro tool

e.

The /var/adm directory contents

The answers are a, c, and d.

3.

Which of the following VxVM packages should be installed first?


a.

VRTSvmman

b.

VRTSob

c.

VRTSvxvm

d.

VRTSvlic

e.

VRTSvmdoc

The answer is d.

3-52

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Configuring VxVM


4.

What is the correct order for the following installation steps?


a.

Install storage firmware patches.

b.

Install VxVM software patches.

c.

Encapsulate the system boot disk.

d.

Install required operating system patches.

e.

Install VxVM software packages.

The correct order is d, a, e, b, and c.

5.

Which slices does VxVM use when it initializes a disk drive?


a.

Slices 6 and 7

b.

Slices 2 and 4

c.

Slices 3 and 4

d.

Slices 0 and 1

e.

Slices 4 and 7

The answer is c.

6.

What is the default amount of space that VxVM requires for a disk
drives private region?
a.

4800 sectors

b.

2048 sectors

c.

1648 sectors

d.

4096 sectors

e.

1024 sectors

The answer is b.

7.

When should you use VxVM disk drive encapsulation instead of


disk drive initialization?
_____________________________________________________________
_____________________________________________________________

The answer is when you want to preserve existing data on the disk drive.

VERITAS Volume Manager Installation


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

3-53

Exercise: Configuring VxVM


8.

What is the primary purpose of the enlr.exclude,


cntrls.exclude, and disks.exclude files?
a.

To make storage devices invisible to all VxVM software

b.

To make storage devices invisible to the AP software

c.

To make storage devices invisible to the vxinstall and


vxdiskadm utilities

d.

To make storage devices invisible to the DMP software

The answer is c.

9.

Which of the following are not VxVM system processes?


a.

vxnotify

b.

vxdiskadm

c.

vxconfigd

d.

vxrecover

e.

vxdctl

The answers are b and e.

10. Which directory contains the VxVM technical manuals?


a.

/opt/VRTSdocs

b.

/etc/vx/docs

c.

/opt/VRTSvxvm/docs

d.

/usr/vx/docs

The answer is c.

3-54

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Configuring VxVM

Task 2 Installing the VxVM Software


To install the VxVM software, complete the following steps:
1.

Log in as user root on the system attached to the storage arrays.

2.

Change to the VxVM installation package location.

3.

Install the license package and the basic VxVM packages.

# pkgadd -d . VRTSvlic VRTSvxvm VRTSvmdoc VRTSvmman


4.

Install the VEA software packages and reference the custom


administration file, VRTSobadmin.

# pkgadd -a ../scripts/VRTSobadmin -d . VRTSob VRTSobgui


5.

Install the remaining VxVM software packages.

# pkgadd -d . VRTSalloc VRTSddlpr VRTSvmpro VRTSfspro

Note There is an additional localization package, VRTSmuob, that adds


some localized VEA information in French, Japanese, or German. The
VRTSmuob package can be added at any time if needed.

VERITAS Volume Manager Installation


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

3-55

Exercise: Configuring VxVM


6.

During the installation, answer yes to all questions unless you feel
there are serious problems.
If there are problems, ask your instructor for advice.

7.

After the package installation is completed, log out and log back in
again.
The installation process usually alters search-path values.

8.

Use the vxinstall utility to initialize VxVM and answer the


following key questions:
Do you want to use enclosure based names for all disks?
[y,n,q,?] (default: n) n
Do you want to setup a system wide default disk group?
[y,n,q,?] (default: y) n

Note If your system does not have Sun SPARCstorage 100 or


Sun StorEdge A5x00 arrays attached, you are prompted for a license key.
9.

Verify that the vxprint status command completes without errors.


# vxprint

10. Verify that the following VxVM command and man page locations
are configured in your environment. Edit the .profile or .cshrc
files as necessary.
# echo $PATH
/usr/sbin:/usr/bin:/opt/VRTS/bin:/opt/VRTSalloc/bin:/op
t/VRTSvlic/bin:/opt/VRTSvmpro/bin:/etc/vx/bin
# echo $MANPATH
/usr/man:/opt/VRTS/man:/opt/VRTSvlic/man
The correct syntax for .profile file entries in the Bourne shell
environment is:
PATH=$PATH:/opt/VRTS/bin:/opt/VRTSalloc/bin:/opt/VRTSvlic/b
in:/opt/VRTSvmpro/bin /etc/vx/bin
MANPATH=/usr/man:/opt/VRTS/man:/opt/VRTSlic/man
export PATH MANPATH
The correct syntax for .cshrc file entries in the C shell environment is:
set path = ($path /opt/VRTS/bin /opt/VRTSalloc/bin
/opt/VRTSvlic/bin /opt/VRTSvmpro/bin /etc/vx/bin)
setenv MANPATH /usr/man:/opt/VRTS/man:/opt/VRTSlic/man

3-56

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Configuring VxVM

Task 3 Verifying the VxVM System Files


Perform the following steps to verify the post-installation files:
1.

Verify that the following directories are present in the /opt directory.

# ls /opt
SUNWits VRTS
VRTSddlpr
SUNWrtvc VRTSalloc VRTSfspro
2.

VRTSvxvm

Verify that the following software is present in the /usr/sbin


directory.

# cd /usr/sbin
# ls vx*
vxadm
vxassist
vxcache
vxclust
vxcmdlog
vxconfigd
vxdco
vxdctl
vxddladm
vxdg
vxdisk
vxdiskadd
vxdiskadm
vxdiskconfig
3.

VRTSob
VRTSvmpro
VRTSvlic VRTSvxms

vxdiskpr
vxdmpadm
vxedit
vxexport
vxibc
vxinfo
vxinstall
vxiod
vxmake
vxmemstat
vxmend
vxnetd
vxnotify
vxplex

vxpool
vxprint
vxrecover
vxrecover.wrap
vxrelayout
vxrlink
vxrsync
vxrvg
vxscriptlog
vxsd
vxsnap
vxsp
vxspcshow
vxstart_vvr

vxstat
vxtask
vxtemplate
vxtrace
vxtranslog
vxtune
vxusertemplate
vxvol
vxvoladm
vxvoladmtask
vxvset

Verify that the following software is present in the /etc/vx/bin


directory.

# ls /etc/vx/bin
egettxt
vxckdiskrm
strtovoff
vxclustadm
ugettxt
vxclustipc
vsshutdown
vxcntrllist
vxa5kchk
vxconfigbackup
vxapslice
vxconfigbackupd
vxbadcxcld
vxconfigrestore
vxbaddxcld
vxconvarrayinfo
vxbootsetup
vxcxcld
vxcached
vxdarestore
vxcap-part
vxdevlist
vxcap-vol
vxdevpromnm
vxcdsconvert
vxdisksetup
vxcheckda
vxdiskunsetup
vxchksundev
vxdxcld

vxedvtoc
vxeeprom
vxencap
vxevac
vxldiskcmd
vxmirror
vxmksdpart
vxnewdmname
vxparms
vxpartadd
vxpartinfo
vxpartrm
vxpartrmall
vxprtvtoc
vxr5check

vxr5vrfy
vxreattach
vxrelocd
vxresize
vxroot
vxrootmir
vxslicer
vxspare
vxsparecheck
vxsplitlines
vxswapctl
vxtaginfo
vxunreloc
vxunroot

VERITAS Volume Manager Installation


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

3-57

Exercise: Configuring VxVM

Task 4 Evaluating the Storage Configuration


Perform the following steps to evaluate your storage configuration:
1.

Type the vxdmpadm listctlr all command and the format


command to evaluate your storage configuration. Use the Control-D
keyboard sequence to exit from the format utility.

2.

Record the controller name, enclosure name, and number of devices


associated with each enclosure.
Controller Number

3-58

Enclosure Name

Number of Devices

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Configuring VxVM

Task 5 Installing the VEA Client Software


Use the following steps to install the VEA client software on your
administration system:
1.

On your administration workstation, change to the location of the


VRTSobgui software package.
Your instructor should have given you the location of the VxVM
software in a previous exercise.

2.

Install the VEA client software on your administration system:

# pkgadd -d . VRTSobgui
Processing package instance <VRTSobgui> from </tmp>
VERITAS Enterprise Administrator
The selected base directory </opt/VRTSob> must exist before
installation is attempted.
Do you want this directory created now [y,n,?,q] y
Using </opt/VRTSob> as the package base directory.
## Processing package information.
## Processing system information.
## Verifying disk space requirements.
## Checking for conflicts with packages already installed.
## Checking for setuid/setgid programs.
This package contains scripts which will be executed with super-user
permission during the process of installing this package.
Do you want to continue with the installation of <VRTSobgui> [y,n,?] y

VERITAS Volume Manager Installation


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

3-59

Exercise: Configuring VxVM

Task 6 Starting the VEA Client Software


To start the VEA client software, complete the following steps:
1.

Start the VEA client software on your administration system:


# vea &

2.

Perform the following steps in the Session Initialization window, as


shown in Figure 3-14:
a.

Type the VxVM server host name.

b.

Type the user name root.

c.

Type the password.

d.

Ensure that the Remember password box is checked.

Figure 3-14 VEA Client Initialization


The VEA GUI initializes after a short delay.

3-60

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Configuring VxVM

Task 7 Customizing the VEA GUI Appearance


Perform the following steps to customize the appearance of the VEA GUI:
1.

Click the left mouse button on the Tools menu, and then select
Preferences from the pop-up menu.

2.

In the Preferences window, click the Appearance tab.

3.

Click Items and select Toolbar.

4.

Click Icons and Text.

5.

Click OK.

Task 8 Navigating the VxVM Technical Manuals


There are several technical manuals installed in the /opt/VRTSvxvm/docs
directory. Perform the following steps to examine the content of the
manuals:
1.

Change to the VxVM manual directory.


# cd /opt/VRTSvxvm/docs
# ls
pitc_ag.pdf
vxvm_hwnotes.pdf
vxvm_ag.pdf
vxvm_ig.pdf
vxvm_cdsag.pdf
vxvm_ispag.pdf

2.

vxvm_tshoot.pdf
vxvm_ug.pdf

View the contents of the PDF manuals using the Adobe Acrobat
Reader (acroread) program
The hypertext links and search features of the Adobe Acrobat Reader
are useful when you are looking for specific information in the
manuals. Use the Control-F sequence to enable the Adobe Acrobat
Find window.

VERITAS Volume Manager Installation


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

3-61

Exercise: Configuring VxVM


3.

Use the Adobe Acrobat Find feature to answer the following


questions for each of the indicated VxVM technical manuals.
a.

Answer the following questions for the vxvm_hwnotes.pdf


VxVM manual:
1.

Which Sun storage array models are supported for boot


disk mirroring?
_____________________________________________________
_____________________________________________________
_____________________________________________________

2.

What is the mirroring configuration requirement when you


upgrade T3B firmware?
_____________________________________________________
_____________________________________________________
_____________________________________________________

b.

Answer the following questions for the vxvm_ig.pdf VxVM


manual:
1.

Is the Solaris OS 2.6 release supported by the current


VxVM software? _____________________________________

2.

When is the installvm script used?


_____________________________________________________
_____________________________________________________
_____________________________________________________

3-62

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Configuring VxVM

Task 9 Using the VxVM Error Numbering System


Perform the following steps to research the use of the VERITAS unique
error numbering system.
Use the Adobe Acrobat Reader (acroread) program to review Chapter 5,
Error Messages, of the VERITAS Volume Manager Troubleshooting Guide.
Answer the following questions:
1.

What are the five fields in the error message format?


_____________________________________________________________
_____________________________________________________________
_____________________________________________________________
_____________________________________________________________
_____________________________________________________________

2.

What are the six severity levels?


_____________________________________________________________
_____________________________________________________________
_____________________________________________________________
_____________________________________________________________
_____________________________________________________________
_____________________________________________________________

3.

What is the suggested action for error number V-5-0-145?


_____________________________________________________________
_____________________________________________________________
_____________________________________________________________
_____________________________________________________________

VERITAS Volume Manager Installation


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

3-63

Exercise Summary

Exercise Summary

Discussion Take a few minutes to discuss what experiences, issues, or


discoveries you had during the lab exercises.

Manage the discussion here based on the time allowed for this module, which was given in the About This
Course module. If you find you do not have time to spend on discussion, then just highlight the key concepts
students should have learned from the lab exercise.

Experiences

Ask students what their overall experiences with this exercise have been. You might want to go over any
trouble spots or especially confusing areas at this time.

Interpretations

Ask students to interpret what they observed during any aspects of this exercise.

Conclusions

Have students articulate any conclusions they reached as a result of this exercise experience.

Applications

Explore with students how they might apply what they learned in this exercise to situations at their workplace.

3-64

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Module 4

VERITAS Volume Manager Basic


Operations
Objectives
Upon completion of this module, you should be able to:

Describe the function of VxVM disk groups

List disk group administrative operations including:

Initialize disk drives for VxVM use

Create disk groups

Add and remove disk drives for a disk group

Import and deport disk groups

Destroy a disk group

Rename VxVM disk drives

Administer disk groups using the vxdiskadm utility

Administer disk groups using command-line programs

Administer disk groups using the VEA GUI

4-1
Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

VxVM Disk Group Functions

VxVM Disk Group Functions


A disk group is a collection of VxVM disk drives that share a common
configuration. Volume structures are created within a disk group and are
confined to the disk drives that are associated with that disk group.

Primary Functions of a Disk Group


Disk groups have two primary functions:

Assist administrative management

Provide increased data availability

Easier Administration
Disk groups enable you to group disk drives into logical collections for
administrative convenience. You can group them according to department
or application. For example, you can create separate disk groups for sales,
finance, and development.

Increased Data Availability


You can move a disk group and its components as a unit from one host
machine to another. This feature provides higher availability of the data in
the following ways:

4-2

If one system fails, another system running VxVM can import the
failed systems disk group and provide access to it.

The first system deports the disk group.

Deporting a disk group disables access to that disk group by the first
system. Another host can then import the disk group and start
accessing all the disk drives in the disk group.

The second system imports the disk group and starts accessing it.

A host can only import disk groups with unique names. Therefore,
all disk groups on all systems should be given unique names, with
the exception of the rootdg disk group.

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

VxVM Disk Group Functions

VxVM Disk Drives


There are two phases to bringing a physical disk drive under VxVM
control. Sometimes both operations are done in one step and you are
unaware that the process is more complex.
When you bring a disk drive under VxVM control, you can:

Add it to a new or existing disk group

Add it to the free-disk pool

The easiest operation is to add a disk drive to the free-disk pool. The
vxdisksetup command repartitions the disk drive into VxVM format,
and then a blank header is written to the disk drive.
If you add a disk drive to a disk group, the disk drive is assigned a
unique media name and it is associated with a disk group object. This
information is then written into the blank header on the disk drive.

Disk Drive Media Names


Unless you intervene, the default media names that are assigned to disk
drives are based on either the disk group name or the logical device path
to the disk drive.
Default disk group-based media names are similar to dgX01 or DGa04. The
default device path-based media names assigned by some command-line
programs are similar to c3t0d16s2 or c5t4d0s2. Device path-based
media names can lead to confusing status and configuration listings.

VERITAS Volume Manager Basic Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

4-3

VxVM Disk Group Functions

Standard VxVM Disk Groups


A disk group is a collection of VxVM disk drives that share a common
configuration. Typically, the disk group contains volumes that are all
related in some way, such as file system volumes that belong to a
particular department or database volumes that are all tables for a single
database.
Each disk group is owned by a single host system. The current ownership
is written into all configuration records. Many of the disk drives in the
disk group have a copy of the configuration record.
A disk group and all its components can be moved as a unit from one
host system to another. Usually, both host systems are connected to the
same dual-ported storage arrays.
As shown in Figure 4-1, even though a second host is attached to the same
storage array, access is allowed only to the current owner of the disk
group. A disk group can be deported from one host and imported by a
different host, but this is generally used as an emergency solution to a
catastrophic host system failure. When a disk group is imported by a
different host, the name of the new host is written into the disk-based
VxVM configuration records.
Host 1
The disk group
is owned by
Host 1

X
A

Host 2

Disk Group

Volume

Volume

Storage Array

Figure 4-1

4-4

VxVM Disk Group Ownership

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

VxVM Disk Group Functions

Shared VxVM Disk Groups


VxVM has an optional shared disk group feature that is licensed. The shared
disk group feature allows two or more host systems simultaneous access to
the same disk group. The host systems are referred to as nodes in the
SunPlex system environment. Shared disk groups are used in the
Sun Cluster software environment to support the ORACLE 9i Real
Application Clusters (RAC) database application. Multiple ORACLE 9i
RAC hosts can access a single database image. As shown in Figure 4-2,
shared disk groups are owned by a cluster_name and not by the name of
a single host.
When a shared disk group is imported by any of the attached nodes, the
name of the Sun Cluster software cluster (cluster_name) is written into
the disk-resident VxVM configuration records, and the disk group is
automatically accessible by all of the attached nodes.
Host 2

Host 1
The disk group
is owned by a
cluster_name

A0 A1

B0 B1

Disk Group

Volume

Volume

Sun StorEdge A5200 Array

Figure 4-2

Shared VxVM Disk Group Ownership

Note To prevent data corruption, all write activity must be coordinated


between the nodes. The ORACLE 9i RAC database uses a global lockmanagement scheme to accomplish this coordination.

VERITAS Volume Manager Basic Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

4-5

VxVM Disk Group Functions

Cross-Platform Data Sharing Disk Groups


A new VxVM disk group format called Cross-Platform Data Sharing
(CDS) allows data residing on multi-host storage arrays to be utilized by
different operating systems.
The CDS feature is not licensed by Sun but, by default, disk drives are
initialized in the CDS format.
The new cdsdisk partition map allocates all disk space to slice 7 as
shown in the following example.
Part
Tag
0 unassigned
1 unassigned
2
backup
3 unassigned
4 unassigned
5 unassigned
6 unassigned
7
-

Flag
wm
wm
wu
wm
wm
wm
wm
wu

Cylinders
0
0
0 - 4923
0
0
0
0
0 - 4923

Size
0
0
8.43GB
0
0
0
0
8.43GB

Blocks
(0/0/0)
0
(0/0/0)
0
(4924/0/0) 17682084
(0/0/0)
0
(0/0/0)
0
(0/0/0)
0
(0/0/0)
0
(4924/0/0) 17682084

Overriding the CDS Format


Technically, the cdsdisk format does not interfere with standard VxVM
operation. However, if you are not comfortable using the cdsdisk format
at your site, you can disable it so that the vxdiskadm utility uses the
sliced format when initializing disk drives.
Before initializing or encapsulating any disk drives for VxVM use, use the
vxdiskadm utility option 22, Change/Display the default disk
layouts, to modify the default disk format and private region size. The
changes are stored in the /etc/default/vxdisk file. Use the following
file format:
# more /etc/default/vxdisk
format=sliced
privlen=2048

4-6

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

VxVM Disk Group Operations

VxVM Disk Group Operations


Basic VxVM administrative functions are those that affect disk drives and
disk groups. The VxVM disk operations are performed using a variety of
programs including command-line and graphical.
The following list briefly summarizes each type of disk-related operation.

Placing disk drives under VxVM control

Removing disk drives from VxVM control

Adding a disk drive to a disk group

Removing disk drives from a disk group

Renaming disk drives

Creating disk groups

Destroying disk groups

Importing disk groups

Deporting disk groups

The most commonly used tools for basic operations include:

The vxdisksetup and vxdiskunsetup commands

The vxdg command

The vxedit command

The vxdiskadm utility

The VEA GUI

VERITAS Volume Manager Basic Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

4-7

Verifying Disk Group Status

Verifying Disk Group Status


The most commonly used disk group status commands are vxdisk and
vxdg. Command-line status is usually more efficient than using the VEA
GUI.

Using the vxdisk Command to Verify Disk Group


Status
Use the following vxdisk command to verify the status of the vxdiskadm
changes. Disks that show a status of online invalid are not under
VxVM control. Disks that show a status of online have been initialized
but are not assigned to a disk group. When online disks are added to a
disk group, they are assigned a name which appears in the DISK column.
By default, the disk name is derived from the name of the disk group.
# vxdisk list
DEVICE
TYPE
c0t0d0s2
auto:none
c0t1d0s2
auto:none
c2t1d0s2
auto:cdsdisk
c2t3d0s2
auto:sliced
c2t5d0s2
auto:simple
c2t16d0s2
auto:sliced
c2t18d0s2
auto:none
c2t20d0s2
auto:none
c2t22d0s2
auto:none
c3t32d0s2
auto:cdsdisk
c3t33d0s2
auto:cdsdisk
c3t35d0s2
auto:cdsdisk
c3t37d0s2
auto:sliced
c3t50d0s2
auto:none
c3t52d0s2
auto:sliced

DISK
a5kdg01
a5kdg02
a5kdg03
newDG01
pdga01
pdga02
pdga03
newDG02
-

GROUP
sdga
sdga
sdga
newDG
pdga
pdga
pdga
newDG
-

STATUS
online
online
online
online
online
online
online
online
online
online
online
online
online
online
online

invalid
invalid

invalid
invalid
invalid

invalid

Using the vxdg Command to Verify Disk Group Status


Use the following vxdg command to verify the current status and unique
ID of all active disk groups.
# vxdg list
NAME
sdga
pdga

4-8

STATE
enabled
enabled

ID
1064619733.28.ns-east-104
1065123027.40.ns-east-104

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Administering Disk Groups Using the vxdiskadm Utility

Administering Disk Groups Using the vxdiskadm Utility


The vxdiskadm utility is a Bourne shell program that provides
menu-based user operations. The following example shows that when the
vxdiskadm utility is started, the following menu operations are available:
# vxdiskadm
Volume Manager Support Operations
Menu: VolumeManager/Disk
1 Add or initialize one or more disks
2 Encapsulate one or more disks
3 Remove a disk
4 Remove a disk for replacement
5 Replace a failed or removed disk
6 Mirror volumes on a disk
7 Move volumes from a disk
8 Enable access to (import) a disk group
9 Remove access to (deport) a disk group
10 Enable (online) a disk device
11 Disable (offline) a disk device
12 Mark a disk as a spare for a disk group
13 Turn off the spare flag on a disk
14 Unrelocate subdisks back to a disk
15 Exclude a disk from hot-relocation use
16 Make a disk available for hot-relocation use
17 Prevent multipathing/Suppress devices from VxVMs view
18 Allow multipathing/Unsuppress devices from VxVMs view
19 List currently suppressed/non-multipathed devices
20 Change the disk naming scheme
21 Get the newly connected/zoned disks in VxVM view
22 Change/Display the default disk layouts
23 Mark a disk as allocator-reserved for a disk group
24 Turn off the allocator-reserved flag on a disk
list
List disk information

?
??
q

Display help about menu


Display help about the menuing system
Exit from menus

Select an operation to perform: 1

VERITAS Volume Manager Basic Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

4-9

Administering Disk Groups Using the vxdiskadm Utility

Functional Overview
The title displayed when the vxdiskadm utility starts up is Volume
Manager Support Operations. The vxdiskadm utility performs a wide
range of support functions, but also offers assistance in performing a
number of common administrative tasks.
The basic disk-related vxdiskadm administrative functions presented in
this module are:

Option 1 Add or initialize one or more disks


You use this operation to add one or more disk drives to a disk
group.
You can add the selected disk drives to an existing disk group or to a
new disk group that is created as a part of the operation.
The disk drives are assigned a default media name based on the
name of the intended disk group. You can override the default disk
drive name if needed.
The selected disk drives can also be initialized without adding them
to a disk group by entering a disk group name of none.

Option 2 Encapsulate one or more disks


You use this operation to bring disks under VxVM control while
preserving existing data.
You can also use this operation if you intend to mirror the data for
increased reliability or stripe the data for increased performance.

Option 3 Remove a disk


You use this operation to remove a disk drive from a disk group and
return it to a pool for reuse. The disk remains in a VxVM format but
its media name is cleared.

Option 8 Enable access to a disk group


You use this option to place a disk group online (imports) that was
previously deported (taken offline).

Option 9 Remove access to a disk group


You use this option to take a disk group offline (deports). None of
the disk group volume structures are available while it is deported.

4-10

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Administering Disk Groups Using the vxdiskadm Utility

Creating a New Disk Group


Before creating a new disk group, you must know the logical address of at
least one disk drive you want to be in the disk group. You should also
select media names for any disk drives you intend to add to the new disk
group.
You use vxdiskadm Option 1, shown in the following example, to create a
new disk group and add one or more disk drives.
Select an operation to perform: 1
Select disk devices to add: [<pattern-list>,all,list,q,?] c2t5d0
Which disk group [<group>,none,list,q,?] (default: rootdg) newDG
Create a new group named newDG? [y,n,q,?] (default: y) y
Use a default disk name for the disk? [y,n,q,?] (default: y) y
Add disk as a spare disk for newDG? [y,n,q,?] (default: n) y
Enter the desired format
[cdsdisk,sliced,simple,q,?] (default: sliced) sliced
Enter desired private region length
[<privlen>,q,?] (default: 2048) 2048
VxVM NOTICE V-5-2-120
Creating a new disk group named newDG containing the disk
device c2t16d0 with the name newDG01.
VxVM INFO V-5-2-305 Setting spare flag for disk newDG01
in disk group newDG.
Add or initialize other disks? [y,n,q,?] (default: n) n

Note For clarity, many informational messages are omitted from the
previous example.

VERITAS Volume Manager Basic Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

4-11

Administering Disk Groups Using the vxdiskadm Utility

Removing a Disk Drive From a Disk Group


You use vxdiskadm Option 3 to remove a disk drive from a disk group, as
shown in the following example. The disk drive is placed in the free-disk
pool and is still under VxVM control.
Select an operation to perform: 3
Remove a disk
Menu: VolumeManager/Disk/RemoveDisk
Enter disk name [<disk>,list,q,?] newDG03
VxVM NOTICE V-5-2-284 Requested operation is to remove disk newDG03
from group newDG.
Continue with operation? [y,n,q,?] (default: y) y
VxVM INFO V-5-2-268 Removal of disk newDG03 is complete.
Remove another disk? [y,n,q,?] (default: n) n

4-12

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Administering Disk Groups Using Command-Line Programs

Administering Disk Groups Using Command-Line


Programs
Although there are many VxVM command-line programs, only a few are
necessary for the most commonly performed operations. This section
describes the basic uses of the following command-line programs:

vxdiskunsetup

vxdisksetup

vxdg

Note Only selected options for each command are described in this
module. When appropriate, other options are described in later modules.

Using the vxdiskunsetup Command


Caution Be careful when using the vxdiskunsetup command. You can
destroy existing data on disk drives.
During the VxVM software installation and initialization, you might see
error messages, such as:
VxVM:vxconfigd:
Disk device not
VxVM:vxconfigd:
Disk device not

WARNING: Disk c3t35d0 in group hanfs:


found
WARNING: Disk c2t18d0 in group hadbms:
found

These errors can indicate that there are disk drives that still contain VxVM
configuration records from a previous installation. You can clear these
disk drives and return them to an uninitialized state by using the
vxdiskunsetup command, as shown in the following example:
# /etc/vx/bin/vxdiskunsetup -C c2t3d0
The vxdiskunsetup command will not clear a disk drive if the VxVM
configuration records indicate it is imported by some other host. The -C
option forces the de-partitioning of the disk drive in such a case. The disk
drives are returned to standard Solaris OS partitioning.

VERITAS Volume Manager Basic Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

4-13

Administering Disk Groups Using Command-Line Programs

Initializing Disk Drives


The vxdisksetup command initializes disk drives for VxVM use, but
does not add them to a disk group. The disk drives remain in a free pool
for future VxVM use. A typical vxdisksetup command is:
# vxdisksetup -i c2t0d0
Without any options, the vxdisksetup command only repartitions the
specified disk drive into the VxVM partition format. With the -i option,
VxVM configuration records are written in the private region.
Note Usually, the vxdisksetup command is not used directly. It is
called by other VxVM programs.

Using the vxdg Command


Although the vxdg command has many options, only a few are presented
in this module. You have already seen how the vxdiskadm utility creates
new disk groups and adds disk drives to a disk group. You can also use
the vxdg command to perform those functions. The vxdg command
options related to basic disk-group administration are:

vxdg init

vxdg adddisk

vxdg rmdisk

vxdg import

vxdg deport

vxdg destroy

Note You must initialize a disk drive before it can be added to a new or
existing disk group using the vxdg command.

4-14

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Administering Disk Groups Using Command-Line Programs

Using the vxdg Command to Create a New Disk Group


To create a new disk group using the vxdg command, you must furnish
the disk drive logical access name (accessname) of at least one disk drive
to be added to the disk group. The VxVM accessname is essentially the
logical path to the disk drive in the form: c3t4d0. You should also specify
a media name for the disk drive. If you do not specify a media name, it
defaults to the accessname. The following shows a typical session to
initialize a new disk drive and add it to a new disk group.
# vxdisk list
DEVICE
c0t0d0s2
c0t1d0s2
c2t1d0s2
c2t3d0s2
c2t5d0s2
c2t16d0s2
c2t18d0s2
c2t20d0s2
c2t22d0s2
c3t32d0s2
c3t33d0s2
c3t35d0s2
c3t37d0s2
c3t50d0s2
c3t52d0s2

TYPE
auto:none
auto:none
auto:none
auto:none
auto:none
auto:none
auto:none
auto:none
auto:none
auto:none
auto:none
auto:none
auto:none
auto:none
auto:none

DISK
-

GROUP
-

STATUS
online
online
online
online
online
online
online
online
online
online
online
online
online
online
online

invalid
invalid
invalid
invalid
invalid
invalid
invalid
invalid
invalid
invalid
invalid
invalid
invalid
invalid
invalid

# vxdisksetup -i c2t1d0
# vxdisksetup -i c2t3d0
# vxdg init newDG ndg-01=c2t1d0 ndg-02=c2t3d0
# vxdisk -g newDG list
DEVICE
c2t1d0s2
c2t3d0s2

TYPE
auto:sliced
auto:sliced

DISK
ndg-01
ndg-02

GROUP
newDG
newDG

STATUS
online spare
online spare

# vxdg list
NAME
newDG

STATE
enabled

ID
1065465185.43.ns-east-104

VERITAS Volume Manager Basic Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

4-15

Administering Disk Groups Using Command-Line Programs

Adding and Removing Disk Drives


The following shows an example of initializing two disk drives, adding
them to an existing disk group, and then removing one of them.
# vxdisksetup -i c2t5d0
# vxdisksetup -i c2t16d0
# vxdg -g newDG adddisk ndg-03=c2t5d0 ndg-04=c2t16d0
# vxdisk list
DEVICE
TYPE
c0t0d0s2
auto:none
c0t1d0s2
auto:none
c2t1d0s2
auto:sliced
c2t3d0s2
auto:sliced
c2t5d0s2
auto:sliced
c2t16d0s2
auto:sliced
c2t18d0s2
auto:none
c2t20d0s2
auto:none
c2t22d0s2
auto:none
c3t32d0s2
auto:none
c3t33d0s2
auto:none
c3t35d0s2
auto:none
c3t37d0s2
auto:none
c3t50d0s2
auto:none
c3t52d0s2
auto:none

DISK
ndg-01
ndg-02
ndg-03
ndg-04
-

GROUP
newDG
newDG
newDG
newDG
-

STATUS
online
online
online
online
online
online
online
online
online
online
online
online
online
online
online

invalid
invalid

invalid
invalid
invalid
invalid
invalid
invalid
invalid
invalid
invalid

# vxdg -g newDG rmdisk ndg-04


The removed disk drive is still initialized and is available for future use. It
is in the free-disk pool and shows a status of auto:sliced and online
with no DISK or GROUP entry.

4-16

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Administering Disk Groups Using Command-Line Programs

Importing and Deporting Disk Groups


At times it is useful to deport a disk group. This function makes the disk
group unavailable and invisible to most commands. If the disk group
resides on dual-ported storage arrays, a different host can import the disk
group. This action is useful if a host system fails. A different host system
that is running VxVM can import the disk group and make the data
available again to users. The following example shows the process of
deportng a disk group.
# vxdg list
NAME
sdga
newDG
pdga

STATE
enabled
enabled
enabled

ID
1064619733.28.ns-east-104
1065465185.43.ns-east-104
1065123027.40.ns-east-104

# vxdg deport newDG


# vxdg list
NAME
sdga
pdga

STATE
enabled
enabled

ID
1064619733.28.ns-east-104
1065123027.40.ns-east-104

When a disk group is deported, the host ID stored on all disk drives in the
disk group is cleared (unless a new host ID is specified with -h).
Therefore, the disk group is not reimported automatically when the
system is rebooted.
The disk group can be deported with the host ID unchanged or you can
change the host ID to another system during the deport operation.
Use the vxdg import command option to import a disk group again. For
example:
# vxdg import newDG

VERITAS Volume Manager Basic Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

4-17

Administering Disk Groups Using Command-Line Programs

Identifying Deported Disk Groups


If you forget the name of a deported disk group, you can use the vxdisk
command to identify currently deported disk groups. The following
example shows the names of currently deported disk groups enclosed in
parenthesis.
# vxdisk -o alldgs list
DEVICE
TYPE
c0t0d0s2
auto:none
c0t1d0s2
auto:none
c2t1d0s2
auto:cdsdisk
c2t3d0s2
auto:sliced
c2t5d0s2
auto:simple
c2t16d0s2
auto:sliced
c2t18d0s2
auto:sliced
c2t20d0s2
auto:sliced
c2t22d0s2
auto:none
c3t32d0s2
auto:cdsdisk
c3t33d0s2
auto:cdsdisk
c3t35d0s2
auto:cdsdisk
c3t37d0s2
auto:sliced
c3t50d0s2
auto:none
c3t52d0s2
auto:sliced

DISK
a5kdg01
a5kdg02
a5kdg03
pdga01
pdga02
pdga03
-

GROUP
sdga
sdga
sdga
(newDG)
pdga
pdga
pdga
(newDG)
-

STATUS
online
online
online
online
online
online
online
online
online
online
online
online
online
online
online

invalid
invalid
nohotuse
nohotuse
nohotuse

invalid

invalid

You can also use the vxdiskadm utility option 8, Enable access to
(import) a disk group, to identify deported disk groups. The
following is an excerpt of the resulting output.
Select disk group to import [<group>,list,q,?] (default:
list) list
GROUP newDG (id: 1065465185.43.ns-east-104)
DEVICES: c2t16d0 c3t37d0

Note The VEA interface also displays deported disk groups.

4-18

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Administering Disk Groups Using Command-Line Programs

Destroying a Disk Group


Before destroying a disk group, you must be absolutely sure that none of
the data on the disk drives is important. A disk group cannot be
destroyed if any volumes in the disk group are in use. You can destroy
only imported disk groups. The following is an example of using the vxdg
command to destroy a disk group.
# vxdg list
NAME
sdga
newDG
pdga

STATE
enabled
enabled
enabled

ID
1064619733.28.ns-east-104
1065465185.43.ns-east-104
1065123027.40.ns-east-104

# vxdg destroy newDG


# vxdg list
NAME
sdga
pdga

STATE
enabled
enabled

ID
1064619733.28.ns-east-104
1065123027.40.ns-east-104

Renaming VxVM Disk Drives


When disk drives are added to a disk group, the disk drives are given
standard VxVM media names, such as dga01, dga02, and dga03. You
might want to rename the disk drives in a group so that there is no
confusion about their intended use.
You can use the following vxedit command to rename VxVM disk
drives:
# vxedit -g sdga rename a5kdg01 sdga01
# vxedit -g sdga rename a5kdg02 sdga02
# vxedit -g sdga rename a5kdg03 sdga03

Note Volume components such as plexes and subdisks are named


according to the VxVM disk media name. If you rename a disk, the plex
and subdisk names do not change accordingly. It is best to rename disks
before creating volume structures on them.

VERITAS Volume Manager Basic Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

4-19

Administering Disk Groups Using the VEA GUI

Administering Disk Groups Using the VEA GUI


The following section describe a series of tasks using the VEA GUI. Only
key points and input forms are presented.

Creating a New Disk Group


You can use the VEA GUI to create a new disk group. Complete the
following steps:
1.

If appropriate, you can pre-select disk drives in the grid area by


simultaneously pressing the Control key and mouse button.

2.

Select the New Disk Group entry from the Actions menu.

3.

Complete the New Disk Group Form, as shown in Figure 4-3.


Disk names default to variations of the disk group name, such as
newDG01 or newDG02. Default disk names are usually acceptable.

Figure 4-3

4-20

New Disk Group Wizard

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Administering Disk Groups Using the VEA GUI


4.

Click Next to continue.


As shown in Figure 4-4, VxVM informs you of potentially dangerous
situations it detects. Click on Yes to continue or No to abort.

Figure 4-4
5.

New Disk Group Warning

Complete the Organization window, as shown in Figure 4-5, by


selecting a disk group organization principle of None.

Figure 4-5

New Disk Group Organization

VERITAS Volume Manager Basic Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

4-21

Administering Disk Groups Using the VEA GUI

Adding and Removing Disk Drives


To add disks to, or remove disk drives from, a disk group, complete the
following steps:
1.

Select the disk group in the object tree, and then click the third
mouse button.

2.

Select Add Disk to Disk Group from the pop-up menu, as shown in
Figure 4-6.

Figure 4-6

4-22

Disk Groups Disk Add Menu Item

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Administering Disk Groups Using the VEA GUI


3.

Complete the Add Disk wizard form, as shown in Figure 4-7.

Figure 4-7

Add Disk to Disk Group Wizard

VERITAS Volume Manager Basic Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

4-23

Administering Disk Groups Using the VEA GUI

Deporting Disk Groups


To deport a disk group, complete the following steps:
1.

Select the disk group in the object tree, and then click the third
mouse button.

2.

Select Deport from the pop-up menu, as shown in Figure 4-8.

Figure 4-8
3.

Disk Groups Deport Menu Item

Complete the Deport Disk Group form, as shown in Figure 4-9.


Usually, you do not modify the disk group name or host system
ownership during a deport operation.

Figure 4-9

4-24

Deport Disk Group Form

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Administering Disk Groups Using the VEA GUI

Importing Disk Groups


To import a disk group, complete the following steps:
1.

Display Disk Groups in the grid area, and then select the deported
disk group.

2.

Click the third mouse button on the deported disk group and select
Import from the pop-up menu, as shown in Figure 4-10.

Figure 4-10 Import Disk Group Menu Item


3.

Complete the Import Disk Group form, as shown in Figure 4-11.


Typically, the Force option is necessary only after a system crash.

Figure 4-11 Import Disk Group Form

VERITAS Volume Manager Basic Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

4-25

Administering Disk Groups Using the VEA GUI

Destroying a Disk Group


A disk group must be imported before it can be destroyed. To destroy a
disk group, complete the following steps:
1.

Select the disk group in the object tree, and then click the third
mouse button.

2.

Select Destroy Disk Group from the pop-up menu, as shown in


Figure 4-12.

Figure 4-12 Disk Group Destroy Menu


If there are mounted volumes associated with the disk group, the
disk group destroy operation fails, as shown in Figure 4-13.

Figure 4-13 Destroy Disk Group Failure

4-26

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Administering Disk Groups Using the VEA GUI

Renaming VxVM Disk Drives


To rename VxVM disk drives, complete the following steps:
1.

To change the media name of a VxVM disk drive, complete the


following steps:
a.

Display the disk group disk drives in the grid area.

b.

Select the disk drive to be renamed, as shown in Figure 4-14.

Figure 4-14 Disk Drive Media Name Listing


2.

Click the third mouse button on the highlighted disk, and then select
Rename from the pop-up menu.

3.

Complete the Rename Disk form, as shown in Figure 4-15.

Figure 4-15 Rename Disk Form

VERITAS Volume Manager Basic Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

4-27

Administering Disk Groups Using the VEA GUI

Displaying VEA Object Properties


Most VxVM objects have associated properties that detail naming
schemes, construction details, and current status.
To display the properties of a VxVM object, complete the following steps:
1.

Click on the object and the third mouse button.

2.

Select Properties from the objects pop-up menu.


The Properties of a disk group is shown in Figure 4-16. Object
Properties windows can have several tabs depending on the
complexity of the object.

Figure 4-16 Disk Group Object Properties Form

4-28

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Performing VxVM Basic Operations

Exercise: Performing VxVM Basic Operations


In this exercise, you complete the following tasks:

Review key lecture points

Verify the VxVM environment

Verify the initial disk drive status

Set the default disk drive format

Initialize disk drives

Create new disk groups

View command logs

Import and deport disk groups

Destroy a disk group

Rename disk drives

Use the vxdiskadm utility to perform basic operations (optional)

Verify ending lab status

VERITAS Volume Manager Basic Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

4-29

Exercise: Performing VxVM Basic Operations

Preparation
If the tasks in this exercise are performed by small groups using disk
drives residing on a central VxVM server, each group must take care to
not interfere with another groups storage resources and structures.
Ask your instructor to provide two unique code letters for your
workgroup (A and B, C and D, E and F, and so on).
Workgroup code letters:

dg ___

dg ___

Copy the logical paths to six disk drives for your work group from the
information in Module 2, Managing Data in Task 9 Selecting Disk
Drives for Use on page 2-32.
Disk: _______________

Disk: _______________

Disk: _______________

Disk: _______________

Disk: _______________

Disk: _______________

Assign each workgroups two code letters (A/B, C/D, E/F) so they can create two unique disk group names,
such as dgA and dgB, or dgE and dgF. A workgroup consists of two or more students working with six disk
drives from one keyboard on an administration workstation.
An antiquated, but useful, management tool is to have each workgroup write the logical paths of their
assigned disk drives on a 3 by 5 card and tape the card to their display monitor.

Many of the tasks are performed twice. The first time using the VEA GUI
and the second time using command-line programs. For most tasks, you
must destroy the structures before creating them again. Destroying and
deleting structures is part of regular VxVM administrative tasks.

4-30

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Performing VxVM Basic Operations

Task 1 Reviewing Key Lecture Points


Answer the following questions about the major areas of concern during
the VxVM installation and initialization process.
1.

What is the primary use for a shared disk group?


a.

Support for load balancing

b.

Support for multi-host storage access

c.

Support for parallel databases

d.

Support for multipath storage access

The answer is c.

2.

Which two answers best describe the VxVM disk-drive initialization


process?
a.

Preserves existing file systems

b.

Places the disk drive in a free disk pool

c.

Adds the disk drive to a disk group

d.

Destroys existing file systems

The answers are b and d.

3.

Which of the following commands is not used to perform disk group


administration?
a.

vxdisk

b.

vxassist

c.

vxdg

d.

vxdiskadm

e.

vxdisksetup

The answer is b.

4.

Which of the following commands is used to rename disk drives?


a.

vxdg

b.

vxassist

c.

vxdisk

d.

vxedit

The answer is d.

VERITAS Volume Manager Basic Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

4-31

Exercise: Performing VxVM Basic Operations

Task 2 Verifying the VxVM Environment


Complete the following steps to verify the environment on the VxVM
administration system and the VxVM server:
1.

Log in to the VxVM server as user root and enter the env shell
command.

2.

Verify that the following environment exists on the server:


PATH=/usr/sbin:/usr/bin:/opt/VRTS/bin:/opt/VRTSalloc/bi
n:/opt/VRTSvlic/bin:/opt/VRTSvmpro/bin:/etc/vx/bin
MANPATH=/usr/man:/opt/VRTS/man:/opt/VRTSlic/man

4-32

3.

If you are working from a remote administration system, log out of


the VxVM server.

4.

On the remote administration system, complete the following steps:


a.

Type the env shell command.

b.

Verify that the /opt/VRTS/bin and /opt/VRTSob/bin


directories are part of the PATH variable.

c.

Verify that the /opt/VRTS/man directory is part of the MANPATH


variable.

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Performing VxVM Basic Operations

Task 3 Verifying the Initial Disk Drive Status


To verify the initial status of your assigned disk drives, complete the
following steps:
1.

Log in as user root to the VxVM server.

2.

Verify the status of your assigned disk drives.


# vxdisk list

3.

Check the output carefully and verify that each disk drive assigned
to your workgroup shows a status of online invalid.
If any of your assigned disk drives show a status other than online
invalid, check with your instructor. If appropriate, use the
vxdiskunsetup command, as shown in the following example, to
return all of your assigned disk drives to an online invalid status.
# /etc/vx/bin/vxdiskunsetup -C c0t22d0

Note You must substitute the VxVM accessname of your disk drives.
Do not proceed until all your assigned disk drives are uninitialized.

VERITAS Volume Manager Basic Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

4-33

Exercise: Performing VxVM Basic Operations

Task 4 Setting the Default Disk Drive Format


Set the default VxVM disk initialization format on the VxVM server by
completing the following steps:
1.

Log in to the VxVM server as user root.

2.

Complete the following steps:


a.

Start the vxdiskadm utility.

b.

Select option 22, Change/Display the Default Disk Layouts.

# vxdiskadm
3.

Select option 1 from the Volume Manager Preferences menu.

4.

Select option 1 from the Disk Initialization Preferences menu and


type sliced for the desired format.

5.

Select option 2 from the Disk Initialization Preferences menu and


type 2048 for the desired private region length.

6.

Type q or quit at least twice to exit the vxdiskadm utility.

7.

Verify the default disk initialization format values are set correctly.
# more /etc/default/vxdisk
format=sliced
privlen=2048

Caution Values entered in the /etc/default/vxdisk file are honored


only by command-line programs such as the vxdiskadm and vxassist
utilities. The VEA GUI does not reference the /etc/default/vxdisk file.

4-34

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Performing VxVM Basic Operations

Task 5 Initializing Disk Drives


Perform the following procedures to initialize your assigned disk drives
for use by VxVM. There are two procedures: one for using the VEA GUI
and one for using command-line programs.

Using the VEA GUI to Initialize Disk Drives


To initialize three of your assigned disk drives using the VEA GUI,
complete the following steps:
1.

Expand the VxVM server node in the object tree and click Disks.

2.

Select three of your assigned disk drives in the grid area by


simultaneously pressing the Control key while using the mouse
button.

3.

Click the third mouse button on one of your highlighted disk drives,
and select Initialize Disk in the pop-up menu.

4.

In the Initialize Disk form, click Yes To All.


After a short delay, three of your assigned disk drives should appear
in the grid area with a status of Free.

5.

Complete the following steps:


a.

Use the vxdiskunsetup command to return each of your


assigned disk drives to an uninitialized state.

b.

Replace the accessname variable in the following command


with the address of your disk drives (for example, c4t3d0).

# vxdiskunsetup -C accessname
6.

Verify that the status of your assigned disk drives is once again
online invalid.
# vxdisk list

VERITAS Volume Manager Basic Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

4-35

Exercise: Performing VxVM Basic Operations

Using Command-Line Programs to Initialize Disk Drives


To initialize your remaining three disk drives using the vxdisksetup
command-line program, complete the following steps:
1.

Use the vxdisksetup command to initialize each of your assigned


disk drives, replacing the accessname with the address of your disk
drives (for example, c4t3d0).
# vxdisksetup -i accessname

2.

Verify the status of your assigned disk drives is online.


# vxdisk list

Task 6 Creating New Disk Groups


In this section, you create a disk group containing three of your assigned
disk drives. Name the disk group according to your first assigned
workgroup letter. For example, if your assigned work group letters are A
and B, then this first disk group should be named dgA.

Using the VEA GUI to Create a New Disk Group


To create a new disk group using the VEA GUI, complete the following
steps:
1.

Click New Group in the Toolbar.


The initial New Disk Group Wizard form appears.

2.

Click the Do not show this page next time box, and then click
Next.
The disk selection form appears.

3.

Complete the disk selection form as follows:


a.

Type the name of your new disk group.

b.

In the Available disk column, select three of your assigned disk


drives, and then click Add.

c.

Do not enter disk names.

d.

Click Next.
The organization principle form appears.

4-36

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Performing VxVM Basic Operations


4.

Complete the organization principle form as follows:


a.

Select the None organization principle.

b.

Click Next.
The summary form appears.

5.

Click Finish on the summary form to create the new disk group.

6.

Verify that your new disk group appears in the VEA object tree.

Using the vxdg Command to Create a Disk Group


Use the vxdg command to create a second disk group that contains your
three remaining assigned disk drives. Name the disk group according to
your second workgroup letter. For example, if your work group letters are
A and B, then this second disk group should be named dgB.
Complete the following steps:
1.

Use the vxdg init command to create a second new disk group and
add one of your remaining assigned disk drives.
# vxdg init dgname medianame=accessname
For example:
vxdg init dgB dgB-01=c2t3d0

2.

Add the two remaining assigned disk drives to the new disk drive
group.
# vxdg -g dgname adddisk medianame=accessname
For example:
vxdg -g dgB adddisk dgB-02=c2t5d0 dgb-03=c2t6d0

3.

Use the vxprint command to verify the status of your new disk
group.

4.

Use the vxdg rmdisk command to remove one of the disk drives
from your new disk group.
# vxdg -g dgname rmdisk medianame
For example: vxdg -g dgB rmdisk dgB-03

5.

Use the vxdg adddisk command to add the same disk drive back
into your disk group.

VERITAS Volume Manager Basic Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

4-37

Exercise: Performing VxVM Basic Operations

Task 7 Viewing Command Logs


The command log records the command-line programs used to
accomplish VEA tasks. It is a good learning tool.
To view the command log file, complete the following steps:
1.

Log in to the VxVM server as user root.

2.

View the command log file.


# cd /var/vx/isis
# ls
alertlog
state
command.log
swap
savedqueries
tasklog
# more ./command.log

vea_portal
vxisis.lock
vxisis.log

Task 8 Importing and Deporting Disk Groups


In this task, you deport and import one or both of your disk groups.

Using the VEA GUI to Import and Deport Disk Groups


To import and deport disk groups using the VEA GUI, complete the
following steps:
1.

Display disk groups in the grid area.

2.

Click the name of one of your disk groups and select Deport Disk
Group from its pop-up menu.
The Deport Disk Group form appears.

Note You can rename a disk group during a deport operation. You can
also assign ownership to a different host. You might do this if you needed
to take the current host down for maintenance and wanted a different
host system to manage the disk group for a while. You might also rename
the disk group if the second host already had a disk group with the same
name.
3.

4-38

Click OK.

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Performing VxVM Basic Operations


4.

Complete the following steps:


a.

Click Browse and highlight the disk group you want to deport.

b.

Click OK.
The status of your disk group should change to Deported in the
grid area display.

5.

Click your deported disk group in the grid area and select Import
from its pop-up menu.

Using Command-Line Programs to Import and Deport Disk


Groups
To import and deport disk groups using command-line programs,
complete the following steps:
1.

Log in to the VxVM server and use the vxdg deport command to
deport either of your disk groups.
# vxdg deport dgname
For example: vxdg deport dgA.

2.

Use the vxdisk command to identify the deported disk group.


# vxdisk -o alldgs list

3.

Use the vxdg import command to import your disk group again.
# vxdg import dgname
For example: vxdg import dgA.

VERITAS Volume Manager Basic Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

4-39

Exercise: Performing VxVM Basic Operations

Task 9 Destroying a Disk Group


In this task, you destroy one of your disk groups.

Using the VEA GUI to Destroy a Disk Group


To destroy a disk group using the VEA GUI, complete the following steps:
1.

Display Disk Groups in the grid area.

2.

Click one of your disk groups and select Destroy Disk Group from
its pop-up menu.

3.

Click New Group in the Toolbar and recreate the disk group you
destroyed.

Using Command-Line Programs to Destroy a Disk Group


To destroy one of your disk groups using command line programs,
complete the following steps:
1.

Use the vxdg command to destroy one of your disk groups.


# vxdg destroy dgname
For example: vxdg destroy dgA.

2.

Recreate the destroyed disk group using the tool of your choice.

Task 10 Renaming Disk Drives


In this task, you rename one or more of your assigned disk drives.

Using the VEA GUI to Rename Disk Drives


To rename disk drives using the VEA GUI, complete the following steps:

4-40

1.

Display the disk drives from one of your disk groups in the grid
area.

2.

Click one of the disk drives and select Rename Disk from its pop-up
menu.

3.

Enter a unique media name for your disk drive.

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Performing VxVM Basic Operations

Using Command-Line Programs to Rename Disk Drives


To rename disk drives using command-line programs, complete the
following step:
Using the vxedit command, restore the original media name of the disk
drive you modified in Using the VEA GUI to Rename Disk Drives on
page 4-40.
# vxedit rename oldname newname
For example: vxedit rename zx12 dgB-03.

Task 11 Using the vxdiskadm Utility to Perform Basic


Operations (Optional)
Ask your instructor if there is enough time to perform this optional task.
To recreate your disk groups using the vxdiskadm utility features,
complete the following steps:
1.

Use the vxdg command to destroy both of your disk groups.


# vxdg destroy dgname

2.

Use the vxdiskadm utility option 1 to recreate both of your assigned


disk groups.

3.

Use the vxdiskadm utility option 3 to remove a disk drive from each
of your disk groups.

4.

Use the vxdiskadm utility option 1 to add the previously removed


disk drives into your disk groups.

5.

Use the vxdiskadm utility option 9 to deport one or both of your


disk groups.

6.

Use the vxdiskadm utility option 8 to import one or both of your


assigned disk groups.

VERITAS Volume Manager Basic Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

4-41

Exercise: Performing VxVM Basic Operations

Task 12 Verifying Ending Lab Status


To verify your system configuration, complete the following steps:
1.

Use the vxdisk list command to ensure that both of your disk
groups are still complete and meet the following guidelines:

You have two disk groups name dgX and dgY.

Each disk group contains three disk drives.

None of your assigned disk drives are designated as hot spares.

Note Substitute your workgroup codes for the X and Y in dgX and dgY.
2.

4-42

Use the vxprint command to verify that there are no subdisk


structures in either of your disk groups.

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise Summary

Exercise Summary

Discussion Take a few minutes to discuss what experiences, issues, or


discoveries you had during the lab exercises.

Manage the discussion here based on the time allowed for this module, which was given in the About This
Course module. If you find you do not have time to spend on discussion, then just highlight the key concepts
students should have learned from the lab exercise.

Experiences

Ask students what their overall experiences with this exercise have been. You might want to go over any
trouble spots or especially confusing areas at this time.

Interpretations

Ask students to interpret what they observed during any aspects of this exercise.

Conclusions

Have students articulate any conclusions they reached as a result of this exercise experience.

Applications

Explore with students how they might apply what they learned in this exercise to situations at their workplace.

VERITAS Volume Manager Basic Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

4-43

Module 5

VERITAS Volume Manager Volume


Operations
Objectives
Upon completion of this module, you should be able to:

Interpret volume structure listings

Describe volume planning activities

Create volumes using the VEA GUI

Create volumes using the vxassist command

Modify volume access attributes

Add file systems to existing volumes

Add and remove volume logs

Use the VEA GUI to analyze volume structures

5-1
Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Interpreting Volume Structure Listings

Interpreting Volume Structure Listings


VERITAS VxVM volume structures are composed of three primary
components: subdisks, plexes, and a volume name. The components are
assigned names based on default conventions or can be manually chosen.

Subdisks
A subdisk is a set of contiguous disk blocks. A subdisk must reside
entirely on a single physical disk drive. The public region of a disk drive
in a disk group can be divided into one or more subdisks. The subdisks
cannot overlap or share the same portions of a public region.
The smallest possible subdisk is a single sector (512 bytes), and the largest
subdisk is the entire VxVM public region.
By default, subdisks are named based on the VxVM media name of the
disk drive on which they reside. This relationship is shown in Figure 5-1.
Disk Group (DGa)
VxVM Disk
Physical Disk

Subdisks

disk01-01

c3t12d0
disk01-02
disk01
VxVM Disk
disk02-01
Physical Disk
c4t33d0

disk02-02
disk02-03
disk02

Figure 5-1

Subdisk Naming Conventions

Ask why spanning storage arrays might be a good idea. One answer is striping for performance or mirroring
for availability.

5-2

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Interpreting Volume Structure Listings

Plexes
The VxVM application uses subdisks to build virtual objects called plexes.
A plex consists of one or more subdisks located on one or more physical
disk drives. Figure 5-2 shows the relationship of subdisks to plexes in a
disk group named DGa.
Disk Group (DGa)

Physical Disk

VxVM Disk

Plex

disk01-01

disk01-01

disk01-02

disk01-02

disk01

vol01-01

VxVM Disk

Plex

disk02-01

disk02-01

disk02-02

disk02-02

c3t12d0

Physical Disk
c4t33d0

disk02-03

vol01-02

disk02

Figure 5-2

Plex Configurations

The data to be stored on the subdisks of a plex can be organized by using


any of the following methods:

Concatenation

Striping

Striping with parity (RAID 5)

Note A plex can contain a maximum of 4096 subdisks.

VERITAS Volume Manager Volume Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

5-3

Interpreting Volume Structure Listings

Volumes
A volume consists of one or more plexes. By definition, a volume with
two plexes is mirrored. Figure 5-3 shows the relationship of plexes in a
mirrored volume in a disk group named DGa.
Disk Group (DGa)
Volume

Physical Disk

VxVM Disk

Plex

disk01-01

disk01-01

disk01-02

disk01-02

disk01

vol01-01

VxVM Disk

Plex

disk02-01

disk02-01

disk02-02

disk02-02

c3t12d0

Physical Disk
c4t33d0

disk02-03

vol01-02

disk02

Figure 5-3

vol01

Mirrored Volume Structure

You should understand the following important points about volumes:

Volumes can have more than two mirrors.

Three mirrors is usually as many as most customers ever have in critical applications.

5-4

RAID-5 volumes cannot be mirrored.

A plex can also be a logging structure that is not used for data
storage.

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Interpreting Volume Structure Listings

Volume Structure Examples


The following vxprint output shows an example of two volumes that
belong to the same disk group. The vol_01 volume is a simple
concatenation and the vol_02 volume is mirrored across two controllers.
The listing indicates the following details:

Both volumes are 2 Gbytes in size (4194304 blocks/sectors)

vol_01 is composed of a single plex and associated subdisk

vol_02 is composed of two plexes each with an associated subdisk

# vxprint -g newDG
TY NAME
ASSOC
dg newDG
newDG

KSTATE
-

LENGTH
-

PLOFFS
-

STATE
-

dm
dm
dm
dm
dm
dm

newDG01
newDG02
newDG03
newDG04
newDG05
newDG06

c2t16d0s2
c2t18d0s2
c2t20d0s2
c3t32d0s2
c3t33d0s2
c3t35d0s2

17679776
17679776
17679776
17679776
17679776
17679776

v vol_01
pl vol_01-01
sd newDG01-01

fsgen
vol_01
vol_01-01

ENABLED
ENABLED
ENABLED

4194304
4194304
4194304

ACTIVE
ACTIVE
-

v
pl
sd
pl
sd

fsgen
vol_02
vol_02-01
vol_02
vol_02-02

ENABLED
ENABLED
ENABLED
ENABLED
ENABLED

4194304
4194304
4194304
4194304
4194304

0
0

SYNC
ACTIVE
ACTIVE
-

vol_02
vol_02-01
newDG04-01
vol_02-02
newDG01-02

The following vxdg output shows the amount of available disk space. The
OFFSET column represents the amount of space currently used and the
LENGTH column represents the amount of free space.
# vxdg -g newDG free
DISK
DEVICE
newDG01
c2t16d0s2
newDG02
c2t18d0s2
newDG03
c2t20d0s2
newDG04
c3t32d0s2
newDG05
c3t33d0s2
newDG06
c3t35d0s2

TAG
c2t16d0
c2t18d0
c2t20d0
c3t32d0
c3t33d0
c3t35d0

OFFSET
8388608
0
0
4194304
0
0

LENGTH
9291168
17679776
17679776
13485472
17679776
17679776

FLAGS
-

VERITAS Volume Manager Volume Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

5-5

Volume Planning

Volume Planning
Creating volume structures is easy to do. It is also easy to make mistakes
unless you understand each aspect of the volume creation process.

Volume Distribution
A common mistake is to place all the disk drives in a single disk group.
The configuration records for a disk group cannot contain information for
more than 4096 objects. Each volume, plex, subdisk, and disk drive is
considered to be an object and requires 256 bytes of space in the private
region. The default private region length is 2048 blocks.
Another reason for organizing disk drives into separate disk groups is
that you might want to deport a disk group and import it to another
connected host system. This action can be part of a disaster recovery plan
or a load balancing measure.
You can design a disk group so that it is better for particular tasks. Each
disk group shown in Figure 5-4 has three disk drives, and each disk drive
is in a different storage array.
Host System

c0

c2

DGa

d1

d2

d3

DGb

d4

d5

d6

DGc

d7

d8

d9

Array

Array

Array

Figure 5-4

5-6

c1

Striped Volume Disk Groups

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Volume Planning
Disk groups organized in this manner are good for creating striped
volume types (such as RAID 5) and for mirrored volumes. The most
important feature is that each disk drive in the disk group is in a separate
enclosure and on a different controller.
Note Exercise care with disk groups that span storage arrays. You must
be sure that the loss of an entire array does not disrupt both mirrors in a
volume or more than one column in a RAID-5 volume.
Another disk group structure, such as the one shown in Figure 5-5, would
be better used with simple concatenated volumes.
Host System

c0

c1

c2

DGa

d1

d2

d3

DGb

d4

d5

d6

DGc

d7

d8

d9

Array

Array

Array

Figure 5-5

Concatenated Volume Disk Groups

If the volumes are large, static, read-only structures that need only a
periodic backup to tape, they do not need any higher level of reliability or
availability.

VERITAS Volume Manager Volume Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

5-7

Volume Planning

Volume Naming Conventions


Within a disk group, the VxVM software requires a unique name for each
new volume that is created. The volume names are usually systematic,
such as vol01, vol02, vol03, and so on. Naming conventions can be used
to reflect volume attributes, such as:

The volume structure

Which department uses them

With which database they are associated

Special purposes within a workgroup

Although naming conventions might not seem to be of much importance,


they can help establish priorities during emergency situations, such as
major power outages.

Space Allocation Planning


Although choosing a general size for a volume is frequently dictated by
the application, administrators frequently want to use as much space as is
practical on a set of disk drives. There are several ways to allocate space
for a volume. Among them are:

Let the VxVM software automatically find the space.

Limit the search for space to selected disk drives in a group.

Research available space with command-line programs.

Note The VEA GUI New Volume Wizard also has limited space research
capabilities during new volume creation.

Automatic Space Allocation


If you do not specify disk resources when creating volumes, the VxVM
software automatically finds portions of unused disk space and assembles
them into a volume. This action can lead to a disorganized structure and
create poor performance for some volume types.

5-8

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Volume Planning

Restricted Space Allocation


Rather than letting VxVM find space anywhere within a disk group, it is
frequently better to direct VxVM to use a particular disk drive for a new
volume.
The disk group shown in Figure 5-6 can be used in several different ways
depending on the type of volume structures you require:

A RAID-5 volume might use disk drives d1, d2, and d3

A concatenated volume might use disk drives d1, d4, and d7.

A mirrored and concatenated volume might use disk drives d1, d4,
and d7 for one mirror and disk drives d3, d6, and d9 for a second
mirror.
Host System
c0

DGa

Figure 5-6

c1

c2

d1

d2

d3

d4

d5

d6

d7

d8

d9

Array

Array

Array

Selecting Disk Drives for a Volume

VERITAS Volume Manager Volume Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

5-9

Volume Planning

Researched Space Allocation


Analyzing free disk space before creating a volume is sometimes
beneficial. Look for patterns of free space that fit your needs. The
following examples show how to research free space in a disk group.
The vxdg command gathers a rough estimate of available disk space. The
following is an example of using vxdg on a 9-Gbyte disk drive.
# vxdg -g newDG free
DISK
DEVICE
ndg-01
c2t1d0s2
ndg-02
c2t3d0s2
ndg-03
c3t32d0s2
ndg-04
c3t33d0s2

TAG
c2t1d0
c2t3d0
c3t32d0
c3t33d0

OFFSET
0
0
0
0

LENGTH
17674902
17674902
17674902
17674902

FLAGS
-

After creating a 6-Gbyte mirrored volume using the ndg-01 and ndg-03
disks, the following disk space is available:
# vxdg -g newDG free
DISK
DEVICE
ndg-01
c2t1d0s2
ndg-02
c2t3d0s2
ndg-03
c3t32d0s2
ndg-04
c3t33d0s2

TAG
c2t1d0
c2t3d0
c3t32d0
c3t33d0

OFFSET
12586455
0
12586455
0

LENGTH
5088447
17674902
5088447
17674902

FLAGS
-

After rough planning, you use the vxassist maxsize command to


research more detailed information. The following examples show how to
use the vxassist command:
# vxassist -g newDG maxsize \
layout=nomirror,nostripe ndg-01 ndg-02 ndg-03 ndg-04
Maximum volume size: 45524992 (22229Mb)
This is a basic concatenation that uses almost all of the available space, that is, 22 Gbytes.

# vxassist -g newDG maxsize \


layout=raid5,nolog ndg-01 ndg-02 ndg-03 ndg-04
Maximum volume size: 20353024 (9938Mb)
RAID-5 volume column size is limited by the size of the smallest available column, which is 5,088,447
blocks in these examples. Additionally, approximately one disk drives worth of space is lost to parity.

The vxassist maxsize command is not usually needed unless you have
especially limited disk drive space and need to maximize its use. It is a
good practice to leave a small amount of space for log placement.

5-10

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Volume Planning

Selecting Volume Types


The VxVM application supports several types of volumes. Each volume
type has a specific layout specifier. The following sections briefly describe
each volume type and its associated layout specification.

Concatenated
The advantages of concatenated volumes are efficient use of storage space
and easier hardware requirements.
Layout specification: concat

Striped
The advantages of striped volumes are improved read and write
performance.
Layout specification: stripe

Mirrored Concatenation
The advantage of a mirrored concatenation volume is improved
availability through data redundancy.
Layout specification: mirror-concat

Mirrored Stripe
The advantages of a mirrored stripe volume are improved availability and
increased read and write performance.
Layout specification: mirror-stripe

VERITAS Volume Manager Volume Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

5-11

Volume Planning

Concatenated Mirror (Layered Volume Type)


The advantages of concatenated mirror volumes are improved availability
and faster recovery time. This is because only a portion of a mirror must
be recovered.
Layout specification: concat-mirror

Striped Mirror (Layered Volume Type)


The advantages of striped mirror volumes are improved performance,
faster recovery times, and high disk drive failure tolerance.
Layout specification: stripe-mirror

RAID 5
The advantage of RAID-5 volumes are somewhat improved availability
with limited negative impact on disk space utilization.
Layout specification: raid5 (or raid5nolog)

5-12

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Creating Volumes Using the VEA GUI

Creating Volumes Using the VEA GUI


A New Volume Wizard is initiated in the VEA GUI using either of the
following methods:

The menu bars Actions menu New Volume item

The toolbars New Volume button

Disk Selection Method


The initial New Volume Wizard form prompts you to select the disk
selection method to be used. The default is to let VxVM decide which
disks to use. However, when creating complex volumes, it is a best
practice to enable manual disk selection, as shown in Figure 5-7.

Figure 5-7

New Volume Wizard Disk Selection Method

VERITAS Volume Manager Volume Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

5-13

Creating Volumes Using the VEA GUI

Using the Disk Selection Form


As shown in Figure 5-8, the New Volume Wizard disk selection form
assists you in locating and selecting suitable disk drives.

Figure 5-8

New Volume Wizard Disk Selection

The Mirror Across and Stripe Across check boxes let you choose how you
want stripes or mirrors distributed across your storage configuration. The
Mirror Across Tray applies to specific storage array models that have
separate disk drive trays in a single array. Unless you later specify a
striped or mirrored volume structure, these features do not perform any
useful function.
The Ordered check box is an advanced function that uses the specified
storage first to create concatenation, then form columns, and finally to
create mirrors. Ordered allocation is an advanced subject presented later
in this course.

5-14

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Creating Volumes Using the VEA GUI

Using the Volume Attributes Form


The New Volume Wizard volume attributes form, shown in Figure 5-9,
assists you in defining the name, size, and type of a volume structure.

Figure 5-9

New Volume Wizard Attributes

Consider the following points when configuring new volume attributes:

Name volumes according to their purpose.

Use Maxsize after you have configured everything else. The Maxsize
function calculates space based on the volume layout specification.
The space available varies widely depending on the target volume
configuration.

Use the Columns and Stripe unit size functions when you are
creating volumes that have columns such as striped or RAID-5
volumes.

Note that the Mirrored check box is automatically enabled when


Concatenated Mirrored or Striped Mirrored structures are selected.
Manually selecting Concatenated, and then selecting Mirrored,
creates a different structure than selecting Concatenated Mirrored.

VERITAS Volume Manager Volume Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

5-15

Creating Volumes Using the VEA GUI

Using the Create File System Form


As shown in Figure 5-10, the New Volume Wizard File System form
prompts for file system type (UNIX file system [UFS] or VxFS) and mount
options. VEA automatically creates the mount point, modifies the
/etc/vfstab file, and initializes and mounts the new file system.
The New File System Details button enables newfs and mkfs option entry.
The Mount File System Details button allows volume ownership and
protection entry.

Figure 5-10 New Volume Wizard File System Form

Note The VEA server logs the commands that perform all functions in
the /var/vx/isis/command.log file. The log file is a useful learning tool.

5-16

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Creating Volumes Using the vxassist Command

Creating Volumes Using the vxassist Command


The vxassist program performs a wide range of volume-related tasks
without the complexity of lower-level programs, such as vxvol. The
vxassist program also protects against many mistakes that can be made
when using lower-level command-line programs.

The vxassist Command


The vxassist command has many options. Most of the options use
default values if values are not explicitly entered.
The most basic form of the vxassist command, which creates a volume,
is:
# vxassist make vol02 50m
The problem with this format is that is assumes the following:

A default disk group, if one is configured

The default volume type is a basic concatenation

Any disk drives that have available space might be used

If you do not specify options, the vxassist command probably cannot


give you what you need. It can also create a poorly performing volume or
contribute to poor performance of other volumes.

Specifying Volume Size


Volume sizes can be specified using the following units:

b (blocks)

g (gigabytes)

k (kilobytes)

m (megabytes)

s (sectors)

VERITAS Volume Manager Volume Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

5-17

Creating Volumes Using the vxassist Command

Using vxassist Command Options


If you furnish even a few options with the vxassist command, the
outcome is more clearly defined. A typical command using limited
options is:
# vxassist -g dg2 make newvol 2g \
layout=raid5,nolog disk01 disk02 disk03
This form of the vxassist command is more explicit and guarantees that
the following are true:

The disk group that is used is dg2.

The name of the volume is newvol.

The size of the volume is 2 Gbytes.

This is a RAID-5 volume without a log and with three columns.

All disk space comes from disk01, disk02, and disk03.

Although there are many vxassist command options, only a few are
commonly used. Some of them require careful study. Always read the
manual (man) pages and related documentation before attempting to use
most of these options.
The vxintro and vxassist man pages contain useful information that is
difficult to find elsewhere.

Default Option Values


Pay close attention to the default values of command options. When
specifying volume layouts, ensure that you understand the default value
or configuration for a particular volume type. The following examples
show some layout specifications for RAID-5 volume types:
layout=raid5 (creates a RAID-5 log by default)
layout=raid5log
layout=raid5,nolog
layout=noraid5log
The first two variations are equivalent and create the same volume
structure. The last two are also equivalent.

5-18

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Creating Volumes Using the vxassist Command

Examples of vxassist Command Options


The following examples show each type of supported volume structure.
The layered volumes require a larger minimum number of disk drives to
implement.
# vxassist -g dg3 make newvol 20m \
layout=concat disk01
# vxassist -g dg3 make newvol 20m \
layout=stripe disk01 disk02
# vxassist -g dg3 make newvol 20m \
layout=mirror-concat disk01 disk02
# vxassist -g dg3 make newvol 20m \
layout=mirror-strip disk01 disk02 disk03 disk04
# vxassist -g dg3 make newvol 20m \
layout=concat-mirror disk01 disk02 disk03 disk04
# vxassist -g dg3 make newvol 20m \
layout=stripe-mirror disk01 disk02 disk03 disk04
# vxassist -g dg3 make newvol 20m \
layout=raid5,nolog disk01 disk02 disk03
The vxassist command can frequently determine the best way to use the
specified disk drives (media names) in a volume structure.

VERITAS Volume Manager Volume Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

5-19

Modifying Volume Access Attributes

Modifying Volume Access Attributes


When new volumes are created in a disk group, they are given a set of
default access attributes that include:

Owner

Group

Mode

The owner, group, and mode are usually those of the root user. For some
volumes, especially raw volumes that are used by a database, the volume
ownership must be modified.
Caution Do not use the chown, chgrp, or chmod command to set raw
volume attributes. This is because the attributes revert to their original
values after each system reboot. Change raw volume attributes using
VxVM commands.

Verifying Volume Ownership


The ownership and permissions of raw volumes can be checked like
ordinary system files. You use the ls command to examine the raw
volume files in the /dev/vx/rdsk/dg_name directories.
# ls -l /dev/vx/rdsk/newDG/testvol
crw------1 root root 199,73000 Feb3 20:37 /dev/vx/rdsk/newDG/testvol

Modifying Volume Ownership and Permissions


To run applications, such as ORACLE 9i RAC, it might be necessary to
change read/write permissions and ownership of the raw volumes. You
use the vxedit command to change the raw volumes permissions or
ownership.
# vxedit -g tpcs set user=oracle group=dba mode=660 acct06

Note You can also use the VEA GUI New Volume Wizard to set volume
ownership and permissions during initial volume creation. Or, later, you
can use the Actions menu File System entry to create a file system and
configure mount information for an existing volume.

5-20

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Adding a UFS File System to Existing Volumes

Adding a UFS File System to Existing Volumes


Adding a UFS file system to a volume is easy to do. You can fill out a form
using the VEA GUI, or you can create a file system from the command
line using standard Solaris OS commands. Both methods are described in
this section.
Note VxVM also supports the VxFS file system type, but the VxFS
features are licensed separately.

Using the VEA GUI to Add a File System


To add a new file system to an existing volume using the VEA GUI,
highlight the target volume in the grid area and select File System New
File System from the Actions menu. The resulting Create File System
Form is identical to the New Volume Wizard Create File System Form.

New File System Details Form


Using the New File System Details Form, shown in Figure 5-11, you can
make fundamental file system changes.

Figure 5-11 New File System Details Form


You enter a comma-separated list of mkfs file system options in the Extra
options text field. Consult the mkfs and mkfs_ufs man pages for more
detailed option information.

VERITAS Volume Manager Volume Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

5-21

Adding a UFS File System to Existing Volumes

Mount Details Form


You can enter valid file system mount options, as shown in Figure 5-12.

Figure 5-12 Mount Details Form

5-22

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Adding a UFS File System to Existing Volumes

Adding a File System From the Command Line


When a new file system is initialized from the command line, you can
adjust certain file system parameters to make more efficient use of
available space. They are:

Minimum file system free space

Number of bytes per inode

File System Free Space


By default, the newfs utility calculates the minimum free space based
on partition size (64 Mbytes partition size 100), rounded down to the
nearest integer. The default value is limited to between 1 percent and
10 percent.
When the file system is full, the free space can only be accessed by user
root. It can act as an emergency overflow.
# newfs -m 10 /dev/vx/rdsk/newDG/vol_01
In very large file systems, you can safely set the minimum free space to a
smaller percentage.

Number of Bytes per Inode


The newfs utility calculates the number of bytes per inode based on file
system size. By default, the newfs utility calculates the number of inodes
as follows:

2048 bytes per inode for 0-1 Gbyte file system size

4096 bytes per inode for 1-2 Gbytes file system size

6144 bytes per inode for 2-3 Gbytes file system size

8192 bytes per inode for file system larger than 3 Gbytes

If you intend to create a large file system that will contain a small number
of very large files, you might be able to decrease the total number of
inodes, for example:
# newfs -i 10240 /dev/vx/rdsk/newDG/vol01

VERITAS Volume Manager Volume Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

5-23

Adding a UFS File System to Existing Volumes

Enabling the Solaris OS UFS Logging Feature


UFS logging is a standard feature of the Solaris 8 OS.
If the logging option is specified for a file system, then logging is enabled
for the duration of the mounted file system. Logging is the process of
storing transactions (changes that make up a complete UFS operation) in a
log before the transactions are applied to the file system. After a
transaction is stored, the transaction can be applied to the file system. This
process prevents file systems from becoming inconsistent, therefore
eliminating the need to run the fsck command. Because the fsck
command can be bypassed, logging reduces the time required to reboot a
system if it crashes or after an unclean halt. The default behavior is no
logging.
The log is allocated from free blocks on the file system, and it is sized at
approximately 1 Mbyte per 1 Gbyte of file system, up to a maximum of
64 Mbytes. Logging can be enabled on any UFS, including root (/). The
log created by UFS logging is continually flushed as it fills up. The log is
totally flushed when the file system is unmounted or when the
lockfs -f command is used.

Example of File System Commands


The following commands summarize the types of operations that must be
performed to manually create a UFS logging file system on an existing
VxVM volume.
# newfs /dev/vx/rdsk/newDG/testvol
# mkdir /Test
# vi /etc/vfstab
/dev/vx/dsk/newDG/testvol
ufs 1 yes logging

/dev/vx/rdsk/newDG/testvol /Test

# mount /Test

Note The default file system behavior is nologging.

5-24

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Administering Volume Logs

Administering Volume Logs


Both mirrored volumes and RAID-5 volumes can have logs. However, the
logs for RAID 5 perform an entirely different function than the mirrored
volume log, which is called a DRL.

Using DRLs
A DRL is a VxVM log file that tracks data changes made to mirrored
volumes. The DRL speeds recovery time when a failed mirror must be
synchronized with a surviving mirror.
A DRL is a small, special-purpose plex attached to a mirrored volume
which has the following features:

It is a log that keeps track of the regions within volumes that have
changed as a result of write operations to a plex. It does this by
maintaining a bitmap and storing this information in a log subdisk.

After a system failure, only the regions marked as changed (dirty) in


the DRL are recovered.

The following example shows a mirrored volume with a DRL. Notice that
the log subdisk does not reside on either of the mirror disk drives.
# vxprint -g newDG mirvol
TY NAME
ASSOC
v mirvol
fsgen
pl mirvol-01
mirvol
sd ndg-01-01
mirvol-01
pl mirvol-02
mirvol
sd ndg-03-01
mirvol-02
pl mirvol-03
mirvol
sd ndg-02-03
mirvol-03

KSTATE
ENABLED
ENABLED
ENABLED
ENABLED
ENABLED
ENABLED
ENABLED

LENGTH
4096
7182
7182
7182
7182
LOGONLY
33

PLOFFS
0
0
LOG

STATE
ACTIVE
ACTIVE
ACTIVE
ACTIVE
-

DRLs are very small compared to the volume data storage.

VERITAS Volume Manager Volume Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

5-25

Administering Volume Logs

Using RAID-5 Logs


RAID-5 logs help prevent data corruption in case a system crashes during
a write operation. Without logging, if a system fails during a write
operation, there is no way to tell if the data and parity were both written
to the disk drives. This could result in corrupted data.
When RAID-5 logging is used, a copy of the data and parity are written to
the RAID-5 log before being written to the disk drives.
RAID-5 logging is optional, but RAID-5 logs are created by default. You
should always run a system with RAID-5 logs to ensure data integrity.
The following example shows a RAID-5 volume with a log. Notice that
the log subdisk does not reside on either of the stripe disk drives.
# vxprint -g newDG raidvol
TY NAME
ASSOC
v raidvol
raid5
pl raidvol-01
raidvol
sd ndg-02-02
raidvol-01
sd ndg-01-03
raidvol-01
sd ndg-03-03
raidvol-01
pl raidvol-02
raidvol
sd ndg-04-02
raidvol-02

KSTATE
ENABLED
ENABLED
ENABLED
ENABLED
ENABLED
ENABLED
ENABLED

LENGTH
4096
7168
3591
3591
3591
3591
3591

PLOFFS
0
0
0
0

STATE
ACTIVE
ACTIVE
LOG
-

The size of RAID-5 logs is automatically set by VxVM. It is dependent on


the stripe width of the volume. The larger the stripe width (not volume),
the larger the RAID-5 log. The RAID-5 log is intended to hold several
full-stripe writes simultaneously.
The default log size for a RAID-5 volume is four times the full stripe
width (the stripe unit size the number of stripe columns).

5-26

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Administering Volume Logs

Planning Log Placement


Although logs can be very beneficial to volume recovery after a system
crash, if they are not properly managed they can create I/O bottlenecks
that negatively impact system performance.
You should plan for both RAID-5 logs and DRLs in advance.
You should take special care with RAID-5 log placement because the data
written to all RAID-5 stripe units is also written to the log.
As shown in Figure 5-13, leaving a small amount of free space at the end
of all disk drives ensures that you always have alternate locations for log
placement or relocation.
Volume 01

Vol02_log

Log Space

Log Space

Log Space

Volume 02

Log Space

Log Space

Log Space

Vol01_log

Figure 5-13 Log Space Allocation


If possible, a log should not reside on the same disk drive as its related
volume.
Note Log sizes are calculated automatically by the VxVM software.

VERITAS Volume Manager Volume Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

5-27

Administering Volume Logs

Adding a Volume Log From the VEA GUI


Usually, DRL and RAID-5 logs are added when a new volume is created.
However, in some cases it might be preferable to run performance tests
before deciding on the placement of logs.
To add a DRL or RAID-5 log to an existing volume, click the volume in
the grid area. Next, click the Actions menu and select Log Add from its
pop-up menu.
As shown in Figure 5-14, you can either let VxVM automatically assign a
suitable log disk or you can enable manual disk assignment.

Figure 5-14 VEA Add Log Form

5-28

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Administering Volume Logs

Adding a Volume Log From the Command Line


You can use the vxassist command to add or remove a log from a
volume. Both DRL and RAID-5 logs can be added this way. By default,
when adding a log, the vxassist command attempts to determine the
correct type of log to add and computes an appropriate size for the log.
Adding a DRL log to a mirrored volume creates a single log subdisk and
a new plex to contain that subdisk. The new plex is then attached to the
volume.

Adding a DRL
To prevent I/O bottlenecks, a DRL should not reside on a disk drive used
by its related volume. It is best to specify the disk drive (media name)
where the DRL should be placed. If the DRL location is not specified, the
vxassist command assesses the available disk space and decides where
to place the log. The following example shows the addition of a DRL to a
mirrored volume.
# vxassist addlog mirvol ndg-02
# vxprint -g newDG mirvol
TY NAME
ASSOC
KSTATE
v mirvol
fsgen
ENABLED
pl mirvol-01
mirvol
ENABLED
sd ndg-01-01
mirvol-01
ENABLED
pl mirvol-02
mirvol
ENABLED
sd ndg-03-01
mirvol-02
ENABLED
pl mirvol-03
mirvol
ENABLED
sd ndg-02-03
mirvol-03
ENABLED

LENGTH
4096
7182
7182
7182
7182
LOGONLY
33

PLOFFS
0
0
LOG

STATE
ACTIVE
ACTIVE
ACTIVE
ACTIVE
-

Adding a RAID-5 Log


The process for adding a RAID-5 log is the same as for a adding a DRL.
The following example shows the addition of a log to a RAID-5 volume.
# vxassist addlog raidvol ndg-04
# vxprint -g newDG raidvol
TY NAME
ASSOC
KSTATE
v raidvol
raid5
ENABLED
pl raidvol-01
raidvol
ENABLED
sd ndg-02-02
raidvol-01
ENABLED
sd ndg-01-03
raidvol-01
ENABLED
sd ndg-03-03
raidvol-01
ENABLED
pl raidvol-02
raidvol
ENABLED
sd ndg-04-02
raidvol-02
ENABLED

LENGTH
4096
7168
3591
3591
3591
3591
3591

PLOFFS
0
0
0
0

STATE
ACTIVE
ACTIVE
LOG
-

VERITAS Volume Manager Volume Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

5-29

Administering Volume Logs

Removing Volume Logs Using the VEA GUI


To remove a volume log using the VEA GUI, click the volume in the grid
area. Next, click the Actions menu and select Log Remove from its pop-up
menu. Volumes can have more than one log. The Remove Log form,
shown in Figure 5-15,allows you to remove logs either by name or by disk
drive.

Figure 5-15 VEA Remove Log Form


The Quantity/Disk removal method provides two options. You can enter
the quantity of logs to be removed, or you can specify one or more disk
drives on which to preserve log copies.

5-30

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Administering Volume Logs

Removing Volume Logs From the Command Line


The vxassist command format to remove volume logs is the same
regardless of the volume type. The following example shows the removal
of logs from both a mirrored volume and a RAID-5 volume.
# vxassist remove log mirvol
# vxassist remove log raidvol

VERITAS Volume Manager Volume Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

5-31

Using the VEA GUI to Analyze Volume Structures

Using the VEA GUI to Analyze Volume Structures


You use the VEA GUI to visually examine volume structure, disk
allocation, and basic performance.

Displaying Volume Layout Details


To display volume layout details, click on a volume in the grid area and
select Layout View from the pop-up menu. As shown in Figure 5-16, use
the right mouse button in the Volume Layout window to select a volume
component and examine its properties.

Figure 5-16 Volume Layout Window

5-32

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Using the VEA GUI to Analyze Volume Structures

Viewing Disk Volume Mapping and Performance


To view volume-to-disk mapping and performance, click on a disk group
in the VEA GUI and select Volume Disk from the pop-up menu.
The Volume Disk Map, shown in Figure 5-17, displays a map of each
volume and its associated disk drives.
You can also enable crude performance monitoring by clicking on the
View menu and selecting Collect Statistics from the pop-up menu. The
disk icons change color to indicate basic levels of I/O activity.

Figure 5-17 Volume Disk Map

VERITAS Volume Manager Volume Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

5-33

Exercise: Creating Volumes and File Systems

Exercise: Creating Volumes and File Systems


In this exercise, you complete the following tasks:

Review key lecture points

Create a volume

Add a volume mirror

Add a file system to a volume

Add a dirty-region log

Resize a volume and file system

Create a RAID-5 volume

Analyze volumes using the VEA GUI

Verify ending lab status

Preparation
The purpose of this lab is to have you create and destroy VxVM objects
using both the VEA GUI and command-line programs. Each method has
advantages.
Ask your instructor for the following information:

A set of unique volume names for your workgroup:


Concat/mirror volume name: __________________________________
RAID-5 volume name: ________________________________________

A unique mount point name for your workgroup:


Mount point name: ___________________________________________

Your workgroups assigned disk drives should be configured as follows:

You have two disk groups name dgX and dgY.

Each disk group contains three disk drives.

None of your assigned disk drives are designated as hot spares.

Note Substitute your workgroup codes for the X and Y in dgX and dgY.

5-34

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Creating Volumes and File Systems

Task 1 Reviewing Key Lecture Points


Answer the following questions about volume planning, volume access,
and volume logging:
1.

Why should you avoid automatic volume naming?


a.

Automatic names are limited to 32 variations.

b.

Automatic names cannot be used by some databases.

c.

Automatic names make administration difficult.

d.

Automatic names can be very long.

The answer is c.

2.

Which of the following commands are useful for space allocation


research?
a.

vxassist maxsize

b.

vxdisksetup -i

c.

vxdisk list

d.

vxdg free

e.

vxdiskadm

The answers are a and d.

3.

Which of the following volume types has the fastest recovery time?
a.

Striped volumes

b.

Layered volumes

c.

Mirrored volumes

The answer is b.

4.

Which of the following volume types has the highest disk drive
failure tolerance?
a.

Concatenated mirror

b.

Mirrored concatenation

c.

RAID 5

d.

Striped mirror

The answer is d.

VERITAS Volume Manager Volume Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

5-35

Exercise: Creating Volumes and File Systems


5.

Which of the following commands modifies volume access


attributes?
a.

vxdisk define

b.

chmod -R

c.

vxedit set

d.

ln -s

The answer is c.

6.

What is the purpose of dirty region logging?


a.

To increase volume write performance

b.

To prevent file system corruption

c.

To speed up volume mirror resynchronization

d.

To control file system access

e.

To prevent internal volume corruption

The answer is c.

7.

What is the purpose of RAID-5 logging?


a.

To increase volume write performance

b.

To prevent file system corruption

c.

To speed up volume mirror resynchronization

d.

To control file system access

e.

To prevent internal volume corruption

The answer is e.

5-36

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Creating Volumes and File Systems

Task 2 Creating a Volume


Perform both variations of this task in the order they occur.

Using the VEA GUI to Create a Volume


To create a volume using the VEA GUI, complete the following steps:
1.

Display your first disk group (dgX) in the grid area.

2.

In the toolbar, select New Volume.

3.

Configure the New Volume Wizard as follows:

Manually select disks to use for the volume.

Select only one of the disks for use in the new volume.

Enter your assigned concatenated volume name.

Select the Concatenated layout.

Select Maxsize.

After the Maxsize calculation has completed, type 200 in the


Size window and select MB from its pull-down menu.

Select No file system.

Review the final configuration parameters and click Finish.

Note The Maxsize feature can be useful when you are trying to
maximize the size of a new volume and when you have limited disk drive
space.
4.

Check the status of the new volume by using the vxprint command.

5.

Verify that your new volume has a single plex with one subdisk and
that the volume and plex are ENABLED and ACTIVE.

6.

Display the new volume in the grid area.

7.

Click the new volume in the grid area and select Properties from its
pop-up menu.

8.

Examine the volumes properties and click Cancel when you are
finished.

VERITAS Volume Manager Volume Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

5-37

Exercise: Creating Volumes and File Systems

Using the Command Line to Create a Volume


To create a volume using the command line, use the man pages, as
necessary, to complete the following steps:
1.

Open a window and use the rlogin or telnet command to log into
the VxVM server as user root.

2.

Stop the volume you created in the previous procedure.


# vxvol -g disk_group stop volume_name

3.

Recursively remove the volume.


# vxedit -g disk_group -rf rm volume_name
What is the purpose of the vxedit -f option?
_____________________________________________________________

4.

5.

Re-create the 200-Mbyte concatenated volume again by using the


vxassist command. You must specify the following items:

The disk group the volume should be in (-g disk_group)

The make option

The name and size of the volume (volname 200m)

The volume layout (layout=concat)

The disk drive (media name) you want to use

Record the command you used to create the volume.


_____________________________________________________________

Note It is not necessary to use the layout= option for a concatenated


volume. The default layout is a concatenation. Any other layout requires a
specific layout option. For example, layout=mirror or layout=raid5.
6.

Use the vxprint command to verify the volume status is showing


ENABLED and ACTIVE.

Note If there is any problem with your new volume, consult with your
instructor.

5-38

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Creating Volumes and File Systems

Task 3 Adding a Volume Mirror


Perform both variations of this task in the order that they occur.

Using the VEA GUI to Add a Mirror


To add a mirror to an existing volume, complete the following steps:
1.

Display your new volume in the grid area and click the volume.

2.

Click the Actions menu and select Mirror Add from its pop-up
menu, as shown in Figure 5-18.

Figure 5-18 Add Mirror Menu

VERITAS Volume Manager Volume Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

5-39

Exercise: Creating Volumes and File Systems


3.

Leave the first Add Mirror form configured with its default values
which should include the following:

Number of mirrors to add: 1

Layout=Concatenated

Let Volume Manager decide which disks to use

4.

On the VxVM server, check the status of the mirror


resynchronization by using the vxtask list command.

5.

On the VxVM server, verify the state of your new mirror by using
the vxprint command.
You should now see two plexes in your volume. Large mirrors take a
while to synchronize.
Until the resynchronization is complete, the related plex is in a
TEMPRMSD state. Consult the vxinfo man page for volume state
definitions.

Note Remember that you can review the command-line operations in


the /var/vx/isis/command.log file on the VxVM server.

Using the Command Line to Add a Volume Mirror


To remove a mirror from a volume, and then create a new mirror,
complete the following steps:
1.

Ensure that both plexes (mirrors) in the volume you added are fully
synchronized and show a status of ENABLED and ACTIVE.

2.

Use the vxassist command to remove one of the mirrors from your
volume.
For example:
# vxassist -g dgX remove mirror vol_01

3.

Use the vxprint command to verify that your volume now has a
single plex and subdisk.

4.

Use the vxassist command to re-create the mirrored volume.

5.

Use the alloc= parameter to specify the disk media name on which
the new mirror is to be created.
For example:
# vxassist -g dgX mirror vol_01 alloc=dgX03

5-40

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Creating Volumes and File Systems


6.

After the vxassist returns, use the vxprint command to verify the
volume has two plexes and its status is ENABLED and ACTIVE.
Consult with your instructor if you are having problems.

Note You can also move a volume mirror to a different disk drive if it is
poorly placed and is causing a performance problem.

Task 4 Adding a File System to a Volume


Perform both variations of this task in the order they occur.

Using the VEA GUI to Add a File System


To add a file system using the VEA GUI, complete the following steps:
1.

Click your mirrored volume in the grid area, and select File System
New File System from the pop-up menu.

2.

Configure the New File System form as follows:

3.

Ensure that the File system type is ufs.

Leave the Allocation at its default value (1024).

Enter your assigned mount name.

Select Create mount point.

Deselect Read only and Honor setuid.

Select Add to file system table and mount at boot.

Set the fsck pass number to 2.

Examine the New file System Details menu

Examine the Mount File System Details menu.

Click OK.

On the VxVM server, verify that the following are true:

The mount point is present in the root directory.

Your file system is mounted.

The mount entry is in the /etc/vfstab file.

The df -kl output seems appropriate for the volume size.

VERITAS Volume Manager Volume Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

5-41

Exercise: Creating Volumes and File Systems


4.

Create some test data in the volumes file system.


# mkfile 2m /mountpoint/myfile

Using the Command Line to Add a File System


Review the command-line operations performed by the VEA software in
the previous section. Complete the following step:
Review the VEA command-line operations recorded in the log file on the
VxVM server.
# tail -45 /var/vx/isis/command.log
The most recent commands are appended to the end of the file. Not all the
details are logged in the command file, such as the edits to the
/etc/vfstab file.
The following is a summary of using the command line to add a file
system to an existing volume.
# mkfs -F ufs /dev/vx/rdsk/dgX/xvol-01 409600
# mkdir /Junk
# vi /etc/vfstab
/dev/vx/dsk/dgX/xvol-01 /dev/vx/rdsk/dgX/xvol-01 /Junk ufs 1 yes logging

# mount -F ufs -o logging /dev/vx/dsk/dgX/xvol-01 /Junk

Note The file system vfstab and mount options enable the UFS logging
feature. UFS logging is not necessary, but it offers additional file system
protection and is part of the Solaris OS.

5-42

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Creating Volumes and File Systems

Task 5 Adding a DRL


Perform both variations of this task in the order they occur.

Using the VEA GUI to Add a DRL


To add a DRL using the VEA GUI, complete the following steps:
1.

Verify there is a disk drive available for the DRL within the same
disk group that does not contain either plex of the mirrored volume.

2.

Display your volume in the grid area and click it with the third
mouse button.

3.

Select Log Add in the volume pop-up menu.

4.

In the Add Log window, complete the following steps:

5.

a.

Click Manual disk assignment.

b.

Select a disk drive that is not part of the mirrored volume.

c.

Click OK.

Return to the command line on the VxVM server and use the
vxprint command to verify the following:

The mirrored volume now has a log plex

The log is not on the same disk drive as either of the volume
mirrors

Note Look at the subdisk entries to determine log placement.


6.

In the VEA GUI, complete the following steps:


a.

Click the volume in the grid area.

b.

Select Properties from the pop-up menu.

c.

Examine both the General and File System tabs.

VERITAS Volume Manager Volume Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

5-43

Exercise: Creating Volumes and File Systems

Using the Command Line to Add a DRL


The following example shows the command sequence to remove and add
a DRL to a volume. Practice removing and adding a DRL from your
volume using the command line.
The disk media name you specify should be on a different disk drive than
the disk drives used by the volume mirrors.
# vxassist remove log volume_name
# vxassist addlog volume_name media_name

5-44

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Creating Volumes and File Systems

Task 6 Resizing a Volume and File System


If a volume has a file system, you can increase the size of both the volume
and the file system by using either the vxresize command or the VEA
GUI.
Note There are other methods of increasing a volumes size and a file
systems size. However, the VEA GUI and vxresize command reliably
increase the size of both the volume and its related file system at the same
time.
The volume to be resized should be of the type fsgen. The vxprint
command displays the volume type.
Perform both variations of this task in the order they occur.

Using the Command Line to Resize File Systems


To resize file systems using the command line, complete the following
steps:
1.

Add 2 Mbytes to the size of your mirrored volume and file system
by using the following command:
# vxresize -F ufs -g disk_group volume_name +2m

Note You can also express the +2m as a new volume length without the
plus sign. There are also -s and -x options that ensure the requested size
value is appropriate. You can also specify disk media names (for example,
disk01, disk02) that you want to be used for the new space.
2.

Examine the new volume and file system to ensure that the changes
have taken place.
Large changes can take a long time.

Note You cannot shrink a volume with a file system unless the file
system is of VxFS type. Read the vxresize man page for a complete
description of restrictions.

VERITAS Volume Manager Volume Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

5-45

Exercise: Creating Volumes and File Systems

Using the VEA GUI to Resize File Systems


To resize file systems using the VEA GUI, complete the following steps:
1.

2.

In the grid area, complete the following steps:


a.

Display your volume.

b.

Click your volume with the third mouse button.

c.

Select Resize Volume in the pop-up menu.

Configure the Resize Volume form, shown in Figure 5-19, as follows:

Enter 2 in the Add By window and select MB from its pulldown menu.

Let VxVM decide which disks to use for the additional space.

Click OK.

Figure 5-19 Resize File System Form


3.

5-46

After the task has completed, verify the results.

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Creating Volumes and File Systems

Task 7 Creating a RAID-5 Volume


In this task you create a three-column RAID-5 volume with a separate log
disk (four disk drives total) in your second disk group.
Complete the following steps:
1.

2.

On the VxVM server, complete the following steps:


a.

Unmount the mirrored volume file system.

b.

Stop the volume.

c.

Remove its mount entry from the /etc/vfstab file.

Destroy the mirrored volume disk group (dgX).


# vxdg destroy disk_group

3.

Click the remaining disk group with the third mouse button and
select Add Disk to Disk Group from its pop-up menu.

4.

Using the Add Disk Wizard, add the three disk drives from the
destroyed disk group to the remaining disk group.

5.

On the VxVM server, calculate the available disk space using four of
the disk drives in a RAID-5 volume with a log.
For example:

# vxassist -g dgB maxsize layout=raid5log dgB01 dgB02 dgB03 dgB04


Maximum volume size: 35348480 (17260Mb)
Caution Do not create a maximum-size volume.
6.

Display the disk drives in your disk group in the grid area.

7.

Select four of the disk drives by simultaneously pressing the Control


key while using the left mouse button.

8.

Select the New Volume button in the toolbar.

9.

Configure the New Volume form as follows:

Enter your assigned RAID-5 volume name.

Enter 200 in the Size field.

Select MB from the Size pull-down menu.

Choose RAID 5 in the Layout area. The Number of Columns


field should automatically be set to 3 with logging is enabled.

Leave the default Stripe Unit Size at 32.

Click Next.

VERITAS Volume Manager Volume Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

5-47

Exercise: Creating Volumes and File Systems

Click Create a File System, and complete the following steps:

Select a FS Type of ufs.

Enter your assigned mount point.

Click Add to File System Table, Mount at Boot and set the fsck
pass to 2.

10. Click Next in the Create File System form.


11. Click Finish in the Summary form after you verify that the
configuration is correct.
12. On the VxVM server, check the status of the new RAID-5 volume by
using the vxprint command.
# vxprint -g dgB raidvol
TY NAME
ASSOC
v raidvol
raid5
pl raidvol-01
raidvol
sd dgB01-01
raidvol-01
sd dgB04-01
raidvol-01
sd dgB05-01
raidvol-01
pl raidvol-02
raidvol
sd dgB06-01
raidvol-02

KSTATE
ENABLED
ENABLED
ENABLED
ENABLED
ENABLED
ENABLED
ENABLED

LENGTH
409600
416512
208278
208278
208278
3591
3591

PLOFFS
0
0
0
0

STATE
ACTIVE
ACTIVE
LOG
-

Note You can add the log later to better control its placement. The disk
drives are not necessarily used in the order you selected them.
13. On the VxVM server, examine the volume creation commands.
# tail -25 /var/vx/isis/command.log
14. In the VEA GUI, click the volume with the third mouse button and
select Properties from its pop-up menu.
15. Examine both the General and File System tabs.
16. Cancel the Volume Properties window.
17. Edit the /etc/vfstab file, and comment out or delete the obsolete
mount entry from the earlier mirrored volume file system mount.
You must remove or comment out obsolete mount entries because
they cause errors at boot time.

5-48

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Creating Volumes and File Systems

Task 8 Analyzing Volumes Using the VEA GUI


The following sections detail different VEA GUI features that are used to
analyze the structure and performance of VxVM volumes.

Displaying Volume Layout Details


To display volume layout details, complete the following steps:
1.

2.

Display your volumes in the grid area and complete the following
steps:
a.

Click your RAID-5 volume with the third mouse button.

b.

Select Layout View from the pop-up menu.

As shown in Figure 5-20, use the third mouse button to highlight a


volume component and examine its properties.

Figure 5-20 Volume Layout Window

VERITAS Volume Manager Volume Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

5-49

Exercise: Creating Volumes and File Systems


3.

Examine the pop-up menus for each of the different components.

4.

Close the Volume Layout window.

Viewing Disk Volume Mapping and Performance


To view volume-to-disk mapping and performance, complete the
following steps:
1.

In the grid area, click your disk group with the third mouse button,
and select Volume/Disk from the pop-up menu.
The Volume/Disk map, shown in Figure 5-21, assists in volume
planning and disk utilization assesment.

Figure 5-21 Volume/Disk Map


2.

5-50

Click the View menu with the third mouse button and select Data
Gathering Options from the pull-down menu.

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Creating Volumes and File Systems


3.

Perform the following steps:


a.

Modify the Refresh Interval to 5 seconds and select Write


Requests from the pull-down menu, as shown in Figure 5-22.

b.

Click OK.

Figure 5-22 Data Gathering Options

VERITAS Volume Manager Volume Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

5-51

Exercise: Creating Volumes and File Systems


4.

Enable performance monitoring by performing the following steps:


a.

Click the View menu with the third mouse button.

b.

Select Collect Statistics from the pop-up menu, as shown in


Figure 5-23.

Figure 5-23 Enabling Performance Monitoring


When performance monitoring is enabled, the relative I/O
activity of each volume disk drive is indicated by color changes
in the small disk icons shown in Figure 5-23.
Red indicates the highest level of I/O activity or slowest
performance. This information can be used to identify basic
performance bottlenecks in disk group volumes.

5-52

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Creating Volumes and File Systems


5.

Use the mkfile command to create some test data into your file
system volume.

6.

Observe the activity levels in the Disk/Volume performance display


For example: mkfile 20m /Test/file1

Figure 5-24 Data Activity Indicators

VERITAS Volume Manager Volume Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

5-53

Exercise: Creating Volumes and File Systems

Task 9 Verifying Ending Lab Status


Type the vxprint command and verify your current VxVM configuration
has the following features:

5-54

One disk group containing six disks

One RAID-5 volume with a log, mounted

Two free disk drives left in the disk group

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise Summary

Exercise Summary

Discussion Take a few minutes to discuss what experiences, issues, or


discoveries you had during the lab exercises.

Manage the discussion here based on the time allowed for this module, which was given in the About This
Course module. If you find you do not have time to spend on discussion, then just highlight the key concepts
students should have learned from the lab exercise.

Experiences

Ask students what their overall experiences with this exercise have been. You might want to go over any
trouble spots or especially confusing areas at this time.

Interpretations

Ask students to interpret what they observed during any aspects of this exercise.

Conclusions

Have students articulate any conclusions they reached as a result of this exercise experience.

Applications

Explore with students how they might apply what they learned in this exercise to situations at their workplace.

VERITAS Volume Manager Volume Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

5-55

Module 6

VERITAS Volume Manager Advanced


Operations
Objectives
Upon completion of this module, you should be able to:

Encapsulate and mirror the system boot disk

Configure a best practice boot disk

Administer hot spares and hot relocation

Evacuate all subdisks from a disk drive

Move a disk drive without preserving data

Move a populated disk drive to a new disk group

Backup and restore a disk group configuration

Describe how to import a disk group after a system crash

Perform a volume snapshot backup

Perform an online volume relayout

Create VxVM layered volumes

Perform basic Intelligent Storage Provisioning administration

Replace a failed disk drive

6-1
Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Boot Disk Encapsulation and Mirroring

Boot Disk Encapsulation and Mirroring


When you install the VxVM software on a system, you can place your
system boot disk under VxVM control. Later, you can mirror the boot disk
to increase availability. You should optimize your boot disk configuration
before starting the encapsulation process.

Optimizing the Boot Disk Hardware Configuration


Although there are many possible boot disk configuration variations, this
section focuses on the preferred boot disk hardware configuration, which
is shown in Figure 6-1.

SCSI
SCSI

SOC

rootdg
disk group

c0
c1

c2

rootvol-01

rootvol-02

newdg
disk group
Storage Array

Figure 6-1

Boot Disk Hardware Configuration

The ideal boot disk hardware configuration has the following features:

6-2

The boot disk and mirror are on separate interfaces.

The boot disk and mirror are not in a storage array.

Only the boot disk and mirror are in the rootdg disk group.

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Boot Disk Encapsulation and Mirroring

Boot Disk Encapsulation Prerequisites


For the boot disk encapsulation process to succeed, the following
prerequisites must be met:

The boot disk must have at least two unused slices.

The boot disk must not have any slices in use other than slice 2 and
the following:

root

swap

var

opt

usr

There must be at least 2048 sectors at the beginning or end of the boot
disk that are not assigned to any partition. These sectors are needed for
the private region. If necessary, VxVM takes the space from the end of the
swap partition, but this can create a difficult-to-manage boot disk
configuration.
The following is an ideal boot disk partition map, before encapsulation,
that has five unconfigured partitions and five cylinders of unassigned
disk space (3875-3879) at the end of the disk.
# format -d c0t0d0
...
...
Part
Tag
Flag
0
root
wm
1
swap
wu
2
backup
wm
3 unassigned
wm
4 unassigned
wm
5 unassigned
wm
6 unassigned
wm
7 unassigned
wm

Cylinders
0 - 3399
3400 - 3874
0 - 3879
0
0
0
0
0

Size
3.50GB
500.98MB
4.00GB
0
0
0
0
0

Blocks
(3400/0/0) 7344000
(475/0/0) 1026000
(3880/0/0) 8380800
(0/0/0)
0
(0/0/0)
0
(0/0/0)
0
(0/0/0)
0
(0/0/0)
0

Note If your boot disk does not have any free cylinders, you boot the
system from CD-ROM in single-user mode. You use the format utility to
modify the swap partition size and relabel the disk.

VERITAS Volume Manager Advanced Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-3

Boot Disk Encapsulation and Mirroring

Encapsulating the System Boot Disk


It is best to use the vxdiskadm utility option 2, Encapsulate one or
more disks, to encapsulate the system boot disk. The following example
shows pertinent excerpts from a typical dialog.
# vxdiskadm
Select an operation to perform: 2
Select disk devices to encapsulate:
[<pattern-list>,all,list,q,?] c0t0d0
Which disk group [<group>,list,q,?] rootdg
There is no active disk group named rootdg.
Create a new group named rootdg? [y,n,q,?] (default: y) y
Use a default disk name for the disk? [y,n,q,?] (default: y) y
A new disk group rootdg will be created and the disk device c0t0d0 will
be encapsulated and added to the disk group with the disk name rootdg01
Enter desired private region length
[<privlen>,q,?] (default: 2048) 2048
The c0t0d0 disk has been configured for encapsulation.
The first stage of encapsulation has completed successfully. You should
now reboot your system at the earliest possible opportunity. The
encapsulation will require two or three reboots which will happen
automatically after the next reboot. To reboot execute the command:
shutdown -g0 -y -i6
This will update the /etc/vfstab file so that volume devices are used to
mount the file systems on this disk device. You will need to update any
other references such as backup scripts, databases,or manually created
swap devices.

6-4

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Boot Disk Encapsulation and Mirroring

Mirroring the System Boot Disk


You use the vxdiskadm utility to add a mirror disk to the rootdg disk
group and to mirror the boot disk. The following examples show
pertinent excerpts from the vxdiskadm dialog.

Adding a Second Disk to the rootdg Disk Group


You use the vxdiskadm utility option 1, Add or initialize one or
more disks, to add a second disk drive to the rootdg disk group. The
following example shows a completed disk group that is ready for
mirroring.
# vxdisk -g rootdg list
DEVICE
TYPE
c0t0d0s2
auto:sliced
c0t1d0s2
auto:sliced

DISK
rootdg01
rootdg02

GROUP
rootdg
rootdg

STATUS
online
online nohotuse

Mirroring the Boot Disk


You use the vxdiskadm utility option 6, Mirror volumes on a disk, to
mirror the encapsulated boot disk. The following example shows
pertinent excerpts from the vxdiskadm dialog.
# vxdiskadm
Select an operation to perform: 6
Mirror volumes on a disk
Enter disk name [<disk>,list,q,?] rootdg01
Enter destination disk [<disk>,list,q,?] (default: any) rootdg02
The requested operation is to mirror all volumes on disk rootdg01
in disk group rootdg onto available disk space on disk rootdg02.
VxVM NOTICE V-5-2-229 NOTE: This operation can take a long time to
complete.
Continue with operation? [y,n,q,?] (default: y) y
VxVM vxmirror INFO V-5-2-22
Mirror volume rootvol ...
VxVM vxmirror INFO V-5-2-22
Mirror volume swapvol ...
VxVM INFO V-5-2-674 Mirroring of disk rootdg01 is complete.
Mirror volumes on another disk? [y,n,q,?] (default: n) n

VERITAS Volume Manager Advanced Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-5

Boot Disk Encapsulation and Mirroring

Verifying the Completed Boot Disk Configuration


The following examples show an encapsulated and mirrored boot disk. A
second boot device alias is automatically configured when the boot disk is
mirrored.

Verifying the VxVM Boot Device Aliases


When a boot disk is encapsulated, the VxVM software automatically
creates boot device aliases for use when booting from the encapsulated
boot disk root partition or the boot disk mirror, for example:
# eeprom nvramrc
nvramrc=devalias vx-rootdg01 /pci@1f,4000/scsi@3/disk@0,0:a
devalias vx-rootdg02 /pci@1f,4000/scsi@3/disk@1,0:a
You cannot boot the system using the device aliases until the OpenBoot
use-nvramrc? variable is set to true. After the variable is enabled, you
can boot from the primary or mirror boot device aliases, for example:
# eeprom use-nvramrc?=true
# init 0
ok boot vx-rootdg01
# init 0
ok boot vx-rootdg02

Verifying the VxVM Reserved Disk Group Variables


VxVM has two reserved variables named defaultdg and bootdg. Unless
special action is taken, both of the variables are set to a value of nodg.
When you encapsulate the system boot disk and place it in the rootdg
disk group, the bootdg variable is automatically updated.
The following example shows the VxVM reserved variables:
# vxdg defaultdg
nodg
# vxdg bootdg
rootdg

6-6

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Boot Disk Encapsulation and Mirroring

Verifying the rootdg Disk Group Structures


The following is an example of a typical encapsulated and mirrored boot
disk volume structure.
# vxprint -g rootdg
TY NAME
ASSOC
dg rootdg
rootdg

KSTATE
-

LENGTH
-

dm rootdg01
dm rootdg02

c0t0d0s2
c0t1d0s2

17674902 17678493 -

NOHOTUSE

v
pl
sd
sd
pl
sd

rootvol
rootvol-01
rootdg01-B0
rootdg01-02
rootvol-02
rootdg02-01

root
rootvol
rootvol-01
rootvol-01
rootvol
rootvol-02

ENABLED
ENABLED
ENABLED
ENABLED
ENABLED
ENABLED

17139843
17139843
1
17139842
17139843
17139843

0
1
0

ACTIVE
ACTIVE
ACTIVE
-

v
pl
sd
pl
sd

swapvol
swapvol-01
rootdg01-01
swapvol-02
rootdg02-02

swap
swapvol
swapvol-01
swapvol
swapvol-02

ENABLED
ENABLED
ENABLED
ENABLED
ENABLED

531468
531468
531468
531468
531468

0
0

ACTIVE
ACTIVE
ACTIVE
-

PLOFFS
-

STATE
-

Verifying the Encapsulated Boot Disk Partitioning


The following is the boot disk partition map after the encapsulation
process is completed. The public region (slice 3) is mapped to the entire
disk. The private region (slice 4) is mapped to the last cylinder. The root
and swap data are in the same location as before the encapsulation.
# format -d c0t1d0
partition> p
Current partition table (original):
Total disk cylinders available: 4924 + 2 (reserved cylinders)
Part
Tag
0
root
1
swap
2
backup
3
4
5 unassigned
6 unassigned
7 unassigned

Flag
wm
wu
wm
wu
wu
wm
wm
wm

Cylinders
0 - 4772
4773 - 4920
0 - 4923
0 - 4923
4922 - 4923
0
0
0

Size
8.17GB
259.51MB
8.43GB
8.43GB
3.51MB
0
0
0

Blocks
(4773/0/0) 17139843
(148/0/0)
531468
(4924/0/0) 17682084
(4924/0/0) 17682084
(2/0/0)
7182
(0/0/0)
0
(0/0/0)
0
(0/0/0)
0

VERITAS Volume Manager Advanced Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-7

Creating a Best Practice Boot Disk Configuration

Creating a Best Practice Boot Disk Configuration


If you encapsulate and mirror your boot disk, you should consider
refining the configuration to improve emergency recovery and
serviceability.
If there are no free cylinders for use as a private region during
encapsulation, VxVM takes them from the swap space and places the
private region (slice 4) in the middle of the public region address space
(slice 3). The private region is protected by a subdisk overlay named
rootdg01Priv. Also, block 0 on the boot disk is protected with a subdisk
overlay named rootdg01-B0. The following example shows an
encapsulated boot disk structure:
# format -d c0t0d0
...
Part
Tag
Flag
0
root
wm
1
swap
wu
2
backup
wm
3
wu
4
wu
5 unassigned
wm
6 unassigned
wm
7 unassigned
wm

Cylinders
0 - 3614
3618 - 3879
0 - 3879
0 - 3879
3615 - 3617
0
0
0

Size
3.72GB
276.33MB
4.00GB
4.00GB
3.16MB
0
0
0

Blocks
(3615/0/0) 7808400
(262/0/0)
565920
(3880/0/0) 8380800
(3880/0/0) 8380800
(3/0/0)
6480
(0/0/0)
0
(0/0/0)
0
(0/0/0)
0

# vxprint
Disk group: rootdg
TY NAME
dg rootdg

ASSOC
rootdg

KSTATE
-

LENGTH
-

PLOFFS
-

STATE
-

dm rootdg01

c0t0d0s2

8380799

sd rootdg01Priv -

ENABLED

6479

v
pl
sd
sd

rootvol
rootvol-01
rootdg01-B0
rootdg01-02

root
rootvol
rootvol-01
rootvol-01

ENABLED
ENABLED
ENABLED
ENABLED

7808400
7808400
1
7808399

0
1

ACTIVE
ACTIVE
-

v swapvol
pl swapvol-01
sd rootdg01-01

swap
swapvol
swapvol-01

ENABLED
ENABLED
ENABLED

565920
565920
565920

ACTIVE
ACTIVE
-

6-8

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Creating a Best Practice Boot Disk Configuration


When the boot disk is mirrored, the structure of the boot disk and the
mirror disk are not identical. This can be confusing and can add difficulty
to service and recovery situations.
The partition maps of a wort-case boot disk and mirror disk configuration
are organized in a different manner, for example:
# format -d c0t0d0
...
Part
Tag
Flag
0
root
wm
1
swap
wu
2
backup
wm
3
wu
4
wu
5 unassigned
wm
6 unassigned
wm
7 unassigned
wm

Cylinders
0 - 3614
3618 - 3879
0 - 3879
0 - 3879
3615 - 3617
0
0
0

Size
3.72GB
276.33MB
4.00GB
4.00GB
3.16MB
0
0
0

Blocks
(3615/0/0) 7808400
(262/0/0)
565920
(3880/0/0) 8380800
(3880/0/0) 8380800
(3/0/0)
6480
(0/0/0)
0
(0/0/0)
0
(0/0/0)
0

# format -d c0t1d0
...
Part
Tag
Flag
0
root
wm
1
swap
wu
2
backup
wu
3
wu
4
wu
5 unassigned
wm
6 unassigned
wm
7 unassigned
wm

Cylinders
3 - 3617
3618 - 3879
0 - 3879
3 - 3879
0 2
0
0
0

Size
3.72GB
276.33MB
4.00GB
3.99GB
3.16MB
0
0
0

Blocks
(3615/0/0) 7808400
(262/0/0)
565920
(3880/0/0) 8380800
(3877/0/0) 8374320
(3/0/0)
6480
(0/0/0)
0
(0/0/0)
0
(0/0/0)
0

The following differences exist between the two disk drives:

The VxVM private region (partition 4) is in the middle of the


primary boot disk (c0t0d0) and is at the beginning of the mirror disk
(c0t1d0).

The root and swap partitions on the primary boot disk still remain in
the same physical location, aligned with the original partition
boundaries. They are offset on the mirror disk.

The boot disk is still directly bootable (ok boot disk). The mirror
disk must be booted using the nvramrc device alias vx-rootdg02.

VERITAS Volume Manager Advanced Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-9

Creating a Best Practice Boot Disk Configuration

Establishing Boot Disk Configuration Policies


Optimized boot disk configurations can make recovery faster and safer.
Administrators can adopt recovery procedures that work for any system
under their control.
If possible, adhere to the following configuration policies:

Document each systems configuration and keep hard copies of the


configurations easily available. Do not rely on having electronic
copies available.

Keep the /usr and /opt directories on the root partition if possible.
The content of these directories is relatively stable and keeping them
on the root partition simplifies the configuration.
Additionally, the /usr directory contains the VxVM recovery
software. Placing the root and /usr software on the same partition
reduces the possibility of having a bootable root file system, but
losing access to the VxVM recovery software residing on a separate
/usr partition.

If possible, also leave the /var directory on the root partition.

Add an additional disk drive to the rootdg disk group as a hot


spare.

Caution Ensure that the mirror disk is initialized in the VxVM sliced
format and not the cdsdisk format. The cdsdisk format must not be
used in the rootdg disk group.
Consult the Sun BluePrints document, Towards a Reference Configuration for VxVM Managed Boot Disks,
for a more detailed explanation.

6-10

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Creating a Best Practice Boot Disk Configuration

Modifying an Existing Boot Disk Configuration


The following section assumes that all disk drives and volume
components follow the default VxVM naming conventions.
After using the vxdiskadm utility to create a standard encapsulated boot
disk and mirror configuration, perform the following steps to reorganize
the disks into a best practice configuration.
1.

Un-mirror the primary boot disk by removing the mirror plexes from
the mirror disk. Use the Bourne shell.
# vxassist -g rootdg remove mirror rootvol !rootdg02
# vxassist -g rootdg remove mirror swapvol !rootdg02

2.

To ensure the correct mirror placement, manually mirror the


rootvol and swapvol volumes again in the order shown.
# vxrootmir rootdg02
# vxassist -g rootdg mirror swapvol rootdg02

3.

Disassociate all primary boot disk plexes and recursively remove


them.
#
#
#
#
#

4.

vxplex
vxplex
vxedit
vxedit
vxedit

-g
-g
-g
-g
-g

rootdg
rootdg
rootdg
rootdg
rootdg

-v
-v
-r
-r
-r

rootvol dis rootvol-01


swapvol dis swapvol-01
rm rootdg01Priv
rm rootvol-01
rm swapvol-01

Verify that only the mirror disk plexes remain.

# vxprint -g rootdg
TY NAME
ASSOC
dg rootdg
rootdg

KSTATE
-

LENGTH
-

PLOFFS
-

STATE
-

dm rootdg01
dm rootdg02

c0t0d0s2
c0t1d0s2

8380799
8374320

NOHOTUSE

v rootvol
pl rootvol-02
sd rootdg02-01

root
rootvol
rootvol-02

ENABLED
ENABLED
ENABLED

7808400
7808400
7808400

ACTIVE
ACTIVE
-

v swapvol
pl swapvol-02
sd rootdg02-02

swap
swapvol
swapvol-02

ENABLED
ENABLED
ENABLED

565920
565920
565920

ACTIVE
ACTIVE
-

VERITAS Volume Manager Advanced Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-11

Creating a Best Practice Boot Disk Configuration


5.

Remove the rootdg01 disk from the rootdg disk group and
complete the following steps:
a.

Reinitialize the rootdg01 disk.

b.

Add the rootdg01 disk back into the rootdg disk group.

# vxdg -g rootdg rmdisk rootdg01


# vxdisksetup -i c0t0d0 format=simple
# vxdg -g rootdg adddisk rootdg01=c0t0d0
6.

Use the vxdiskadm option 6, Mirror volume on a disk, to mirror


rootvol and swapvol on the rootdg02 disk drive back to the newly
initialized rootdg01 disk drive.
# vxdiskadm
...
At the prompt below, supply the name of the disk
containing the volumes to be mirrored.
Enter disk name [<disk>,list,q,?] rootdg02
Enter destination disk [<disk>,list,q,?] (default: any)
rootdg01

7.

Modify the boot-device parameter and enable nvramrc usage.


# eeprom boot-device="rootdisk rootmirror"
# eeprom "use-nvramrc?"=true

8.

Use the prtvtoc command and the format utility to verify that the
primary and mirror boot disk drives have exactly the same partition
maps.

9.

Boot the system from each of the available system devices.


ok
ok
ok
ok

boot
boot
boot
boot

disk0
disk1.
vx-rootdg01
vx-rootdg02

You can now replace a defective boot disk drive in the same manner as
any other VxVM disk drive, and then resynchronize the mirrors.

6-12

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Administering Hot Devices

Administering Hot Devices


Depending on how the /etc/rc2.d/S95vxvm-recover file is configured,
either the older hot-spare daemon, vxsparecheck, starts at boot time or
the newer hot-relocation daemon, vxrelocd, starts at boot time.
The functionality of the two daemons is different. By default, hot
relocation is enabled. Either mode of operation requires that the failed
object is redundant. Effectively, the hot-device feature makes a copy of a
surviving object mirror.

Selecting Hot Device Operational Mode


You can configure either of two hot device modes of operation. Most
administrators prefer the newer hot-relocation mode.

Hot-Spare Functionality (Legacy Mode of Operation)


In early versions of VxVM, the hot-spare daemon, vxsparecheck,
detected and reacted to total disk media failures. It did this by moving all
redundant objects on the failed disk drive to a pre-designated spare disk
drive in the disk group.
Hot-sparing is an older mode of operation, but can still be enabled, if
necessary, by editing the /etc/rc2.d/S95vxvm-recover file and
commenting out the vxrelocd root & line, and then uncommenting the
#vxsparecheck root & line.

Hot-Relocation Functionality
The hot-relocation daemon, vxrelocd, detects and reacts to partial disk
media failures. It does this by copying the affected subdisk mirror to free
space on a different disk drive in the group.
Free space can be found on disk drives that have been designated as hot
spares. If there are no designated hot spares, VxVM uses available free
space on any disk drive in the disk group that does not have the
nohotuse flag set.
Hot relocation can also be performed for subdisks that are part of a
RAID-5 volume.

VERITAS Volume Manager Advanced Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-13

Administering Hot Devices


Hot relocation is enabled by default and goes into effect, without system
administrator intervention, when a failure occurs.
As shown in Figure 6-2, when a subdisk failure is detected, the contents of
the subdisk are reconstructed on the designated hot spare. The volume
continues to function with its original full redundancy.
Volume

Hot
spare

Private

Private

Primary
Subdisk

Mirror
Subdisk

Figure 6-2

Private
Copy

New
Subdisk

Subdisk Relocation

The hot-relocation daemon, vxrelocd, detects and reacts to the following


types of failures:

Disk drive failure


This is first detected as an I/O failure from a VxVM object. VxVM
attempts to correct the error. If the error cannot be corrected, VxVM
tries to access configuration information in the private region of the
disk drive. If it cannot access the private regions, it considers the
disk drive to have failed.
All plexes associated with the failed disk drive are eventually
detached. The output of vxprint shows the plex in an IOFAIL state.
This state is a hard failure that typically requires replacement of the
disk drive.

Plex failure
This is detected as an uncorrectable I/O error in the plex. For
mirrored volumes, the plex is detached.
Typically, this type of failure is caused by a block read or write error
that cannot be recovered with a series of retry operations.
It is a good idea to periodically monitor for these types of errors.
They might show up as correctable errors for a while and then
become hard errors. Sometimes, recoverable read/write errors are
spurious and not seen again.

6-14

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Administering Hot Devices

RAID-5 subdisk failure


This is detected as an uncorrectable I/O error in one of the
RAID-5 subdisks. The subdisk is detached. The failure is caused by
the same situation as plex failures: unrecoverable block read/write
errors.

Hot relocation is not possible if any of the following are true:

Subdisks do not belong to a mirrored or RAID-5 volume.

Not enough spare disk drive space is available.

The only available space for relocation is on a disk drive that


contains any portion of the surviving mirror or RAID-5 volume.

A mirrored volume has a DRL subdisk as part of its data plex:


subdisks belonging to that plex cannot be relocated.

The failure is a log plex: a new log plex is created and there is no
relocation.

VERITAS Volume Manager Advanced Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-15

Administering Hot Devices

Evaluating Hot-Device Configurations


You can verify the hot-relocation and hot-spare status of disk drives from
the command line. The following example shows command-line
verification:
# vxdg -g dgY nohotuse
DISK
DEVICE
TAG
dgY04
c2t1d0s2
c2t1d0

OFFSET
0

LENGTH
17674902

FLAGS
n

# vxdg -g dgY spare


DISK
DEVICE
TAG
dgY05
c2t3d0s2
c2t3d0

OFFSET
0

LENGTH
17674902

FLAGS
s

You can also verify and modify disk drive hot-device status in the VEA
grid area, as shown in Figure 6-3.

Figure 6-3

6-16

VEA Hot-Device Flags

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Administering Hot Devices

Administering Hot Devices Using Command-Line


Programs
You use the vxedit command-line program to administer hot devices.

Designating Hot-Spare Disk Drives


A disk designated as a spare is used only for hot relocation. The vxassist
utility will not allocate a subdisk on that disk unless forced to by
command-line arguments. You designate disk drive as spares using the
vxedit command, and you verify the spare status of the disk drives with
the vxdisk command. The following example shows the command-line
process:
# vxedit -g
# vxdisk -g
DEVICE
c2t1d0s2
c2t3d0s2
c2t5d0s2
c3t32d0s2
c3t33d0s2
c3t35d0s2

dgY set spare=on dgY05


dgY list
TYPE
DISK
GROUP
sliced
dgY04
dgY
sliced
dgY05
dgY
sliced
dgY01
dgY
sliced
dgY06
dgY
sliced
dgY02
dgY
sliced
dgY03
dgY

STATUS
online
online spare
online
online
online
online

Note If a disk drive is marked as a hot spare, the vxassist utility does
not create a subdisk on that disk drive unless the disk drive is specifically
designated in command-line arguments.

Blocking Hot Relocation


If hot relocation is enabled (the default), you can use any disk drive with
free space during the relocation of a failed subdisk if there is no hot-spare
space available. If you do not want a disk drive to be used for hot
relocation, you can mark it for no hot use as follows:
# vxedit -g dgY set nohotuse=on dgY04
# vxdisk -g dgY list
DEVICE
c2t1d0s2
c2t3d0s2
c2t5d0s2
c3t32d0s2
c3t33d0s2
c3t35d0s2

TYPE
sliced
sliced
sliced
sliced
sliced
sliced

DISK
dgY04
dgY05
dgY01
dgY06
dgY02
dgY03

GROUP
dgY
dgY
dgY
dgY
dgY
dgY

STATUS
online nohotuse
online spare
online
online
online
online

VERITAS Volume Manager Advanced Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-17

Administering Hot Devices

Administering Hot Devices Using the VEA GUI


Display a disk groups disk drives in the grid area, click a disk drive with
the third mouse button, and then select Set Disk Usage from its pop-up
menu. Set or reset Spare, No hot use, Reserved, or Reserved for Allocator,
as shown in Figure 6-4.
The Spare and No hot use boxes are mutually exclusive.

Figure 6-4

VEA Hot-Device Administration

The Reserve flag, if set, prevents automatic space allocation by utilities,


such as the vxassist program, unless the disk drive is specified on the
command line.
The Reserved for Allocator flag, if set, reserves a disk for exclusive use by
ISP utilities such as the vxpool and vxvoladm commands.

6-18

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Administering Hot Devices

Controlling Relocation Recovery Time


You can reduce the impact of recovery on system performance by
instructing the vxrelocd daemon to increase the delay between the
recovery of each region of a volume. Use the vxrelocd daemon as
follows:
# vxrelocd -o slow=500 &
The value of slow is passed on to the vxrecover command. The default
value is 250 milliseconds.

Monitoring Errors
By default, the vxrelocd daemon sends email notification of errors to the
server root account. You can modify the account name in the vxrelocd
root & line in the etc/rc2.d/S95vxvm-recover file.
You can also examine system error logs for evidence of disk drive
problems, but the email notification to root is usually sufficient.

VERITAS Volume Manager Advanced Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-19

Evacuating a Disk Drive

Edited by Foxit Reader


Copyright(C) by Foxit Software Company,2005-2007
For Evaluation Only.

Evacuating a Disk Drive


The volume structures on a disk drive that is starting to experience
recoverable data errors can be evacuated to a different disk drive before
the disk drive fails entirely. Evacuation can reduce the risk of data loss.
You can also use evacuation to reduce or eliminate performance
bottlenecks that have been identified.
Evacuation can only be performed on disk drives within the same group.

Identifying Evacuation Conflicts


Before you proceed with disk drive evacuation, carefully investigate the
configuration of both the failing disk drive and the new disk drive.
You should verify that the evacuation process is not going to create any of
the following conflicts:

Both volume mirrors are on the same physical disk drive.

More than one stripe column of a striped or RAID-5 volume is on the


same disk drive.

Preparing for Evacuation


Before starting the evacuation process, you must:

6-20

Find out with what volume the failing plex or subdisk is associated,
and determine the name of the disk drives that are associated with it.

Find out what disk group is associated with the failing disk drive.

Determine if any other volumes are associated with the failing disk
drive.

Find a new disk drive with enough free space to perform the
evacuation.

Check for any volume conflicts associated with the new disk drive.

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Evacuating a Disk Drive

Identifying Suitable Evacuation Disk Drives


The following example shows how to prepare for disk drive evacuation if
you are notified of a correctable read error on a subdisk named dgX02-01
in disk group dgX.
# vxprint -g dgX
TY NAME
ASSOC
dg dgX
dgX

KSTATE
-

LENGTH
-

PLOFFS
-

STATE
-

dm
dm
dm
dm
dm
dm

dgX01
dgX02
dgX03
dgX04
dgX05
dgX06

c2t16d0s2
c2t18d0s2
c2t1d0s2
c2t3d0s2
c2t5d0s2
c2t20d0s2

17674896
17674896
17674896
17674896
17674896
17674896

NOHOTUSE
SPARE

v
pl
sd
sd
sd
sd
sd

r5demo
r5demo-01
dgX03-01
dgX04-01
dgX05-01
dgX01-01
dgX02-01

raid5
r5demo
r5demo-01
r5demo-01
r5demo-01
r5demo-01
r5demo-01

ENABLED
ENABLED
ENABLED
ENABLED
ENABLED
ENABLED
ENABLED

61440
61440
15360
15360
15360
15360
15360

0
0
0
0
0

ACTIVE
ACTIVE
-

v
pl
sd
sd
sd
sd
sd

stdemo
stdemo-01
dgX03-02
dgX04-02
dgX05-02
dgX01-02
dgX02-02

fsgen
stdemo
stdemo-01
stdemo-01
stdemo-01
stdemo-01
stdemo-01

ENABLED
ENABLED
ENABLED
ENABLED
ENABLED
ENABLED
ENABLED

61440
61440
12288
12288
12288
12288
12288

0
0
0
0
0

ACTIVE
ACTIVE
-

An analysis of the preceding vxprint output indicates the following:

The failing subdisk, dgX02-01, is on disk drive dgX02.

The dgX02-02 subdisk in another volume is also on dgX02.

Disk dgX06 appears to be unused, is marked as a spare, and is a


suitable candidate for evacuating disk dgX02.

For this scenario, evacuate the disk dgX02 to disk dgX06. After the
evacuation has completed, replace disk dgX02. You might mark the new
dgX02 disk drive as the spare and unmark dgX06. You could also migrate
the stripe data from dgX06 back to the new dgX02 disk drive.

VERITAS Volume Manager Advanced Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-21

Evacuating a Disk Drive

Evacuation Using the vxevac Command


The vxevac command moves subdisks from the specified disk drive to a
new disk drive. If a destination disk drive is not specified, the vxevac
command uses any available disk drive. The following example shows a
typical vxevac procedure.
# vxevac -g dgX dgX02 dgX06
# vxprint -g dgX
TY NAME
ASSOC
dg dgX
dgX

KSTATE
-

LENGTH
-

PLOFFS
-

STATE
-

dm
dm
dm
dm
dm
dm

dgX01
dgX02
dgX03
dgX04
dgX05
dgX06

c2t16d0s2
c2t18d0s2
c2t1d0s2
c2t3d0s2
c2t5d0s2
c2t20d0s2

17674896
17674896
17674896
17674896
17674896
17674896

NOHOTUSE
SPARE

v
pl
sd
sd
sd
sd
sd

r5demo
r5demo-01
dgX03-01
dgX04-01
dgX05-01
dgX01-01
dgX06-01

raid5
r5demo
r5demo-01
r5demo-01
r5demo-01
r5demo-01
r5demo-01

ENABLED
ENABLED
ENABLED
ENABLED
ENABLED
ENABLED
ENABLED

61440
61440
15360
15360
15360
15360
15360

0
0
0
0
0

ACTIVE
ACTIVE
-

v
pl
sd
sd
sd
sd
sd

stdemo
stdemo-01
dgX03-02
dgX04-02
dgX05-02
dgX01-02
dgX06-02

fsgen
stdemo
stdemo-01
stdemo-01
stdemo-01
stdemo-01
stdemo-01

ENABLED
ENABLED
ENABLED
ENABLED
ENABLED
ENABLED
ENABLED

61440
61440
12288
12288
12288
12288
12288

0
0
0
0
0

ACTIVE
ACTIVE
-

6-22

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Evacuating a Disk Drive

Evacuation Using the VEA GUI


To perform an evacuation using the VEA GUI, complete the following
steps:
1.

Select the disk drive that contains the objects and data to be moved.

2.

Select Evacuate Disk from the disk drives pop-up menu.

3.

Configure the Evacuate Disk form, as shown in Figure 6-5.

Figure 6-5

VEA Evacuate Disk Form

VERITAS Volume Manager Advanced Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-23

Evacuating a Disk Drive

Evacuation Using the vxdiskadm Utility


To perform an evacuation using the vxdiskadm utility, complete the
following steps:
1.

Start the vxdiskadm utility, and select Option 7, Move volumes


from a disk.

2.

Type the media name of the disk drive to be evacuated.

3.

Type the media name of the destination disk drive.

The following example shows a summary of the process.


# vxdiskadm
Select an operation to perform: 7
Move volumes from a disk
Enter disk name [<disk>,list,q,?] dgX06
Enter disk name [<disk>,list,q,?] dgX02
VxVM NOTICE V-5-2-283
Requested operation is to move all volumes from disk dgX06 in
group dgX.
NOTE: This operation can take a long time to complete.
Continue with operation? [y,n,q,?] (default: y) y
VxVM vxevac INFO V-5-2-24
Move volume r5demo ...
VxVM vxevac INFO V-5-2-24
Move volume stdemo ...
VxVM INFO V-5-2-188 Evacuation of disk dgX06 is complete.
Move volumes from another disk? [y,n,q,?] (default: n) n

6-24

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Moving Disk Drives Without Preserving Data

Moving Disk Drives Without Preserving Data


You might want to move a VxVM disk drive to a different disk group
because the destination disk group needs the additional disk space. As
long as the disk drive does not contain any functional data that you need
to preserve, the process is fairly easy.

Moving a Disk Drive Using the Command Line


If the disk drive you want to move contains an active volume and you do
not care if the data is lost, complete the following steps to move the disk
drive to a different disk group:
1.

Use the vxprint command to verify that there are no unexpected


volumes, or portions of volumes, associated with the disk drive you
want to move.

2.

Disable any applications related to the volume.

3.

Unmount any file systems related to the volume.

4.

Stop the volume.


You stop volumes using the vxvol command as follows:
# vxvol -g old_dg stop vol-02

5.

Delete the volume configuration.


Recursively delete all objects in a volume as follows:
# vxedit -g old_dg -rf rm vol-02

6.

Remove the disk drive from the disk group.


You use the vxdg command to remove a disk drive from a disk
group as follows:
# vxdg -g old_dg rmdisk olddg-12
Even after the vxdg rmdisk operation, the disk drive is still
initialized for VxVM use. The vxdiskunsetup command completely
removes a disk drive from VxVM control.

7.

Add the disk drive to a different disk group.


You use the vxdg command to add the disk drive to a different disk
group as follows:
# vxdg -g new_dg adddisk newdg-02=c1t3d0

VERITAS Volume Manager Advanced Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-25

Moving Disk Drives Without Preserving Data

Moving a Disk Drive Using the VEA GUI


Moving a disk drive to a new disk group is easy using the VEA GUI. You
use the following process to reduce errors:
1.

In the grid area, click the third mouse button on the disk group and
select Disk/Volume Map from the pop-up menu.

2.

Examine the Volume to Disk Mapping display, shown in Figure 6-6,


and verify that there are no unexpected volume structures associated
with the disk drive you want to move.

Figure 6-6
3.

Analyzing Volume to Disk Relationships

Use the pop-up menus available on the volume name and disk drive
name to perform any of the following actions:

Stop any volumes on the disk drive that you want to remove.

Delete the volume or volumes.

Remove the disk from the disk group.


The disk drive is returned to the free disk pool and can now be
added to a different disk group.

6-26

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Moving Populated Disk Drives to a New Disk Group

Moving Populated Disk Drives to a New Disk Group


Moving populated VxVM disk drives to a new or different disk group is a
technique you might use occasionally. One reason to use this technique is
if you have mistakenly created all of your volumes in a single disk group.
To resolve a performance problem, you would move some of the volumes
and their related disk drives to a different disk group.
Caution Do not perform this operation on a production system without
first backing up the data on all associated volumes. If this process fails,
there is no way to recover data without backup tapes.
In this section, a volume called mirvol is moved from a disk group
named old_dg to a new disk group named new_dg.
The process of moving populated disks is easier if you have VERITAS FastResync. Currently, Sun does not
sell, license, or support this option. VERITAS FastResync enables the vxdg move/split/join options.

Evaluating Disk Drive Involvement


Before you take any action, you must determine which physical disk
drives are part of your target volume. You must also ensure that the disk
drives are not being used by other volumes.

Determining Disk Drive Involvement From the Command Line


Use the vxprint command as follows to determine the volume to disk
drive mapping:
# vxprint -g old_dg
TY NAME
ASSOC
dg old_dg
old_dg

KSTATE
-

LENGTH
-

dm olddg-01
dm olddg-02
dm olddg-03

c2t16d0s2
c3t37d0s2
c3t52d0s2

17674902 17674902 17674902 -

v
pl
sd
pl
sd

fsgen
mirvol
mirvol-01
mirvol
mirvol-02

ENABLED
ENABLED
ENABLED
ENABLED
ENABLED

204800
208278
208278
208278
208278

ACTIVE
ACTIVE
ACTIVE
-

mirvol
mirvol-01
olddg-01-01
mirvol-02
olddg-02-01

PLOFFS
-

0
0

STATE
-

VERITAS Volume Manager Advanced Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-27

Moving Populated Disk Drives to a New Disk Group


In the preceding example, the volume mirvol contains two plexes with a
single subdisk associated with each plex. The volume is associated with
two disk drives, olddg-01 and olddg-02.
Note The VEA GUIs Disk/Volume Map display can be very helpful
when you are trying to determine volume involvement with specific disk
drives.

Saving the Configuration


Use the vxprint command to save the volume configuration of the
mirvol volume in a file named save_mirvol.
# vxprint -hmQqrL -g old_dg mirvol > \
/var/tmp/save_mirvol
Caution If you are saving layered volumes that have sub-volumes (such
as striped mirror structures), you must add the r and L options to the
vxprint command. If you fail to do this, the saved configuration
information is incomplete.
The vxprint -m option provides detailed configuration information in a
format that can be used later by the vxmake utility. Table 6-1 shows the
vxprint command options used in this example.
Table 6-1 The vxprint Command Options

6-28

Command

Function

-h

Lists complete hierarchies.

-m

Displays information in a format that can be used as


input to the vxmake utility.

-Q

Suppresses the disk group header that separates each


disk group.

-q

Suppresses headers (in addition to disk group header).

-r

Displays related records of a volume containing subvolumes.

-L

Affects record grouping when used with the -r option.

-g

Specifies the disk group.

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Moving Populated Disk Drives to a New Disk Group

Moving the Disk Drives to a New Disk Group


Moving the disk drives to a new disk group requires several steps that
you have seen earlier in this course. They are:
1.

Unmount appropriate file systems and stop any processes that are
accessing the mirvol volume directly.
# umount /Data

2.

Stop the volume.


# vxvol -g old_dg stop mirvol

3.

Remove the definitions of the structures (volume, plexes, and


subdisks) from the configuration database.
# vxedit -g old_dg -rf rm mirvol
The vxedit command removes the definitions of the volume,
plexes, and subdisks from the configuration database for the old disk
group, old_dg.

Note Removing the definitions does not affect the data: it only removes
selected records from the configuration database. The -r option
recursively removes the volume and all associated plexes and subdisks.
4.

Remove the disk drives from the original disk group.


# vxdg -g old_dg rmdisk olddg-01 olddg-02

5.

If the new disk group, new_dg, does not exist, initialize it using one
of the disk drives to be moved (disk old_dg-01, in this example).
# vxdg init new_dg olddg-01=c2t16d0

Caution It is critical that all of the disk drives retain their original media
names when they are added to the new disk group.
6.

Add the remaining disk drives to the new disk group.


# vxdg -g new_dg adddisk olddg-02=c3t37d0

7.

Verify that the disk drives have been added to the new disk group.
# vxdisk list | grep new_dg
c2t16d0s2 sliced olddg-01
c3t37d0s2 sliced olddg-02
c3t52d0s2 sliced newdg-01

new_dg
new_dg
new_dg

VERITAS Volume Manager Advanced Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

online
online
online

6-29

Moving Populated Disk Drives to a New Disk Group

Reloading the Volume Configuration


To reload the volume configuration, complete the following steps:
1.

Use the vxmake command to reload the saved configuration for the
volume vol01.
# vxmake -g dgY -d /var/tmp/save_mirvol
Earlier the volume configuration was saved in the save_mirvol
file. The -d option specifies the description file to use for building
subdisks, plexes, and volumes.

Caution If the disk drives do not have their original media names, the
configuration reload fails.
2.

Check the volume status. The volume and plexes should be in a


DISABLED state.

# vxprint -g new_dg mirvol


TY NAME
ASSOC
KSTATE
LENGTH
v mirvol
fsgen
DISABLED 204800
pl mirvol-01
mirvol
DISABLED 208278
sd olddg-01-01 mirvol-01 ENABLED
208278
pl mirvol-02
mirvol
DISABLED 208278
sd olddg-02-01 mirvol-02 ENABLED
208278
3.

PLOFFS
0
0

STATE
EMPTY
EMPTY
EMPTY
-

Use the vxvol command to bring the volumes back online.


# vxvol -g new_dg init active mirvol

Note An alternative to this procedure is to create a new volume in


another disk group, and either dump a backup tape onto it or perform a
direct copy from the old volume.
4.

Edit the /etc/vfstab mount information for the volumes file


system and change the disk group in the logical paths.

# vi /etc/vfstab
/dev/vx/dsk/new_dg/mirvol
5.

/dev/vx/rdsk/new_dg/mirvol /Data ufs 1 yes -

Mount and test the file system.


# mount /Data
# ls /Data
lost+found

6-30

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Backing Up and Restoring Disk Group Configurations

Backing Up and Restoring Disk Group Configurations


The VxVM software automatically saves disk group configuration
backups. You can also manually backup disk group configurations.

Automatic Configuration Backup


VxVM keeps updated copies of the configuration status both in memory
and written on one or more storage disk drives. This status is updated
any time there is a change in the VxVM configuration or availability due
to errors or administrative changes. A backup copy of the configuration is
automatically updated on the system boot disk when administrative
changes are made to the configuration.
The primary elements involved are:

The vxio software driver

The vxconfigd configuration daemon

The vxconfigbackupd daemon

Figure 6-7 shows how the kernel configuration table is checked by the
vxio driver before the driver attempts to access a virtual structure. The
disk-resident copies do not need to be examined.
Consult
Before
Access
vxio
Driver

Update

Device
Access
Error

Administrative
Modifications
Kernel
Configuration
Table

vxconfigbackupd

vxconfigd
/etc/vx/cbr/bk
Storage Array
Update
configdb
Data

Figure 6-7

Configuration Monitoring Components

VERITAS Volume Manager Advanced Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-31

Backing Up and Restoring Disk Group Configurations

Initial Volume Configuration


When VxVM starts, the vxconfigd daemon imports disk groups that
belong to the VxVM server.
When disk groups are imported, the kernel configuration table is created
by the vxconfigd daemon, which reads the disk-resident configdb
records.

Status Configuration Changes


When the vxio driver is notified of a hard-device error, it disables the
volume it was trying to access and updates the VxVM kernel
configuration table. The vxio driver also signals the vxconfigd daemon
that a configuration change has taken place.

Administrative Configuration Changes


System administrator changes to the configuration, such as creating a new
disk group, are automatically recorded on the system boot disk by the
vxconfigbackupd daemon. A backup directory for each disk group is
maintained in the /etc/vx/cbr/bk directory. The following example
shows backup files on a system with two disk groups named pridg and
secdg.
# cd /etc/vx/cbr/bk
# ls
pridg.1064606899.21.ns-east-115
secdg.1064865853.29.ns-east-115
#
# cd pridg.1064606899.21.ns-east-115
# ls
1064606899.21.ns-east-115.binconfig
1064606899.21.ns-east-115.cfgrec
1064606899.21.ns-east-115.dginfo
1064606899.21.ns-east-115.diskinfo

Note If all array-resident configuration copies are lost, the


vxconfigrestore command can be used to rebuild the original
structures. The data on the structures is not preserved.

6-32

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Backing Up and Restoring Disk Group Configurations

Manual Configuration Backup and Restore


The vxconfigbackupd daemons are started by the
etc/rc2.d/S95vxvm-recover script and record any VxVM disk group
configuration changes in the /etc/vx/cbr/bk directory.
You use the vxconfigbackup command to backup the current disk group
configuration information. You restore a disk group configuration using
the vxconfigrestore command.
If there is damage to the disk group configuration records that are stored
in the private regions of one or more disk drives in a disk group, the disk
group import operation might fail. The vxconfigrestore command is
used to automatically correct the damaged configuration records or to
recreate a disk group from the beginning.
When restoring or repairing damaged disk group records, you must meet
the following criteria:

Failed disk drives must be replaced prior to using the


vxconfigrestore command.

Replacement disk drives must be initialized for VxVM use prior to


using the vxconfigrestore command.

All disk drives must have the same physical configuration and
logical addresses as when the configuration backup was performed.

Caution The vxconfigbackup command does not preserve data. File


system and raw volume data must be preserved separately using backup
commands or applications. Volumes with redundant data, such as
mirrored or RAID-5 structure, can recover from the loss of a mirror or a
RAID-5 stripe. Non-redundant volume types must be reinitialized for file
system use and the data recovered from backup media.

VERITAS Volume Manager Advanced Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-33

Backing Up and Restoring Disk Group Configurations

Using the vxconfigbackup Command


The vxconfigbackup command is used to backup one or more disk
groups. If the name of a disk group is not specified with the command, all
disk groups are backed up.
You use the -l option to specify a backup file location other than the
default location in /etc/vx/cbr/bk. The following example shows the
use of the vxconfigbackup command.
# vxdg list
NAME
STATE
rootdg
enabled
dgX
enabled

ID
1066871088.21.ns-east-104
1066748899.279.ns-east-104

# vxconfigbackup -l /etc/vx/cbr/mybackup dgX


Start backing up diskgroup dgX to
/etc/vx/cbr/mybackup/dgX.1066748899.279.ns-east-104 ...
VxVM NOTICE V-5-2-3100 Backup complete for diskgroup: dgX
# ls /etc/vx/cbr/mybackup
dgX.1066748899.279.ns-east-104
# ls /etc/vx/cbr/mybackup/dgX.1066748899.279.ns-east-104
1066748899.279.ns-east-104.binconfig
1066748899.279.ns-east-104.cfgrec
1066748899.279.ns-east-104.dginfo
1066748899.279.ns-east-104.diskinfo

6-34

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Backing Up and Restoring Disk Group Configurations

Using the vxconfigrestore Command


When you use the vxconfigrestore command to restore a disk group
and its associated structures, the process is performed in two stages:
precommit and commit.
# mount |grep R5demo
/R5demo on /dev/vx/dsk/dgX/r5demo
# ls -l /R5demo
total 28736
-rw------T
1 root
-rw------T
1 root
-rw------T
1 root
drwx-----2 root
#
#
#
#

other
other
other
root

1048576 Oct 27 11:37 file1


3145728 Oct 27 11:37 file2
10485760 Oct 27 19:40 file3
8192 Oct 27 11:35 lost+found

umount /R5demo
vxvol -g dgX stop r5demo
vxvol -g dgX stop stdemo
vxdg deport dgX

# vxdisksetup c2t3d0 (reinitialize the disk)


# vxdg import dgX
VxVM vxdg ERROR V-5-1-587 Disk group dgX: import failed: Disk for disk
group not found
# vxconfigrestore -p -l /etc/vx/cbr/mybackup dgX (precommit stage)
Installing volume manager disk header for c2t18d0s2 ...
Installing volume manager disk header for c2t1d0s2 ...
Installing volume manager disk header for c2t20d0s2 ...
Installing volume manager disk header for c2t16d0s2 ...
Installing volume manager disk header for c2t3d0s2 ...
Installing volume manager disk header for c2t5d0s2 ...
dgXs diskgroup configuration is restored
Diskgroup can be accessed in read only and can be examined using
vxprint in this state.
# vxconfigrestore -l /etc/vx/cbr/mybackup -c dgX (commit stage)
Committing configuration restoration for diskgroup dgX ....
dgXs diskgroup configuration restoration is committed.

VERITAS Volume Manager Advanced Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-35

Importing Disk Groups After a System Crash

Importing Disk Groups After a System Crash


Depending on the state of a disk group, there are several variations of the
import operation that might be useful. The basic import operation is not
complicated and can be done using either the vxdg command, the
vxdiskadm utility, or the VEA GUI.
If a disk group must be imported after a system crash, the process can be
more difficult. Following are some of the possible variations.

Performing a typical import of a clean disk group:


# vxdg import disk_group_name

Importing a disk group to another system after a crash:


# vxdg -C import disk_group_name
The -C option is necessary to clear the old host IDs that were left on
the disk drives after the crash.
# vxdg -fC import disk_group_name

Caution The -f option forces an import in the event that all the disk
drives are not usable. This option can be dangerous on dual-hosted
storage arrays because the disk group might also be imported to another
host. A disk group that is imported to two host systems can become
corrupted.
# vxrecover -g disk_group_name -sb
You execute this command after a crash to start the volumes and
perform a recovery process. This is done automatically during a
reboot.

Importing a disk group with a duplicate name:


# vxdg -t -n new_disk_group import disk_group_name
The -t option makes the new disk group name temporary.

Note When a disk group is imported, the volumes can be in a disabled


state. You can use the vxvol start command to start the volumes.

6-36

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Importing Disk Groups After a System Crash

Importing the rootdg Disk Group After a Crash


After a crash it might be necessary to import the rootdg disk group to
another system to perform repair operations. This process is a little more
complicated because you cannot have two rootdg disk groups on a
system. This procedure works only if the boot disk can be easily attached
to a different system.
Importing a second rootdg disk group requires the use of multiple
options to:

Assign a new temporary disk group name to rootdg.

Clear the original hostid ownership.

Use the unique rootdg group identifier:


# vxdg -tC -n new_disk_group import group_id

The difficult part is that you must know the unique rootdg group
identifier. This value must be known in advance. You can determine the
rootdg group identifier with the vxdisk command as follows:
# vxdisk -s list
Disk:
c0t2d0s2
type:
sliced
flags: online ready private autoconfig autoimport
imported
diskid: 791000525.1055.boulder
dgname: rootdg
dgid:
791000499.1025.boulder
hostid: boulder
The disk group is renamed by the importing host.
Note The vxdisk -s list command lists information for all attached
disk drives. Disk drives that belong to a cleanly deported disk group have
a blank hostid entry.

VERITAS Volume Manager Advanced Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-37

Volume Snapshot Operations

Volume Snapshot Operations


When you need to back up the data on a volume, such as a file system
volume, you can use the VxVM snapshot function to create a copy of the
volume. You can then back up the new copy to tape without disrupting
service to the original volume.

Snapshot Process
You must satisfy the following prerequisites before the snapshot process
can be started:

You must know the name of the volume to be backed up.

You must provide a name for the new temporary snapshot volume.

You can specify a specific disk drive to use for the snapshot copy.

You must have sufficient unused disk space for the snapshot.

The snapshot process is a two-step process. First, a snapshot mirror of a


volume is created. If no disk drives are specified for use by the snapshot
operation, VxVM finds free disk space anywhere it can. The resulting
snapshot mirror can be poorly constructed. Because the snapshot mirror is
only a temporary structure, this might not be a problem. However, it
could temporarily interfere with the performance of other volumes.
When the snapshot mirror is fully synchronized, its state changes to
SNAPDONE. The snapshot mirror continues to maintain synchronization
with the parent volume until it is detached.
The final operation detaches the temporary snapshot mirror and attaches
it to a regular volume with a name of your choosing.
You can then perform standard backup procedures on the snapshot
volume.

6-38

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Volume Snapshot Operations

Using the VEA GUI to Create a Snapshot


You can use the snapshot feature from the VEA GUI to create a snapshot.
Complete the following steps:
1.

2.

In the grid area, complete the following steps:


a.

Click on the volume with the third mouse button.

b.

Select Snap, and then Snap Start from its pop-up menu.

Complete the Snap Start Volume form, as shown in Figure 6-8.


It is usually safe to let VxVM decide which disks to use for the
Snapshot mirror.
The process of creating and synchronizing a new mirror with the
existing volume data can take some time depending on the volume
size.

Figure 6-8

VEA Snap Start Volume Form

VERITAS Volume Manager Advanced Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-39

Volume Snapshot Operations


3.

In the grid area, complete the following steps:


a.

Click the volume name with the third mouse button.

b.

Select Snap, and then Snap Shot from its pop-up menu.

c.

Complete the Snap Shot Volume form as shown in Figure 6-9.


The new mirror is detached and a separate volume is created
from it based on a default naming scheme. You can change the
default snapshot name if needed.

Figure 6-9

6-40

VEA Snap Shot Volume Form

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Volume Snapshot Operations


4.

Use the vxprint command to verify the status of the new snapshot
volume.
In the following case, the snapshot mirror/volume is named
SNAP-stdemo. The parent volume is named stdemo.

# vxprint -g dgX
TY NAME
ASSOC
dg dgX
dgX

KSTATE
-

LENGTH
-

PLOFFS
-

STATE
-

dm
dm
dm
dm
dm
dm
dm
dm
dm
dm
dm
dm

dgX01
dgX02
dgX03
dgX04
dgX05
dgX06
dgX07
dgX08
dgX09
dgX10
dgX11
dgX12

c2t16d0s2
c2t18d0s2
c2t1d0s2
c2t3d0s2
c2t5d0s2
c2t20d0s2
c3t32d0s2
c3t33d0s2
c3t35d0s2
c3t37d0s2
c3t50d0s2
c3t52d0s2

17674896
17674896
17674896
17674896
17674896
17674896
17674896
17674896
17674896
17674896
17674896
17674896

NOHOTUSE
-

v
pl
sd
sd
sd
sd
sd

SNAP-stdemo
stdemo-02
dgX06-01
dgX07-01
dgX08-01
dgX09-01
dgX10-01

fsgen
SNAP-stdemo
stdemo-02
stdemo-02
stdemo-02
stdemo-02
stdemo-02

ENABLED
ENABLED
ENABLED
ENABLED
ENABLED
ENABLED
ENABLED

61440
61440
12288
12288
12288
12288
12288

0
0
0
0
0

ACTIVE
ACTIVE
-

v
pl
sd
sd
sd
sd
sd

r5demo
r5demo-01
dgX03-01
dgX04-01
dgX05-01
dgX01-01
dgX02-01

raid5
r5demo
r5demo-01
r5demo-01
r5demo-01
r5demo-01
r5demo-01

ENABLED
ENABLED
ENABLED
ENABLED
ENABLED
ENABLED
ENABLED

61440
61440
15360
15360
15360
15360
15360

0
0
0
0
0

ACTIVE
ACTIVE
-

v
pl
sd
sd
sd
sd
sd

stdemo
stdemo-01
dgX03-02
dgX04-02
dgX05-02
dgX01-02
dgX02-02

fsgen
stdemo
stdemo-01
stdemo-01
stdemo-01
stdemo-01
stdemo-01

ENABLED
ENABLED
ENABLED
ENABLED
ENABLED
ENABLED
ENABLED

61440
61440
12288
12288
12288
12288
12288

0
0
0
0
0

ACTIVE
ACTIVE
-

VERITAS Volume Manager Advanced Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-41

Volume Snapshot Operations


5.

Create a temporary mount point, and complete the following steps:


a.

Mount the new snapshot volume.

b.

Back it up to tape.

# mkdir /Temp
# mount /dev/vx/dsk/dgX/SNAP-stdemo /Temp
# tar tvf /dev/rmt/0 /Temp
6.

Unmount and delete the snapshot volume.


# umount /Temp
# vxedit -g dgX -rf rm SNAP-stdemo
The VEA GUI volume snapshot menu has additional functions, Snap
Back and Snap Clear.
As shown in Figure 6-10, the Snap Back function rejoins the snapshot
mirror to the original volume. It also resynchronizes the data in
either direction between the original volume and the snapshot
mirror.

Figure 6-10 VEA Snap Back Volume Form


The Snap Clear function disassociates the detached snapshot mirror
from the original volume. The Snap Back feature cannot be used
after the Snap Clear operation has completed.

6-42

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Volume Snapshot Operations

Using the Command Line to Create a Snapshot


Using the command line to make snapshots is useful for performing
backups from script files. The following example shows the
command-line snapshot process.
1.

Create a snapshot of the stdemo volume and verify it has completed.

# vxassist -g dgX -b snapstart stdemo


# vxprint -g dgX stdemo
ASSOC
KSTATE
LENGTH
TY NAME
v
pl
sd
sd
sd
sd
sd
pl
sd
sd
sd
sd
sd

stdemo
stdemo-01
dgX03-02
dgX04-02
dgX05-02
dgX01-02
dgX02-02
stdemo-02
dgX06-01
dgX07-01
dgX08-01
dgX09-01
dgX10-01

fsgen
stdemo
stdemo-01
stdemo-01
stdemo-01
stdemo-01
stdemo-01
stdemo
stdemo-02
stdemo-02
stdemo-02
stdemo-02
stdemo-02

2.

ENABLED
ENABLED
ENABLED
ENABLED
ENABLED
ENABLED
ENABLED
ENABLED
ENABLED
ENABLED
ENABLED
ENABLED
ENABLED

61440
61440
12288
12288
12288
12288
12288
61440
12288
12288
12288
12288
12288

PLOFFS
0
0
0
0
0
0
0
0
0
0

STATE
ACTIVE
ACTIVE
SNAPDONE
-

Use the vxassist snapshot option to detach the temporary


snapshot mirror (plex) and associate it with a new volume named
SNAP-stdemo.

# vxassist -g dgX snapshot stdemo-02 SNAP-stdemo


3.

Process the volume with the fsck utility to check its integrity, and
complete the following steps:
a.

Mount the volume.

b.

Perform a backup of the snapshot volume.

c.

Unmount and delete the volume.

# fsck -y /dev/vx/rdsk/dgX/SNAP-stdemo
# mkdir /Temp
# mount /dev/vx/dsk/dgX/SNAP-stdemo /Temp
# tar cvf /dev/rmt/0 /Temp
# umount /Temp
# vxedit -g dgX -rf rm SNAP-stdemo

VERITAS Volume Manager Advanced Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-43

Online Volume Relayout

Online Volume Relayout


Online volume relayout is a tool that you can use to correct configuration
mistakes or to enhance the performance or reliability of a volume.
An important feature of online volume relayout is that a volume and its
file system can remain available during the relayout operation.
The relayout feature can be used to perform many operations, such as:

Adding more stripe columns to a RAID-5 volume

Changing the stripe unit size of a volume

Changing the type of volume from RAID 5 to mirrored or


concatenated

Note Read the Changing a VxVM Volume Layout section of the


VERITAS Enterprise Administrator Users Guide carefully before attempting
a relayout of a production volume.

Volume Relayout Prerequisites


You must research the following information before starting an online
volume relayout process:

Identify the new volume layout.


This includes concatenated, striped, RAID 5, concatenated mirror,
striped mirror, additional columns, and new stripe widths.

Research additional permanent disk space that might be needed by


the new volume layout.

Research the temporary disk space that might be needed during the
volume layout.

Note You can also use the vxassist relayout command to accomplish
online volume relayout. Unless you explicity allocate storage, the
vxassist command automatically determines where to get the
permanent and temporary disk space that might be needed.

6-44

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Online Volume Relayout

Volume Relayout Using the Command Line


The following examples show the basic process to add an extra column to
an existing RAID-5 volume name r5demo. There is enough additional disk
space on the current disk drives being used by the RAID-5 volume for any
temporary space needs. For performance reasons, you should specify
which disk drive to use for the new column.
# vxprint -g dgX r5demo stdemo
TY NAME
ASSOC
KSTATE
v r5demo
raid5
ENABLED
pl r5demo-01
r5demo
ENABLED
sd dgX03-01
r5demo-01
ENABLED
sd dgX04-01
r5demo-01
ENABLED
sd dgX05-01
r5demo-01
ENABLED
sd dgX01-01
r5demo-01
ENABLED
sd dgX02-01
r5demo-01
ENABLED

LENGTH
61440
61440
15360
15360
15360
15360
15360

PLOFFS
0
0
0
0
0

STATE
ACTIVE
ACTIVE
-

v
pl
sd
sd
sd
sd
sd

61440
61440
12288
12288
12288
12288
12288

0
0
0
0
0

ACTIVE
ACTIVE
-

stdemo
stdemo-01
dgX03-02
dgX04-02
dgX05-02
dgX01-02
dgX02-02

fsgen
stdemo
stdemo-01
stdemo-01
stdemo-01
stdemo-01
stdemo-01

# vxdg -g dgX free


DISK
DEVICE
dgX01
c2t16d0s2
dgX02
c2t18d0s2
dgX03
c2t1d0s2
dgX04
c2t3d0s2
dgX05
c2t5d0s2
dgX06
c2t20d0s2

ENABLED
ENABLED
ENABLED
ENABLED
ENABLED
ENABLED
ENABLED

TAG
c2t16d0
c2t18d0
c2t1d0
c2t3d0
c2t5d0
c2t20d0

OFFSET
27648
27648
27648
27648
27648
0

LENGTH
17647248
17647248
17647248
17647248
17647248
17674896

FLAGS
n
-

The RAID 5 and RAID 0 stripes are using a total of 27,648 blocks on each
disk drive or about 14 Mbytes. All disk drives in the disk group have
more than enough space left for use during the relayout operations.
The following example adds an additional column to the RAID-5 volume
and specifies that the new column is to be on the disk dgX06.
# vxassist -g dgX relayout r5demo layout=raid5 alloc="dgX06 ncol=6

VERITAS Volume Manager Advanced Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-45

Online Volume Relayout

Completed RAID 5 Volume Relayout


After the completion of the previous volume relayout example, the
RAID-5 volume now has the following configuration.
# vxprint -g dgX r5demo
TY NAME
ASSOC
v r5demo
raid5
pl r5demo-Dp02 r5demo
sd dgX06-03
r5demo-Dp02
pl r5demo-01
r5demo
sd dgX03-04
r5demo-01
sd dgX04-03
r5demo-01
sd dgX05-03
r5demo-01
sd dgX01-03
r5demo-01
sd dgX02-03
r5demo-01
sd dgX06-02
r5demo-01

KSTATE
ENABLED
ENABLED
ENABLED
ENABLED
ENABLED
ENABLED
ENABLED
ENABLED
ENABLED
ENABLED

LENGTH
61440
1920
1920
61440
12288
12288
12288
12288
12288
12288

PLOFFS
0
0
0
0
0
0
0

STATE
ACTIVE
LOG
ACTIVE
-

By default, the vxassist layout=raid5 specification adds a log to the


RAID-5 volume. Due to performance degradation, RAID-5 volume logs
should not reside on the same disk drives as the RAID-5 data.
To prevent the unintentional addition of a RAID-5 log, the vxassist
layout specification must be more specific.
# vxassist -g dgX relayout r5demo layout=raid5,nolog alloc="dgX06 ncol=6

6-46

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Online Volume Relayout

Volume Relayout Using the VEA GUI


To perform the same process as in the previous section, complete the
following steps:
1.

Display the RAID-5 volume in the grid area, and select Change
Layout from its pop-up menu.

2.

Complete the Change Volume Layout form, as shown in Figure 6-11.

Figure 6-11 VEA Change Volume Layout Form


In the preceding example, the volume Layout was set to RAID 5 and
the Number of Columns was increased to 6. Use the Show Options
button to enter additional relayout criteria, such as which disks
drives to use.

VERITAS Volume Manager Advanced Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-47

Online Volume Relayout

Relayout Status Monitor


After you fill out the Change Volume Layout form and start the relayout
process, a Relayout Status window, shown in Figure 6-12, appears. You
use the controls in the Relayout Status window to:

Temporarily stop the relayout process (pause)

Abort the relayout process

Continue the process after a pause

Undo the relayout changes (reverse)

The VEA Relayout Status Monitor window also displays the percentage
complete status.

Figure 6-12 VEA Relayout Status Monitor Window

Note The relayout task could fail if the target volume was not originally
created using the VEA GUI or the vxassist command.

6-48

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Creating Layered Volumes

Creating Layered Volumes


A layered volume is built on one or more other volumes. The underlying
volumes are mirrored. Layered volume are RAID 1+0 format and are
called striped mirror or concatenated mirror configurations.

Layered Volume Disk Requirements


By default, the mirrors in layered volumes are created with DRLs. For best
performance and reliability, all components of a layered volume should be
on different disk drives.
A striped mirror volume, shown in Figure 6-13, requires a minimum of
four disk drives to implement. Placing the DRLs on separate disk drives
increases the disk requirement to six.
Volume
Stripe

Log

Mirror

Stripe

Mirror

Log

Mirror

Mirror

Figure 6-13 Striped Mirror Disk Requirements


The disk requirements for a concatenated mirror volume structure, shown
in Figure 6-14, are the same as for striped mirror structures.
Volume
Subdisk

Mirror

Mirror

Log

Mirror

Mirror

Log

Subdisk

Figure 6-14 Concatenated Mirror Disk Requirements

VERITAS Volume Manager Advanced Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-49

Creating Layered Volumes

Evaluating Available Disk Space


There are several methods available for evaluating available disk space.
The vxdg free command is a good tool for performing a basic
evaluation. In the following example, each of the disk drives in the dgX
disk group have approximately 8 Gbytes of available space. To calculate
the available free space in Mbytes on a given disk drive, divide its LENGTH
value by 2048 (17674896/2048 = 8630.32 Mbytes = 8.63 Gbytes).
# vxdg -g dgX free
DISK
DEVICE
dgX01
c2t16d0s2
dgX02
c2t18d0s2
dgX03
c2t1d0s2
dgX04
c2t3d0s2
dgX05
c2t5d0s2
dgX06
c2t20d0s2

TAG
c2t16d0
c2t18d0
c2t1d0
c2t3d0
c2t5d0
c2t20d0

OFFSET
0
0
0
0
0
0

LENGTH
17674896
17674896
17674896
17674896
17674896
17674896

FLAGS
n
-

You can also use the vxassist maxsize option as follows to calculate
maximum available space for a specific structure. The example defaults to
two mirrors and logging.
# vxassist -g dgX maxsize \
layout=stripe-mirror ncolumn=2 \
dgX01 dgX02 dgX03 dgX04 dgX05 dgX06
Maximum volume size: 35348480 (17260Mb)

Note If you use the VEA GUI New Volume wizard to configure layered
volumes, you can use the Max Size button to estimate maximum available
space.

6-50

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Creating Layered Volumes

Creating Layered Volumes From the Command Line


If you create layered volumes using the vxassist command, you should
specify the layout, number of columns, and whether logging is to be
configured. The default is to not create DRLs.
An example of using the vxassist command to create a layered volume
follows.
# vxassist -g dgX make strmirr 500m \
layout=stripe-mirror ncolumn=2 \
dgX01 dgX02 dgX03 dgX04 dgX05 dgX06

Use the vxprint command as follows to display the layered volume


structure. Output not related to the volume is omitted for clarity.
# vxprint -g dgX
...
...
v strmirr
fsgen
pl strmirr-03
strmirr
sv strmirr-S01 strmirr-03
sv strmirr-S02 strmirr-03

ENABLED
ENABLED
ENABLED
ENABLED

1024000
1024000
512000
512000

0
0

ACTIVE
ACTIVE
-

v
pl
sd
pl
sd

strmirr-L01
strmirr-P01
dgX03-02
strmirr-P02
dgX05-02

fsgen
strmirr-L01
strmirr-P01
strmirr-L01
strmirr-P02

ENABLED
ENABLED
ENABLED
ENABLED
ENABLED

512000
512000
512000
512000
512000

0
0

ACTIVE
ACTIVE
ACTIVE
-

v
pl
sd
pl
sd

strmirr-L02
strmirr-P03
dgX04-02
strmirr-P04
dgX01-02

fsgen
strmirr-L02
strmirr-P03
strmirr-L02
strmirr-P04

ENABLED
ENABLED
ENABLED
ENABLED
ENABLED

512000
512000
512000
512000
512000

0
0

ACTIVE
ACTIVE
ACTIVE
-

Note Layered volume structures are difficult to understand in the


vxprint output format. Use the VEA GUI Layout View function to see a
visual representation of layered volume structures.

VERITAS Volume Manager Advanced Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-51

Creating Layered Volumes

Creating Layered Volumes Using the VEA GUI


Use the VEA GUI New Volume wizard, show in Figure 6-15, to configure
layered volumes. Select either Concatenated Mirrored or Striped Mirrored
in the Layout section.
By default, the New Volume wizard configures two columns, two mirrors
for each column, and enables logging (DRLs).

Figure 6-15 Creating Layered Volumes Using the VEA GUI

6-52

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Creating Layered Volumes

Identifying Layered Volume Subcomponents


The striped mirror volume components, shown in Figure 6-16, consist of
several layers. The lower levels of the layered volumes are ready-made
configurations designed to provide the highest level of availability
without increasing the administrative complexity.

Volume
vol01
10 GB

Striped Plex
vol01-03
10 GB
Col/Width:2x64

Subdisk
vol01-S01
Column: 0

Sub-Volume
vol01-L01
5 GB

Subdisk
vol01-S02
Column: 1

Sub-Volume
vol01-L02
5 GB

Sub-Plex
vol01-P01
5 GB

Subdisk
disk01-01
5 GB

Sub-Plex
vol01-P02
5 GB

Subdisk
disk02-01
5 GB

Sub-Plex
vol01-P03
5 GB

Subdisk
disk03-01
5 GB

Sub-Plex
vol01-P04
5 GB

Subdisk
disk04-01
5 GB

Figure 6-16 Striped Mirror Volume Structure


The concatenated mirror volume components shown in Figure 6-17 also
consist of several layers.

Volume
vol01
10.5 GB

Concat Plex
vol01-03
10.5 GB

Subdisk
vol01-S01
1.6 GB

Sub-Volume
vol01-L01
1.6 GB

Subdisk
vol01-S02
8.9 GB

Sub-Volume
vol01-L02
8.9 GB

Sub-Plex
vol01-P01
1.6 GB

Subdisk
disk01-01
1.6 GB

Sub-Plex
vol01-P02
1.6 GB

Subdisk
disk02-01
1.6 GB

Sub-Plex
vol01-P03
8.9 GB

Subdisk
disk03-01
8.9 GB

Sub-Plex
vol01-P04
8.9 GB

Subdisk
disk04-01
8.9 GB

Figure 6-17 Concatenated Mirror Volume Structure

VERITAS Volume Manager Advanced Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-53

Basic Intelligent Storage Provisioning Administration

Basic Intelligent Storage Provisioning Administration


The VxVM Intelligent Storage Provision (ISP) feature is specifically
targeted at managing large storage installations composed of a huge
number of hardware RAID LUNs or SAN devices.

Primary ISP Components


Traditional VxVM volume creation, using software such as the vxassist
command or the VEA GUI, relies on the administrator. The administrator
must analyze storage resources and configure volumes in a manner that
satisfies the required performance or reliability needs.
With the advent of hardware RAID LUN technology, such as the Sun
StorEdge 3510/6020/9910 models and related SAN technology, a system
administrator might be faced with analyzing thousands of devices whose
underlying characteristics and capabilities are hidden and unknown.
The ISP system provides a storage allocation engine that automatically
chooses which designated storage to use based on the capabilities you
specify when creating new ISP volumes (called application volumes).
The components, commands, volumes, and storage designated for ISP use
cannot be used by traditional VxVM commands such as vxassist,
vxdiskadm, and vxvol.
Basic ISP components include the following:

Data pools (only for data volumes)

Clone pools (only for snapshots)

Application volumes

Pre-defined templates

All ISP operations are performed using the vxvoladm command, the
vxpool command, or the VEA GUI. There are no other commands or
tools used to create ISP pools or application volumes.

6-54

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Basic Intelligent Storage Provisioning Administration

ISP Storage Pools


You configure both data pools and clone pools within a disk group.
A data storage pool is created within a standard disk group. One or more
LUNs from a disk group are assigned to a named storage pool. Any
subsequent storage pools created in the same disk group are
automatically defined as clone pools.
Clone pools are used only to hold full-sized instant snapshots of data pool
volumes in the same disk group. If the instant snapshot feature is not
licensed on your system, clone pools have no use.

ISP Application Volumes


Application volumes reside only in an ISP data storage pool. You create
application volumes using either the vxvoladm command or VEA GUI.

Predefined ISP Templates


You associate ISP templates with storage pools so that volumes created in
a storage pool are restrained by a fixed set of configuration rules. The
hierarchy of ISP templates is shown in Figure 6-18.
Storage Pool Template Sets (6)

Policies

2 Each
Storage Pool Templates (11)

autogrow
selfsufficient

5-15 Each
Volume Templates (21)
1-2 Each
Capabilities (25)
0-2 Each
Variables (8)

Figure 6-18 ISP Template Hierarchy

VERITAS Volume Manager Advanced Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-55

Basic Intelligent Storage Provisioning Administration

Using Storage Pool Set Templates


Storage pool set templates describe both data pool and clone pool
characteristics within a disk group. There are six pre-defined storage pool
sets. By default, the first pool created in a disk group is a data pool and all
subsequent pools created in a disk group are clone pools.
Each storage pool set provides two storage pool definitions. For example,
the storage pool set, mirrored_data_striped_clones, provides the
mirrored_volumes storage pool definition for the data pool and the
striped_volumes storage pool definition for the clone pool.

Creating Storage Pool Sets Using the VEA GUI


As shown in Figure 6-19, the VEA GUI Organize Disk Group Wizard
organizes existing disk groups using one of the pre-defined storage pool set
templates. Default data pool and clone pool names can be modified. You
assign disk group disks to the pools afterwards.

Figure 6-19 Selecting Storage Pool Sets

6-56

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Basic Intelligent Storage Provisioning Administration

Creating Storage Pool Sets Using the vxpool Command


The vxpool command associates a storage pool set template with an
existing disk group. In the following example, the storage pool set
template mirrored_data_striped_clones is associated with the dgSP
disk group.
# vxpool listpoolsets
mirrored_data_striped_clones
mirrored_prefab_raid5_data_mirrored_clones
mirrored_prefab_stripe_data_striped_clones
prefab_mirrored_data_prefab_striped_clones
stripe_mirrored_data_striped_clones
striped_prefab_mirrored_data_striped_clones
# vxpool -g dgSP organize mirrored_data_striped_clones
By default, the data and clone pool names are a variation of the pool
templates, mirrored_volumes and striped_volumes.
After initial pool creation, you use the vxpool command to associate disk
group media names with each pool. You can also modify the default
storage pool names. An example follows.
# vxpool -g dgSP adddisk mirrored_volumes1 \
dm=dgSP01,dgSP03,dgSP04,dgSP02,dgSP07,dgSP06
# vxpool -g dgSP adddisk striped_volumes1 \
dm=dgSP08,dgSP12,dgSP13,dgSP09,dgSP10,dgSP11
# vxpool -g newDG2
# vxpool -g newDG2

rename mirrored_volumes1 dp_01


rename striped_volumes1 cp_01

VERITAS Volume Manager Advanced Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-57

Basic Intelligent Storage Provisioning Administration

Using Storage Pool Templates


When first created, data pools or clone pools are associated with a single
storage pool template such as mirrored_volumes or raid5_volumes. Each
storage pool template references a list of predefined volume templates. You
can create storage pools without assigning a template, and later use the
vxpool command to associate a particular template.
Each storage pool template references 5-15 volume templates.
The following example shows the use of the vxpool command to create a
storage pool, assign disks to it, and associate a template with the pool.
The first pool created in a disk group is automatically a data pool.
# vxdisk -g dgX list
DEVICE
TYPE
c2t1d0s2
auto:cdsdisk
c2t3d0s2
auto:cdsdisk
c2t5d0s2
auto:cdsdisk
c2t16d0s2
auto:cdsdisk
c2t18d0s2
auto:cdsdisk
c3t32d0s2
auto:cdsdisk
c3t33d0s2
auto:cdsdisk
c3t35d0s2
auto:cdsdisk
c3t37d0s2
auto:cdsdisk
c3t50d0s2
auto:cdsdisk

DISK
dgX03
dgX04
dgX05
dgX01
dgX02
dgX06
dgX07
dgX08
dgX09
dgX10

GROUP
dgX
dgX
dgX
dgX
dgX
dgX
dgX
dgX
dgX
dgX

STATUS
online
online
online
online
online
online
online
online
online
online

# vxpool listpooldefinitions
any_volume_type
mirror_stripe_volumes
mirrored_prefab_raid5_volumes
mirrored_prefab_striped_volumes
mirrored_volumes
prefab_mirrored_volumes
prefab_raid5_volumes
prefab_striped_volumes
raid5_volumes
stripe_mirror_volumes
striped_prefab_mirrored_volumes
striped_volumes
# vxpool -g dgX create r5pool \
dm=dgX01,dgX02,dgX03,dgX06,dgX07,dgX08 \
pooldefinition=raid5_volumes

6-58

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Basic Intelligent Storage Provisioning Administration


# vxpool -g dgX list
r5pool

# vxpool -g dgX print


TY NAME
TYPE
st r5pool
data
dm dgX08
dm dgX07
dm dgX06
dm dgX03
dm dgX02
dm dgX01
-

AUTOGROW
diskgroup
-

SELFSUFFICIENT
pool
-

DESCRIPTION
Volume has parity
-

By default, the autogrow policy for pools is set to 2 (diskgroup). The pool
can be grown by bringing in additional storage from the disk group
outside of the storage pool.
By default, the selfsufficient policy is set to level 1 (pool). It only allows
the use of templates that have been manually assigned to the storage pool.
Storage pool attributes can be modified after initial pool creation. See the
vxpool man page for more details.
The vxprint command provides a method for evaluating current storage
pools. In the following example, a disk group named dgX, containing 10
disk drives, has a data pool named r5pool that uses 6 of the disk groups
drives.
# vxprint -g dgX
TY NAME
ASSOC
dg dgX
dgX

KSTATE
-

LENGTH
-

PLOFFS
-

STATE
ALLOC_SUP

dm
dm
dm
dm

dgX04
dgX05
dgX09
dgX10

c2t3d0s2
c2t5d0s2
c3t37d0s2
c3t50d0s2

17679776
17679776
17679776
17679776

st
dm
dm
dm
dm
dm
dm

r5pool
dgX01
dgX02
dgX03
dgX06
dgX07
dgX08

c2t16d0s2
c2t18d0s2
c2t1d0s2
c3t32d0s2
c3t33d0s2
c3t35d0s2

17679776
17679776
17679776
17679776
17679776
17679776

DATA
-

VERITAS Volume Manager Advanced Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-59

Basic Intelligent Storage Provisioning Administration

Using Application Volume Templates


There are currently 21 predefined application volume templates. Each
storage pool template references 5-15 volume templates.
Each predefined volume template references one or two volume
capabilities. For instance, the storage pool template raid5_volumes refers
to a set of ten volume templates.
Most of the volume templates contain the name of a single capability.
Many of the capabilities use the same name as the volume templates. The
capabilities define one or more variables that can be used when creating
application volumes. Many of the variables have default values. The
following are capabilities for the Raid5Volume template.
Volume Template: Raid5Volume
Provides capabilities: Raid5Capability, Raid5LogMirroring
Variables:

ncols

Minimum number of columns

nlogs

Number of logs

nmaxcols

Maximum number of columns

The Raid5Capability capability provides the ncols and nmaxcols


variables which define the minimum and maximum number of columns
to create for a RAID-5 volume. The default value for ncols is 8 and for
nmaxcols is 20.
The Raid5LogMirroring capability provides the nlogs variable which
defines the number of logs to create for a RAID-5 volume. The default
value is 1.
Another volume template named LogsOnSeparateComponents has the
following capabilities:
Volume Template: LogsOnSeparateComponents
Provides capabilities: LogsOnSeparateComponents
Variables:

component Name of component

The component variable can be set to Controller or Enclosure to


separate multiple logs. The default is Enclosure.

6-60

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Basic Intelligent Storage Provisioning Administration

Deriving Volume Template Capabilities


Table 6-2 shows capabilities resulting from applying the
mirrored_volumes storage pool template to a disk group. Many of the
variables have default values.
Table 6-2

Volume Capability Derivation


Storage Pool: mirrored_volumes

Associate Volume
Templates

Capability
Name

ArrayProductId

Same name

Volume uses storage with the same


productid

ConfineLogs
ToSimilarStorage

Same name

Logs confined to same storage by


variable: name
default: Enclosure

ConfineMirrors
ToSimilarStorage

Same name

Mirrors confined to same storage by


variable: name default: Enclosure

Confine
ToSimilarStorage

Same name

Volumes confined to same storage by


variable: name default: VendorName

Confine
ToSpecificStorage

Same name

Volume confined to specific storage by


variable: name (no default)
variable: value (no default)

DataMirroring

Same name

Number of mirror set by


variable: nmirs default: 2

DCOLogMirroring

Same name

Number of DCO log mirrors set by


variable: nlogs default:1

InstantSnapshottable

Same name

This volume supports instant snapshots.

LogsOn
SeparateComponents

Same name

Separate volume logs by component.


variable: component default:
Enclosure

MirrorsOn
SeparateComponents

Same name

Separate volume mirrors by component.


variable: component default:Enclosure

MultipathingThrough
MultiplePaths

Same name

Number of paths that can fail.


variable: npaths default=2

Capability Variables

VERITAS Volume Manager Advanced Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-61

Basic Intelligent Storage Provisioning Administration

Creating Application Volumes Using the vxvoladm


Command
You use the vxvoladm command to create application volumes. The
following example shows the use of the vxvoladm command to create a
default RAID-5 volume in a storage pool named r5pool.
# vxvoladm -g dgX make r5vol 10m volume_template=Raid5Volume
# vxprint -g dgX r5vol
TY NAME
ASSOC
v r5vol
raid5
pl r5vol-01
r5vol
sd dgX03-01
r5vol-01
sd dgX05-01
r5vol-01
sd dgX04-01
r5vol-01
sd dgX02-01
r5vol-01
sd dgX06-01
r5vol-01
sd dgX08-01
r5vol-01
sd dgX07-01
r5vol-01
sd dgX09-01
r5vol-01
sd dgX10-01
r5vol-01
pl r5vol-02
r5vol
sd dgX01-01
r5vol-02

KSTATE
ENABLED
ENABLED
ENABLED
ENABLED
ENABLED
ENABLED
ENABLED
ENABLED
ENABLED
ENABLED
ENABLED
ENABLED
ENABLED

LENGTH
20480
20480
2560
2560
2560
2560
2560
2560
2560
2560
2560
8640
8640

PLOFFS
0
0
0
0
0
0
0
0
0
0

STATE
ACTIVE
ACTIVE
LOG
-

The initial results are unexpected because the autogrow policy allowed
the use of disk group disks outside of the pool and the default for the
nmaxcols variable is 20. The result is a 9 column RAID-5 volume that
uses all of the disk group disks, even those outside of the defined pool.
# vxvoladm -g dgX remove volume r5vol
When the r5vol volume is removed, the extra disks are automatically
removed from the r5pool.
# vxpool -g dgX getpolicy r5pool
AUTOGROW
SELFSUFFICIENT
diskgroup
pool
# vxpool -g dgX setpolicy r5pool autogrow=pool
# vxvoladm -g dgX make r5vol 10m volume_template=Raid5Volume \
capability=Raid5Capability(nmaxcols=4)

6-62

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Basic Intelligent Storage Provisioning Administration

Creating Application Volumes Using the VEA GUI


You create application volume using the VEA GUI by clicking a disk
group that has storage pools configured and selecting New Volume from
its pop-up menu.
When you create a new volume in a disk group that contains configured
storage pools, the New Volume Wizard is aware that a storage pool exists.
It automatically displays all possible volume configuration capabilities
based on templates associated with the data pool in the disk group.
Initially, none of the capabilities are enabled. If you do not enable any of
the capabilities, the completed volume is a simple concatenation.

Selecting Volume Capabilities


As shown in Figure 6-20, some capabilities display variables, such as
number of mirrors, that you can modify if needed.

Figure 6-20 Selecting Volume Capabilities

VERITAS Volume Manager Advanced Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-63

Basic Intelligent Storage Provisioning Administration

Saving Custom Volume Templates


The next application volume window display, shown in Figure 6-21,
summarizes your capability choices and offers the opportunity to save the
capability configuration as a custom volume template.
If you save a custom template, the next time you create a volume, you are
offered the opportunity to use your custom template and bypass the
manual capability process.

Figure 6-21 Saving Custom Volume Templates

6-64

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Basic Intelligent Storage Provisioning Administration

Interpreting Application Volume Configurations


As shown in the following example, disk group disks, pools, and their
related application volumes are grouped together by the vxprint
command.
# vxprint -g dgSP
TY NAME
ASSOC
dg dgSP
dgSP

KSTATE
-

LENGTH
-

dm dgSP05

c2t22d0s2

17679776 -

st
dm
dm
dm
dm
dm
dm

mirrored_volumes1 dgSP01
c2t16d0s2
dgSP02
c2t18d0s2
dgSP03
c2t1d0s2
dgSP04
c2t20d0s2
dgSP06
c2t3d0s2
dgSP07
c2t5d0s2

17679776
17679776
17679776
17679776
17679776
17679776

DATA
-

v
pl
sd
pl
sd

datapool_vol01 fsgen
ENABLED 2097152 datapool_vol01-01 datapool_vol01 ENABLED 2097152 dgSP04-01
datapool_vol01-01 ENABLED 2097152 0
datapool_vol01-02 datapool_vol01 ENABLED 2097152 dgSP01-01
datapool_vol01-02 ENABLED 2097152 0

ACTIVE
ACTIVE
ACTIVE
-

st
dm
dm
dm
dm
dm
dm

striped_volumes1 dgSP08
c3t32d0s2
dgSP09
c3t33d0s2
dgSP10
c3t35d0s2
dgSP11
c3t37d0s2
dgSP12
c3t50d0s2
dgSP13
c3t52d0s2

CLONE
-

17679776
17679776
17679776
17679776
17679776
17679776

PLOFFS
-

STATE
ALLOC_SUP

VERITAS Volume Manager Advanced Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-65

Replacing Failed Disk Drives

Replacing Failed Disk Drives


One of the most common VxVM errors that you might see is detached
plex messages. This error message usually means that one of the mirrors
in a volume has encountered a disk drive error while being accessed.
You must identify the physical path to the failed disk drive before you
proceed. The most common tools you use to do this are:

The vxprint command

The vxdisk command

The /var/adm/messages file

Failure Behavior
A plex (mirror) is detached if a persistent I/O error is encountered. There
are several things to be aware of before proceeding:

Disk block read errors might affect one subdisk, while other subdisks
on the same physical disk drive remain functional.

Errors are not detected until read or write operations are attempted.

Severe disk drive errors, such as general access failures, result in


relocation of all redundant subdisks associated with the failed disk.
They are relocated to either designated hot spare disks or to any disk
that does not have the nohotuse flag set.

Hardware RAID Behavior


Hardware RAID storage units, such as the Sun StorEdge T3 array, present
LUNs to attached systems. Each LUN is actually a portion of a hardware
RAID structure that is monitored internally in the storage array for disk
failures. Typically, the internal RAID volumes are redundant, such as
RAID 5 or RAID 1, and the storage array internally relocates the failing
data to a designated spare drive. Hardware RAID internal failures are
usually transparent to the VxVM software. Hardware RAID storage
devices usually notify the user root mail account of internally detected
problems.

6-66

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Replacing Failed Disk Drives

Evaluating Failure Severity


By default, the VxVM software sends failure notification email to the user
root account. The extent of failures must be evaluated in a timely
manner.
It is important to assess failures in a timely manner before a problem
escalates to a much worse situation. The first line of defence is to examine
system error logs and verify volume status.

Use root Email Notification


Typically, a series of root account email error notifications follow a device
failure. After the initial error notification, several more message are
usually sent as VxVM attempts to relocate failed volume components. The
following example shows an initial error notification.
# mail
?
From root@ns-east-104.East.Sun.COM Fri Nov 7 21:11:59 2003
Date: Fri, 7 Nov 2003 21:11:58 -0500 (EST)
From: Super-User <root@ns-east-104.East.Sun.COM>
Message-Id: <200311080211.hA82BwQX027415@ns-east-104.East.Sun.COM>
To: root@ns-east-104.East.Sun.COM
Subject: Volume Manager failures on host ns-east-104
Content-Length: 239
Failures have been detected by the VERITAS Volume Manager:
failed disks:
dgX04
failed plexes:
mirvol_01-02
The Volume Manager will attempt to find spare disks, relocate failed
subdisks and then recover the data in the failed plexes.

VERITAS Volume Manager Advanced Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-67

Replacing Failed Disk Drives

Examining Errors in the /var/adm/messages File


The following messages are seen when a disk drive is experiencing access
problems, such as hard write errors.
Nov 7 21:11:40 ns-east-104 vxio: [ID 245403 kern.warning] WARNING: VxVM
vxio V-5-0-151 error on Plex mirvol_01-02 while writing volume mirvol_01
offset 16 length 4
Nov 7 21:11:40 ns-east-104 vxio: [ID 786473 kern.warning] WARNING: VxVM
vxio V-5-0-4 Plex mirvol_01-02 detached from volume mirvol_01
Nov 7 21:11:40 ns-east-104 vxio: [ID 628984 kern.warning] WARNING: VxVM
vxio V-5-0-386 dgX04-01 Subdisk failed in plex mirvol_01-02 in vol
mirvol_01
Nov 7 21:11:40 ns-east-104 vxvm:vxconfigd: [ID 976563 daemon.notice] V5-1-768 Offlining config copy 1 on disk c2t3d0s2:
Nov 7 21:11:40 ns-east-104 vxvm:vxconfigd: [ID 672837 daemon.notice]
Reason: Disk write failure
Nov 7 21:11:41 ns-east-104 vxvm:vxconfigd: [ID 905431 daemon.notice] V5-1-7909 Detached disk dgX04

Checking Volume Status


The vxprint command is the easiest way to check the status of all volume
structures. In the following excerpt, the status of two plexes in a volume is
bad. One of the plexes is a log.
# vxprint
Disk group: sdg0
TY NAME
dg sdg0

ASSOC
sdg0

KSTATE
-

LENGTH
-

PLOFFS
-

STATE
-

dm disk0
dm disk7

c4t0d0s2
c5t0d0s2

8368512
8368512

v
pl
sd
pl
sd
pl
sd

fsgen
vol0
vol0-01
vol0
vol0-02
vol0
vol0-03

ENABLED
DISABLED
ENABLED
ENABLED
ENABLED
DISABLED
ENABLED

524288
525141
525141
525141
525141
LOGONLY
5

0
0
LOG

ACTIVE
IOFAIL
ACTIVE
IOFAIL
-

6-68

vol0
vol0-01
disk0-01
vol0-02
disk7-01
vol0-03
disk0-02

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Replacing Failed Disk Drives

Note If a hot spare is available and the volume is redundant, the


structure is relocated to a different disk drive and you never see such
volume states.
When VxVM detects a disk drive failure, it can place a failed plex in a
number of different states. The two most common states for a failed plex
are:

DETACHED/IOFAIL

DISABLED/NODEVICE

The DETACHED/IOFAIL state is not so severe. It is relatively easy to


identify the failed device path by examining the vxprint -ht command
output as follows:
# vxprint -ht | grep sd0b-01
sd sd0b-01 lv0b-01 disk7 0

525141

c5t0d0

ENA

In the previous example, the VxVM media name is disk7 and the
physical path is c5t0d0.
When the VxVM software loses complete contact with a disk drive, the
physical path in the vxprint -ht commands output might be blank. At
those times, you must determine the media name of the failed disk drive
from the vxprint command, and then use the vxdisk list command to
associate the media name with the physical device.
# vxdisk list
DEVICE
TYPE
c0t0d0s2
auto:sliced
c0t1d0s2
auto:sliced
c2t1d0s2
auto:sliced
c2t3d0s2
auto:sliced
c2t5d0s2
auto:sliced
c2t16d0s2
auto:sliced
c2t18d0s2
auto:sliced
c2t20d0s2
auto:sliced
dgX04

DISK
rootdg01
rootdg02
dgX03
dgX05
dgX01
dgX02
dgX06
dgX

GROUP
STATUS
rootdg
online
rootdg
online nohotuse
dgX
online
online
dgX
online
dgX
online
dgX
online
dgX
online
failed nohotuse was:c2t3d0s2

When a disk drive fails and becomes detached, the VxVM software cannot
find the disk drive, but it still knows the physical path. This information is
the origin of the failed was status. This status means that the disk drive
has failed and that the physical path is the value displayed in the STATUS
column.

VERITAS Volume Manager Advanced Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-69

Replacing Failed Disk Drives

General Disk Drive Replacement Process


When a disk drive containing redundant volumes fails, usually all of the
subdisks are relocated to a hot spare and the redundant volume continues
to function normally. Subdisk relocation messages are sent to the user
root email account.
The VxVM software requires that a specific recovery process is followed
to replace the failed disk drive, and then relocate the subdisks back to the
new disk drive.
Some storage arrays also require that specific disk replacement processes
are followed.
Disk drive failures in hardware RAID storage, such as the Sun StorEdge
T3 arrays, are frequently transparent to VxVM. This is because their
internal LUN structures are redundant and an internal spare is
automatically substituted for the failed drive.
The overall replacement process is typically a mixture of software and
hardware replacement procedures. Following is a summary of the
process.
1.

If appropriate, try to resolve temporary problems before taking more


complex action.
If the disk drive is detached due to a secondary problem, such as a
loose cable, you can try to reattach it using the vxreattach
command.
Reattachment might resolve all problems. If you feel the disk is still
at risk, you can evacuate it and perform a hardware replacement.

2.

Remove the failed disk drive from VxVM control.


Use the vxdiskadm utility option 4, Remove a disk for
replacement, to remove the physical disk from its disk group while
retaining its media name.
If you intend to hot-swap the failed disk without rebooting the
system, you might also use the vxdiskadm utility option 11, Disable
(offline) a disk device, to stop all VxVM access, such as
polling.

3.

Perform the storage-specific hardware replacement procedure.


The hardware replacement procedures vary widely depending on
the storage type and model. Always consult the technical manuals
for your particular array model.

6-70

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Replacing Failed Disk Drives


4.

Return the failed disk drive to VxVM control.


Use the vxdiskadm utility option 10, Enable (online) a disk
device, to enable VxVM access to the replacement disk.
Use the vxdiskadm utility option 5, Replace a failed or
removed disk, to associate the new disk with its media name. You
initially select None as a replacement device. You are then asked if
you want to initialize the new disk drive.

Note In some cases you might need to scan for new disk drives using
either the vxdiskconfig or vxdctl enable command.
5.

Use the vxdiskadm utility option 14, Unrelocate subdisks back


to a disk, to move the relocated volume components back to the
original disk location.
The only information you furnish is the media name of the
replacement disk, VxVM.

Note If a failed volume is non-redundant, you cannot recover it. You


must manually recreate the volume structure, using tools such as
vxconfigrestore, and restore the data from backup tapes.

VERITAS Volume Manager Advanced Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-71

Exercise: Performing Advanced Operations

Exercise: Performing Advanced Operations


In this exercise, you complete the following tasks:

Review key lecture points

Encapsulate the system boot disk

Mirror the system boot disk

Perform an online volume relayout

Evacuate a disk drive

Move a populated volume

Perform a snapshot backup

Create a layered volume

Replace a failed disk drive

Use Intelligent Storage Provisioning

Configure a best practice boot disk

Preparation
If your lab environment uses a central VxVM server instead of standalone
workstations, the Encapsulating the System Boot Disk section on page
6-4 and the Mirroring the System Boot Disk section on page 6-5 must be
performed as a demonstration on the VxVM server.
The demonstration is typically performed by the instructor. The tasks
require a second internal disk drive that is the same model and capacity
as the system boot disk.

6-72

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Performing Advanced Operations

Task 1 Reviewing Key Lecture Points


Answer the following questions about boot disk mirroring, hot devices,
and layered volumes.
1.

Which of the following are features of a preferred boot disk


configuration?
a.

The boot disk should be located in a storage array.

b.

The boot disk should be mirrored.

c.

The boot disk should not be part of the rootdg disk group.

d.

The boot disk should have at least two unused slices.

e.

The boot disk should be at least 18 Gbytes in size.

The answers are b and d.

2.

Which answer most accurately describes the best practice boot disk
configuration process?
a.

Initialize, copy, mirror, delete

b.

Encapsulate, mirror, delete, copy

c.

Copy, delete, encapsulate, mirror

d.

Mirror, encapsulate, initialize, copy

The answer is b.

3.

What is the primary benefit of a best-practice boot disk


configuration?
a.

It improves boot disk performance

b.

It standardizes recovery procedures

c.

It simplifies boot disk prerequisites

The answer is b.

4.

What is the key feature of hot relocation?


a.

Whole disk drive relocation

b.

Cylinder group relocation

c.

Subdisk relocation

d.

Sub-volume relocation

The answer is c.

VERITAS Volume Manager Advanced Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-73

Exercise: Performing Advanced Operations


5.

Can designated hot spares be used for hot relocation?


a.

Yes

b.

No

The answer is a.

6.

What is a key prerequisite for both hot spares and hot relocation?
a.

Volumes must be striped

b.

Volumes must be failure tolerant

c.

Volumes must be mirrored

d.

Volumes must be striped mirror

The answer is b.

7.

Can a designated hot spare be used for typical volume creation?


a.

Yes

b.

No

The answer is b.

8.

What is the default hot-device mode of operation?


a.

Hot relocation

b.

Hot spare

The answer is a.

9.

Which of the following commands performs hot-device


administration?
a.

vxdisk set

b.

vxtask set

c.

vxedit set

d.

vxdisksetup

The answer is c.

6-74

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Performing Advanced Operations


10. Which of the following commands starts a layered volume?
a.

vxvol

b.

vxrecover

c.

vxdctl

d.

vxdiskadm

The answer is b.

11. What is the primary purpose of the ISP software?


a.

To create complex volume structures

b.

To increase overall system performance

c.

To manage complex SAN environments

The answer is c.

12. Which of the following commands is used to create ISP application


volumes?
a.

vxvol

b.

vxvoladm

c.

vxassist

d.

vxdiskadm

The answer is b.

VERITAS Volume Manager Advanced Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-75

Exercise: Performing Advanced Operations

Task 2 Encapsulating the System Boot Disk


If your lab configuration uses a central VxVM server, this task must be
performed as a demonstration (probably by the instructor).
1.

Before proceeding with this task, ensure that your boot disk meets
the following requirements:

The boot disk must have at least two unused slices.

Ideally, the boot disk does not have any slices in use other than
the root and swap partitions.

There should be a minimum of 2048 sectors (blocks) at the


beginning or end of the boot disk that are not assigned to a
partition.
A standard lab boot disk should either have one or two
unassigned cylinders or have them assigned to an unused
partition, usually slice 7. If necessary, use the format utility to
zero out the small partition at the end of the boot disk. Do not
proceed until your boot disk meets this requirement.

2.

Start the vxdiskadm utility, select option 2, and answer the


preliminary questions as follows. Substitute the logical path to your
disk drive.

# vxdiskadm
Select an operation to perform: 2
Encapsulate one or more disks
Menu: VolumeManager/Disk/Encapsulate

Select disk devices to encapsulate:


[<pattern-list>,all,list,q,?] c0t0d0
Which disk group [<group>,list,q,?] rootdg
There is no active disk group named rootdg.
Create a new group named rootdg? [y,n,q,?] (default: y) y
Use a default disk name for the disk? [y,n,q,?] (default: y) y
A new disk group rootdg will be created and the disk device c0t0d0 will
be encapsulated and added to the disk group with the disk name rootdg01
Enter desired private region length
[<privlen>,q,?] (default: 2048) 2048

6-76

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Performing Advanced Operations


3.

Reboot the Solaris OS.


# init 0
ok boot

4.

Observe the following system messages as the encapsulation process


proceeds through reconfiguration reboots.

VxVM vxvm-startup2 INFO V-5-2-503 VxVM general startup...


VxVM vxvm-reconfig INFO V-5-2-324 The Volume Manager is now
reconfiguring (partition phase)...
VxVM vxvm-reconfig INFO V-5-2-499 Volume Manager: Partitioning c0t0d0
as an encapsulated disk.
/dev/dsk/c0t0d0s1 was dump device -invoking dumpadm(1M) -d swap to select new dump device
VxVM vxvm-reconfig INFO V-5-2-323 The Volume Manager is now
reconfiguring (initialization phase)...
VxVM vxvm-reconfig INFO V-5-2-497 Volume Manager: Adding rootdg01
(c0t0d0) as an encapsulated disk.
VxVM vxcap-vol INFO V-5-2-89 Adding volumes for c0t0d0.
..
Starting new volumes...
VxVM vxcap-vol INFO V-5-2-444 Updating /etc/vfstab...
Remove encapsulated partitions...
VxVM vxroot INFO V-5-2-328 The Volume Manager will now set up your Boot
Disk as a managed disk.
VxVM vxroot INFO V-5-2-290 Saving original configuration...
80 blocks
VxVM vxvm-reconfig NOTICE V-5-2-393 The system will now be rebooted.
syncing file systems... done
rebooting...
Rebooting with command: boot
...
VxVM INFO V-5-2-3247 starting special volumes ( swapvol rootvol )...
VxVM vxvm-startup2 INFO V-5-2-503 VxVM general startup...
vxvm: NOTE: Setting partition /dev/dsk/c0t0d0s1 as the dump device.

VERITAS Volume Manager Advanced Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-77

Exercise: Performing Advanced Operations


5.

Log into the system as user root and verify the boot disk
environment is correctly configured.
# vxprint
# eeprom nvramrc
# eeprom use-nvramrc?=true
# vxdg defaultdg
nodg
# vxdg bootdg
rootdg

Task 3 Mirroring the System Boot Disk


Perform the following steps only if you have already encapsulated your
boot disk and have a disk drive available that is identical to the primary
system boot disk.
In a production system, it would be best to take the system offline before
starting a procedure such as the following. Single-user mode would be
ideal.
Caution This procedure assumes two identical disk drives. The primary
boot disk address is c0t0d0, and the mirror disk is c0t1d0. Ensure that
you are using the correct address or disk media name for each step in this
procedure.
1.

Locate an appropriate mirror disk and add it to the rootdg disk


group using the VxVM medianame rootdg02.
# vxdisk list
# vxdisksetup -i c0t1d0
# vxdg -g rootdg adddisk rootdg02=c0t1d0

2.

Start the vxdiskadm utility and select option 6, Mirror Volumes on


a disk.

# vxdiskadm
Select an operation to perform: 6
Mirror volumes on a disk
At the prompt below, supply the name of the disk containing the volumes
to be mirrored.
Enter disk name [<disk>,list,q,?] list
Disk group: rootdg

6-78

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Performing Advanced Operations


DM NAME
dm rootdg01
dm rootdg02

DEVICE
c0t0d0s2
c0t1d0s2

TYPE
auto
auto

PRIVLEN
2048
4063

PUBLEN
8380799
8374320

STATE
-

Enter disk name [<disk>,list,q,?] rootdg01


Enter destination disk [<disk>,list,q,?] (default: any) rootdg02
VxVM vxmirror INFO V-5-2-22
Mirror volume rootvol ...
VxVM vxmirror INFO V-5-2-22
Mirror volume swapvol ...
3.

Use the vxprint command to examine your completed boot disk


configuration.

# vxprint -g rootdg
ns-east-104# vxprint -g rootdg
TY NAME
ASSOC
KSTATE
dg rootdg
rootdg
-

LENGTH
-

dm rootdg01
dm rootdg02

c0t0d0s2
c0t1d0s2

17674902 17678493 -

NOHOTUSE

v
pl
sd
sd
pl
sd

rootvol
rootvol-01
rootdg01-B0
rootdg01-02
rootvol-02
rootdg02-01

root
rootvol
rootvol-01
rootvol-01
rootvol
rootvol-02

ENABLED
ENABLED
ENABLED
ENABLED
ENABLED
ENABLED

17139843
17139843
1
17139842
17139843
17139843

0
1
0

ACTIVE
ACTIVE
ACTIVE
-

v
pl
sd
pl
sd

swapvol
swapvol-01
rootdg01-01
swapvol-02
rootdg02-02

swap
swapvol
swapvol-01
swapvol
swapvol-02

ENABLED
ENABLED
ENABLED
ENABLED
ENABLED

531468
531468
531468
531468
531468

0
0

ACTIVE
ACTIVE
ACTIVE
-

4.

PLOFFS
-

STATE
-

Examine the OpenBoot PROM boot device aliases and verify that an
alias for the boot disk mirror was added.
# eeprom nvramrc
nvramrc=devalias vx-rootdg01
/pci@1f,4000/scsi@3/disk@0,0:a
devalias vx-rootdg02
/pci@1f,4000/scsi@3/disk@1,0:a

5.

Halt the Solaris OS and boot from the VxVM boot disk device alias.
# init 0
...
ok boot vx-rootdg01

VERITAS Volume Manager Advanced Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-79

Exercise: Performing Advanced Operations

Task 4 Performing an Online Volume Relayout


In this task, you use the volume relayout feature of the VEA GUI to add
two columns to an existing 3-column RAID-5 volume. This task can also
be accomplished using the command line, but the syntax is complex.
It is best to unmount the volume file system before performing an online
relayout operation.
To perform a volume relayout, complete the following steps:
1.

Click the third mouse button on your RAID-5 volume in the VEA
GUI grid area and select Change Layout from the pop-up menu.
Let the VxVM software select additional disk drives.

2.

In the Change Volume Layout form, change the Columns entry to 5


(as shown in Figure 6-22) and click OK.

Figure 6-22 Change Volume Layout Form

6-80

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Performing Advanced Operations


3.

When the status window appears, as shown in Figure 6-23, try


pausing and continuing the relayout operation.

Figure 6-23 Relayout Status Monitor


4.

After the relayout has completed, examine the new volume structure
using the vxprint command and verify that the results are what
you anticipated.

5.

Unmount the RAID-5 volume and delete it.

6.

Create a two-disk mirrored volume, 200 Mbytes in size, with a


mounted file system.

7.

Use the mkfile command to create some test files in the mirrorvolumes file system.
# mkfile 10m /Test/file1
# mkfile 20m /Test/file2

VERITAS Volume Manager Advanced Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-81

Exercise: Performing Advanced Operations

Task 5 Evacuating a Disk Drive


To evacuate a disk drive, complete the following steps:
1.

Start the vxdiskadm utility on the VxVM server, and select option 7,
Move volumes from a disk.

2.

Move the contents of one of the disk drives in your new mirrored
volume to another unused disk drive in your disk group.

Caution If you do not specify a destination disk drive for the move,
VxVM uses any available disk space in the disk group. This might result
in a poorly configured volume with performance problems.

Task 6 Moving a Populated Volume


In this task, you move your mirrored file system volume into another
existing disk group. You must first create a second disk group. Complete
the following steps:
1.

On the VxVM server, use the vxprint command to determine the


media names and logical paths of the two disk drives being used in
your mirrored file system.
# vxprint -g old_dg -ht

2.

Record the media names and logical paths of the two disk drives
being used in your mirrored volume. For example: dgB01, c3t4d0.
Media name: __________

__________

Logical path: __________

__________

3.

Remove an unused disk drive from your current disk group and use
it to create a second disk group.

4.

Use the vxprint command to save the mirrored volume


configuration.
# vxprint -hmQq -g old_dg volumename > volumename.save

Note Do not locate the volumename.save file on a disk drive that is


being relocated.
5.

Unmount any file systems associated with the mirrored volume.

6.

Stop the mirrored volume.


# vxvol -g old_dg stop volumename

6-82

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Performing Advanced Operations


7.

Remove the definitions of the structures (volume, plexes, and


subdisks) from the configuration database.
# vxedit -g old_dg -r rm volumename

8.

Remove the disk drives from the original disk group.


# vxdg -g old_dg rmdisk olddg01 olddg02

Note The disk drives should now be in the free disk pool.
9.

Add the two disk drives to the new disk group.

10. Rename the disk drives so that they have their original media names.
Caution Step 10 is critical. If the disk drives do not have their original
media names, the configuration reload fails.
11. Use the vxmake command to reload the saved configuration for the
volumename volume.
# vxmake -g new_dg -d volumename.save
The -d option specifies the description file to use for building
subdisks, plexes, and volumes.
12. Use the vxvol command to bring the mirrored volume back online.
# vxvol -g new_dg init active volumename
13. Mount the mirrored volume file system to return the mirrored
volume to service.
14. Unmount the mirrored volume and destroy its disk group.
15. Add all of your disk drives into a single disk group again.

VERITAS Volume Manager Advanced Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-83

Exercise: Performing Advanced Operations

Task 7 Performing a Snapshot Backup


To perform a snapshot backup, complete the following steps:
1.

Create a two-disk mirrored volume, 200 Mbytes in size, with a


mounted file system.

2.

Use the mkfile command to create some test files in the mirrorvolumes file system.
# mkfile 10m /Test/file1
# mkfile 20m /Test/file2

3.

Click the mirrored volume in the VEA GUI grid area, and select
Snapshot Interactive from the pop-up menu.

Note Although it is not absolutely necessary, you can assign disk drives
for temporary use in the Volume Snapshot form.
4.

Click Snapstart on the Volume Snapshot form to start the snapshot


process.
This can take some time depending on the volume size.

5.

Click Snapshot when the mirror copy is completed.


This detaches the new mirror and creates a separate volume
from it.

6.
v SNAP-vol02
pl vol02-02
sd dgY07-01

Use the vxprint command to verify that a volume structure similar


to the following is present:
fsgen
SNAP-vol02
vol02-02

7.

204800
208278
208278

ACTIVE
ACTIVE
-

fsck -y /dev/vx/rdsk/dgY/SNAP-vol02
mkdir /vol02_backup_081202
mount /dev/vx/dsk/dgY/SNAP-vol02 /vol02_backup_081202
cd /
tar cvf /dev/rmt/0 ./vol02_backup_081202

Unmount and delete the snapshot volume.


# umount /vol02_backup_081202
# vxedit -g dgY -rf rm SNAP-vol02

6-84

Back up the new snapshot volume to tape (if possible). The following
example shows the process.
#
#
#
#
#

8.

ENABLED
ENABLED
ENABLED

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Performing Advanced Operations

Task 8 Creating a Layered Volume


To perform this task, you need a minimum of five unused disk drives in a
disk group.
To create a striped-mirror RAID 1+0 volume, complete the following
steps:
1.

Unmount all volume-related file systems.

2.

Stop all volumes in your disk group and delete them.

3.

Display the disk drives in your disk group in the VEA GUI grid area,
and select all six of them by simultaneously clicking the mouse
button and pressing the Control key.

4.

Click New Volume in the toolbar.

5.

Fill out the New Volume Wizard form as follows:

6.

a.

Enter a volume name, such as strmir_vol01.

b.

Enter a size of 200m or larger.

c.

Select the Stripe Mirrored layout.

d.

Leave the number of columns at 2.

e.

Leave the number of mirrors at 2.

f.

Leave logging enabled

g.

Add a file system that mounts at boot time.

Click Next and Finish in the next two forms.


It can take quite a while to complete a large layered volume. Keep
checking the status of the volume completion either from the
command line or by using the Task Monitor.

7.

Use the vxprint command to verify that your RAID 1+0 volume is
structured using sub-volume components.

8.

Click the layered volume in the VEA GUI grid area and select
Layout View from its pop-up menu.

9.

Resize the Volume Layout window until you can see all of the
layered volume components.

10. Close the Volume Layout window when you finish examining the
volumes structure.
11. Test your new volumes file system by copying data into it.

VERITAS Volume Manager Advanced Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-85

Exercise: Performing Advanced Operations

Task 9 Replacing a Failed Disk Drive


You can decided whether or not to perform this procedure as a group demonstration.

To simulate a failed disk drive replacement, complete the following steps:


1.

Unmount and delete your RAID 1+0 volume.

Caution Before proceeding, ensure that all of the disks in the disk group
are initialized in a sliced format. If the disks are in a cdsdisk format, the
failure simulation will not work. If necessary, destroy the disk group and
recreate it using the vxdiskadm utility.
2.

Create a two-disk mirrored volume, 200 Mbytes in size, with a


mounted file system, with logging disabled (no DRLs).

3.

Use the mkfile command to create a test file in the mirrored


volumes file system.
# mkfile 10m /Test/file1

4.

Double-click on the volume in the VEA GUI object tree to display


detailed information about the volumes disk drives.

5.

Complete the following steps:


a.

Simulate a disk drive failure by zeroing-out the VTOC of one of


the mirrored volume disks.

b.

Substitute the physical path to one of your mirrored volume


disks.

# fmthard -s /dev/null /dev/rdsk/c2t3d0s2


6.

Create a second test file in your mirrored volumes file system.


# mkfile 20m /Test/file2
You should see some plex-related errors.

7.

Complete the following steps:


a.

Type mail on the VxVM server and press the Return key to
view each new message.

b.

Type q when you are done.


Unless you disabled hot-relocation use on all of your disk drive,
VxVM relocates the mirror on the failed disk drive to a different
disk drive.

6-86

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Performing Advanced Operations


8.

Use the vxprint command to search for disabled plexes.


The mirrored volume should show a normal status because the
failed plex was relocated to a different disk drive.

9.

Complete the following steps:


a.

Use the vxdisk list command to verify the failed disk drive.

b.

Record the logical path to the failed disk drive.

Failed disk drive logical path: __________________________________


10. Remove the failed disk drive from VxVM control.
11. Use the vxdiskadm utility option 4, Remove a disk for
replacement, to remove the physical disk from its disk group while
retaining its media name. Do not select a replacement disk.
Note If you intend to hot-swap the failed disk without rebooting the
system, you might also use the vxdiskadm utility option 11, Disable
(offline) a disk device, to stop all VxVM access, such as polling.
12. Perform the storage-specific hardware replacement procedure.
The hardware replacement procedures vary widely depending on
the storage type and model. Always consult the technical manuals
for your particular array model.
13. Return the failed disk drive to VxVM control.
a.

Use the vxdiskadm utility option 10, Enable (online) a


disk device, to enable VxVM access to the replacement disk.

b.

Use the vxdiskadm utility option 5, Replace a failed or


removed disk, to associate the new disk with its media name.
You initially select none as a replacement device. You are then
asked if you want to initialize the new disk drive.
Do not use FMR for plex resync.

Note In some cases you might need scan for new disk drives using the
vxdctl enable command.

VERITAS Volume Manager Advanced Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-87

Exercise: Performing Advanced Operations


14. Use the vxdiskadm utility option 14, Unrelocate subdisks back
to a disk, to move the relocated volume components back to the
original disk location and complete the following steps:

The only information you furnish is the media name of the


replacement disk, VxVM

Answer no to Unrelocate to a new disk.

Answer no to Use -f option.

15. Use the vxprint command to verify the mirrored volume is


returned to its original configuration.
The letters UR are added to the relocated subdisk name.

Task 10 Using Intelligent Storage Provisioning


Perform the following steps to configure ISP storage pools and ISP
application volumes.
1.

Unmount and delete all volumes in your disk group.

2.

Click your disk group in the VEA GUI object tree and select
Organize DiskGroup from its pop-up menu.

3.

Click on the Mirrored Data and Striped Snapshot organization


category in the Organize DiskGroup Wizard window.

4.

Use the default data pool and clone pool names.

5.

Review the summary information and click Finish when you are
satisfied with the configuration.

6.

Display your disk group in the VEA GUI grid area and click the
Storage Pools tab.

7.

Complete the following steps:

8.

6-88

a.

Click the mirrored_volumes1 data pool with the third mouse


button and select Properties from its pop-up menu.

b.

Examine the properties and capabilities associated with the


mirrored_volumes1 data pool.

Complete the following steps:


a.

Click the mirrored_volumes1 data pool with the third mouse


button and select Add/Remove Disks from its pop-up menu.

b.

Add all six of your disk drive to the mirrored_volumes1 data


pool.

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Performing Advanced Operations


9.

Click your disk group in the VEA GUI object tree and select New
Volume from its pop-up menu.

10. Create a mirrored ISP application volume as follows:


a.

Click Next in the Select User Template window.

b.

Click Data Redundancy and Data Mirroring capabilities, and


then click Next.

c.

Leave the Let Volume Manager Decide button enabled, and


then click Next.

d.

Complete the following steps:


1.

Enter a volume name.

2.

Set the size to 200 Mbytes

3.

Click Next.

e.

Do not create a file system on the volume.

f.

Complete the following steps:


1.

Enter a user template name.

2.

Click Save.

3.

Click Finish.

g.

Use the VEA GUI and the vxprint command to examine the
resulting volume structure and storage pool organization.

h.

Delete the mirrored ISP application volume.

11. Create another application volume in your data storage pool using
additional capabilities.
a.

Click Next in the Select User Template window.

b.

Click Mirrored DCO Logs, Data Redundancy and Data


Mirroring capabilities, and then click Next.

c.

Leave the Let Volume Manager Decide button enabled, and


then click Next.

d.

Complete the following steps:


1.

Enter a volume name.

2.

Set the size to 200 Mbytes.

3.

Click Next.

e.

Do not create a file system on the volume.

f.

Click Finish.

VERITAS Volume Manager Advanced Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-89

Exercise: Performing Advanced Operations

Note DCO logging is associated with a feature not currently licensed or


supported by Sun Microsystems.
12. Click your disk group in the VEA GUI object tree and select New
Storage Pool from its pop-up menu.
13. Create another storage pool in your disk group as follows:
a.

Enter a storage pool name and click Next.

b.

Click the RAID-5 storage pool template, and then click Next.

c.

Examine the summary information, and then click Finish.

14. Display your storage pools in the VEA GUI grid area and verify that
the new storage pool is a clone pool.
15. Delete all of your storage pools and organize your disk group again
using a different storage pool set template, such as the Striped
Mirrored Data and Striped Snapshots template.
16. Create one more application volume using the new storage pool
capabilities.

6-90

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Performing Advanced Operations

Task 11 Configuring a Best Practice Boot Disk


Do not perform this task unless you have encapsulated and mirrored your
boot disk in an earlier task.
The following section assumes that all disk drives and volume
components follow the default VxVM naming conventions.
To reorganize the disks into a best practice configuration, complete the
following steps:
1.

Un-mirror the primary boot disk by removing the mirror plexes from
the mirror disk. Use the Bourne shell.
# vxassist -g rootdg remove mirror rootvol !rootdg02
# vxassist -g rootdg remove mirror swapvol !rootdg02

2.

To ensure the correct mirror placement, manually mirror the


rootvol and swapvol volumes again in the order shown.
# vxrootmir rootdg02
# vxassist -g rootdg mirror swapvol rootdg02

3.

Disassociate all primary boot disk plexes and recursively remove


them.
#
#
#
#
#

4.

vxplex
vxplex
vxedit
vxedit
vxedit

-g
-g
-g
-g
-g

rootdg
rootdg
rootdg
rootdg
rootdg

-v
-v
-r
-r
-r

rootvol dis rootvol-01


swapvol dis swapvol-01
rm rootdg01Priv
rm rootvol-01
rm swapvol-01

Verify that only the mirror disk plexes remain.

# vxprint -g rootdg
TY NAME
ASSOC
dg rootdg
rootdg

KSTATE
-

LENGTH
-

PLOFFS
-

STATE
-

dm rootdg01
dm rootdg02

c0t0d0s2
c0t1d0s2

8380799
8374320

NOHOTUSE

v rootvol
pl rootvol-02
sd rootdg02-01

root
rootvol
rootvol-02

ENABLED
ENABLED
ENABLED

7808400
7808400
7808400

ACTIVE
ACTIVE
-

v swapvol
pl swapvol-02
sd rootdg02-02

swap
swapvol
swapvol-02

ENABLED
ENABLED
ENABLED

565920
565920
565920

ACTIVE
ACTIVE
-

VERITAS Volume Manager Advanced Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-91

Exercise: Performing Advanced Operations


5.

Remove the rootdg01 disk from the rootdg disk group, and
complete the following steps:
a.

Reinitialize the rootdg01 disk.

b.

Add the rootdg01 disk back into the rootdg disk group.

# vxdg -g rootdg rmdisk rootdg01


# vxdisksetup -i c0t0d0 format=simple
# vxdg -g rootdg adddisk rootdg01=c0t0d0
6.

Use the vxdiskadm option 6, Mirror volume on a disk, to mirror


rootvol and swapvol on the rootdg02 disk drive back to the newly
initialized rootdg01 disk drive.
# vxdiskadm
...
At the prompt below, supply the name of the disk
containing the volumes to be mirrored.
Enter disk name [<disk>,list,q,?] rootdg02
Enter destination disk [<disk>,list,q,?] (default: any)
rootdg01

7.

Modify the boot-device parameter and enable nvramrc usage.


# eeprom boot-device="rootdisk rootmirror"
# eeprom "use-nvramrc?"=true

8.

Use the prtvtoc command and the format utility to verify that the
primary and mirror boot disk drives have exactly the same partition
maps.

9.

Boot the system from each of the available system devices.


ok
ok
ok
ok

boot
boot
boot
boot

disk0
disk1.
vx-rootdg01
vx-rootdg02

You can now replace either the primary boot disk drive or its mirror in the
same manner as any other VxVM disk drive, and just resynchronize the
mirrors.

6-92

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise Summary

Exercise Summary

Discussion Take a few minutes to discuss what experiences, issues, or


discoveries you had during the lab exercises.

Manage the discussion here based on the time allowed for this module, which was given in the About This
Course module. If you find you do not have time to spend on discussion, highlight just the key concepts
students should have learned from the lab exercise.

Experiences

Ask students what their overall experiences with this exercise have been. Go over any trouble spots or
especially confusing areas at this time.

Interpretations

Ask students to interpret what they observed during any aspects of this exercise.

Conclusions

Have students articulate any conclusions they reached as a result of this exercise experience.

Applications

Explore with students how they might apply what they learned in this exercise to situations at their workplace.

VERITAS Volume Manager Advanced Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-93

Module 7

VERITAS File System Basic Operations


Objectives
Upon completion of this module, you should be able to:

Describe basic VxFS features

Install the VxFS software

Create VxFS file systems

Use extended VxFS mount options

Perform online VxFS administration tasks

7-1
Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Basic VxFS Features

Basic VxFS Features


The VxFS software is targeted at commercial environments where high
performance and availability are important, and large volumes of data
must be managed.
The VxFS extent-based space allocation scheme increases performance by
reducing the number of I/O operations required to read and write large
amounts of data.
The VxFS intent log feature provides fast recovery following a system
crash or reboot. A file system check can be completed in seconds,
regardless of the file system size.

Extent-Based Space Allocation


Standard UFS file systems use block-based allocation schemes and
provide good random access to files and reasonable latency on small files.
For larger files, however, this block-based architecture limits throughput.
The VxFS file system improves file system performance by using a
different allocation scheme name extent-based allocation.
Disk space is allocated by the system in 512-byte sectors, which are
grouped together to form a logical block. VxFS supports logical block
sizes of 1024, 2048, 4096, or 8192 bytes. The default block sizes vary
according to file system size as follows:

1024 block size for file systems up to 4-Tbytes in size

2048 block size for file systems up to 8-Tbytes in size

4096 block size for file systems up to 16-Tbytes in size

8192 block size for file systems beyond 16-Tbytes in size

An extent is one or more adjacent blocks of data within a file system. It is


presented as an address-length pair, which identifies the starting block
address and the length of the extent (in blocks). When data is added to a
file on a VxFS system, it is grouped in extents as opposed to being
allocated a block at a time (as is done with UFS file systems).
By allocating disk space in extents, disk I/O to and from a file can be done
in units of multiple blocks considerably faster than block-at-a-time
operations.

7-2

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Basic VxFS Features

File System Intent Logging


After a system failure, the structural integrity of UNIX file systems is
verified by the fsck program during the system reboot operation. This
means checking the entire structure of a file system, verifying that it is
intact, and correcting any inconsistencies that are found. This process can
be very time consuming.
The VxFS file system provides recovery only seconds after a system
failure by using a tracking feature called intent logging. Intent logging is a
logging scheme that records pending changes to the file system structure.
During system recovery from a failure, the intent log for each file system
is scanned and operations that were pending are completed. The file
system can then be mounted without a full structural check of the entire
system.
When the disk has a hardware failure, the intent log might not be enough
for recovery and, in such cases, a full fsck check must be performed.
However, when the failure is due to software rather than hardware, a
system can be recovered in seconds.
The default intent log size is currently 64 Mbytes. The fsadm command
can be used to dynamically modify the intent log size. Larger intent logs
can improve system performance because they reduce the number of
times the log wraps around. An intent log that is too large can increase file
system recovery time after a system failure.
Note See the fsadm_vxfs and mkfs_vxfs man pages for more
information.

VERITAS File System Basic Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

7-3

Installing the VxFS Software

Installing the VxFS Software


You use the pkgadd command to install the VxFS software. The VxFS
packages should be installed in the order shown.
# pkgadd -d .
The following packages are available:
1 VRTSalloc
VERITAS Volume Manager: VERITAS Intelligent Storage
Provisioning
(sparc) 4.0,REV=12.06.2003.03.07
2 VRTSap
VERITAS Action Provider
(sparc) 2.00.015
3 VRTSddlpr
VERITAS Device Discovery Layer Services Provider
(sparc) 4.0,REV=12.06.2003.02.03
4 VRTSfppm
VERITAS File Placement.
(sparc) 4.0,REV=GA03
5 VRTSfsdoc
VERITAS File System Documentation
(sparc) 4.0,REV=GA04
6 VRTSfsman
VERITAS File System - Manual Pages
(sparc) 4.0,REV=GA04
7 VRTSfspro
VERITAS File System Management Services Provider
(sparc) 4.0,REV=GA04
8 VRTSmuob
VERITAS Enterprise Administrator Service Localized
Package
(sparc) 3.2.514.0
9 VRTSob
VERITAS Enterprise Administrator Service
(sparc) 3.2.514.0
10 VRTSobgui
VERITAS Enterprise Administrator
(sparc) 3.2.514.0
11 VRTStep
VERITAS Task Exec Provider
(sparc) 1.20.025
12 VRTSvlic
VERITAS License Utilities
(sparc) 3.02.005d
13 VRTSvmdoc
VERITAS Volume Manager (user documentation)
(sparc) 4.0,REV=12.06.2003.01.34
14 VRTSvmman
VERITAS Volume Manager, Manual Pages
(sparc) 4.0,REV=12.06.2003.01.34
15 VRTSvmpro
VERITAS Volume Manager Management Services Provider
(sparc) 4.0,REV=12.06.2003.01.35
16 VRTSvxfs
VERITAS File System
(sparc) 4.0,REV=GA04
17 VRTSvxvm
VERITAS Volume Manager, Binaries
(sparc) 4.0,REV=12.06.2003.01.35
Select package(s) you wish to process (or all to process
all packages). (default: all) [?,??,q]: 16 4 5 6 11 2

7-4

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Creating VxFS File Systems

Creating VxFS File Systems


You use the same process as with the UNIX UFS file system to create VxFS
file systems. To create VxFS file system on top of VxVM volumes, you can
use either the VEA GUI New File System Wizard or the standard file
system command-line programs, such as mkfs.
The primary difference is that you must specify a file system type of vxfs.
You create a VxFS file system using the mkfs command as follows.
# mkfs -F vxfs /dev/vx/rdsk/dgX/mirvol
The following mkfs command options can affect system performance
when used appropriately:

mkfs -F vxfs -o bsize


You use the bsize option to decrease or increase the file system
block size during initial file system creation. For most applications it
is best to use the default VxFS block size, which is automatically
calculated based on file system size.

mkfs -F vxfs -o logsize


You use the logsize option to decrease or increase the size of the
intent log during initial VxFS file system creation. Increasing the size
of the intent log can improve performance, but the log must never be
larger than half of virtual memory (physical memory + swap).

mkfs -F vxfs -o largefiles/nolargefiles


If the largefiles option is enabled, a persistent largefiles bit is
written on the file system. The largefiles bit (flag) allows the
creation of files larger than 2 Gbytes.
The default mode of operation is largefiles.

Note Consult the mkfs_vxfs man pages for more information on other
VxFS mkfs command options, including inosize and ninode.

VERITAS File System Basic Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

7-5

Extended VxFS Mount Options

Extended VxFS Mount Options


The VxFS software provides a number of mount options that might be
appropriate under special circumstances. You should not use many of the
options unless there is a clearly defined need. The most commonly used
VxFS mount options control intent log behavior and how the VxFS
software responds to media errors.

Intent Log Behavior


The log, delaylog, tmplog, logiosize, and nodatinlog mount options
have varying effects on VxFS file system performance and integrity,
depending on specific needs.

mount -F VxFS -o log


The log option guarantees all file system changes are logged before
a system call returns. Recent changes are not lost.

mount -F VxFS -o delaylog


The default logging mode is delaylog. Most system call changes are
logged before returning. Some changes are logged shortly after the
system call returns. After a system failure, the most recent change
might be lost. This behavior is similar to UNIX UFS file system
behavior.

mount -F VxFS -o tmplog


When the tmplog mode of operation is enabled, most intent log
changes are made after the system calls return. This mode can
increase file system performance, but places recent data changes at a
higher level of risk.

mount -F VxFS -o logiosize=size


The logiosize option can increase performance by adjusting intent
log write size to match the stripe size of read-modify-write (RAID 5)
devices. Valid size values are 512, 1024, 2048, 4096, or 8192.

mount -F VxFS -o nodatinlog


The nodatainlog option should not be used on systems capable of
relocating bad disk blocks. The Solaris OS uses bad block relocation
to relocate disk blocks that are having recoverable read or write
errors. If enabled, nodatainlog decreases the file system performance
by approximately 50 percent.

7-6

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Extended VxFS Mount Options

Error Handling Behavior


The ioerror option has several sub-options that are designed to handle
errors in a way that is appropriate for the related storage technology.

mount -F VxFS -o ioerror=disable


The disable sub-option disables all file system data, and metadata
reads and writes when an error is detected. You use this sub-option
when the underlying storage is redundant, such as mirroring or
RAID 5.
You should use the disable policy for cluster file systems.

mount -F VxFS -o ioerror=nodisable


The nodisable sub-option does not disable reads or writes on error
detection. It posts appropriate error messages and sets numerous file
system error flags.
This behavior is similar to previous versions of VxFS.

mount -F VxFS -o ioerror=wdisable/mwdisable


The wdisable sub-option disables all write operations to file system
and metadata, but read status is set to degraded.
The mwdisable sub-option disables writes to metadata, and all other
read and write status is degraded.
The wdisable and mwdiable sub-options are used in environments
where read errors are more likely to persist than write errors, such as
when using non-redundant storage.
The mwdisable policy is the default for locally mounted file systems.

Other VxFS Mount Options


There is also a blkclear option that is designed for high security
environments. The blkclear mode of operation guarantees that
uninitialized storage (old data) never remains in a file system. This mode
of operations can degrade file system performance by as much as
10 percent.
Note Consult the mount_vxfs man pages for further details about other
available mount options, some of which relate to VxFS features that are
not licensed by Sun.

VERITAS File System Basic Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

7-7

Online File System Administration

Online File System Administration


You can defragment, resize, and back up VxFS file systems while they are
online and in use.

Online Defragmentation
The UFS software uses the concept of cylinder groups to limit
fragmentation. These are self-contained sections of a file system that are
composed of inodes, data blocks, and bitmaps, that indicate free inodes
and data blocks. Allocation strategies in UFS attempt to place inodes and
related data blocks near each other. This strategy reduces fragmentation,
but does not eliminate it. Over time, the original ordering of free resources
can be lost. As files are added and removed, gaps between used areas of
the disk can still occur.
The VxFS software fsadm utility is used to defragment a disk without
requiring that the disk be unmounted first. The fsadm utility removes
unused space from directories, makes small files contiguous, and
consolidates free blocks for use.

Online Resizing
When UFS file systems are too small or become too large for their
assigned disk space, the following methods are used to correct the
problem:

Users are moved to new or different file systems.

Subdirectories are moved to other file systems.

The file systems are backed up and restored to a different file system.

VxFS with VxVM enables a file system to be expanded or reduced in size


while it is being accessed.

Online Backup and Restore


You use the VxFS vxdump utility to dump volume snapshots to tape.
Volumes are restored using the vxrestore utility.

7-8

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Configuring VxFS

Exercise: Configuring VxFS


In this exercise, you complete the following tasks:

Review key lecture points

Install the VxFS software

Create a VxFS file system

Resize a VxFS file system

Defragment a VxFS file system

Backup and restore a VxFS file system

Use VxFS extended mount options

Preparation
VxFS is a separately licensed option. You must install a license key to
activate the software. Ask your instructor for a VxFS temporary license
key. Record the temporary key.
VxFS license key: __________________________________________________
You can get VxVM, VxFS, and Shared Disk Group temporary licenses at the Sun Business Partners Web site
at http://webhome.ebay/partnersoftware/.

VERITAS File System Basic Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

7-9

Exercise: Configuring VxFS

Task 1 Reviewing Key Lecture Points


Answer the following questions about VxFS features and mount options.
1.

What is the primary advantage of extent-based space allocation?


a.

Faster recovery times after system failures

b.

Reduced data fragmentation

c.

Increased throughput in write-intensive applications

d.

Reduced data loss after system failures

The answer is c.

2.

What is the primary advantage of intent logging?


a.

Simplified file system administration

b.

Enhanced space utilization

c.

Improved file system performance

d.

Faster recovery time after system failures

The answer is d.

3.

Which of the following VxFS mount options is used to increase data


security?
a.

log

b.

delaylog

c.

blkclear

d.

nodatinlog

e.

convosync

The answer is c.

4.

Which of the following VxFS mount options should be used with


cluster file systems?
a.

log

b.

delaylog

c.

blkclear

d.

nodatinlog

The answer is a.

7-10

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Configuring VxFS

Task 2 Installing the VxFS Software


To install the VxFS software, complete the following steps:
1.

Log in as user root on the system attached to the storage arrays.

2.

Change to the VxVM installation package location furnished by your


instructor.

3.

Install the VxFS packages in the order shown and answer yes to all
questions.
# pkgadd -d . VRTSvxfs VRTSfppm VRTSfsdoc VRTSfsman \
VRTStep VRTSap

4.

Install the VxFS license key furnished by your instructor.


# vxlicinst
VERITAS License Manager vxlicinst utility version
3.02.005
Copyright (C) VERITAS Software Corp 2002. All Rights
reserved.
Enter your license key : 3JZU-YBPO-CR28-RWWP-P5RO-P12

5.

Reboot the VxFS server.


There are no VxFS-specific boot messages.

6.

Log in to the VxFS server and verify that the VxFS program files are
installed.
# ls /opt/VRTSvxfs/sbin
cp
qiostat
cpio
qlogadm
fcladm
qlogckd
fsadm
qlogclustadm
fsapadm
qlogdb
fscat
qlogdetach
fsckpt_restore qlogdisable
fsckptadm
qlogenable
fsclustadm
qlogmk
fsvoladm
qlogprint
getext
qlogrm
ls
qlogstat
mv
qlogtrace
qioadmin
setext
qiomkfile
vxdump

vxedquota
vxenablef
vxfsckd
vxfsconvert
vxfsstat
vxlsino
vxquot
vxquota
vxquotaoff
vxquotaon
vxrepquota
vxrestore
vxtunefs
vxupgrade

VERITAS File System Basic Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

7-11

Exercise: Configuring VxFS


7.

Verify that the VxFS technical manuals are installed.


# ls /opt/VRTS/docs
vxfs_ag.pdf vxfs_ig.pdf

8.

Verify that the VxFS man pages have been added to the
/opt/VRTS/man directory.
# man vxquot
# man vxdump

9.

Verify that the VxFS driver is loaded.


# modinfo | grep vxfs

10. Verify that the special file system programs that are required early in
the boot process are present.
# ls /etc/fs/vxfs
mount
qlogattach

qlogck
qlogrec

system.preinstall

11. Move to the VxFS administration guide directory.


# cd /opt/VRTS/docs
12. View the contents of the manuals with the Adobe Acrobat Reader
(acroread).
13. Complete the following steps:
a.

Examine Appendix B, Kernel Messages, of the VERITAS File


System 4.0 Administrators Guide.

b.

Examine the Description and Action sections associated with


each of the error messages.

Note Use the Control-F sequence to enable the Adobe Acrobat Find
window.

7-12

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Configuring VxFS

Task 3 Creating a VxFS File System


You use VEA to perform the following file system operations. Study VxFS
options in the VERITAS File System 4.0 Administrators Guide as necessary.
To create a VxFS file system, complete the following steps:
1.

Delete all volumes in your disk group and add all of your disks to
the group.

2.

Use the VEA GUI to create a mirrored volume as follows:


a.

Use all 6 of your disk drives for the volume.

b.

Click Concatenated layout and select the Mirrored check box.

c.

Set the total number of mirrors at 2.

d.

Disable logging.

e.

Click Max Size to calculate the maximum size and round-off the
result that is returned.

f.

Do not create a file system on the volume.

3.

Click your new volume and select File System then New File System
from its pop-up menu.

4.

Configure the New File System window as follows:


a.

Set the file system type to vxfs.

b.

Leave the block size set to the default value (1024).

c.

Enter a file system mount point.

d.

Click Add to file system table check box, and complete the
following steps:

e.
5.

1.

Select Mount at boot.

2.

Set the fsck pass number to 2.

Click OK.

In the VEA GUI grid area, complete the following steps:


a.

Click your VxFS volume and select Layout View from its popup menu.

b.

Examine the concatenated mirror volume structure and close


the Layout View window when you have finished.

VERITAS File System Basic Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

7-13

Exercise: Configuring VxFS


6.

On the VxVM server, complete the following steps:


a.

Examine the VEA GUI command log.

b.

Observe the volume size and the log size information in the
Output section of the mkfs command.

# tail -40 /var/vx/isis/command.log


c.

Verify that the largefiles option was specified with the mkfs
command.

Task 4 Resizing a VxFS File System


A unique feature of VxFS is the ability to decrease the size of an existing
file system that is overly large for its intended application.
To decrease the size of the VxFS file system created previously in this
exercise, complete the following steps.
1.

Use the df -kl command to verify the amount of space available in


your new VxFS file system.

# df -kl
Filesystem
kbytes
....
/dev/vx/dsk/dgX/mirvol 26214400

used

avail

capacity

Mounted on

39890

24538611

/VXFS

In the preceding example, the file system is approximately 26-Gbytes


in size and about 40-Mbytes of space is in use by inode and intent
log space.
2.

Use the mount command to verity that your VxFS file system was
mounted using the delaylog, largefiles, and
ioerror=mwdisable mount options.

3.

Verify that you can create a file larger than 2-Gbytes in your file
system.
# mkfile 2500m /VXFS/file1

7-14

4.

Delete all test files from your VxFS file system.

5.

Click on the VxFS volume in the VEA GUI Object tree, and select
Resize Volume from its pop-up menu.

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Configuring VxFS


6.

Complete the Resize Volume form as follows:


a.

Enter a new volume size of 1 Gbyte.

b.

Let volume manager decide which disks to use.

c.

Click on the OK button to complete the resize operation.

d.

Verify the new VxFS file system size by using the df -kl
command.

Task 5 Defragmenting a VxFS File System


See the fsadm_vxfs man page for further information on the fsadm
defragmentation utility.
Perform the following steps to defragment your VxFS file system:
1.

Use the fsadm command to verify the current fragmentation in your


VxFS file system. Substitute the name of your file system.

# fsadm -D /VXFS
Directory Fragmentation Report
Dirs
Total
Searched
Blocks
total
2
0

Immed
Dirs
0

Immeds
to Add
2

Dirs to
Reduce
0

Blocks to
Reduce
0

The Blocks to Reduce value is a general indicator that


defragmentation might be of value. Some directories are fragmented
in a way that does not respond to defragmentation.
2.

Perform the following command sequence to create some


fragmentation in your VxFS file system. Substitute the name of your
file system.
#
#
#
#
#
#

3.

cp
cp
rm
cp
cp
cp

-r
-r
-r
-r
-r
-r

/usr/dt /VXFS
/usr/appserver /VXFS
/VXFS/dt
/usr/iplanet /VXFS
/usr/share /VXFS
/usr/dt /VXFS

(151 Mbytes)
(75 Mbytes)
(65 Mbytes)
(78 Mbytes)

Verify the level of fragmentation and available space in your VxFS


file system.
# fsadm -D /VXFS
# df -kl

VERITAS File System Basic Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

7-15

Exercise: Configuring VxFS


4.

Defragment your VxFS file system. Substitute the name of your file
system.
# fsadm -de /VXFS

5.

Verify your VxFS file systems fragmentation and available space


have improved.
# fsadm -D /VXFS
# df -kl

Task 6 Backing Up and Restoring a VxFS File


System
To use the Snapshot feature to prepare for a VxFS file system backup,
complete the following steps:
1.

In the VEA grid area, click your VxFS volume and select Snap then
Snap Start from its pop-up menu.

2.

Let volume manager decide what disks to use for the Snapshot, and
then click OK.

3.

On the VxVM server, use the vxprint command to verify that the
Snapshot structure is complete.

4.

In the VEA grid area, click your VxFS volume and select Snap then
Snap Shot from its pop-up menu.

5.

Examine the Snap Shot Volume form and complete the following
steps:
a.

Enter a custom volume name or use the default volume name.

b.

Click OK.
The Snapshot mirror is detached and used to create the
Snapshot volume.

6.

In the VEA GUI grid area, click the Snapshot volume and select File
System then Mount File System from its pop-up menu.

7.

Enter a mount point name in the Mount File System form and
click OK.

8.

Verify that the Snapshot volumes file system is mounted and


available.
# df -kl

7-16

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Configuring VxFS


9.

Ask your instructor if there is a tape unit available to perform the


following dump and restore procedure. Substitute the name of your
Snapshot file system and VxFS file system.
# vxdump -0f /dev/rmt/0 /SNAP
# cd /VXFS
# rm -r appserver dt iplanet share
# ls
lost+found
# vxrestore -f /dev/rmt/0

Note The vxdump and vxrestore utilities are functionally the same as
the Solaris OS ufsdump and ufsrestore utilities.

Task 7 Using VxFS Extended Mount Options


To investigate the effects of selected VxFS file system mount options,
complete the following steps:
1.

If you have not done so already, remove all of the files from your
VxFS file system.
# cd /VXFS
# rm -r appserver dt iplanet share
# ls
lost+found
# cd /

2.

Create a test file in your VxFS file system and record the amount of
time it takes to complete the operation.
# ptime mkfile 100m /VXFS/file1
real
7.348
user
0.076
sys
3.459
# rm /VXFS/file1

3.

Unmount your VxFS file system and mount it again using the
blkclear security option.
# umount /VXFS
# mount -F vxfs -o blkclear /VXFS

VERITAS File System Basic Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

7-17

Exercise: Configuring VxFS


4.

Use the mount command to verify that the VxFS file systems mount
options are now read, write, setuid, blkclear, delaylog,
largefiles, and ioerror=mwdisable.

5.

Run a performance test on your VxFS file system again.


# ptime mkfile 100m /VXFS/file1
real
1:10.047
user
0.078
sys
5.145
# rm /VXFS/file1

6.

Unmount your VxFS file system and mount it again with no special
options.
# mount -F vxfs /VXFS

7-18

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise Summary

Exercise Summary

Discussion Take a few minutes to discuss what experiences, issues, or


discoveries you had during the lab exercises.

Manage the discussion here based on the time allowed for this module, which was given in the About This
Course module. If you find you do not have time to spend on discussion, then just highlight the key concepts
students should have learned from the lab exercise.

Experiences

Ask students what their overall experiences with this exercise have been. You might want to go over any
trouble spots or especially confusing areas at this time.

Interpretations

Ask students to interpret what they observed during any aspects of this exercise.

Conclusions

Have students articulate any conclusions they reached as a result of this exercise experience.

Applications

Explore with students how they might apply what they learned in this exercise to situations at their workplace.

VERITAS File System Basic Operations


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

7-19

Module 8

VERITAS Volume Manager Performance


Management
Objectives
Upon completion of this module, you should be able to:

Describe performance improvement techniques

Use the vxstat and vxtrace performance analysis tools

Describe RAID-5 write performance characteristics

8-1
Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Performance Improvement Techniques

Performance Improvement Techniques


Periodic reassessment of volume performance is necessary on any system.
The access to any data structure can degrade over time to the point of
poor performance.

Data Assignment Strategies


When deciding where to locate a file system, you usually attempt to
balance I/O load among available disk drives. The success of this process
is limited by the difficulty of anticipating future usage patterns. Figure 8-1
shows how data assignment mistakes can lead to a performance problem.

Controller

Controller

c3

c4

Array

Array
Heavy-Use
Volume

Low-Use
Volume

Heavy-Use
Volume

Low-Use
Volume

Heavy-Use
Volume

Low-Use
Volume

Heavy-Use
Volume

Low-Use
Volume

Figure 8-1

Data Assignment Bottleneck

In general, do not place file systems that have heavy I/O loading on the
same disk drives. Separate them into different storage arrays on various
controllers.
Also, the placement of logs can be critical to performance. This is
especially true of RAID-5 logs.

8-2

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Performance Improvement Techniques


The following solutions can be used to resolve the problem demonstrated
in Figure 8-1 on page 8-2:

Swap some of the heavy-use volumes with the low-use volumes.

Move one of the heavy-use subdisks to a different location.

Note Swapping volume locations is probably a better solution because it


eliminates having two heavily used volumes on a single disk drive.
Another type of performance problem can occur when a log plex is placed
on the same disk drive as its associated data plex. In the case of RAID-5
logs, you should always consider that the data written to all RAID-5
columns must also be written to the log.
In a six-column RAID-5 volume, this configuration could increase the I/O
rate of the log disk drive by as much as 600 percent.
As shown in Figure 8-2, leaving unused space on all disk drives ensures
that you always have alternate locations to which to move logs. This is
why the Maxsize calculations of the vxassist command or VEA might
not be wise to use.
Volume 01

Column 0

Column 1

Column 2

Column 3

Vol02_log

Log Space

Log Space

Log Space

Volume 02

Column 0

Column 1

Column 2

Column 3

Log Space

Log Space

Log Space

Vol01_log

Figure 8-2

RAID-5 Log Placement

The log placement shown in Figure 8-2 would not work well if both
volumes were heavily accessed. The configuration would work best if at
least one of the volumes has low write activity.

VERITAS Volume Manager Performance Management


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

8-3

Performance Improvement Techniques

Volume Structure Strategies


Sometimes, performance problems are not due to physical volume
locations and can be greatly reduced by reconfiguring the volume
structures. In many cases, this can be accomplished using the VxVM
Volume Relayout feature.

Striping
Striping distributes data across multiple devices to improve access
performance. Striping improves performance for both read and write
operations.
If you can identify the most heavily accessed volumes (for file systems or
database tables) during the initial design stages, then you can eliminate
performance bottlenecks by striping them across several devices. The
example in Figure 8-3 shows a volume (Hot_Vol) that was identified as
being a data-access bottleneck. The volume is striped across four disk
drives, leaving the remainder of those four disk drives free for use by less
heavily used volumes.

Hot_Vol
Stripe 0

Hot_Vol
Stripe 1

Light Use

Light Use

Hot_Vol
Stripe 2
Light Use

Hot_Vol
Stripe 3
Light Use

Light Use

Figure 8-3

8-4

Using Striping to Improve Performance

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Performance Improvement Techniques

Mirroring
Mirroring stores multiple copies of data on a system. Mirroring is
primarily used to protect against data loss due to physical media failure.
It also improves the chance of data recovery in the event of a system
crash.
In some cases, mirroring can also be used to improve system
performance. Mirroring heavily accessed data not only protects the data
from loss due to disk drive failures, but it can also improve I/O
performance. Unlike striping, however, performance gained through the
use of mirroring depends on the read/write ratio of the disk drive
accesses. If the system workload is primarily write-intensive (for example,
greater than 30 percent writes), then mirroring can result in somewhat
reduced performance.

RAID 0+1
RAID 0+1 is also referred to as a mirror-stripe configuration. When used
together, mirroring and striping provide the advantage of both spreading
the data across multiple disk drives and providing redundancy of data.

Layered Volumes
Striped mirror (RAID 1+0) volume configurations can improve
performance significantly. Additionally, striped mirror configurations can
tolerate a higher percentage of disk drive failures and recovery times are
greatly reduced.

RAID 5
RAID 5 provides the advantage of read performance that is similar to that
of striping, while also providing data protection using a distributed parity
scheme. The disadvantage of RAID 5 is relatively slow write performance.
RAID 5 is not generally seen as a performance improvement mechanism
except in cases of read-intensive applications. However, RAID-5 volume
structures can be adjusted to dramatically improve performance for some
write-intensive applications.

VERITAS Volume Manager Performance Management


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

8-5

Performance Improvement Techniques

Read Policy Strategies


To provide optimal read performance for different types of mirrored
volumes, you can select the following read policies:

The round-robin read policy (round)


Read requests to a mirrored volume are satisfied in a round-robin
manner from all plexes in the volume. You select this policy when
there is no significant performance advantage by using any
particular mirror.

The preferred-plex read policy (prefer)


Read requests to a mirrored volume are satisfied from one specific
plex (presumably the plex with the highest performance), unless that
plex has failed.

The default read policy (select)


The appropriate read policy is automatically selected for the
configuration. For example, preferred-plex is selected when there is
only one striped plex associated with the volume, and round-robin is
selected in most other cases.

In the example in Figure 8-4, you set the read policy of the volume labeled
Hot_Vol to prefer for the striped plex labeled Plex 1. In this way, read
requests are directed to the striped plex that has the best performance
characteristics.

Hot_Vol
Stripe 0
Plex 1

Hot_Vol
Stripe 1
Plex 1

Hot_Vol
Stripe 2
Plex 1

Disk 1

Disk 2

Disk 3

Figure 8-4

Hot_Vol
Plex 2

Disk 4

Preferred-Plex Read Policy

You can change volume read policies either from the command line or
using the VEA GUI.

8-6

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Performance Improvement Techniques


In the VEA GUI, highlight the volume in the grid area, and click Props in
the toolbar. In the General properties tab, you can choose one of following
fixed read policy options:

Based on Layout

Round Robin

Prefer (preconfigured)

From the command line, use the vxvol command as follows:


# vxvol -g dgY rdpol prefer vol01 vol01-03

Hardware Configuration Strategies


For increased performance, availability, or both, striping and mirroring
should be done across system boards, controllers, and targets. You can
gain the highest level of performance or reliability by striping or
mirroring across system boards, as shown in Figure 8-5.
Host System
System Board

Controller c3

System Board

Controller c4

Perferred stripe or
mirror configuration

t1

t1
t2

t2

t3
Array

Figure 8-5

t3
Array

High Availability and Performance Cabling

VERITAS Volume Manager Performance Management


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

8-7

Using Performance Analysis Tools

Using Performance Analysis Tools


The VxVM software continuously gathers performance statistics about all
devices and objects under its control. The types of information include:

A count of operations

The number of blocks transferred

The average operation time (which reflects the total time through the
VxVM software and is not suitable for comparison against other
statistics programs)

The statistics include the number of reads, writes, atomic copies, verified
reads, verified writes, plex reads, and plex writes for each volume. As a
result, one write to a two-plex volume results in at least five operations:
one for each plex, one for each subdisk, and one for the volume.
VxVM also maintains other statistical data, such as information about
read and write failures. The statistics are continuously gathered starting
with the system boot operation. The statistics are reset prior to a testing
operation.

Gathering Volume Performance Statistics


The vxstat command displays statistical information about different
types of VxVM physical and logical objects. You can use the following
options to control the output:

vxstat -g disk_group
Displays volume statistics for the specified disk group.

vxstat -g disk_group vol01


Displays statistics for the specified volume.

vxstat -g disk_group -d
Displays disk drive level statistics for the specified disk group.

vxstat -g disk_group -d disk01


Displays statistics for the specified disk drive.

After performance data has been gathered, you can analyze it to


determine, and to optimize, your system configuration for efficient use of
system resources.

8-8

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Using Performance Analysis Tools


A volume or disk drive with elevated read or write access times is not
necessarily a problem. If the slow response is not causing any apparent
problems for users or applications, then there might not be anything that
needs fixing.

Preparing for Analysis


Before obtaining statistics, clear (reset) all existing statistics by using the
vxstat -r command. Clearing statistics eliminates any differences
between volume or disk drives due to volumes being created. It also
removes statistics that are not typically of interest, such as information
about booting.
After clearing the statistics, allow the system to run during typical system
activity. When monitoring a system that is used for multiple purposes, try
not to exercise any one application more than it would usually be
exercised.
It can also be beneficial to take periodic snapshots of the volume statistics
to help identify the source of irregular system-load problems.
A single volume with excessive I/O rates can cause performance
degradation on other volumes associated with the same disk drives.

Analyzing Volume Statistics


Use the vxstat command as follows to help identify volumes with an
unusually large number of operations or excessive read or write times:
# vxstat -g bench
OPERATIONS
BLOCKS
AVG TIME(ms)
TYP
NAME
READ
WRITE
READ
WRITE
READ WRITE
vol
acct
473
11
57252
44
4.0
20.9
vol
brch
23
11
92
44
33.0 20.0
vol
ctrl
92773
121312064799783 78102463
276.9 457.2
vol
hist1
23
11
92
44
97.0 24.5
vol
hist2
23
11
92
44
54.8 22.7
vol
hist3
23
11
92
44
103.5 25.5
vol
log1
9
27217
9
409716
16.7 21.1
vol
log2
7
8830
7
159769
15.7 24.3
vol
rb1
123
13
492
52
30.7 83.1
vol
rb2
23
11
92
44
149.1 25.5
vol
sys
26933
86156
17768834463215.0310.7
vol
t11r
23
11
92
44
39.6 24.5

VERITAS Volume Manager Performance Management


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

8-9

Using Performance Analysis Tools

Analyzing Disk Drive Statistics


You can also use the vxstat command to summarize operations
according to physical disk drives. For example:
# vxstat -g bench -d
OPERATIONS
TYP NAME
READ
WRITE
dm c3t98d0
14330
140370
dm c3t100d0
13881
140370
dm c3t113d0
0
0
dm c3t115d0
0
0
dm c3t117d0
0
0

8-10

BLOCKS
READ
WRITE
120348 986785
117971 986785
0
0
0
0
0
0

AVG TIME(ms)
READ WRITE
15.4 185.6
15.4 187.7
0.0
0.0
0.0
0.0
0.0
0.0

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Using Performance Analysis Tools

Gathering Application Performance Statistics


The vxtrace command displays detailed trace information about errors
or I/O operations. This level of detail is generally not necessary, but is
included here for completeness. You can use the following options to
control the display:

vxtrace -o disk
Traces all physical disk drive I/O operations.

vxtrace -o disk c3t98d0


Traces all I/O operations to the physical disk drive c3t98d0.

vxtrace hist2
Traces all virtual device I/O operations associated with the volume
hist2.

vxtrace -o dev hist2


Traces virtual disk device I/O to the device associated with volume
hist2.

After identifying a volume that has an I/O-related problem, you can use
the vxtrace command to determine which system process is responsible
for the I/O requests. The volume of interest in this example is named
ctrl.
# vxtrace -o dev ctrl
40122 START write vdev ctrl block 16 len
40122 END write vdev ctrl op 40122 block
40123 START write vdev ctrl block 16 len
40123 END write vdev ctrl op 40123 block
40124 START write vdev ctrl block 16 len
40124 END write vdev ctrl op 40124 block
40125 START write vdev ctrl block 16 len
40125 END write vdev ctrl op 40125 block
^C

4 concurrency
16 len 4 time
4 concurrency
16 len 4 time
4 concurrency
16 len 4 time
4 concurrency
16 len 4 time

1
1
1
2
1
4
1
0

pid 10689
pid 10689
pid 10689
pid 10689

# ps -ef |grep 10689


oracle 10689 1 0 20:05:21 ? 0:03 ora_ckpt_bench

VERITAS Volume Manager Performance Management


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

8-11

RAID-5 Write Performance

RAID-5 Write Performance


The RAID-5 write process is controlled according to how much data is
going to be written into a full stripe width. The optimum write
performance is obtained when full stripes are written.

Read-Modify-Write Operations
When less than 50 percent of the data disk drives are undergoing write
operations in a single I/O, the read-modify-write sequence is used.
This is the default operation for RAID-5 volumes.
As shown in Figure 8-6, the read-modify-write sequence involves several
steps, including:
1.

The stripes to be modified are read into a buffer.

2.

The parity information is read into a buffer.

3.

Exclusive OR (XOR) operations are performed.

4.

The new data and parity are written in a single write operation.
New Data
1

XOR

Stripe
Unit 0

Stripe
Unit 1

Stripe
Unit 2

Stripe
Unit 3

Stripe
Unit 4

Parity

Figure 8-6

8-12

Read-Modify-Write Operation

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

RAID-5 Write Performance


At least three I/O operations are necessary in the example shown in
Figure 8-6 on page 8-12. Also, additional XOR calculations are
necessary to account for the data in stripe units 2, 3, and 4 that was
not read.
Generally, the read-modify-write method is the least efficient way of
writing to RAID-5 structures.

Reconstruct-Write Operations
If more than 50 percent of the data stripe will be modified, use the
reconstruct-write method.
As shown in Figure 8-7, the reconstruct-write method involves different
steps, including:
1.

Only unaffected data is read into a buffer.

2.

XOR is applied to the new data and the unaffected data.

3.

New parity and data are written in a single write operation.


New Data
0

XOR
0

Stripe
Unit 0

Stripe
Unit 1

Stripe
Unit 2

Stripe
Unit 3

Stripe
Unit 4

Parity

Figure 8-7

Reconstruct-Write Operation

Only two I/O operations are necessary in the example shown in


Figure 8-7.
Generally, the reconstruct-write operation is more efficient than the
read-modify-write sequence.

VERITAS Volume Manager Performance Management


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

8-13

RAID-5 Write Performance

Full-Stripe Write Operations


When large write operations that cover an entire data stripe are issued,
the read-modify-write and reconstruct-write procedures are bypassed in
favor of a full-stripe-write operation. A full-stripe write operation is faster
than the other RAID-5 write procedures because it does not require any
read operations. As shown in Figure 8-8, a full-stripe write operation
consists of the following steps:
1.

XOR is applied to the new data to produce new parity.

2.

The new data and parity are written in a single write operation.
New Data
0

XOR
0

Stripe
Unit 0

Stripe
Unit 1

Stripe
Unit 2

Stripe
Unit 3

Stripe
Unit 4

Parity

Figure 8-8

Full-Stripe-Write Operation

Only a single write operation is necessary in the example shown in


Figure 8-8.
Note In some cases, it is beneficial to reduce the number of RAID-5
columns to force more full-stripe write operations. This reduction can
enhance overall write performance for some applications that use
random-length write operations.

8-14

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Demonstrating Performance Differences

Exercise: Demonstrating Performance Differences


In this exercise, you complete the following tasks:

Review key lecture points

Perform a RAID-5 write performance test

Perform a striped volume write performance test

Preparation
Unless your instructor says otherwise, the instructor conducts the
performance demonstrations on the VxVM server system.
First complete the questions in Task 1 Reviewing Key Lecture Points
on page 8-16.

VERITAS Volume Manager Performance Management


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

8-15

Exercise: Demonstrating Performance Differences

Task 1 Reviewing Key Lecture Points


Answer the following questions about performance planning and
monitoring.
1.

What are the two most important areas of interest when researching
volume placement?
a.

Volume structure

b.

Volume activity levels

c.

Volume size

d.

Volume log locations

The answers are b and d.

2.

Why is it important to leave a small amount of free space on each


disk drive?
a.

To relocate the private region

b.

To adjust cylinder boundaries

c.

To relocate log space

d.

To create snapshot volumes

The answer is c.

3.

Which RAID structure usually has the best write performance?


a.

RAID 5

b.

Mirrored

c.

Concatenated

d.

Striped

The answer is d.

4.

When is the preferred-plex read policy used?


a.

When a RAID-5 stripe is on a faster disk drive

b.

When one mirror has better read performance

c.

When no mirror has a performance advantage

The answer is b.

8-16

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Demonstrating Performance Differences


5.

Which of the following commands sets read policies?


a.

vxedit

b.

vxvol

c.

vxdisk

The answer is b.

6.

What are the advantages of mirroring across controllers?


a.

Availability

b.

Performance

c.

Administration

d.

Tuning

The answers are a and b.

7.

What is the primary VxVM command for displaying performance


statistics?
a.

vxtrace

b.

vxtask

c.

vxstat

The answer is c.

8.

What should be done prior to starting a VxVM volume performance


test?
a.

Clear the VxVM statistics

b.

Empty the system logs

c.

Initialize the syslogd daemon

The answer is a.

9.

What data pattern gives the best RAID-5 write performance?


a.

Less than 30 percent stripe-width writes

b.

Near 80 percent stripe-width writes

c.

Full stripe-width writes

The answer is c.

VERITAS Volume Manager Performance Management


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

8-17

Exercise: Demonstrating Performance Differences

Task 2 Performing a RAID-5 Write Performance Test


The following demonstration is performed by the instructor unless you
have been given other directions. The performance test uses the dd
command to write directly to an unmounted raw VxVM volume.
This demonstration is designed to be performed on a 30-Mbyte RAID-5
5+0 volume named r5vol that is in a disk group named dgX, and a
5-column striped volume named stdemo that is also 30 Mbytes in size.
There is a lab demonstration file named r5demo.sh that is used to
perform a series of performance tests. Alternately, you can enter the
command manually to perform the performance test.

Create the Demonstration Volume and Test Data


Complete the following steps:
1.

Log in to the VxVM server as user root.

2.

Complete the following steps:

3.

a.

Create a disk group named dgX containing a minimum of 5


disks.

b.

Use the vxdiskadm utility to ensure the disk type is sliced.

Create a 5-column, no log, RAID-5 volume with 30 Mbytes.


# vxassist -g dgX make r5demo 30m \
layout=raid5,nolog \
dgX01 dgX02 dgX03 dgX04 dgX05

Note The default stripe unit size is 32 blocks (16 Kbytes).


4.

Check for space, and then create a 20-Mbyte test file.


# df -kl /
# mkfile 20m /testfile

8-18

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Demonstrating Performance Differences

Enable Classroom-Wide Display


Complete the following steps:
Note In this task, all steps are directed toward the instructor. The words
you and your refer to the instructor, not the students.
The instructor should complete the following steps:
1.

Verify that the students remotely log in to the VxVM server.

2.

Verify that each workgroup gives you their terminal identifier by


typing the tty command. Record the identifiers here:
__________ __________ __________
__________ __________ __________

3.

Direct the output of your window to all of the students terminals.


For example:

# script /dev/null | tee /dev/pts/2 | tee /dev/pts/3

VERITAS Volume Manager Performance Management


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

8-19

Exercise: Demonstrating Performance Differences

Perform a RAID 5 Volume Demonstration


Complete the following steps:
1.

Run the r5demo.sh demonstration test file on the VxVM server.


You can also manually repeat the following command sequence,
each time incrementing the block size of the dd command in the
following sequence: 17, 33, 49, 65, 80, 64.

# vxstat -g dgX -r r5demo


# /usr/proc/bin/ptime dd if=/testfile \
of=/dev/vx/rdsk/dgX/r5demo bs=17k
# vxstat -g dgX -f MWF -v r5demo

Note As you move past the 50-percent stripe write into full-stripe write,
the I/O should move through the three write categories (M, W, and F). The
full stripe width is 16 Kbytes times 5 columns, which is equal to
80 Kbytes.
Tell the class that a full-stripe write is only four of the five columns. One column is always used for parity. This
is evident in the last test loop using 64-Kbyte transfer size.

The vxstat -f MWF options have the following meanings:

2.

8-20

Read-modify-write statistics

Reconstruct write statistics

Full-stripe write statistics

Kill the tee processes to end the demonstration. Do not kill the
script process, it terminates after the tee processes are gone.

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Demonstrating Performance Differences

Task 3 Performing a Striped Volume Write


Performance Test
If time permits, create a second volume in a striped format with
5 columns, and run the performance test again. Use the same test file data.
Complete the following steps:
1.

Use the vxassist command to create a 5-column striped volume.


# vxassist -g dgX make stdemo 30m \
layout=striped \
dgX01 dgX02 dgX03 dgX04 dgX05

2.

Run the r5demo.sh script again, but substitute the name and path of
the new striped volume, stdemo.

Enter disk group name (default: dgX)


Enter the name of the demo volume (default: r5demo) stdemo
Enter the raw path to the demo volume (default: /dev/vx/rdsk/dgX/r5demo)
/dev/vx/rdsk/dgX/stdemo
Enter data file location (default: /testfile)

Note The vxstat command does not display any statistics for the
striped volume, but the ptime results are informative.

VERITAS Volume Manager Performance Management


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

8-21

Exercise Summary

Exercise Summary

Discussion Take a few minutes to discuss what experiences, issues, or


discoveries you had during the lab exercises.

Manage the discussion here based on the time allowed for this module, which was given in the About This
Course module. If you find you do not have time to spend on discussion, highlight just the key concepts
students should have learned from the lab exercise.

Experiences

Ask students what their overall experiences with this exercise have been. Go over any trouble spots or
especially confusing areas at this time.

Interpretations

Ask students to interpret what they observed during any aspects of this exercise.

Conclusions

Have students articulate any conclusions they reached as a result of this exercise experience.

Applications

Explore with students how they might apply what they learned in this exercise to situations at their workplace.

8-22

VERITAS Volume Manager 4.0 Administration


Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D