A Complete Blog for Unix System Administrators

Wednesday, 28 November 2012

Netbackup Important Commands and Explanations
Master Server 1) Check the license details /usr/openv/netbackup/bin/admincmd/get_license_key 2) Stop and Start the netabackup services i) /etc/init.d/netbackup stop (start) > graceful stop and start ii) /usr/openv/netbackup/bin/bp.kill_all > Stop backup including GUI sessions, ungraceful iii) /usr/openv/netbackup/bin/bp.start_all > Start the backup iv) /usr/openv/netbackup/bin/initbprd > starts the master server v) /usr/openv/netbackup/bin/vmd > starts the media server vi) /usr/openv/netbackup/bin/jnbSA > Starts the GUI sessions 3) Scan the tape devices #sgscan (in Solaris) #/usr/openv/volmgr/bin/scan (in AIX) 4) Display all the netbackup process #bpps x 5) Check the backup status In GUI > Activity monitor In CLI > #bpdbjobs -report 6) Lists the server errors #bperror #bperrror U problems hoursago 1 #bperror U backstat -by_statcode -hoursago 1 7) Display information about the error code #bperror S <statuscode> 8) Reread bp.conf file without stop/start the netbackup #bprdreq -rereadconfig

Media Server (starts with bpxxx ) 1) List the storage units #bpstulist U 2) List media details # /usr/openv/netbackup/bin/goodies/available_media This cmd is retrieving all info about media that is available/full/expired/Frozen 3) List all the netbackup jobs #bpdbjobs report <hoursago> 4) Freeze or Unfreeze media In CLI, #bpmedia unfreeze [-freeze] ev <media ID> 5) List media details #bpmedialist -ev <media ID> 6) List the media contents #bpmedialist U mcontents m <mediaID> 7) List the information about NB images on media #bpimmedia mediaid <ID> -L 8) List backup image information #bpimagelist -U (general) # bpimagelist -media U (for media) 9) Expire a tape # bpexpdate d 0 ev <mediaID> -force 10) Expire client images #bpimage cleanup allclients 11) Which tapes are used for taking backup In GUI, Backup and Restore > Find the Filesystem > Preview Media Button In CLI, #bpimagelist media U Volume Commands (starts with vmxxx) 1) Tape Drive (vmoprcmd) 1) List the drive status, detail drive info and pending requests In GUI, Device mgmt

In CLI, #vmoprcmd #vmoprcmd d ds (status) #vmopcrmd d pr (pending requests) 2) Control a tape device In GUI, Device mgmt In CLI, #vmoprcmd [-reset] [-up] [-down] <drive number> 2) Tape Media commands (vmpool,vmquery,vmchange,vmdelete) 1) List all the pools In CLI, #vmpool listall -bx 2) List the scratch pool available #vmpool -list_scratch 3) List tapes in pool In CLI, #vmquery pn <pool name> -bx 4) List all tapes in the robot In CLI, #vmquery rn 0 bx 5) List cleaning tapes In CLI, #vmquery mt dlt_clean bx 6) List tape volume details #vmquery m <media ID> 7) Delete a volume from the catalog #vmdelete m <mediaID> 8) Changes a tapes expiry date #vmchange -exp 12/31/2012 hr:mm:ss m <media ID> 9) Changes a tapes media pool #vmchange -p <pool number> -m <media ID> 3) Tape/Robot commands (starts with tpxxx) 1) List the tape drives #tpconfig d 2) List the cleaning times on drives #tpclean -L

3) Clean a drive #tpclean C <drive number> Client Commands i) List the clients #bpplclients Policy Commands i) List the policies #bppllist U ii) List the detailed information about the policies #bpplist U -allpolicies

Netbackup Cheat Sheet Master Server Daemons/Processes Request daemon bprd Scheduler bpsched (started with bprd) Netbackup database manager bpdbm (started with bpsched) Job Monitor bpjobd (started with bpdbm) Media Server Daemons/Processes Communications daemon bpcd Backup and restore manager bpbrm (started with bpcd) Tape Manager bptm (started with bpbrm) Disk Manager bpdm (started with bpbrm) Media Manager ltid Bar code reader avrd (started with ltid) Remote device management/ controls volume database vmd (started with ltid) Roboticdaemon (one on each media server) talks to tldcd tldd (started with ltid) Robotic control daemon talks to the robot directl via scsi tldcd (started with ltid) Catalogs

Master Server Information about backed-up files Storage Unit, Global Configuration, Catalog backup configuration. Backup Policy information Job status information Netbackup logs with error and status information Information on volumes, volume pools, scratch pool and volume groups Media Server Tracks assigned volumes (media that has data them) Information about devices managed by the media server Log and Information Files Netbackup and Patch versions Media Version Patch Level history Buffer size Number of buffers Network Buffer Size Java GUI authorisation Catalog type (binary or ASCII) Netbackup and media manager parameter files Corrupt Database image files (5.0 and above) Server Commands Check license details Start Netbackup

image - /opt/openv/netbackup/db config - /opt/openv/netbackup/db class - /opt/openv/netbackup/db jobs - /opt/openv/netbackup/db error - /opt/openv/netbackup/db volume /opt/openv/volmgr/database media - /opt/openv/netbackup/db device /opt/openv/volmgr/database

/opt/openv/netbackup/bin/version /opt/openv/volmgr/version /opt/openv/netbackup/patch/patch.history /opt/openv/netbackup/db/config/SIZE_DATA_BUFFERS /opt/openv/netbackup/db/config/NUMBER_DATA_BUFFERS /opt/openv/netbackup/NET_BUFFER_SZ (default = 32) /opt/openv/java/auth.conf /opt/openv/netbackup/db/config/cat_format.cfg /opt/openv/netbackup/bp.conf /opt/openv/volmgr/vm.conf /opt/openv/netbackup/db.corrupt

/opt/openv/netbackup/bin/admincmd/get_license_key netbackup start /opt/openv/netbackup/bin/initbprd (master) /opt/openv/volmgr/bin/vmd (media) netbackup stop /opt/openv/netbackup/bin/admincmd/bprdreq -terminate (master) /opt/openv/netbackup/bin/bpdbm -terminate (master)

Stop Netbackup (does not disconnect GUI sessions)

Stop Netbackup and kill all GUI sessions Start the GUI

/opt/openv/netbackup/bin/goodies/bp.kill_all

/opt/openv/netbackup/bin/jnbSA sgscan (solaris) Scan for tape devices ioscan (HPUX) Display all Netbackup processes bpps -a bperror -U -problems -hoursago <number of hours> lists servers errors bperror -U -backstat -by_statcode -hoursago <number of hours> display information on a error code bperror -statuscode <statuscode> [-recommendation] Reread bp.conf file without bprdreq -rereadconfig stopping Netbackup bpdbm -consistency 1 bpdbm -consistency 2 Check database consistency Check for the below lines: Bad image header Does not exist

Netbackup Recovery bprecover -l -m <media ID> -d dlt (listing) bprecover -r -m <media ID> -d dlt (recovering) bprecover -l -tpath <tape_path> (listing) Device catalog is gone or corrupted bprecover -r -tpath <tape_path> (recovering) bprecover -l -dpath <disk_path> (listing) Disk backups bprecover -r -dpath <disk_path> (recovering) Device catalog is intact Volume Commands Tape Drive and Inventory Commands List drive status, detail drive info and pending requests List the tape drive status List the pending requests Control a tape device List all changes in the robot(but do not update) Empty the robot and reinventory (using barcodes) Tape Media Commands List all pools

vmoprcmd vmoprcmd -d ds vmoprcmd -d pr vmoprcmd [-reset][-up][-down] <drive number> vmupdate -recommend -rt tld -rn 0 vmcheckxxx -rt tld -rn 0 -recommend vmupdate -rt tld -rn <robot number> -rh <silo slave> -vh <host> -nostderr -use_barcode_rules -use_seed -empty_ie vmpool -listall -bx

List tapes in pool List all tapes in the robot List cleaning tapes List tape volume details Delete a volume from the catalog Change a tapes expiry date Change a tape's media pool Media commands

vmquery -pn <pool name> -bx vmquery -rn 0 -bx |grep 'TLD' | sort +4 vmquery -mt dlt_clean -bx vmquery -m <media ID> vmdelete -m <media ID> vmchange -exp 12/31/06 23:59:58 -m <media ID> vmchange -p <pool number> -m <media ID>

List the storage units Freeze or unfreeze media List media details List media contents List backup Image Information Expire client images Expire a tape List all netbackups jobs Move media from one media server to another Tape/Robot commands List tape drives List cleaning times on drives clean a drive change a drives cleaning frequency set a drives cleaning time to zero

bpstulist -U bpmedia [-freeze][-unfreeze] -ev <media ID> bpmedialist -ev <media ID> bpmedialist -U mcontents -m <media ID> bpimagelist -backupid <image ID> bpimage -cleanup -allclients bpexpdate -d 0 -ev <media ID> -force bpdbjobs -report [-hoursago] bpmedia -movedb -newserver <media server> -oldserver <media server>

tpconfig -d tpclean -L tpclean -C <drive number> tpclean -F <drive> <frequency> tpclean -M <drive> robtest commands that can be used are as follows:

ss (show slots) sd (show drives) Move tapes within robot using robtest si (show load port) m s250 d5 (move tape from slot 250 into drive 5) uload d5 (unload tape from drive 5) m d5 s250 (move tape from drive 5 to slot 250) m s250 i1 (mov tape from slot 250 to load port 1)

List load port tapes List all slot contents List tape drive contents Move a tape in s100 to drive 1 Move a tape to load port 1 Archiving Commands

echo "s i q" | tldtest -r /dev/sg/c0t4l0 echo "s s q" | tldtest -r /dev/sg/c0t4l0 echo "s d q" | tldtest -r /dev/sg/c0t4l0 echo "m s100 d1" | tldtest -r /dev/sg/c0t4l0 echo "m s100 i1" | tldtest -r /dev/sg/c0t4l0

bpcatlist -client all -before Jul 01 2006 bpcatlist -client all -before Aug 01 2006 archive and remove images bpcatlist -before Jul 01 2006 | bpcatarc | bpcatrm restore archive files bpcatlist -before Jul 01 2006 | bpcatres list archive info Client commands bpclntcmd [-ip <ip addres>] bpclntcmd [-hn <hostname>] test client connectivity bpclntcmd [-pn] bpclntcmd [-sv]

veritas netbackup tricks and commands Netbackup Tips and Tricks (Unix)

Glossary Term Definition CLI Command Line Interface GUI Graphical User Interface Media Server

Master Server

Starting and Stopping Netbackup Stopping Netbackup /usr/openv/netbackup/bin/K77netbackup --> graceful shutdown /usr/openv/netbackup/bin/bpps -a --> check for any remaining processes /usr/openv/netbackup/bin/goodies/bp.kill_all ---> kills all remaining netbackup processes, not necessarily graceful /usr/openv/netbackup/bin/bpps -a --> check for any remaining processes kill -9 for any remaining. NOTE: unkillable processes may require a reboot Starting Netbackup /usr/openv/netbackup/bin/S77netbackup --> after bp.kill_all, to restart /usr/openv/volmgr/bin/vmd Common Tasks Starting the Administration GUI java from the windows client

 x-windows from the server - /usr/openv/netbackup/bin/xnb & Checking Backup Status Activity Monitor or /usr/openv/netbackup/bin/admincmd/bpdbjobs -report Cleaning a tape manually Identify the drive name to be cleaned o tpclean -L Manually clean the drive: o tpclean -C Determining what tapes were used for a backup GUI o Backup and Restore --> Find the file system --> Preview Media Button CLI o Find the correct backup images bpimagelist -U -client -d -e o Find the media used for those images bpimagelist -U -client -d -e -media

Listing the files in a backup

Find the tape(s) used (above procedure using bpimagelist) cd /usr/openv/netbackup/db/jobs/done Run the following script and redirect it's output to a text file: for file in `grep MOUNTING *|grep |awk '{print $1}'|sed 's/:MOUNTING//'` do echo $file grep PATH_WRITTEN $file|awk '{print $3}' echo " " echo "==========================================En d of Image======================================" echo " " done

This process works for NBU V3.4:

 cd /usr/openv/netbackup/db/images/ ls -ltr --> this will identify the directory with the proper date verify directory with "bpdbm -ctime cd ls -ltr --> lists all of the backups for this client on this date cat __.f | awk '{print $10}' --> this prints out the files in the backup

For NBU > V3.4 bpflist --help --> undocumented netbackup command to list files from a binary .f file

Inventory the Robot Inventory Robot --> /opt/openv/volmgr/bin/vmcheckxxx -rt robot_type -rn robot_number -list (where robot_type is tld, acs, . . .) Inventory Robot and Update Configuration --> /opt/openv/volmgr/bin/vmupdate -rt robot_type -rn robot_number -list (where robot_type is tld, acs, . . .)

Listing Properties of the Volume Pools vmpool -listall Scratch Tapes Count scratch tapes: /usr/openv/volmgr/bin/vmquery -pn Scratch | grep -c "robot slot" Moving tapes to the scratch pool o If Needed - Expire the tape bpexpdate -ev -d 0 -force -host o Move the tape vmchange -p 2 -m Checking Drive Usage /usr/openv/volmgr/bin/vmoprcmd Taking a drive down or up /usr/openv/volmgr/vmoprcmd -down /usr/openv/volmgr/vmoprcmd -up

Performing a Restore From the GUI o user backup & restore --> configuration --> client

o user backup & restore --> configuration --> client to restore o directory to search o directory depth o date range o file --> browse backups for restore Adding New Tapes to the Library Using the GUI o Media Management --> Actions --> New --> Single Volume . . --> o Media Type (ie DLT) o Robot Type (ie TLD) o Media ID (from Inventory) o Slot Number (from Inventory) o Robot Number (ie 0) o Volume Group o Volume Pool (ie Scratch) Using the CLI o vmadd -m -mt -verbose -rt -b -rn -rc1 -p -mm

vmpool -listall --> lists all pools, both name and number o For example: vmadd -m 000151 -mt dlt -verbose -rt tld -b 000151 -rn 0 -rc1 8 -p 2 -mm 0 Re-using Tapes from other systems or older Netbackups Expire the media o bpexpdate -ev MEDIA_ID -d 0 -force -host HOST Deassign the media o vmquery -deassignbyid MEDIA_ID 4 0 Move to the scratch pool o vmchange -m MEDIA_ID -p POOL# Relabel the media o bplabel -ev CIM572 -d dlt -p Scratch Changing the attributes of media Changing the barcode o vmchange -barcode CYM100D -m CYM100 Changing the Volume Pool o vmchange -m MEDIA_ID -p POOL#

To expire media

 bpexpdate -ev -d 0 -force -host To unfreeze media List the frozen media o /usr/openv/netbackup/bin/goodies/available_media | grep -i FROZEN Unfreeze the media o bpmedia -unfreeze -ev -h To relabel a tape bplabel -ev -d -p bplabel -ev 000687 -d dlt -p TriVrgt_OFFSITE To remove media from the Netbackup database Verify that there are no images on the tape o bpimmedia -mediaid 000687 -L Expire the tape o bpexpdate -ev 000687 -d 0 -host scorpius -force Get the status and pool number of the tape o vmquery -m 000687 Deassign the tape o vmquery -deassignbyid

o vmquery -deassignbyid 000687 4 0x0 Delete the tape o vmdelete -m 000687

Installing the Netbackup Client /update_clients -ForceInstall -ClientList /tmp/clients.lst o requires that TMPDIR and TEMPDIR be set correctly Excludng files from backup on a client Create /usr/openv/netbackup/exclude_list Put the file specifications of the files/directories to be excluded o /mnt/directory/* Displaying Information about a Tape vmquery -m --> Displays attributes about a particular tape bpmedialist -U -mcontents -ev 000687 --> Displays media contents bpmedialist -U -mlist --> List of all media bpmedialist -U -mlist -ev CYM966 --> Listing of a particular media id

 bpimmedia -mediaid 000687 -L --> Listing of images on a tape Robtest Commands Starting robtest o robtest o 1 --> to select TLD 0 Getting help o? Looking at contents of the tape drives osd Looking at the contents of the library oss Moving a tape from a drive to a library slot o s d --> to identify drive number that has tape (Contains Cartridge = yes, Barcode=XXXXXX) o s s --> to identify an empty slot in the tape library (Netbackup will need to be re-inventoried) o m d# s# --> from from drive # to slot # o s d --> verify the tape drive is empty o s s --> verify the library slot has the tape

Configuration Files /usr/openv/netbackup/bp.conf configuration file, sets backup server and backup clients force statement must be correct client to browse from client to restore to /usr/openv/volmgr/vmconf Logfiles To utilize logfiles, create the corresponding directory in /usr/openv/netbackup/logs

Server Logfile directories: admin - adminstrative commands bpbrm - backup and restore manager bpcd - client daemon bpdbjobs - database manager program process bpdm - disk manager process

 bpjava-msvc - Java application server authentication service bpjava-usvc - process that services Java requests bprd - request daemon process bpsched - scheduler process that runs on master servers bptm - tape/optical media management process user-ops - required directory for use by Java programs xbpadm - X based administration utility xbpmon - X based job monitor process Client Logfile directories: bp - client user interface process bparchive - archive program bpbackup - backup program bpbkar - program that generates golden images bpcd - client daemon bpjava-msvc - Java application server authentication service bpjava-usvc - process that services Java requests bplist - program that lists backed up and archived files

 bpmount - program that determines local mountpoints and wildcard expansion for multiple streams bphdb - Oracle database backup program start process db_log - database specific extension log tar - tar process log during restores user_ops Media Manager logging automatically goes to the system log using syslogd logging facility .Logging will only occur if these directories are created. These directories will generate a lot of data and should be deleted when no longer necessary.

To increase the amount of logging information set VERBOSE=2 in /usr/open/netbackup/bp.conf (default is VERBOSE=1) Processes ltid acsd vmd

Useful Commands

bpcllist - list classes bpclinfo -L --> displays info about a class vmpool - volume pools vmpool -listall vmpool -listscratch bplabel -ev -d hcart bpbackup db --> backs up the catalog bpclclients --> lists the clients for a particular policy (class) Troubleshooting bperror -statuscode <-- displays information about the netbackup error.

No Backups are running: Check system log file for error messages Stop and restart all the netbackup processes Look for a downed drive o /usr/openv/volmgr/bin/vmoprcmd

o /usr/openv/volmgr/bin/vmoprcmd -up 0 --> this will bring up drive 0 if it's control shows as down Look for pending requests o /usr/openv/volmgr/bin/vmoprcmd or gui --> device management o If there is a pending request either re-assign it to a drive, or deny the request Downed drive does not come back up or does not stay up Check for a hardware problem by looking for messages on the tape library Make sure there is not a tape stuck in the drive o Use robtest (described above) to look at the drives If there is a tape stuck in the drive, try to remove it using robtest If robtest fails, then you must manually remove it. Verify the Client is communicating properly: bpclncmd -ip --> from both client and server bpclntcmd -hn --> from both client and server bpclntcmd -pn --> from client only

Device Actions

Device Management --> info about tape drives dlt hcart (ultrium) Media Actions Media id must agree with # of the tape Create a media id actions -->new-->single volume-->dlt cart (not dlt2) put it into the "netbackup" volume pool Netbackup Client

To check things out do this:

It could be a couple things. Mostly DNS, bp.conf, or something stupid. On the client run this command

/usr/openv/netbackup/bin/bpclntcmd -pn

/usr/openv/netbackup/bin/bpclntcmd -server "server name"

/usr/openv/netbackup/bin/bpclntcmd ip "ip_address"

One of these usually fails and your able to fix it right off

1074 ./bpclntcmd -hn corpbu1 1075 ./bpclntcmd -ip 10.194.1.129 1076 ping 10.194.1.129 1077 ./bpclntcmd -hn corpldv1 1078 ./bpclntcmd -hn corpbu1.corporate.vox.net 1079 ping corpldv1 1080 ./bpclntcmd -ip 10.194.1.120

Must be able to resolve correctly from the master server and the client or it will not work!!!

Netbackup Command Master Server Daemons/Processes Request daemon bprd Scheduler bpsched (started with bprd) Netbackup database manager bpdbm (started with bpsched) Job Monitor bpjobd (started with bpdbm) Media Server Daemons/Processes Communications daemon bpcd Backup and restore manager bpbrm (started with bpcd) Tape Manager bptm (started with bpbrm) Disk Manager bpdm (started with bpbrm) Media Manager ltid Bar code reader avrd (started with ltid) Remote device management/ controls volume database vmd (started with ltid) Roboticdaemon (one on each media server) talks to tldcd tldd (started with ltid) Robotic control daemon talks to the robot directl via scsi tldcd (started with ltid) Catalogs

Master Server Information about backed-up files image /opt/openv/netbackup/db Storage Unit, Global Configuration, Catalog backup configuration. config - /opt/openv/netbackup/db Backup Policy information class - /opt/openv/netbackup/db Job status information jobs - /opt/openv/netbackup/db Netbackup logs with error and status information error /opt/openv/netbackup/db Information on volumes, volume pools, scratch pool and volume groups volume - /opt/openv/volmgr/database Media Server Tracks assigned volumes (media that has data them) media /opt/openv/netbackup/db Information about devices managed by the media server device - /opt/openv/volmgr/database Log and Information Files Netbackup and Patch versions /opt/openv/netbackup/bin/version Media Version /opt/openv/volmgr/version Patch Level history /opt/openv/netbackup/patch/patch.history

Buffer size /opt/openv/netbackup/db/config/SIZE_DATA_BUFFERS Number of buffers /opt/openv/netbackup/db/config/NUMBER_DATA_BUFFE RS Network Buffer Size /opt/openv/netbackup/NET_BUFFER_SZ (default = 32) Java GUI authorisation /opt/openv/java/auth.conf Catalog type (binary or ASCII) /opt/openv/netbackup/db/config/cat_format.cfg Netbackup and media manager parameter files /opt/openv/netbackup/bp.conf /opt/openv/volmgr/vm.conf Corrupt Database image files (5.0 and above) /opt/openv/netbackup/db.corrupt Server Commands Check license details /opt/openv/netbackup/bin/admincmd/get_license_key Start Netbackup netbackup start

/opt/openv/netbackup/bin/initbprd (master) /opt/openv/volmgr/bin/vmd (media)

Stop Netbackup (does not disconnect GUI sessions) netbackup stop /opt/openv/netbackup/bin/admincmd/bprdreq -terminate (master) /opt/openv/netbackup/bin/bpdbm -terminate (master) Stop Netbackup and kill all GUI sessions /opt/openv/netbackup/bin/goodies/bp.kill_all Start the GUI /opt/openv/netbackup/bin/jnbSA Scan for tape devices sgscan (solaris) ioscan (HPUX) Display all Netbackup processes bpps -a lists servers errors bperror -U -problems -hoursago bperror -U -backstat -by_statcode -hoursago display information on a error code bperror -statuscode [recommendation] Reread bp.conf file without stopping Netbackup bprdreq rereadconfig Check database consistency bpdbm -consistency 1 bpdbm -consistency 2 Check for the below lines: Bad image header

Does not exist Netbackup Recovery Device catalog is intact bprecover -l -m -d dlt (listing) bprecover -r -m -d dlt (recovering) Device catalog is gone or corrupted bprecover -l -tpath (listing) bprecover -r -tpath (recovering) Disk backups bprecover -l -dpath (listing) bprecover -r -dpath (recovering) Volume Commands Tape Drive and Inventory Commands List drive status, detail drive info and pending requests vmoprcmd List the tape drive status vmoprcmd -d ds List the pending requests vmoprcmd -d pr Control a tape device vmoprcmd [-reset][-up][-down] List all changes in the robot(but do not update) vmupdate recommend -rt tld -rn 0 vmcheckxxx -rt tld -rn 0 -recommend

Empty the robot and re-inventory (using barcodes) vmupdate -rt tld -rn -rh -vh -nostderr -use_barcode_rules use_seed -empty_ie Tape Media Commands List all pools vmpool -listall -bx List tapes in pool vmquery -pn -bx List all tapes in the robot vmquery -rn 0 -bx |grep 'TLD' | sort +4 List cleaning tapes vmquery -mt dlt_clean -bx List tape volume details vmquery -m Delete a volume from the catalog vmdelete -m Change a tapes expiry date vmchange -exp 12/31/06 23:59:58 -m Change a tape's media pool vmchange -p -m Media commands List the storage units bpstulist -U Freeze or unfreeze media bpmedia [-freeze][-unfreeze] -ev List media details bpmedialist -ev List media contents bpmedialist -U mcontents -m List backup Image Information bpimagelist -backupid

Expire client images bpimage -cleanup -allclients Expire a tape bpexpdate -d 0 -ev -force List all netbackups jobs bpdbjobs -report [-hoursago] Move media from one media server to another bpmedia movedb -newserver -oldserver Tape/Robot commands List tape drives tpconfig -d List cleaning times on drives tpclean -L clean a drive tpclean -C change a drives cleaning frequency tpclean -F set a drives cleaning time to zero tpclean -M Move tapes within robot using robtest robtest commands that can be used are as follows:

s s (show slots) s d (show drives) s i (show load port) m s250 d5 (move tape from slot 250 into drive 5) uload d5 (unload tape from drive 5)

m d5 s250 (move tape from drive 5 to slot 250) m s250 i1 (mov tape from slot 250 to load port 1)

List load port tapes echo "s i q" | tldtest -r /dev/sg/c0t4l0 List all slot contents echo "s s q" | tldtest -r /dev/sg/c0t4l0 List tape drive contents echo "s d q" | tldtest -r /dev/sg/c0t4l0 Move a tape in s100 to drive 1 echo "m s100 d1" | tldtest -r /dev/sg/c0t4l0 Move a tape to load port 1 echo "m s100 i1" | tldtest -r /dev/sg/c0t4l0 Archiving Commands list archive info bpcatlist -client all -before Jul 01 2006 bpcatlist -client all -before Aug 01 2006 archive and remove images bpcatlist -before Jul 01 2006 | bpcatarc | bpcatrm restore archive files bpcatlist -before Jul 01 2006 | bpcatres Client commands test client connectivity bpclntcmd [-ip ] bpclntcmd [-hn ]

bpclntcmd [-pn] bpclntcmd [-sv]

1.To expire the tape

rakshak:/usr/openv/netbackup/bin/admincmd [23] > ./bpexpdate -m SVOB06 -d 0

rakshak:/usr/openv/volmgr/bin [40] > ./vmdelete -m SVOB02

/var/adm [1] >> media

/var/adm [2] >> bpps -a NB Processes -----------root 536 1 0 Dec 23 ? 0:00 /bin/sh /usr/openv/netbackup/bin/admincmd/nbdbdmon --user=root

root 548 536 0 Dec 23 ? 0:00 /usr/openv/db/bin/nbdbd -basedir=/usr/openv/db --datadir=/usr/openv/db/var --u root 441 1 0 Dec 23 ? 0:17 /usr/openv/netbackup/bin/bprd root 476 1 0 Dec 23 ? 3:23 /usr/openv/netbackup/bin/bpdbm root 12691 441 0 23:30:19 ? 0:12 /usr/openv/netbackup/bin/bpsched -ppid 441 /var/adm [12] >> bpps -a to kill the GUI of rakshak

root 11142 11120 0 09:25:25 pts/4 1:06 /usr/openv/java/jre/bin/java Dvrts.NBJAVA_CONF=/usr/openv/java/nbj.conf -Djava root 27301 27279 0 Jan 11 pts/2 5:31 /usr/openv/java/jre/bin/java Dvrts.NBJAVA_CONF=/usr/openv/java/nbj.conf -Djava

To check the media list

/usr/openv/netbackup/bin/admincmd [15] > media

Netbackup reboot Hi Jerome,

Following timing where I am going to run the backup: ( Please check the below ones, because it's full backup and I have not run from last 2 months )

ZIN05VIEW01:10:15 Sat morning ZIN05VIEW02:10:30 Sat morning ZIN05VIEW03:10:45 Sat morning

Rakshak:23:00 Sat night ZIN05VOB03:23:00 Sat Night

Check the below steps , If you rebooting the vobserver and veiew servers.

A. Telnet to rakshak B. Set the display C. Go to /usr/openv/netbackup/bin and then run ./jnbSA &

D. Gui will open then on the password tab give the rakshak root passowrd. E. Check any active job is running by clicking activity monitor. F. If at all any jobs pertaining to critical servers are running, then select the jobs and cancel.

If you are rebooting VIEW01 VIEW02 VIEW03 VOB03 Then check the follwing process are running or not

1. Go to /usr/openv/volmgr/bin/sgscan

zin05view01:/usr/openv/volmgr/bin # ./sgscan # #WARNING: detected StorEdge Network Foundation connected devices not in

# SG configuration file: # # Device World Wide Port Name 21000004cf277475 # Device World Wide Port Name 21000004cf9b645a # Device World Wide Port Name 21000004cfa17ac8 # Device World Wide Port Name 5080020000197169 # Device World Wide Port Name 21000004cfa17b08 # Device World Wide Port Name 500000e010136591 # Device World Wide Port Name 21000004cfa17211 # Device World Wide Port Name 21000004cf9b61fb # Device World Wide Port Name 21000004cf9b6ada # Device World Wide Port Name 21000004cf9b6aee # Device World Wide Port Name 21000004cf9b6898 # Device World Wide Port Name 21000004cf9b61df # Device World Wide Port Name 22000004cf9b645a # Device World Wide Port Name 22000004cf277475 # Device World Wide Port Name 22000004cfa17ac8 # Device World Wide Port Name 508002000019716a # Device World Wide Port Name 22000004cf9b61fb

# Device World Wide Port Name 22000004cfa17211 # Device World Wide Port Name 22000004cf9b6ada # Device World Wide Port Name 22000004cf9b6aee # Device World Wide Port Name 22000004cf9b6898 # Device World Wide Port Name 22000004cf9b61df # Device World Wide Port Name 22000004cfa17b08 # Device World Wide Port Name 500000e010136592 # # See /usr/openv/volmgr/MediaMgr_DeviceConfig_Guide.txt chapter # "Special configuration for "Sun StorEdge Network Foundation" HBA/Driver" # for information on how to use sg.build and sg.install to # configure these devices #

/dev/sg/c0tw2100001086103bacl0: Changer: "ATL M2500" /dev/sg/c0tw2100001086103bacl1: Tape (/dev/rmt/3): "QUANTUM SDLT320"

/dev/sg/c0tw2100001086103bacl2: Tape (/dev/rmt/2): "QUANTUM SDLT320" /dev/sg/c0tw2100001086104420l0: Tape (/dev/rmt/1): "QUANTUM SDLT320" /dev/sg/c0tw2100001086104420l1: Tape (/dev/rmt/0): "QUANTUM SDLT320"

If it's not showing then do

/etc/rc2.d/S99M2500 start

Again run the sgscan command

2. go to /usropenv/netbackup/bin/bpps -a

NB Processes -----------root 27315 1 0 Sep 22 ? 0:00 /bin/sh /usr/openv/netbackup/bin/admincmd/nbdbdmon --user=root root 27327 27315 0 Sep 22 ? 0:00 /usr/openv/db/bin/nbdbd -basedir=/usr/openv/db --datadir=/usr/openv/db/var --u

MM Processes -----------root 29066 28896 0 Sep 22 ? 0:00 avrd root 27238 1 0 Sep 22 ? 0:01 vmd root 28896 1 0 Sep 22 ? 0:02 /usr/openv/volmgr/bin/ltid root 29064 28896 0 Sep 22 ? 0:00 tldd root 526 27238 0 Sep 22 ? 0:01 rdevmi -sockfd 6 -r root 29903 27238 0 Sep 22 ? 0:01 rdevmi -sockfd 6 -r root 66 27238 0 Sep 22 ? 0:01 rdevmi -sockfd 6 -r root 15380 27238 0 11:07:44 ? 0:00 rdevmi -sockfd 6 -r

If it's not running then go to the below path to start the netbackup service.

/usr/openv/netbackup/bin/goodies/netbackup

3. On vob03 check vxtask list whether the flashsnap is running or not.

Hope you will understand my problem....

Regards,

Ragha.....

To freeze the tapes you need to login to the respective media servers

cd /usr/openv/netbackup/bin/admincmd zin05view03:/usr/openv/netbackup/bin/admincmd # ./bpmedia -freeze -m SVEW03

To check the media list 2 methods

1. avilable media it will displays only what all the media available and which media which belongs 2.bpmedialisd give all the information about what all the media availbles and media which belongs to the pools and when it allocated and expiration last date updated and read etc.. with respect to media servers.

1.rakshak:/ [1] > cd /usr/openv/netbackup/bin/admincmd/ rakshak:/usr/openv/netbackup/bin/admincmd [2] > ./bpmedialist

Server Host = rakshak

id rl images allocated last updated density kbytes restores vimages expiration last read <------- STATUS -------> ------------------------------------------------------------------------------DMON01 8 100 06/14/2004 15:13 N/A dlt2 2212471 0 MPX 10/19/2005 16:19 11/03/2004 19:13 IMPORTED

SDEL01 24 395 04/23/2004 09:45 12/09/2004 17:29 dlt2 211914147 2 MPX 12/07/2014 17:29 12/29/2004 12:18

SDLY01 5 2611 08/26/2004 21:05 01/12/2005 21:57 dlt2 323247463 0

Server Host = zin05view01

id rl images allocated last updated density kbytes restores vimages expiration last read <------- STATUS -------> ------------------------------------------------------------------------------SVEW02 8 3 12/31/2004 23:37 01/12/2005 23:30 dlt2 192245499 0 MPX 01/12/2006 23:30 N/A

SVEW06 8 317 12/22/2004 11:29 12/31/2004 23:37 dlt2 421554297 0 MPX 12/31/2005 23:37 N/A FULL

Server Host = zin05view02

id rl images allocated last updated density kbytes restores vimages expiration last read <------- STATUS -------> ------------------------------------------------------------------------------SVEW01 8 3 01/01/2005 02:40 01/12/2005 23:54 dlt2 225510259 0 MPX 01/12/2006 23:54 N/A

SVEW13 8 229 12/18/2004 01:02 01/01/2005 02:40 dlt2 398799500 0 MPX 01/01/2006 02:40 N/A FULL

SVEW14 8 210 12/18/2004 01:02 12/25/2004 01:37 dlt2 408836258 0 MPX 12/25/2005 01:37 N/A FULL

SVEW15 8 1 12/25/2004 01:37 12/25/2004 01:37 dlt2 13287112 0

Server Host = zin05view03

id rl images allocated last updated density kbytes restores vimages expiration last read <------- STATUS -------> ------------------------------------------------------------------------------SVEW03 8 2 01/11/2005 09:23 01/13/2005 00:55 dlt2 101592944 MPX 01/13/2006 00:55 N/A

SVEW10 8 14 12/18/2004 11:53 12/18/2004 12:04 dlt2 18070585 0 MPX 12/18/2005 12:11 N/A

SVEW11 8 44 12/18/2004 11:02 01/11/2005 09:23 dlt2 391753568 0 MPX 01/11/2006 09:23 N/A FULL

Prerequisites
Before attending this course, you should have: * One year of NetBackup UNIX administration * Two years experience as a UNIX system administrator (including networking) * Familiarity with backup and restore concepts and SAN concepts

Syllabus

Course Content
Upgrading NetBackup

Preparing for an Upgrade Upgrading to NetBackup 6.x

Analyzing and Detecting Problems

Analyzing Problems Detecting Problems Using the Activity Monitor and Reports Detecting Problems Using NBSupport Detecting Problems Using E-mail Notification Detecting Problems Using NOM

Process Flow

NetBackup Architecture and Processes Review Backup Process Flow Restore Process Flow

Using Debug Logs

Log Management Capturing Debug Logs Viewing Logs

NetBackup Database Troubleshooting

The NetBackup Relational Database (NBDB) NBDB Creation Process The EMM Domain EMM Server Backup Process Flow Protecting the NBDB Maintaining the NBDB

Troubleshooting Devices

Device Error Status Codes DOWN Drives: Isolating Tape Drive Problems Troubleshooting Shared Storage Tape Alert Drive Diagnostics

Troubleshooting Media

Media-Related Status Codes Troubleshooting Frozen and Suspended Media

Troubleshooting Network Issues

PBX and VNETD Usage NetBackup and TCP/IP Communications Communication Errors Physical Network Failures NetBackup Configuration Errors (?)

Performance Tuning

Performance Concepts Isolating Bottlenecks Addressing Bottlenecks Netbackup Cheat Sheet Master Daemons | Media Daemons | Catalog | Log Files | Server | Volume | Media | Tape/Robot | Archive | Client Master Server Daemons/Processes bprd bpsched (started with bprd) bpdbm (started with bpsched) bpjobd (started with bpdbm)

Request daemon Scheduler Netbackup database manager Job Monitor

Media Server Daemons/Processes bpcd bpbrm (started with bpcd) bptm (started with bpbrm) bpdm (started with bpbrm) ltid avrd (started with ltid)

Communications daemon Backup and restore manager Tape Manager Disk Manager Media Manager Bar code reader

Remote device management/ controls vmd (started with ltid) volume database Roboticdaemon (one on each media server) talks to tldcd Robotic control daemon talks to the robot directl via scsi

tldd (started with ltid) tldcd (started with ltid)

Catalogs Master Server image - /opt/openv/netbackup/db config - /opt/openv/netbackup/db class - /opt/openv/netbackup/db jobs - /opt/openv/netbackup/db error - /opt/openv/netbackup/db volume - /opt/openv/volmgr/database Media Server

Information about backed-up files Storage Unit, Global Configuration, Catalog backup configuration. Backup Policy information Job status information Netbackup logs with error and status information Information on volumes, volume pools, scratch pool and volume groups

Tracks assigned volumes (media that media - /opt/openv/netbackup/db has data them) Information about devices managed by the media server

device - /opt/openv/volmgr/database

Log and Information Files /opt/openv/netbackup/bin/version /opt/openv/volmgr/version /opt/openv/netbackup/patch/patch.history /opt/openv/netbackup/db/config/SIZE_DATA_BUFFERS /opt/openv/netbackup/db/config/NUMBER_DATA_BUFFERS /opt/openv/netbackup/NET_BUFFER_SZ (default = 32) /opt/openv/java/auth.conf /opt/openv/netbackup/db/config/cat_format.cfg /opt/openv/netbackup/bp.conf /opt/openv/volmgr/vm.conf

Netbackup and Patch versions Media Version Patch Level history Buffer size Number of buffers Network Buffer Size Java GUI authorisation Catalog type (binary or ASCII) Netbackup and media manager parameter files

Corrupt Database image files (5.0 and above)

/opt/openv/netbackup/db.corrupt

Server Commands /opt/openv/netbackup/bin/admincmd/get_license_key netbackup start

Check license details Start Netbackup

/opt/openv/netbackup/bin/initbprd (master) /opt/openv/volmgr/bin/vmd (media) netbackup stop

Stop Netbackup (does not disconnect GUI sessions)

/opt/openv/netbackup/bin/admincmd/bprdreq -terminate (master) /opt/openv/netbackup/bin/bpdbm -terminate (master) /opt/openv/netbackup/bin/jnbSA sgscan (solaris) ioscan (HPUX) bpps -a bperror -U -problems -hoursago <number of hours> bperror -U -backstat -by_statcode -hoursago <number of hours> bperror -statuscode <statuscode> [-recommendation] bprdreq -rereadconfig bpdbm -consistency 1 bpdbm -consistency 2

Stop Netbackup and kill all GUI sessions /opt/openv/netbackup/bin/goodies/bp.kill_all Start the GUI Scan for tape devices Display all Netbackup processes lists servers errors display information on a error code Reread bp.conf file without stopping Netbackup

Check database consistency

Check for the below lines: Bad image header Does not exist Netbackup Recovery bprecover -l -m <media ID> -d dlt (listing) bprecover -r -m <media ID> -d dlt (recovering) bprecover -l -tpath <tape_path> (listing) bprecover -r -tpath <tape_path> (recovering) bprecover -l -dpath <disk_path> (listing) bprecover -r -dpath <disk_path> (recovering)

Device catalog is intact Device catalog is gone or corrupted Disk backups

Volume Commands

Tape Drive and Inventory Commands List drive status, detail drive info and pending requests List the tape drive status List the pending requests Control a tape device List all changes in the robot(but do not update) Empty the robot and re-inventory (using barcodes) List all pools List tapes in pool List all tapes in the robot List cleaning tapes List tape volume details Delete a volume from the catalog Change a tapes expiry date Change a tape's media pool

vmoprcmd vmoprcmd -d ds vmoprcmd -d pr vmoprcmd [-reset][-up][-down] <drive number> vmupdate -recommend -rt tld -rn 0 vmcheckxxx -rt tld -rn 0 -recommend vmupdate -rt tld -rn <robot number> -rh <silo slave> -vh <host> -nostderr -use_barcode_rules -use_seed -empty_ie

Tape Media Commands vmpool -listall -bx vmquery -pn <pool name> -bx vmquery -rn 0 -bx |grep 'TLD' | sort +4 vmquery -mt dlt_clean -bx vmquery -m <media ID> vmdelete -m <media ID> vmchange -exp 12/31/06 23:59:58 -m <media ID> vmchange -p <pool number> -m <media ID>

Media commands bpstulist -U bpmedia [-freeze][-unfreeze] -ev <media ID> bpmedialist -ev <media ID> bpmedialist -U mcontents -m <media ID> bpimagelist -backupid <image ID> bpimage -cleanup -allclients bpexpdate -d 0 -ev <media ID> -force bpdbjobs -report [-hoursago] bpmedia -movedb -newserver <media server> -oldserver <media server>

List the storage units Freeze or unfreeze media List media details List media contents List backup Image Information Expire client images Expire a tape List all netbackups jobs Move media from one media server to another

Tape/Robot commands tpconfig -d

List tape drives

List cleaning times on drives clean a drive change a drives cleaning frequency set a drives cleaning time to zero

tpclean -L tpclean -C <drive number> tpclean -F <drive> <frequency> tpclean -M <drive> robtest commands that can be used are as follows:

ss (show slots) sd (show drives) Move tapes within robot using robtest s i (show load port) m s250 d5 (move tape from slot 250 into drive 5) uload d5 (unload tape from drive 5) m d5 s250 (move tape from drive 5 to slot 250) m s250 i1 (mov tape from slot 250 to load port 1)

List load port tapes List all slot contents List tape drive contents Move a tape in s100 to drive 1 Move a tape to load port 1

echo "s i q" | tldtest -r /dev/sg/c0t4l0 echo "s s q" | tldtest -r /dev/sg/c0t4l0 echo "s d q" | tldtest -r /dev/sg/c0t4l0 echo "m s100 d1" | tldtest -r /dev/sg/c0t4l0 echo "m s100 i1" | tldtest -r /dev/sg/c0t4l0

Archiving Commands bpcatlist -client all -before Jul 01 2006 bpcatlist -client all -before Aug 01 2006 bpcatlist -before Jul 01 2006 | bpcatarc | bpcatrm bpcatlist -before Jul 01 2006 | bpcatres

list archive info archive and remove images restore archive files

Client commands bpclntcmd [-ip <ip addres>] bpclntcmd [-hn <hostname>] bpclntcmd [-pn] bpclntcmd [-sv]

test client connectivity

Daily tasks of Backup Admins

In this article I would like to share what Symantec consulting recommends that Netbackup Administrator perform the following tasks to ensure a properly funtioning Netbackup domain. This information has been taken from from Symantec Netbackup Configuration Report document. 1. Monitor all NetBackup daemons are running on the master and all media servers. This can be monitored from the Console > Activity Monitor >Daemons. 2. Monitor all NetBackup processes are running on the master and all media servers. This can be done from the Console > Activity Monitor > Processes tab. 3. Monitor all backup jobs and investigate any failed jobs. This can be done from the Console > Activity Monitor > Jobs tab. 4. Monitor the health of all backup devices. This can be done from the Console > Media and Device Management > Device Monitor 5. If the drives are in a down or unknown state, investigate the causes through system logs or Event Viewer logs. Downed drives can be upped from Console > Media and Device Management > Device Monitor > select one or multiple drives > Actions > select Up Drive. 6. If the drives are in an unknown state, reset the drive to NetBackups known state from Console > Media and Device Management > Device Monitor > select one or more device > Actions > Reset Drive. 7. Resolve any pending tape mount requests for restores or backups. When NetBackup cannot automatically service a tape mount request due to a particular media not being available, operator action is required to complete the request. This can be seen from Console > Device and Media Management > Device Monitor > Drives Tab. After the requested media is inserted into an available drive, from the Actions > select Assign Request. 8. Monitor availability of Media in the scratch pool for the daily backup operations. This can be done from the Console > Media and Device Management > Media and selecting the Scratch Pool. 9. Pre-labeling media will speed up mounting of tapes during backups. To pre-label tapes, from the Console > Media and Device Management > Media > select the volumes to be labeled on the right pane > Actions > select Label and check Verify media label before operation. 10. Monitor for Frozen and inactive media. This can be done from Console > NetBackup Management > Reports > Tape Reports > Tape Summary. A frozen media is unavailable for backups and never expires. To unfreeze the media, from the Console > Media and Device Management > Media > select the media to be unfreezed and Actions > Unfreeze. 11. Monitor the NetBackup Problems Reports or All Log Enteries reports for NetBackup Errors. This can be done from Console > NetBackup Management > Reports > Problems or All Log Entries. The Troubleshooter can be launched by selecting a particular line in the reports pane and right clicking to get additional menu items as shown below. Troubleshooter provides explanation and tips for resolutions of the problems.

1. Monitor the disk space for NetBackup Catalogs and log files. If necessary add additional space before the file system gets full. 2. Periodically run the check_coverage script from the /usr/openv/netbackup/bin/goodies directory to verify that there are no gaps in client file system backups.

Performance metrics of Symantec Netbackup Server

Symantec consulting recommends the following metrics be tracked on a regular basis to ensure an optimally performing NetBackup infrastructure. Master Server: 1. 2. 3. 4. 5. 6. 7. 8. 9. CPU load: Recommend that it be less than 80% throughout the backup window. Memory usage: Less than 90% usage all through the backup window. Catalog Disk Space: Less than 90% full. Daily Backup Volumes: Trends point to impending problems with the capacity of the infrastructure. Backup Jobs outside the window: Points to schedule adjustments. Backup Failures: Recommend less than 5%. Backup Failure Code Trends: Points to other environmental problems in the enterprise. Media in the scratch pool: Recommended to have at least twice the number of media required for a weekly full. Restore Start Times: Points to tape drive resource contentions.

Media Servers: 1. 2. 3. 4. 5. CPU load: Recommended to be less than 90% Memory usage: Recommended to be less than 90% HBA Saturation: Recommended to be greater than 85% NIC Saturation: Recommended to be greater than 70% Tape Drive Utilization: Recommended to be evenly used across all tape drives

Please refer the following link to access the document. https://symiqforpartners.com/sites/sdrk/dp/Service%20Offering%20Library/Supplement_NBU_ ConfigReport_Implementation_A4_v1.00.docx