Symbol Technologies PLC: University of Gondar Netapp Storage Implementation For Uog - Ucs Project As-Built Documentation

University of Gondar
NetApp Storage Implementation for UoG - UCS Project
As-Built Documentation
Symbol Technologies PLC

For Smart Solutions!
Addis Ababa, Ethiopia,
Tel: +251-116-630564
Fax: +251-116-630602
Cel: +251-911-005275
Email: info@symboltech.net
URL: www.symboltech.net
Ethiopia, Gondar
Date: 5/Jan-2013
Pages: 36 1|Page
UAT for NetApp Implementation Version: 1.0
Symbol Technologies PLC © 2012- Confidential

Table of Content
Document Control......................................................................................................................................... 3
About the NetApp FAS3240 storage controller .................................................................................... 4
About the NetApp DS4243 disk shelf .................................................................................................... 4
Ethernet Interfaces ............................................................................................................................... 4
FC Interfaces ......................................................................................................................................... 4
Management ......................................................................................................................................... 5
Installation Steps ................................................................................................................................... 5
Initial Setup of system from console .................................................................................................... 6
Accessing controllers via GUI ................................................................................................................ 7
Creating Aggregates .............................................................................................................................. 9
Volume Creation ................................................................................................................................. 11
LUN CREATION .................................................................................................................................... 12
Configuration parameters................................................................................................................... 16
Snapshots ............................................................................................................................................ 22
Snap protect ........................................................................................................................................ 25
Restore from Snap protect backup ..................................................................................................... 31
Config Advisor (Wire Gauge Tool)....................................................................................................... 33
Troubleshooting .................................................................................................................................. 37
2|Page
UAT for NetApp Implementation

University of Gondar / UCS Project –NetApp Storage System As-Built Documentation
Document Control
Ver. Date Author/s Remark
1.0 Sept 3/2013 Nebiyou Yirsaw – Symboltech Final Version
Tewodros Kifle - Symboltech

Table 1 : Document History
Ver. Date Reviewers Remark
1.0 Sept 5 /2013 Naod Misslae – Symboltech
Tewodros Kifle - Symboltech
3|Page

About the NetApp FAS3240 storage controller
In UoG we have two FAS3240 controllers, top (UoG-TED-CtrlTop) and bottom

(UoG-TED-CtrlBot) controllers. The top one is responsible for SATA shelves and
the bottom is responsible for SAS shelves. In case of a power fluctuation or
general failure on one controller, the controller will be in takeover mode, it will
automatically switch controller to partner cluster.
About the NetApp DS4243 disk shelf
In UoG we have two SAS disk shelfs of 600GB, a total of 28 TB and three SATA disk
shelfs of 2TB, a total of 144 TB. We are using RAID-DP so we have a net usable
space of 19 TB for SAS and 83 TB of SATA, a total of 102 TB usable spaces.
Ethernet Interfaces
 We have ten gigabit Ethernet interfaces(c0a and c0b) which we use for
cluster to listen the heart beat of each controller so that in case if one
controller fails the other one will take over and there will not be service
disruption.
FC Interfaces
 We have two 10 gigabit Ethernet interfaces (0c and 0d) which we use for
connecting the controller to the nexus 5k, so that the disks will be available
to the VM’s as a LUN.
4|Page

Management
 We have two gigabit Ethernet interfaces (e0a and e0b) which are
responsible for local management. They are configured like a port channel
(in NetApp VIF) and we use 10.139.8.30 for top controller and 10.139.8.32
for bottom controller.
 We have another gigabit Ethernet interface responsible for out-of-band
management (service processor) so that the controllers are accessible
remotely for management and maintenance. We use 10.139.8.35 for the
Top controller and 10.139.8.36 for Bottom controller.
Installation Steps
Below is a document showing configuration steps done on UoG storage and how
you can be able to make changes whenever necessary.
5|Page

Initial Setup of system from console

Login to the controllers using console cable and type the default username and
password (root). Type setup and configure all controllers, as shown below, for
initial setup.
6|Page

Accessing controllers via GUI
Run the NetApp Oncommand System manager setup and accept all defaults. Once
done, run the program and it will launch via browser. From the Oncommand
home page, click on the ADD tab and add the controllers using their IP addresses.
Select the MORE drop down button and click on Credentials tab. Click add to
complete task. Being a dual controller, it will search for controller 2 and ask for IP
address and credentials to be added. Provide details and finish to complete
addition of controllers.
7|Page

Double click on any controller name so as to access configuration dashboard.

Once expanded, use the drop down button on required controller and you will
access the storage configuration tabs as shown below;
8|Page

Creating Aggregates
Click on the aggregates tab. You will notice aggr0 is created by default and this is
done during the setup process of the NetApp Ontap OS. It takes 3 disks for the OS
and RAID creation. NetApp uses RAID –DP, which means it will use 2 disks for
parity and 1 for spare, so as to guard against 2 disk failures. While still on the
aggregate page, you will notice an add disks tab. This will be used whenever you
want to add disks to an aggregate, say when you purchase a new shelf. In your
case, aggregates have already been created and disks added.
9|Page

You will notice a presence of 2 aggregates, aggr0 and aggr_sas0. Your

environment houses over 16TB of data, and with that in mind, Netapp came up
with a 64 bit aggregate for disks over 16TB in usable capacity. In order to house
these disks, we use a 64 bit aggregate, because these disks couldn’t all fit in the
aggr0 32 bit aggregate.
10 | P a g e

Volume Creation
To create a new volume, click on the volume menu, and then create volume.
Provide volume name details and as for storage type, use NAS option for creating
NAS specific volumes (File Shares) and SAN for the DB block level volumes.
Provide total size and click next to create the volume. Once this is done, right click
on the created volume and edit. Under security style, chose either NTFS or UNIX
depending on OS to reside on volume.
11 | P a g e

LUN CREATION
On the LUN menu, create new. Provide name, OS type and size. On the LUN
container, pick the select an existing volume tab and map the new lun to the
aggregate and volume you want lun to reside. This will lead you to igroup
creation. For FC you will need to add the wwpn numbers of the servers and for
iscsi you will need to add iqn number of the server
12 | P a g e

13 | P a g e

Above document shows a mapped LUN, its wwpn and igroup its mapped to.
Once mapping is done, go to your server, device manager and scan hardware
changes. Under disk drives the new drive will appear as “NETAPP LUN disk
device”. Go to Disk management and initialize it. It will then be seen as a drive on
your computer. It is to be noted for this installation that since we are presenting
VMFS Datastores to VMware, the discovery of the presented LUNS will be done
from the VM host.
N/B: Note that in order to access LUNS, you must start the fcp or iscsi services
form console command line as shown below:
14 | P a g e

UoG-TED-CtrlBot>fcp start
fcp start: FCP service is already running. To know the wwpn, type fcp show
initiator on the prompt, and the logged initiators will be displayed. E.g. will display
portname and mapped group
UoG-TED-CtrlBot>fcp show initiators

Initiators connected on adapter 0c:
Portname Group
-------- -----
20:00:00:25:b5:02:0a:01 UOG-ESXi2; UoG_FC
WWPN Alias(es): UoG-ESXI2-HBA1
20:00:00:25:b5:01:0a:01 UoG-ESXi1; UoG_FC
WWPN Alias(es): UoG-ESXI1-HBA1
Initiators connected on adapter 0d:

Portname Group
-------- -----
20:00:00:25:b5:02:0b:02 UOG-ESXi2; UoG_FC
20:00:00:25:b5:01:0b:02 UoG-ESXi1; UoG_FC
And
UoG-TED-CtrlBot>iscsi start
iSCSI service is already running. . To know the iqn, type iscsi initiator show on the
prompt, and the logged initiators will be displayed. To know whether there are
any connected initiators, type the below;
UoG-TED-CtrlBot>iscsi initiator show
For connected sessions;
UoG-TED-CtrlBot>iscsi session show
It will show you the connected sessions you have currently.
15 | P a g e

Configuration parameters
To access storage serial number, log in via SSH and on prompt, type sysconfig.
This will display system configurations of your storage. An example is shown
below:
UoG-TED-CtrlBot> sysconfig
NetApp Release 8.1.2 7-Mode: Tue Oct 30 19:56:51 PDT 2012
System ID: 1575087585 (UoG-TED-CtrlBot); partner ID: 1575087058 (UoG-TED-CtrlTop)
System Serial Number: 200000561658 (UoG-TED-CtrlBot)
System Rev: F6
System Storage Configuration: Multi-Path HA
System ACP Connectivity: Full Connectivity
slot 0: System Board
Processors: 4
Processor type: Intel(R) Xeon(R) CPU L5410 @ 2.33GHz
Memory Size: 8192 MB
Memory Attributes: Bank Interleaving
Hoisting
Chipkill ECC
Service Processor Status: Online
IOXM Status: present
Power: Good
Agent version: N/A
Serial Number: 5013538771
Part Number: 111-00647
Version: B0
slot 0: Dual 10G Ethernet Controller T320E-SFP/KR
c0a MAC Address: 00:a0:98:1d:ae:66 (auto-10g_twinax-fd-up)
c0b MAC Address: 00:a0:98:1d:ae:67 (auto-10g_twinax-fd-up)
slot 0: Internal BGE 10/100 Ethernet Controller
e0M MAC Address: 00:a0:98:1d:ae:68 (auto-100tx-fd-up)
e0P MAC Address: 00:a0:98:1d:ae:69 (auto-100tx-fd-up)
slot 0: Dual 10/100/1000 Ethernet Controller G20
e0a MAC Address: 00:a0:98:1d:ae:64 (auto-1000t-fd-up)
16 | P a g e

e0b MAC Address: 00:a0:98:1d:ae:65 (auto-100tx-fd-up)

slot 0: SAS Host Adapter 0a
72 Disks: 122073.5GB
3 shelves with IOM3
slot 0: SAS Host Adapter 0b
48 Disks: 26880.0GB
2 shelves with IOM3
slot 0: Fibre Channel Target Host Adapter 0c
slot 0: Fibre Channel Target Host Adapter 0d
slot 0: Intel ICH USB EHCI Adapter u0a (0xdfa01400)
boot0 Micron Technology Real SSD eUSB 2GB, class 0/0, rev 2.00/11.10, addr 2 1936MB 512B/sect
(0FF0022700173455)
slot 1: Dual 10G Ethernet Controller T320E-SFP+
e1a MAC Address: 00:07:43:08:90:10 (auto-unknown-down)
e1b MAC Address: 00:07:43:08:90:11 (auto-unknown-down)
slot 2: Flash Cache 512 GB
State: Enabled
slot 3: SAS Host Adapter 3a
48 Disks: 26880.0GB
2 shelves with IOM3
slot 3: SAS Host Adapter 3b
slot 3: SAS Host Adapter 3c
slot 3: SAS Host Adapter 3d
72 Disks: 122073.5GB
3 shelves with IOM3
To know your storage IP, you can either go to prompt and type ifconfig –a or use
the Oncommand software menu list as shown below. Access configurations via
the configuration drop down menu.
UoG-TED-CtrlBot> ifconfig -a
e0a: flags=0xaf0c867<BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
ether 02:a0:98:1d:ae:64 (auto-1000t-fd-up) flowcontrol full
trunked VIF1
e0b: flags=0xaf0c867<BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
ether 02:a0:98:1d:ae:64 (auto-100tx-fd-up) flowcontrol full
trunked VIF1
e1a: flags=0x170e866<BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
ether 00:07:43:08:90:10 (auto-unknown-down) flowcontrol full
e1b: flags=0x170e866<BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
17 | P a g e

ether 00:07:43:08:90:11 (auto-unknown-down) flowcontrol full

c0a: flags=0x1f4e867<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 9000 PRIVATE
inet 192.168.1.249 netmask 0xffffff00 broadcast 192.168.1.255
ether 00:a0:98:1d:ae:66 (auto-10g_twinax-fd-up) flowcontrol full
c0b: flags=0x1f4e867<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 9000 PRIVATE
ether 00:a0:98:1d:ae:67 (auto-10g_twinax-fd-up) flowcontrol full
e0M: flags=0x2b0c866<BROADCAST,RUNNING,MULTICAST,TCPCKSUM,MGMT_PORT> mtu 1500
e0P: flags=0x2b4c867<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM,ACP_PORT> mtu 1500
PRIVATE
inet 192.168.2.91 netmask 0xfffffc00 broadcast 192.168.3.255 noddns
lo: flags=0x1b48049<UP,LOOPBACK,RUNNING,MULTICAST,TCPCKSUM> mtu 8160
inet 127.0.0.1 netmask 0xff000000 broadcast 127.0.0.1
ether 00:00:00:00:00:00 (RNIC Provider)
losk: flags=0x40a400c9<UP,LOOPBACK,RUNNING> mtu 9188
inet 127.0.20.1 netmask 0xff000000 broadcast 127.0.20.1
VIF1: flags=0x22f4c863<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
partner VIF1 (not in use)
ether 02:a0:98:1d:ae:64 (Enabled interface groups)
18 | P a g e

From the System tools menu, you are able to configure your Autosupport, Date,
Check your licenses and also shutdown/ reboot the system.
Autossuport is a key feature to help in troubleshooting and knowing system

status and utilization so it is a must that it should be enabled.To configure your
autossuport, you will need to have your mailhost configured and enter at least 3
people who will be able to receive the autosupport notification. For UOG
netappsupport@uog.edu.et contains email addresses of UoG ICT persons. As for
transport protocol, you should use SMTP. Make sure the enable button is on so as
to be able to configure this feature.
From Command prompt;
19 | P a g e

UoG-TED-CtrlBot> options autosupport

autosupport.content complete (value might be overwritten in takeover)
autosupport.enable on (value might be overwritten in takeover)
autosupport.from netapp1@uog.edu.et (value might be overwritten in takeover)
autosupport.local_collection on (value might be overwritten in takeover)
autosupport.mailhost 10.139.5.20 (value might be overwritten in takeover)
autosupport.to netappsupport@uog.edu.et (value might be overwritten in takeover)
autosupport.support.to autosupport@netapp.com (value might be overwritten in takeover)
20 | P a g e

21 | P a g e

Snapshots
To create snapshots, click on the volume you want from the OncommandGUI .
From the snapshot copies tab, select create. To configure snapshot schedules,
click on the configure tab.
22 | P a g e

In order to perform a restore, go to the LUN menu, select the LUN you want to
restore and unmap it from its igroup. On the Host server, go to device manager
and select scan hardware changes to ensure LUN has been unmapped. Return to
your Oncommand GUI, under snapshot copies, click restore and pick the snapshot
you want to restore to. After this, go back to igroup, lun map and scan from your
server to ensure volume is back.
23 | P a g e

N/B: The above is for a basic Netapp lun level restores, for UoG environment, we
will be using a NetApp application aware software called SnapProtect.
24 | P a g e

Snap protect
Login to Netapp oncommand core (10.139.8.40) to access snap protect software,
on the desktop open ‘commCell console GUI’ shortcut. Type the username and
password. To configure backups, a VMware instance must be created. On client
computers go to netappcore, Right-click Virtual Server iDA and Create New Instance.
Give the instance a name in UoG we give it ‘VCenter’ and select VMware for the vendor
type.
After creating the VMware instance, create a new backup set. Right-click the
instance(VCenter) All Tasks Create New Backup Set, Give the backup set a name
and select the storage policy. In UoG we created a backup set called ‘Test’.
25 | P a g e

Create a new subclient in the backup set: right-click the backup set (Test)  All
Tasks New Subclient.
Give the subclient a name; in our case it is called ‘UoG_SC’
26 | P a g e

Click the Storage Device tab and create a storage policy. Give the name for the storage
policy, In UoG we created a storage policy called ‘storage’. Then Choose location to
store indices, in our case there is 14TB allocated for back up. Select that and click next.
Then you will be redirected to data aging window, this is what expires backups and
deletes snapshot copies, configure it based on your environment and click next. Enter
the ip address of the data fabric manager, and click next, and then it will give you a
chance to review what is configured click finish to accept the configuration.
After finishing and creating the storage policy, select it like the picture below, in our case
the policy name is ‘storage’.
27 | P a g e

Under the SnapProtect Operations tab, select an ESX server to be responsible for
mounting Snapshot
copies as needed. Click OK.
28 | P a g e

Under the Content tab, click configure then click on discover which will list you all
available VM’s that can you apply the subclient (in our case UoG_SC). Select the VM
you are going to back up. For UoG select VCenter to back up all VM’s. Click apply.
29 | P a g e

To schedule backup, right click storage policy and select add so that you can set
full and incremental schedule backup types. For UoG we schedule an incremental
backup type on Monday, Tuesday, Wednesday, and Thursday, and a weekly full
backup type on Friday at 6 pm.
30 | P a g e

Restore from Snap protect backup
In order to show you how snap protect restore works, we create a VM backup
called Test 6 and after the backup is completed, we delete it. Here is how you can
restore in case if such things happen.
If you want to restore a single file or an entire VM, right click the subclient (in our
case, UoG_SC) and click browse backup data.
Choose your mediaAgent in UOG it is netappcore and specify the type of resource
you are going to restore, you can restore
31 | P a g e

 Individual file/folders,
 Entire volume, or
 Entire VM.
Then Click ok to go to the next step.
Then you will get a window that will list all backups, choose the file you want to
restore, in this case we want to restore Test6, select test 6, it will highlight all
contents in blue, and click ‘recover all selected’.
32 | P a g e

Config Advisor (Wire Gauge Tool)
For checking status of the system for any errors in cabling or disk configuration, a
NetApp Tool known as Config Advisor formerly known as Wiregauge is used. Once you
install it on your client machine, it will require credentials to access the storage
controllers.To run configuration query to check on system, click on the collect data tab,
then ok. It will check the system for any errors or misconfigurations and give a report as
shown below;
33 | P a g e

Click on the collect data tab so as to gather data from the storage about configuration
checks, system health status and any misconfigurations.
34 | P a g e

35 | P a g e

From the status tab, it shows pass for the configuration. In case of an error, it will
display fail. This should be done for both TOP and bottom controllers.
36 | P a g e

Troubleshooting
Some of the common problems you may face include:
1. E0a Ethernet port losing its ip. Remedy to this is enter via console
and type below command:
Ifconfig e0a (storageip) netmask 255.255.255.0 up
Same applies if e0b also loses its ip. Once the IP addresses have
been entered, it’s recommended to enter the IP addressed via GUI
(Oncommand) so as to solve this problem
2. Controller being in takeover mode, either due to power

fluctuations or general failure, will automatically switch controller
to partner cluster. When you enter prompt and see message
“ready for takeover”, switch to the partner controller and on its
command prompt, enter cf giveback -f. This will return disk
ownership that was taken by partner back to original controller
37 | P a g e

Symbol Technologies PLC: University of Gondar Netapp Storage Implementation For Uog - Ucs Project As-Built Documentation

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Symbol Technologies PLC: University of Gondar Netapp Storage Implementation For Uog - Ucs Project As-Built Documentation

Uploaded by

Copyright:

Available Formats

University of Gondar

NetApp Storage Implementation for UoG - UCS Project

Symbol Technologies PLC

UAT for NetApp Implementation Version: 1.0

Symbol Technologies PLC © 2012- Confidential

UAT for NetApp Implementation

Symbol Technologies PLC © 2012- Confidential

Ver. Date Author/s Remark

1.0 Sept 3/2013 Nebiyou Yirsaw – Symboltech Final Version

Tewodros Kifle - Symboltech

Ver. Date Reviewers Remark

1.0 Sept 5 /2013 Naod Misslae – Symboltech

Tewodros Kifle - Symboltech

UAT for NetApp Implementation

Symbol Technologies PLC © 2012- Confidential

About the NetApp FAS3240 storage controller

In UoG we have two FAS3240 controllers, top (UoG-TED-CtrlTop) and bottom

UAT for NetApp Implementation

Symbol Technologies PLC © 2012- Confidential

UAT for NetApp Implementation

Symbol Technologies PLC © 2012- Confidential

Initial Setup of system from console

UAT for NetApp Implementation

Symbol Technologies PLC © 2012- Confidential

Accessing controllers via GUI

UAT for NetApp Implementation

Symbol Technologies PLC © 2012- Confidential

Double click on any controller name so as to access configuration dashboard.

UAT for NetApp Implementation

Symbol Technologies PLC © 2012- Confidential

UAT for NetApp Implementation

Symbol Technologies PLC © 2012- Confidential

You will notice a presence of 2 aggregates, aggr0 and aggr_sas0. Your

UAT for NetApp Implementation

Symbol Technologies PLC © 2012- Confidential

UAT for NetApp Implementation

Symbol Technologies PLC © 2012- Confidential

UAT for NetApp Implementation

Symbol Technologies PLC © 2012- Confidential

UAT for NetApp Implementation

Symbol Technologies PLC © 2012- Confidential

UAT for NetApp Implementation

Symbol Technologies PLC © 2012- Confidential

UoG-TED-CtrlBot>fcp show initiators

Initiators connected on adapter 0d:

UoG-TED-CtrlBot>iscsi initiator show

For connected sessions;

UoG-TED-CtrlBot>iscsi session show

It will show you the connected sessions you have currently.

UAT for NetApp Implementation

Symbol Technologies PLC © 2012- Confidential

UAT for NetApp Implementation

Symbol Technologies PLC © 2012- Confidential

e0b MAC Address: 00:a0:98:1d:ae:65 (auto-100tx-fd-up)

UAT for NetApp Implementation

Symbol Technologies PLC © 2012- Confidential

ether 00:07:43:08:90:11 (auto-unknown-down) flowcontrol full

UAT for NetApp Implementation

Symbol Technologies PLC © 2012- Confidential

Autossuport is a key feature to help in troubleshooting and knowing system

From Command prompt;

UAT for NetApp Implementation

Symbol Technologies PLC © 2012- Confidential