IBM SAP International Competence Center

CIBER creates next-generation IT infrastructure solution for SAP hosting services

“Through the 100 percent virtualization you gain a dynamic infrastructure and you get the ability for an easy billing system.”
Carl-Philipp Müller Lead Sales Operations Services CIBER Managed Services GmbH

“XIV is simply amazing. The usage of XIV real time copy, which is able to copy many terabytes of data very fast, reduces the downtime for provisioning drastically. The main focus as a service provider is the sector of customer satisfaction, which is especially improved by the usage of the new XIV system.”
Markus Peltz Data center Manager CIBER Managed Services GmbH

CIBER creates next-generation IT infrastructure solution for SAP hosting services
About this paper CIBER Managed Services GmbH built a dynamic Cloud Infrastructure for SAP landscapes, based on a combination of IBM server, storage, database, and system management software. The new storage infrastructure, which consists of the IBM XIV Storage System and IBM SAN Volume Controller (SVC), offers individual service classes to suit different customer needs, maximizing operational efficiency and minimizing expenses. The SAP application landscapes run on both IBM Power Systems servers and IBM System x Blade Server technologies.

IB

Intern M SA P

ational

Compete

nce Cen

ter

n neratio ext-ge n creates solution for CIBER ructure s infrast IT service osting SAP h

“IBM was chosen as a partner, because of the very good longterm experience with them. Especially their integrated concepts and their highly skilled consultants for very special challenges, plus the capability to implement the required standards within time and budget convinced us to do this project with IBM.”
Mats Johansson, Head of Operations, CIBER Managed Services GmbH

Customer Objectives

Installed six IBM System x3650 M2 systems, powered by quad-core 2.26 GHz Intel Xeon Processor L5520 series processors, running Novell SUSE Linux Enterprise Server 10 and 11 as well as Windows Server 2003 and 2008

Provide a highly scalable and dynamic infrastructure to host customer-oriented SAP landscapes for small- and mediumsized businesses in a private cloud environment that will support the future growth of the business

Implemented a next generation three service class storage solution based on IBM SAN Volume Controller 5.x, IBM XIV Storage System and IBM System Storage DS3400

Establish high-performance infrastructure, featuring transparent and easy billing models for high customer satisfaction

Implemented IBM Tivoli Storage Manager and two IBM System Storage TS3500 tape library systems, each with five LTO4 tape drives, 260 slots, and about 200 cartridges (migration speed to tape of 350 GB/h).

• •

Reduce CIBER’s IT operational costs without sacrificing service quality Operate a complete range of SAP Business Suite components on an integrated and shared storage infrastructure and powered by either IBM Power servers or IBM System x BladeServer technology depending on customer requirements

Customer Benefits

Ability to provide faster and more flexible and scalable SAP hosting services to businesses, with 99.9 percent virtualization and improved Service Level Agreements.

• •

Introduce reliable and comprehensive backup, resilience, and disaster recovery solutions Increase the speed of new SAP application provisioning, to help respond faster to business requests.

Improved business continuity and higher availability even across two locations, using IBM SVC storage virtualization and the IBM XIV Storage System


IBM Solution

Reduced workload for IT staff through simplified setup and administration of the XIV Storage Systems

Worked with IBM and SVA, an IBM Premier Business Partner, to design and implement a highly flexible and cost-efficient infrastructure and a next-generation storage solution for SAP in only four months

Reduced operating costs with improved service quality supported by three levels of storage service classes, an integrated and virtualized storage architecture, and new server hardware

Set up two IBM Power 560 servers, each with two POWER6 processors with four cores, and two IBM Power 750 servers, each with four POWER7 processors with eight cores, running SAP applications on IBM AIX in about 40 LPARs, sharing processor resources and direct mappings of LPARs to LUNs. The two POWER7 machines provide 170,000 SAPS and process 12,500 sales order positions per hour between them

Reduced time for provisioning and deployment of new systems from weeks to hours with the new storage solution based on IBM XIV technology, using the XIV Snapshot functionality and the SVC FlashCopy functionality.

• • •

Improved database performance with DB2 by a factor of eight in online operations Achieved higher level of automation in database environment Gained disk space savings and reduced data growth by deploying DB2 compression and the reclaimable Storage features (reduced size of tablespaces from about 1 TB down to 300 MB for SAP NetWeaver Business Warehouse)

Installed two BladeCenter H, both with six HS22 BladeServers powered by quad-core 2.40 GHz Intel Xeon L5530 processors, and four HS22 Blade servers powered by six-core 2.60 GHz Intel Xeon X5640 processors, running VMware vSphere 4, forming several SAP clusters and hosting guest operating systems Novell SUSE Linux Enterprise Server 10 and 11 as well as Windows Server 2003 and 2008

The BladeCenter technology allows an option to upgrade online to current BladeServer models, such as HX5 and also offers hot swap technology for blades to ensure high availability with the BladeCenter Open Fabric Manager, if needed.

DB2 for Linux, UNIX and Windows is the most appropriate database platform to support the SAP applications at CIBER

4

About CIBER
Background, starting point and objectives CIBER is a full service IT partner founded in 1974 and headquartered in Denver, USA. It employs more than 8,500 staff at 85 locations in 18 countries, and generates annual revenues of about $1.037 billion. CIBER specializes in implementing and supporting complex SAP and similar solutions for a broad customer base in multiple industry sectors. Application Services provide comprehensive SAP-functional and business process-related support. With Applications Services, CIBER supports customers in their daily business. Furthermore Application Services support the continuous improvement process. Additionally, CIBER renders SAP Software Support for all SAP Software Maintenance customers. For these customers, the CIBER support organization works as an integral part of the Global SAP support organization.

Service Management ensures comprehensive and reliable service delivery. CIBER’s customers benefit from the efficiency of individually assigned Service Teams delivering the agreed services. A dedicated Service Manager coordinates the delivery More than 400 of CIBER’s total of 1,300 SAP consultants work for CIBER Germany, which has particular expertise in retail and consumer products, chemicals and pharmaceuticals, automotive industry, and financial services. With more than 20 years of SAP solution implementation experience, CIBER Germany achieved sales of €55 million in 2009. of the agreed services, provides transparent reporting of all activities and KPIs, and represents the customer and their needs inside the CIBER service organization.

CIBER Managed Services GmbH is based in Freiburg, Germany, and was established in January 2010 to provide SAP application and operation services for small- and medium-sized businesses from its two data centers, with more than 120 experienced consultants.

The Managed Services business specializes in postimplementation-services, designed to preserve the initial investments that customers make in their SAP implementations. Managed Services consist of Operations Services, Application Services, Software Support and comprehensive Service Management. As a VMware Solution Provider with a special expertise in SAP implementations, CIBER has competencies and proven processes in architecture and sizing, and also in deployment Operations Services provide customers with a full range of SAP system maintenance capabilities, covering the provision and operation of modern data centers, the necessary IT infrastructure, and the administration and maintenance of SAP and IT-related applications. 5 and analyzing complex virtualized SAP landscapes. Figure 1: CIBER Services

Initial IT environment In order to offer operation services for SAP as a cloud provider, CIBER Managed Services needed to build up two completely new data centers. The new and flexible technologies allow CIBER to satisfy growing customer needs. As a future service, CIBER wants to implement a self-service portal to allow customers to request SAP services. Subsequently, the necessary server capacity, operating system, applications and storage could be provisioned and charged back automatically. This would keep costs low and enable CIBER to serve smaller clients cost-effectively with enterprise-class solutions. With very positive experience of IBM System x and IBM Power Systems servers, CIBER chose to continue on the same strategic path, and selected IBM XIV Storage Systems with IBM SAN CIBER selected SAP Solution Manager and its own CIBER Solution Suite to provide the central interface, which includes a Volume Controller to address performance, high availability scenarios, and premium support for data center management. full service desk based on ITIL® V3. The CIBER tool and SAP Solution Manager enable the creation of a virtual server (either using VMware vSphere 4 technology on IBM System x or using LPARs on IBM Power servers), the installation of the operating system, assignment of the LUNs for storage and the deployment of the SAP applications. The CIBER team needed next to consider the best infrastructure to support this business strategy.

Redundant FC connection

Figure 2: SAP private cloud

6

Business challenges and project objectives The thinking behind the technology strategy was based on the view that although customer workload was growing continuously, the nature of the demand was highly volatile. Providing an industry-standard IT services solution with server capacity sold in fixed amounts would not deliver the kind of flexibility that customers wanted, and CIBER wanted to create a solution that could scale up and down in performance – and allow customers to pay on a more cost-effective usage basis. Continuing the theme of continuous availability, CIBER uses Live As a starting point, CIBER needed an infrastructure capable of hosting 120 SAP instances with about 8,000 SAP users, and up to 20 systems for one customer in a private cloud environment. The customers are offered free choice among different operating systems and platforms, from IBM AIX on IBM Power Systems to SUSE Linux Enterprise Server or Windows Server on IBM System x BladeServer technology. While CIBER recommends that its customers use IBM DB2 for Linux, UNIX and Windows, some customers run MaxDB, Oracle, and MS SQL Server databases. The infrastructure is capable of running all these operating systems and databases, and the CIBER team can support all of these systems. Alongside the scale-up scale-down flexibility for individual customers, CIBER planned for overall systems growth that would equip them to respond to the total business demand. The solutions must therefore be able to expand in capacity as a CIBER operates the complete range of SAP ERP software components, as well as SAP applications such as Supplier Relationship Management (SRM), SAP ERP Human Capital Management (HCM), Advanced Planning and Optimization (APO), Customer Relationship Management (CRM), SAP NetWeaver Business Warehouse (BW) and SAP NetWeaver Portal. Additionally, CIBER offers its own solutions based on SAP All-in-One software, including Rapid Retail, QuickWear, Fabricated Metals, Industrial Machinery and Components, Aerospace and Defense, and CIBER Solution Suite, a new intuitive front-end for SAP Solution Manager. Each SAP instance runs in its own LPAR. Knowledge of new customer contracts, the customer queue, and planned growth predicted that the infrastructure at the current data centers would reach capacity in less than six months. CIBER planned to build the new data center in less than four months to allow for sufficient transition time and maintain the delivery of high service levels for its SAP hosting customers. whole, while preserving the ability to offer each customer exactly the right services. Partition Mobility, available on Power servers with POWER6 processors and upwards. Live Partition Mobility enables the migration of entire running AIX partitions and hosted applications from one physical server to another without disrupting services and loads. Combined with LPARs, SAN Volume Controller and other virtualization technologies, Live Partition Mobility will further enhance CIBER’s ability to respond to customer demands. If an additional Java Stack is needed, this is installed in a separate LPAR. The largest SAP environment provides up to 170,000 SAPS in a total of 20 instances, processing up to 12,500 sales order positions per hour. On customer request the environment is set up as a Power HA for AIX cluster to support high availability of logical partitions for business critical systems in a two-node cluster with automatic failover.

7

Technical solution
Server architecture Working with SVA and IBM, the CIBER team designed and implemented a highly flexible and fully virtualized environment, capable of hosting almost every kind of SAP application landscape. Ultimately, however, CIBER’s clients select the platform: either Power Systems and AIX or System x and Linux or Windows, depending on the combination of reliability, scalability and predicted workload.

SVA, an IBM Premier Business Partner, provided implementation services. SVA installed the SAN Volume Controller, XIV, x3650 and BladeCenters. Ten SVA and IBM technicians completed the implementation on time and on budget in a six-week period – during part of which one of the data centers did not have a fully functioning electricity supply. IBM System x and IBM BladeCenter CIBER runs IBM BladeCenter H chassis featuring IBM BladeServer HS22, each of them powered by two 2.40 GHz quad-core Intel Xeon Processor E5530 series processors and four HS22 Blade servers powered by hexa-core Intel Xeon Processor X5640 series processors In essence, everything is virtualized – the servers, storage and network. When an application grows in workload and functional reach, its processes are distributed among multiple virtual machines to ensure performance is maintained, using either VMware virtual machines on System x or logical partitions (LPARs) on Power servers. VMware delivers the freedom to be fully supported for SAP production environments on both Linux and Windows operating system and multiple x86-Hardware. CIBER uses IBM System x3650 M2 machines powered by 2.26 GHz quad-core Intel Xeon Processor L5520 series processors. These x3650 M2 servers provide VMware virtual machines, whereby each allows at most 8 virtual CPUs and 255 GB RAM, are smaller than the virtual machines available in LPARs, which support maximum of 256 virtual CPUs and 1 TB RAM. When applications grow, this scale-out distribution of processes happens earlier on System x than on Power servers. infrastructure services, running Novell SUSE Linux Enterprise Server 10 and 11, as well as Microsoft Windows 2003 and 2008. These servers run without virtualization, and monitor the virtualized systems with applications such as IBM Director, cloud monitoring tools, and SAP monitoring applications. at 2.6 GHz. The blades run multiple virtual servers – collectively around 500 systems – using a combination of VMware vSphere 4 as hypervisor, and Novell SUSE Linux Enterprise Server, and Windows Server as the SAP platform.

The result is that CIBER has gathered a great deal of experience in the scale-out of SAP applications, particularly on System x. Examples include separation of database and application server (that is, moving from two-tier to three-tier configurations), separation into multiple application servers, and separation of SAP Central Services. The SAP Central Services include the unique SAP application system services, including enqueue server and message server.

IBM Power Systems CIBER uses two IBM Power 560 and two IBM Power 750 servers, running the IBM AIX operating system and IBM DB2 for Linux, UNIX and Windows. Both Power 750 systems are equipped with four POWER7 processors with 8 cores each and 512 GB of memory. The IBM Power 560 servers are equipped with two POWER6 processors with 4 cores each and 128 GB of memory. The servers each host around 40 logical partitions (LPARs).

8

Data center 1
RESOURCE POOL
SAP SAP SAP SAP SAP SAP SAP SAP Non SAP Non SAP SAP SAP

Data center 2

SAP

SAP

SAP

SAP

SAP

SAP

SAP

Win/MaxDB SAP ERP 6.0

SAP

SAP Intel/DB2

SAP

Linux/DB2 SAP ERP 6.0
SAP SAP

SAP

Non SAP

SAP

Win/Oracle SAP ERP 6.0

SAP

Intel/DB2 SAP

SAP

Win/MS SQL SAP ERP 6.0  SAP

SAP

SAP

SAP

SAP

SAP

SAP

SAP

Non SAP

Non SAP

SAP

SAP

SAP

SAP

SAP

SAP

SAP

SAP

ESX

ESX

ESX

ESX

ESX

<<<  ESX HA  /  VMWare  Vmotion >>>

ESX

ESX

ESX

ESX

ESX

LPAR

LPAR

LPAR

LPAR

LPAR

PowerVM

LPAR

LPAR

LPAR

LPAR

LPAR

Direct Mapping non productive 2.5 TB

Mirrored vDisks Productive 1 TB

Direct Mapping non productive 2.5 TB

SVC MDISK SAP CRM SAP  BI DEV SAP SolMan D SAP ERP P …

SVC MDISK SAP ERP  SAP EP DEV SAP NW QA SAP .. …

Figure 3: High Availability Concept and storage cloud operating system

IBM BladeCenter and System x landscapes For many customers, CIBER provides SAP hosting services on IBM System x BladeServers.

Virtualization using VMware in the Intel processor-based systems Almost all applications, perhaps 99.9 percent, on the Intel processor-based platforms are virtualized using VMware vSphere 4. The only exceptions are monitoring and systems

For systems powered by Intel processors, CIBER follows a rule, based on experience: There is an optimum relationship between processor capacity consumed and memory required, which means that more memory does not necessarily increase the performance of the SAP applications hosted on that server. CIBER found that 8 GB memory per core is sufficient. In the future, CIBER may upgrade the blades with IBM BladeServer HX5 technology.

management applications.

The Blade Servers host guest operating systems such as Novell SUSE Linux Enterprise Server 10 and 11, as well as Microsoft Windows Server 2003 and 2008.

For the System x servers, CIBER recommends the Linux operating system to customers for the following reasons:

Provisioning of new SAP instances is completed using VMware templates, making it fast and easy to set up standardized systems ready for immediate customer deployment. Additional tools such as TSM backup agents, monitoring agents, and special SAP configurations are already integrated for quick deployment.

• •

Lower license costs Linux installation can be configured to comprise just the software components needed, creating a low-footprint operating system with relatively low maintenance requirements (Windows includes many software components not actually required by the applications, increasing the operating system footprint and increasing the

CIBER uses VMware vCenter Converter (also named vConverter) to automate physical to virtual machine conversions as well as conversions between virtual machine formats. In particular, the converter is used to migrate native physical systems into CIBER’s virtual environments. 9 •

patch requirements). CIBER believes that Linux Logical Volume Manager (LVM) is more flexible than the Microsoft Logical Disk Manager; for example, Linux LVM allows data replication (mirroring) at server level

The conversion takes place either at the customer site or in CIBER’s central data center. CIBER uses Acronis True Image to create a backup image of the customer’s application systems environment and transport the image files from the client’s site to the data center, which eliminates the need for risky physical movement of hardware from place to place.

For disaster tolerance and disaster recovery capabilities, as well as for high availability across locations, CIBER implemented an enterprise-class storage design. It consists mainly of the IBM SAN Volume Controller (SVC) and the IBM XIV Storage System. The SVC has been set up with the Split I/O Group design to achieve high availability even if one location fails completely.

Once customer systems have been virtualized, CIBER uses VMware Virtual Symmetric Multi-Processing (SMP) to enable a single virtual machine to use multiple physical processors simultaneously. This allows CIBER to maximize utilization of its physical processor resources, and provide the virtual machines with the capacity required easily and quickly.

In 2011, CIBER plans to deploy VMware vCloud Director data center management software, which pools virtual infrastructure resources and delivers them to users as a catalog-based service. The target users are internal consultants who administer the local data center systems.

CIBER also plans to implement VMware vCenter Chargeback, With the applications and servers hosted by CIBER, the next question is how to deliver them to the customers. VMware View is built on VMware vSphere for Desktops. CIBER runs its own data center management and administration applications using VMware View virtualized desktops.
Customer 1

which logs resource requests and consumption and enables usage-based chargeback and accounting for hosted data center operations.

VMware vCloud Director

For data storage, security and management, CIBER uses VMware vStorage VMFS (virtual machine storage file system), a high-performance cluster file system optimized for VMware virtual machines. A VMware virtual machine stores the contents of a virtual machine’s hard disk drive in a virtual disk file with the extension .vmdk (virtual machine disk). The vmdk specification describes and documents the virtual machine environment, and how the contents of the hard disk drives is stored. CIBER uses .vmdk files rather than storing the virtual disks on raw devices to ensure isolation of the data stored in separate virtual disks.

CIBER Solution Suite
Win/MaxDB ERP 6.0

Catalogs
Linux/ DB2 BS7 Linux/SQL BI

Security

Datacenter Configuration(Gold)

VMware vShield

Datacenter Configuration (Silver)

VMware vCenter Server VMware vSphere

VMware vCenter Server VMware vSphere

VMware Storage vMotion enables Live Migration of .vmdk files – in other words, a .vmdk file can be moved from one location to another even while it is in use. This ability helps CIBER to minimize downtime during disk array migrations and storage upgrades, or when the virtual machines require larger-capacity storage devices.

Figure 4: Providing and consuming IT as a service

10

IBM Power Systems landscapes Typically, for larger customers CIBER hosts SAP ERP and SAP NetWeaver applications on IBM Power systems. For existing clients, CIBER has two IBM Power 560 and two IBM Power 750 servers, running the IBM AIX operating system and IBM DB2 for Linux, UNIX and Windows. The VIOS provides full virtual paths from each LPAR to the switched fabric in the data center. Switched fabric network topology allows nodes to connect with each other via one or more network switches, which provides both high total capacity and a high degree of resilience if a network switch fails.

LPARs are subsets of a computer’s hardware resources, virtualized as a separate computer using the hypervisor. A physical machine can be partitioned into multiple LPARs, each hosting a separate AIX operating system. The hypervisor assigns a specific range of virtual processors to each of the LPARs. CIBER does not configure dedicated assignments of processors, and shares uncapped processor resources across all the LPARs.

CIBER has assigned a separate LPAR for storage management, with a dedicated Converged Network Adapter to support concurrent LAN (TCP/IP) and SAN (FCoE, iSCSI) traffic over a shared 10 Gb Ethernet link. CIBER currently uses only TCP/IP and iSCSI protocols, and wants to be ready for FCoE in the future.

PowerHA for AIX (previously named High Availability Cluster Multiprocessing HACMP) provides high availability of logical partitions (LPARs) in a two-node cluster by way of automatic failover. When one server node goes down, the other server node takes over. Depending on clients’ requirements, most production systems and some of the quality assurance systems running on AIX use PowerHA to ensure continuous availability.

For large SAP systems, CIBER prefers to use POWER technology and virtualization over VMware vSphere. This is for two reasons: first, because AIX LPARs can scale to higher total processor and memory limits; and second, because AIX enables much higher processor utilization. By using AIX LPARs, the SAP application can grow much larger before it is necessary to scale-out over multiple servers.

CIBER uses IBM Virtual I/O Server (VIOS) to enable the sharing of physical I/O resources between client logical partitions within the server. The total physical bandwidth can be allocated in fixed amounts to each server, or policies set to enable full capacity to be used by high-priority workload at peak times.

The migration from IBM POWER5 to POWER6 and POWER7 was without any problems, CIBER simply connected the new servers to the SAN storage system and restarted the new servers and the SAN storage system.

11

Storage architecture CIBER implemented an enterprise-class storage solution, based on two IBM XIV enterprise storage systems, IBM SAN Volume Controller (SVC) as Split I/O Group, IBM System Storage DS3400 for the SVC quorum disk, and IBM Tivoli Storage Manager. The storage infrastructure has been designed to provide a combination of performance and resilience, as well as the ability to allocate storage volumes to multiple customers exactly as their needs dictate with the least possible administration and expense. CIBER evaluated different solutions and chose XIV technology because of better IO performance, improved administrative handling, and easier maintenance support than its competitors. CIBER implemented the two XIV systems, located at the primary and secondary data centers. XIV technology offers high, consistent performance and hotspot-free service due to mass parallelism and the distribution of logical volumes across all hard disks in the system, eliminating distribution planning.
Datacenter 1
1 5 9 13 17 21 25 29 33 37
Cisco Nexus 5020

At the core of the storage landscape are two IBM XIV model A14 storage systems, each equipped with 79 TB storage. The XIV systems act as the principal SAP application data storage. CIBER can handle almost three-fold storage growth as customer data volumes grow. In practice, Tivoli Storage Manager enables sophisticated management of data, moving archive and less active data onto tape, to help reduce the need to expand disk systems.

Datacenter 2
1 5 9 13 17 21 25 29 33 37
Cisco Nexus 5020

SLOT2

SLOT2

PS1

2 STAT 3

6

10

14

18

22

26

30

34

38 STAT

2

6

10

14

18

22

26

30

34

38

L1

MGMT 0

7

11

15

19

23

27

31

35

39

L1

MGMT 0

3

7

11

15

19

23

27

31

35

39

PS2

L2

MGMT 1

CONSOLE

4

8

12

16

20

24

28

32

36

40

200-240v-6A 50~60Hz

1

5

9

13

17

21

25

29

33

37

Cisco Nexus 5020

10GE LAN Backbone

10GE LAN Backbone

L2

MGMT 1

CONSOLE

SLOT3

SLOT3

PS2
200-240v-6A 50~60Hz

4

8

12

16

20

24

28

32

36

40

PS1

SLOT2

SLOT2

PS1

2 STAT 3

6

10

14

18

22

26

30

34

38 STAT

2

6

10

14

18

22

26

30

34

38

L1

MGMT 0

7

11

15

19

23

27

31

35

39

L1

MGMT 0

3

7

11

15

19

23

27

31

35

39

PS2

L2

MGMT 1

CONSOLE

4

8

12

16

20

24

28

32

36

40

200-240v-6A 50~60Hz

L2

MGMT 1

CONSOLE

SLOT3

SLOT3

PS2
200-240v-6A 50~60Hz

4

8

12

16

20

24

28

32

36

40

BladeCenter H

1

2

3

4

5

6

7

8

9

10

11

12

13

14

BladeCenter H

1

2

3

4

5

6

7

8

9

10

11

12

13

14

PS1

` `

`
Cisco Nexus 5020

1

5

9

13

17

21

25

29

33

37

`

System Stor age

1

2

3

4

5

6

7

8

9

10

11

12

13

14

SVC Node
TotalStor age
Storage Engine 336

1

2

3

4

5

6

7

8

9

10

11

12

13

14

Bladecenter H
01 9 8 7 6 5 4 3 2 1
2498-B24 2498-B24

74 32 64 22

4 Y LPP US R EWOP

54 12 44 02 34 91 24 81 14

1

2

3

4

5

6

7

8

9

10

8 GB FC Backbone
POWE R SU PPL Y 4

0 4 1 5 2 6 3 7 8 12 9 13 10 14

0

71

4 1 5 2 6 3 7

04 61 93 51 83 41 73

3 Y LPP US R EWOP
8 12 9 13 10 14

31 63 21 53

47

11

23

43

11 15

11 15

46

01

22

33

45

Dark Fiber

9

16

16

21

20

20

23

44

8

17

17

20

21 18

21

13

18

43

22

22

7

19

03

19

19

42

23

23

6

18

92

41

5

17

2 Y LPP US R EWOP

82

40

4

16

72

39

3

15

62

38

2

14

52

37

13

POWE R SU PPL Y 3

1 42

36

0

34

11

35

12

9

33

10

1 + 3 Y LPPUS R EWOP

2 + 4 Y LPPUS R EWOP

1 Y LPP US R EWOP

5

29

6

30

7

31

8

32

4

28

POWE R SU PPL Y 2

POWER SUPPLY 4 + 2

POWER SUPPLY 3 + 1

POWE R SU PPL Y 1

FC CWDM
2498-B24 2498-B24

FC CWDM
01 9 8 7 6 5 4 3 2 1
74 32 64 22

FC 850nm

0

24

1

25

2

26

3

27

4 Y LPP US R EWOP

54 12 44 02 34 91 24 81 14 71 04

22

D

61 93

0 4

0 4

51

1

2

3

4

5

6

7

8

9

10

83

1 5 2 6

1 5 2

BladeCenter H

41 73

POWE R SU PPL Y 4
8 12

Dark Fiber

3 Y LPP US R EWOP

6

31

47

63

3

3

23

7

7

21

46

1

2

3

4

5

6

7

8

9

10

11

12

13

14

53

45

8

11

12

21

43

44

9 13

9 13

01

20

33

10 14

10 14

9

43

23

19

11 15

11 15

8

42

15

2 Y LPP US R EWOP
19 19 23 23

13

POWE R SU PPL Y 3

XIV Storage 2
BladeCenter H
2 + 4 Y LPPUS R EWOP

18

13

41

16 20

16

7

20

17

03

40

17 21

17

6

21

16

92

18

18

5

39

22

22

82 4

14

38

72

37

3 62

36

2

12

52 1

35

42

11

0

32

9

33

10

34

XIV Storage 1
0 1 2 3 4 5 6 7 8 9 10 11
1 3 2 4

FC 850nm
POWER SUPPLY 4 + 2

1 + 3 Y LPPUS R EWOP

31

8

5

29

6

30

1 Y LPP US R EWOP

1

2

3

4

5

6

7

8

9

10

11

12

13

14

7

4

28

POWE R SU PPL Y 2

8 GB FC Backbone
633 enign E egarotS

0

1

2

3

4

5

6

7

8

9

10

11

24

1

25

2

26

3

27

1
1 3 2 4

2

3

4

5

6

7

8

9

10

11

12

13

14

0

System x3650 M2

POWER SUPPLY 3 + 1

POWE R SU PPL Y 1

0

1

2

3

4

5

6

7

8

9

10

11

System x3650 M2

1 3

2 4

System x3650 M2

0

1

2

3

4

5

6

7

8

9

10

11

ega rotS lato T

0

1

2

3

4

5

6

7

8

9

10

11

1 3

2 4

System x3650 M2

0

1

2

3

4

5

6

7

8

9

10

11

1 3

2 4

System x3650 M2

Infrastructure Mgmt

System Stor age

Datacenter 3
2498-B24
0 4 1 5 2 6 3 7 8 12 9 13 10 14 11 15 16 20 17 21 18 22 19 23

SVC Node

1
1 3 2 4

2

3

4

5

6

7

8

9

10

11

12

13

14

System x3650 M2

Infrastructure Mgmt

Bladecenter H

Power 750 Power 750

Infrastrukturserver
D3 D4 D5 D6 D7 D8 D9 D10
0 4 1 5 2 6 3 7 8 12 9 13 10 14 11 15 16 20 17 21 18 22 19 23 24 28

FC CWDM
25 29 26 30 27 31 32 36 33 37 34 38 35 39 0 4 1 5 2 6 3 7 8 12 9 13 10 14 11 15 16 20 17 21 18 22 19 23 24 28 25 29 26 30 27 31 32 36 33 37 34 38 35 39

Infrastrukturserver
D3 D4 D5 D6 D7 D8 D9 D10

2498-B40

2498-B40

560 560
System Stor age

1

4

5

8

9

12

DS3400

Quorum Power 750

Power 750

Fiberchannel Single Mode

Fiberchannel Multi Mode

Dark Fiber

Local Area Network

Figure 5: Data center landscape

Working with the service partner Netzwerktechnik Röver, CIBER implemented a redundant and geographically separated metro data center connection based on CWDM technology. All storage mirroring is performed through multiplexed Dark Fiber with a bandwidth of 60 GBit/s, which can be scaled up to 300 GBit/s. 12

The XIV system is controlled through an intuitive graphical user interface (GUI), and a powerful command line interface (CLI) for complex scripting. Training on XIV systems usage is usually a matter of hours rather than weeks, saving costs and speeding time to full production.

The very high I/O, reliability and availability performance is enabled by a grid-based controller and disk architecture with active-active component redundancy, which enables nondisruptive hot-swapping of hardware and firmware. XIV systems provide incremental scaling from partial to full rack, with 27-79 TB usable, and with every capacity increase the performance scales linearly. Automatic re-distribution of volumes ensures that the high performance and fast access continues even as data volumes grow. There is no tuning necessary, and the XIV system does not need to tune itself during operation.

The storage landscape is completed with two IBM System Storage TS3500 tape library systems, each with five LTO-4 drives, 260 slots and about 200 cartridges. These systems provide a data transfer speed of 350 GB per hour.

Database architecture For CIBER, DB2 for Linux, UNIX and Windows is the most appropriate database platform to support the SAP applications. Small and medium-sized businesses tend to underestimate their data growth, and easily reach sizes where DB2 offers significant advantages, such as Deep Compression that helps to reduce

Using the SVC’s FlashCopy function and XIV point-in-time copy CIBER clones SAP systems as customers request new instances. Combined with automated SAP post-processing, CIBER is able to create a completed cloned SAP instance in about 30 minutes. XIV Snapshot gives CIBER the ability to respond to customer demands in ways and at a speed that even in recent years would have been impossible to achieve.

the database size and data growth.

DB2 also offers close integration with Tivoli Storage Manager, providing a comprehensive backup solution that protects DB2 and related data reliably and cost-effectively.

The new index compression feature within DB2 is the next step to improve database performance and reduce storage costs. In

In selecting XIV systems, a new technology for the CIBER team, they soon discovered that only three hours’ training was needed to understand the easy-to-use XIV system’s graphical interface, compared with older systems that had taken weeks to fully learn (redundant).

addition, compression for temporary tables for SAP systems is activated ‘under the cover’ to reduce disk costs. DB2, compared with other database compression algorithms, can handle data and indexing more efficiently. CIBER experienced a compression rate in the high double-digit area, and performance improvements of 60 percent with DB2.

The DS3400 storage system serves two purposes: • As a storage device to host the active quorum disk (tie breaker) for the SVC clusters • As a storage pool for the TSM server Besides performance, DB2 also makes administration tasks easier and faster by offering a high level of automation with Self Tuning Memory Management, Automatic Storage, automatic runstats, and Real Time Statistics. CIBER uses IBM Tivoli Storage Manager, running in a separate LPAR on the Power servers, to manage backup, archive and restore. A dedicated physical 10 Gbps Ethernet adapter allows data transfer to run on fully virtualized end-to-end connections powered by Nexus switches from Cisco, connecting to the IBM System Storage TS3500 tape library. This arrangement offers very fast system backup and restore processes while ensuring that the data traffic does not impact production systems. 13 For example, CIBER uses the autoresize functionality so that tablespaces can enlarge themselves if needed, while the Container Management is completed manually to preserve a certain degree of control over the data management by the database administrators.

Automatic provisioning of SAP with Cloud services Automatic reorganization offers time-saving advantages for database administrators, because it checks if a reorganization of data and indices is needed and if so, the Reorg utility is automatically called and reorganizes the data and indices in the background. CIBER also plans to use the reclaimable storage feature for its SAP NetWeaver Business Warehouse to reduce the tablespaces from about 1 TB down to 300 MB Defining customer requirements will become an interactive, face to face activity, and once requirements have been defined, the SAP landscapes will be provisioned automatically. CIBER’s ultimate goal is to deliver Cloud services to its customers, which means being able to provision, upgrade and clone SAP instances very quickly using automated systems.

With the new tablespace option of reclaimable storage, introduced in DB2 version 9.7, it is now much easier to reduce the size of the tablespace and return unused space to the file system, including unused space below the High-Water Mark (HWM).

The Service Request Tool is the “CIBER Solution Suite”, which CIBER developed in-house, based on SAP Solution Manager. The backend (workflows) were developed in ABAP, while CIBER uses Adobe Flex™ (Fx) as GUI front end. This tool enables the CIBER SAP Consultant simply to select the correct SAP instance template, and then kick off the automated provisioning.

DB2 Migration: Opportunities for optimization It is possible to activate compression features during a SAP migration project to DB2, which eliminates the need for subsequent compression tasks. To clone SAP instances, CIBER uses XIV point-in-time copy. To do so, CIBER used the “Sampled” option, which comes with the R3Load Version 7. This option causes every ‘N’th row of the target table to be loaded first, followed by a reorg to create the Compression Dictionary based on the data loaded. The count of data used for the sampling process is provided by the environment variables DB6LOAD_SAMPLING_FREQUENCY and DB6LOAD_MAX_SAMPLE_SIZE. Subsequently, R3load is restarted to load the rest of the table in a compressed format. Because CIBER does not need to install the SAP instance, there are no pre- or post-processing steps, and a new clone can Transition to Unicode If Unicode is required on the target solution, then recommended practice is to perform the DB2 migration in conjunction with an SAP software upgrade and/or Unicode conversion in a single project. This combination eliminates redundant project tasks, keeping the cost of the migration as low as possible. Around 90 percent of the Unicode conversion steps are also part of the DB2 migration process. By combining the two processes, both costs and workload can be reduced, eliminating duplication of effort. therefore be provisioned and made available within 30 minutes. CIBER clones (snapshots) the entire SAP system, including EXE, DATA, and LOG files, while keeping the SID. The new clone is started in its own VLAN segment and will be automatically reconfigured and customized. This ensures that the new clone with the same SID does not interfere with the existing SAP landscape. CIBER used the standard OS tools, such as NIM & NIS (Network Installation Manager & Services), to install the operating system.

“IBM has brought in major contribution to the quick success of CIBER – to develop such a high customer rate within such a short timeframe.”
Stefan Rother Lead Infrastructure Services CIBER Managed Services GmbH

14

Project achievements
Performance improvements The new SAP infrastructure allows CIBER to improve efficiency for its SAP managed services business significantly by offering a solution that scales in performance and capacity, while providing a very cost-effective model. CIBER plans to take advantage of Active Memory Expansion, available on Power servers with POWER7 processors and upwards, which employs memory compression technology to transparently compress in-memory data, allowing more data to be placed into memory and thus expanding the memory The virtualization capabilities of the storage and server infrastructure allow CIBER to offer customers a very flexible service with real end-to-end virtualization, able to scale up quickly in times of peak workload and back again as required. For example, being able to clone a SAP system inside 30 minutes, or to activate another 1 TB of storage on the XIV system within an hour, mean that CIBER can cope with almost any customer demand. The result is that CIBER can offer “pay as you grow” SAP hosting as software as a service (SaaS) backed by a powerful cloud infrastructure that encourages customers to make the leap without having to predict workload or commit to specific infrastructure spending. For improved data center management and new services, CIBER plans to apply VMware vCloud Director in 2011. vCloud Director is a data center management software, which pools virtual infrastructure resources and delivers them to users as a The combined performance improvements available from the IBM Power servers, IBM System x BladeServers, and IBM storage infrastructure have led to increased user satisfaction. Many customers have noticed greatly improved response times for online operation of the DB2 database – in some cases recording an eight-fold improvement. The enhanced response times contribute to increased user productivity, and help build the case for using hosted SAP applications rather than in-house, as CIBER is able to make the kind of technology investment that might be out of reach of a single customer. catalog-based service. For cost models that are customized for accounting of resource requests and consumption, CIBER will implement VMware vCenter Chargeback. CIBER plans to roll out IBM DB2 Deep Compression and Row Compression features on all of its systems, and deploy IBM Tivoli Provisioning Manager and IBM Tivoli FlashCopy to enhance data security and performance. capacity of POWER7 systems. Utilizing Active Memory Expansion can improve system utilization and increase a system’s throughput.

“We recommend the DB2 database for new customers who start with SAP, because DB2 is very cost-effective and brings performance benefits and DB2 and SAP simply fit together.”
Holger Stasch SAP Senior Consultant CIBER Managed Services GmbH

15

Authors Markus Fehling, IBM, markus.fehling@de.ibm.com Elke Hartmann-Bakan, IBM, elke.hartmann-bakan@de.ibm.com Frank Neumann, IBM, frank.m.neumann@de.ibm.com Carsten Siegler, IBM, csiegler@de.ibm.com Hans-Achim Muscate, VMware, hamuscate@vmware.com Markus Peltz, CIBER, markus.peltz@ciber.com

© Copyright IBM Corp. 2010 All Rights Reserved. IBM Deutschland GmbH D – 71137 Ehningen ibm.com Produced in Germany IBM, the IBM logo, ibm.com, i5/OS, DB2, Domino, FlashCopy, Lotus, Notes, POWER, POWER4, POWER5, POWER6, System i, System x, and Tivoli are trademarks of International Business Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms are marked on their first occurrence in this information with a trademark symbol (® or ™), these symbols indicate U.S. registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of other IBM trademarks is available on the Web at: http://www.ibm.com/legal/copytrade. shtml UNIX is a registered trademark of The Open Group in the United States and other countries. Linux is a trademark of Linus Torvalds in the United States, other countries, or both. Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both. Other company, product or service names may be trademarks, or service marks of others. This brochure illustrates how IBM customers may be using IBM and/or IBM Business Partner technologies/services. Many factors have contributed to the results and benefits described. IBM does not guarantee comparable results. All information contained herein was provided by the featured customer/s and/or IBM Business Partner/s. IBM does not attest to its accuracy. All customer examples cited represent how some customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics will vary depending on individual customer configurations and conditions. This publication is for general guidance only. Photographs may show design models.

For more information To learn more about the solutions from IBM and SAP, visit: ibm-sap.com

For more information about SAP products and services, contact an SAP representative or visit: sap.com

For more information about IBM products and services, contact an IBM representative or visit: ibm.com

For more information about CIBER products and services, contact a CIBER representative or visit: ciber.com

Contacts Markus Fehling, markus.fehling@de.ibm.com Markus Peltz, markus.peltz@ciber.com

For feedback and further questions please contact the IBM SAP International Competency Center via isicc@de.ibm.com

SPC03307-DEEN-00 (December 2010)

Master your semester with Scribd & The New York Times

Special offer for students: Only $4.99/month.

Master your semester with Scribd & The New York Times

Cancel anytime.