Centera To ECS 2.2.1 Native Migration (Transformation) : Vasily Pantyukhin

Centera to ECS 2.2.
1
Native Migration (Transformation)
Vasily Pantyukhin
http://oldhenhut.com

Contents
1. The overall procedure .............................................................................................................. 2

2. Preparation & Configuration .................................................................................................... 2
2.1. Centera preparation ............................................................................................................................................................... 2
2.2. Authentication into ECS ........................................................................................................................................................ 7
2.3. Get the Replication Group ID .............................................................................................................................................. 7
2.4. Create transformation instance ......................................................................................................................................... 8
3. Discovery & Precheck ............................................................................................................... 9
3.1. Get profile mappings .............................................................................................................................................................. 9
3.1. Set profile mapping ............................................................................................................................................................... 11
3.2. Set transformation sources ............................................................................................................................................... 12
3.3. Run transformation pre-check ........................................................................................................................................ 12
3.4. Check the ECS components created ............................................................................................................................... 15
3.4.1. Namespace ............................................................................................................................................................................. 15
3.4.2. Users ......................................................................................................................................................................................... 16
3.4.3. Buckets .................................................................................................................................................................................... 20
3.5. Pre-check conflicts ................................................................................................................................................................ 21
3.6. Switch application to ECS ................................................................................................................................................... 22
4. Run Enumeration ................................................................................................................... 22
5. Run data Indexing .................................................................................................................. 23
6. Run data migration ................................................................................................................ 24
7. Run a transformation data reconciliation ............................................................................... 25
8. Delete the previous Transformation job ................................................................................. 27
9. Appendix ............................................................................................................................... 28
9.1. Centera to ECS mappings ................................................................................................................................................... 28
9.2. Access control rules .............................................................................................................................................................. 28

1. The overall procedure

The Transformation and Migration contains several phases:
• Preparation & Configuration
• Discovery & Pre-check
• Enumeration
• Indexing
• Data Migration
• Reconciliation
2. Preparation & Configuration
2.1. Centera preparation
• Start Centera Viewer and then access the CLI commands

• Get the list of Centera Pools
Config# show pool list

Pool Name ID Profiles Mask
-------------------------------------------------------------------------------
…
NASforapp_T2 4abb97ce-1dd2-11b2-97bc-955cedfd9a8d-13 2 rdqeDcw-
NASforapp_T3 4abb97ce-1dd2-11b2-97bc-955cedfd9a8d-14 2 rdqeDcw-
-------------------------------------------------------------------------------

• Check the details of Centera Pools
Config# show pool detail NASforapp_T2
Centera Pool Detail Report
------------------------------------------------------
Generated on 9 June 2016 13:47:19 CEST

Pool Name: NASforapp_T2
Pool ID: 4abb97ce-1dd2-11b2-97bc-955cedfd9a8d-13
Pool Mask: rdqeDcw-
Cluster Mask: rdqe-cw--h

Pool Quota Alert: 500 GB
Pool Quota Hard Stop: --
Used Pool Capacity: 17 MB
Free Pool Capacity (Alert): 500 GB
2

Free Pool Capacity (Hard Stop): --

Number of C-Clips: 3
Number of Files: 6
Number of scheduled tasks: 0

Granted Rights to Access Profiles:

Profile Name Granted Effective Monitor Cap Enabled Home Pool
-------------------------------------------------------------------------------
NASforapp_T2 rdqeDcw- rdqe-cw- no yes yes
replication rdqeDcw- rdqe-cw- yes yes no
-------------------------------------------------------------------------------

Pool Mappings: none
Pool being replicated: yes

------------------------------------------------------

Pool Mask: rdqeDcw-

Used Pool Capacity: 0 GB
Number of C-Clips: --
Number of Files: --


-------------------------------------------------------------------------------
-------------------------------------------------------------------------------

Pool Mappings: none
• Check details for the specific Pools

------------------------------------------------------

Pool Mask: rdqeDcw-

Used Pool Capacity: 18 MB
Number of C-Clips: 3
Number of Files: 6


-------------------------------------------------------------------------------
-------------------------------------------------------------------------------

Pool Mappings: none
• Find the PEA file for CTA pool

• To generate new PEA file use the command update profile <profilename>
<pea version="1.0.0">
<defaultkey name="NASforapp_T1">
<credential id="csp1.secret" enc="base64">xxxxxx</credential>
</defaultkey>
<key type="cluster" id="4abb97ce-1dd2-11b2-97bc-955cedfd9a8d" name="NASforapp_T1">
<credential id="csp1.secret" enc="base64">xxxxxx</credential>
</key>
</pea>
• Check .pea file with JCenteraVerify

• Check JCenteraVerify logs
**************************************************************
************** Start of JCenteraVerify log ******************
**************************************************************

--------------------- General information -----------------------

Time: Tue Jun 07 16:10:03 CEST 2016
Tool Version: 3.2.23
SDK Version: 3.2.607

--------------------- User configuration ------------------------

TOOL SETTINGS
Number Of Files: 3
FileSize (KB): 1, 10, 100
Working Dir: .\Temp
Restore Dir: .\Temp\Retrieve
Log File: .\JCenteraVerify.log
Delete temp Files: true
Delete restore Files: true

SDK SETTINGS
Number of Retries: -1
Retry Sleep: -1

----------------- JCenteraVerify test results -------------------

ACCESS NODE CONNECTIVITY CHECK

192.168.0.6?C:\zz_business_temp\NASforapp_T1.pea is available.

CLUSTER INFORMATION

ClusterId: 4abb97ce-1dd2-11b2-97bc-955cedfd9a8d
CentraStar: 4.2.2-4682-2-0
Available Capacity: 23,240 GB
Total Free Capacity: 26,82 GB
Replica Address: 192.168.0.28

CAPABILITIES

Read allowed: true
Write allowed: true
Delete allowed: true
Exists allowed: true
Purge allowed: false
Monitor allowed: false
Query allowed: true
PrivDelete allowed: false

Writing the file 1
1 File(s) written successfully. CA: 1V2115AP198S2e01GENR0VQFVS1
Reading the file 1. CA: 1V2115AP198S2e01GENR0VQFVS1
1 File(s) read successfully
Deleting the file 1. CA: 1V2115AP198S2e01GENR0VQFVS1
1 File(s) deleted successfully
Writing the file 2
2 File(s) written successfully. CA: DDI527S7SB9L5eF51GSNOKM4UIO
Reading the file 2. CA: DDI527S7SB9L5eF51GSNOKM4UIO
Deleting the file 2. CA: DDI527S7SB9L5eF51GSNOKM4UIO
Writing the file 3
3 File(s) written successfully. CA: 10M3R9KL1OS96eAEJ1LA10VPFVC
Reading the file 3. CA: 10M3R9KL1OS96eAEJ1LA10VPFVC
Deleting the file 3. CA: 10M3R9KL1OS96eAEJ1LA10VPFVC

**************************************************************
***************** End of JCenteraVerify log ****************
**************************************************************
2.2. Authentication into ECS
• Authenticate with ECS keeping the token in the cookie file

admin@host:/opt/emc> curl -k https://root:xxx@10.10.10.1/login > /tmp/token
admin@host:/opt/emc> cat /tmp/token
Bxxxxxxxxxxxxxxxx0A8=

• To make further the curl commands shorter, specify some HTTP headers in the bash valiable
admin@host:/tmp> export HEADERS="-H X-SDS-AUTH-TOKEN:$(cat /tmp/token) -H
Accept:application/xml -H Content-Type:application/json"

2.3. Get the Replication Group ID
• Check all available RGs in ECS GUI

• We are going to use the RG distributed across 2x VDCs
• Get the ID for the RG

admin@host:/tmp> curl -s -k -XGET ${HEADERS} https://localhost:4443/vdc/data-service/vpools |
xmllint --format -
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<data_service_vpools>
…
<data_service_vpool>
<creation_time>1465560162090</creation_time>
<id>urn:storageos:ReplicationGroupInfo:f3fdeb40-0b14-4896-bdda-2af1710227ec:global</id>
<inactive>false</inactive>
<isAllowAllNamespaces>true</isAllowAllNamespaces>
<description/>
<enable_rebalancing>true</enable_rebalancing>
<isFullRep>false</isFullRep>
<name>rg_ri_st</name>
<varrayMappings>
<name>urn:storageos:VirtualDataCenterData:3acbaff8-a2b1-4f0c-b281-27844140e69e</name>
<value>urn:storageos:VirtualArray:53dd93b6-54e8-4827-bafb-b493d637299d</value>
</varrayMappings>
<varrayMappings>
<name>urn:storageos:VirtualDataCenterData:79c6761c-eadd-4fa8-a05e-69998a49d888</name>
<value>urn:storageos:VirtualArray:c365c750-b45a-4fcf-993a-a36a6491d941</value>
</varrayMappings>
</data_service_vpool>
</data_service_vpools>

2.4. Create transformation instance
• Create the config file

admin@host:/tmp> vi transformation.json
{
"type": "centera",
"name": "Centera Transformation",
"description": "Centera transformation",
"replication_group": "urn:storageos:ReplicationGroupInfo:f3fdeb40-0b14-4896-bdda-
2af1710227ec:global",
"admin": "admin",
"password": "centera",
"management_ip": "192.168.0.6",
"port": 3682,
"access_ip": "192.168.0.6",
"datagram_port": 3218
}

• Create the new Transformation job

admin@host:/tmp> curl –s -k -XPOST ${HEADERS} -d @transformation2.json
https://localhost:4443/object/transformation | xmllint --format -
<transformation>
<transformation_id>urn:Transformation:Centera:4abb97ce-1dd2-11b2-97bc-
955cedfd9a8d</transformation_id>
</transformation>

• Check the new Transformation job
admin@host:/tmp> curl -s -k -XGET ${HEADERS} https://localhost:4443/object/transformation |
xmllint --format -
<transformationList>
<transformations>
<id>urn:Transformation:Centera:4abb97ce-1dd2-11b2-97bc-955cedfd9a8d</id>
<namespace>centera_4abb97ce-1dd2-11b2-97bc-955cedfd9a8d</namespace>
<replication_group>urn:storageos:ReplicationGroupInfo:f3fdeb40-0b14-4896-bdda-
2af1710227ec:global</replication_group>
</transformations>
</transformationList>

• To make the curl commands shorter, specify the common part of the URL in the shell variable
admin@host:/tmp> export
TRANSFORMATION_BASE_URI=https://localhost:4443/object/transformation/urn:Transformation:
Centera:4abb97ce-1dd2-11b2-97bc-955cedfd9a8d

3. Discovery & Precheck
3.1. Get profile mappings
A Profile Mapping defines a mapping between a pair of data instances defined as a Centera Pool and
Profile mapped to a to a pair of data instances defined as an ECS bucket and an ECS user.
Auto-calculated Profile Mappings are less authoritative than manually created ones. If a target_user or
target_bucket of an auto-calculated Profile Mapping matches an already existing ECS user or ECS bucket,
ECS marks that auto-calculated Profile Mapping as a conflicting one. You cannot start a transformation
if conflicting Profile Mappings exist. You can over-ride a conflicting auto-calculated Profile Mapping by
setting a manual Profile Mapping entry for Centera source profile.

• List of currently available Profile Mapping entries.
admin@host:/tmp> curl -k -XGET ${HEADERS} ${TRANSFORMATION_BASE_URI}/profile/mapping |
xmllint --format –
<profile_mappings>
…
<mappings>
<centera_profile>
<mask>rdqe-cw-</mask>
<pool>NASforapp_T2</pool>
<profile>replication</profile>
<source_id>NASforapp_T2/replication</source_id>
</centera_profile>
<profile_mapping>
<target_bucket>NASforapp_T2</target_bucket>
<target_namespace>centera_4abb97ce-1dd2-11b2-97bc-955cedfd9a8d</target_namespace>
<target_user>replication</target_user>
</profile_mapping>
<conflict>
<conflict_type>userExists</conflict_type>
</conflict>
</mappings>
<mappings>
<centera_profile>
<mask>rdqe-cw-</mask>
<pool>NASforapp_T2</pool>
<profile>NASforapp_T2</profile>
<source_id>NASforapp_T2/NASforapp_T2</source_id>
</centera_profile>
<profile_mapping>
<source_id>NASforapp_T2/NASforapp_T2</source_id>
<target_user>nasforapp_t2</target_user>
</profile_mapping>
</mappings>
</profile_mappings>

10

3.1. Set profile mapping
Manual naming assignment and pre-check conflicts must be resolved manually by setting Profile
Mappings.
• Specify how source Centera Pool:Profile should be mapped to ECS user and backet in the .json file.
admin@host:/tmp> vi mapping-modif.json
{
"mappings": [
{
"source_id": "NASforapp_T2/replication",
"target_user": "replication3",
"target_bucket": "NASforapp_T2",
"target_namespace": "centera_4abb97ce-1dd2-11b2-97bc-955cedfd9a8d"
},
{
"source_id": "NASforapp_T3/replication",
"target_user": "replication4",
"target_bucket": "NASforapp_T3",
"target_namespace": "centera_4abb97ce-1dd2-11b2-97bc-955cedfd9a8d"
}
]
}

• Change the Accept:application header from xml to json
• Assign the customized mapping
Accept:application/json -H Content-Type:application/json"
admin@host:/tmp> curl -s -k -XPOST ${HEADERS} -d @mapping-modif.json
${TRANSFORMATION_BASE_URI}/profile/mapping

• For better xml structure readability change the Accept:application header from json back to xml
• Checke the modified mapping
Accept:application/xml -H Content-Type:application/json"
admin@host:/tmp> curl -k -XGET ${HEADERS} ${TRANSFORMATION_BASE_URI}/profile/mapping |
…
<profile_mapping>
<target_user>replication3</target_user>
</profile_mapping>
</mappings>
<profile_mapping>
<target_user>replication4</target_user>
</profile_mapping>
11

3.2. Set transformation sources
ECS need to have one or several CAS pools to be selected for Transformation. Selection is made by
supplying a list of Centera profile source_id values via Set Transformation Sources REST API. Only one
Centera profile per pool can be selected as a source. Selected source profile should have Read and Query
capabilities
• Specify the Centera Pool to be transformed in the .json file
admin@host:/tmp> vi transformationsources.json
{
"source_ids" : [
"NASforapp_T2/NASforapp_T2",
"NASforapp_T3/NASforapp_T3"
]
}

• Assign transformation sources
admin@host:/tmp> curl -s -k -XPOST ${HEADERS} -d @transformationsources.json
${TRANSFORMATION_BASE_URI}/transformationSources | xmllint --format -
<transformation>
</transformation>

• Check the assigned transformation sources
admin@host:/tmp> curl -s -k ${HEADERS} ${TRANSFORMATION_BASE_URI} | xmllint --format -
<transformation>
<source_ids>NASforapp_T2/NASforapp_T2</source_ids>
<source_ids>NASforapp_T3/NASforapp_T3</source_ids>
</transformation>

3.3. Run transformation pre-check
During the pre-check phase, ECS will make sure the data from these pools can be migrated and will
create the different components (namespace, buckets, users).
The result of the pre-check must be read carefully to ensure the migration can be performed

12

List of the checks executed:

Check Description
SOURCE_ADMIN_HAS_CAPABILITIES_CHECK Provisioned Centera MAPI user has enough privileges
DATA_INTEGRITY_CHECK Centera reports data integrity incompleteness
HW_VERSION_CHECK Centera hardware version does not met requirements
UPGRADE_STATUS_CHECK Centera upgrade is in progress
SW_VERSION_CHECK Centera software version does not met requirements
COMPLIANCE_MODE_CHECK Unsupported compliance mode
PROFILE_MAPPING_CONFLICTS_RESOLVED_
There are still unresolved Profile Mapping conflicts
CHECK
SOURCE_PROFILES_ARE_CONFIGURED No source pools are added to the Transformation
SOURCE_PROFILES_AUTH_CHECK Configured source pools is not accessible by profile given
NAMESPACE_CREATED_CHECK Ensures that target namespace for Transformation buckets exists,
namespace will be created if missing. New namespace admin user
will be created automatically for newly created namespace
BUCKETS_CREATED_CHECK Ensures that target buckets exist, buckets will be created if
missing
USERS_CREATED_CHECK Ensures that ECS users exist, users will be created if missing
RETENTION_CLASSES_IMPORT_CHECK Ensures that ECS CAS retention classes match Centera, ECS CAS
retention policies will be created if missing
COMPLIANCE_MODE_PERSISTED_CHECK Ensures that configured Centera compliance mode is stored in
Transformation metadata
RECONCILIATION_USERS_CREATED_CHECK Ensures that miscellaneous data user accounts needed at
reconciliation phase are created

• Start the Precheck job
admin@host:/tmp> curl -s -k -XPOST ${HEADERS} ${TRANSFORMATION_BASE_URI}/precheck |
xmllint --format -
<transformation>
</transformation>

• Check the status of the pre-checks
admin@host:/tmp> curl -s -k -XGET ${HEADERS} ${TRANSFORMATION_BASE_URI}/precheck | xmllint
--format –
<precheck_report>
<processing_speed>0.0</processing_speed>
<processing_speed_for_last_day>0.0</processing_speed_for_last_day>
<processing_speed_for_last_hour>0.0</processing_speed_for_last_hour>
<progress>100.0</progress>
<start_time>2016-06-09T10:15:58.531</start_time>
<status>Succeeded</status>
<prechecks>
<message/>
<name>DATA_INTEGRITY_CHECK</name>
<status>OK</status>
</prechecks>
<prechecks>
13

<message/>
<name>HW_VERSION_CHECK</name>
<status>OK</status>
</prechecks>
<prechecks>
<message/>
<name>UPGRADE_STATUS_CHECK</name>
<status>OK</status>
</prechecks>
<prechecks>
<message/>
<name>SW_VERSION_CHECK</name>
<status>OK</status>
</prechecks>
<prechecks>
<message/>
<name>COMPLIANCE_MODE_CHECK</name>
<status>OK</status>
</prechecks>
<prechecks>
<message/>
<name>ADVANCED_RETENTION_CHECK</name>
<status>OK</status>
</prechecks>
<prechecks>
<message/>
<name>PROFILE_MAPPING_CONFLICTS_RESOLVED_CHECK</name>
<status>OK</status>
</prechecks>
<prechecks>
<message/>
<name>RECONCILIATION_USERS_SETUP_CHECK</name>
<status>OK</status>
</prechecks>
<prechecks>
<message/>
<name>SOURCE_PROFILES_ARE_CONFIGURED</name>
<status>OK</status>
</prechecks>
<prechecks>
<message/>
<name>SOURCE_PROFILES_AUTH_CHECK</name>
<status>OK</status>
</prechecks>
<prechecks>
<message/>
<name>NAMESPACE_CREATED_CHECK</name>
<status>OK</status>
</prechecks>
<prechecks>
<message/>
<name>BUCKETS_CREATED_CHECK</name>
<status>OK</status>
</prechecks>
<prechecks>
14

<message/>
<name>USERS_CREATED_CHECK</name>
<status>OK</status>
</prechecks>
<prechecks>
<message/>
<name>RETENTION_CLASSES_IMPORT_CHECK</name>
<status>OK</status>
</prechecks>
<prechecks>
<message/>
<name>COMPLIANCE_MODE_PERSISTED_CHECK</name>
<status>OK</status>
</prechecks>
<prechecks>
<message/>
<name>RECONCILIATION_USERS_CREATED_CHECK</name>
<status>OK</status>
</prechecks>
</precheck_report>

3.4. Check the ECS components created
All necessary ECS components have been created automatically on ECS.

3.4.1. Namespace
• Centera Cluster is mapped to ECS Namespace

• Namespace naming convention is centera_<Centera ID>
15

• Quotas and Access During Outage are disabled for the Namespace
3.4.2. Users
• Object users for data access and reconciliation are automatically created

16

• User centera-<Centera ID>

• CAS access is not configured for this user

• Object user for every Pool / Bucket

• That kind of users always configured to get an object access
• Password and corresponding .pea file are generated
• Transformation MetaInfo attribute is specified
17

• Separate reconciliation user is used to check data consistency on the final step of the of data
migration
18

• We are not going to use the user which corresponds to the 2nd Centera Profile for the data access
• Despite on that, the corresponding user is created following the mapping rules

19

3.4.3. Buckets
• Buckets are 1:1 created corresponding to the Centera Pools

• Bucket owner is root cause an authorization is configured via ACL

• Soft Quota is assigned according to the Centera’s Pool Quota Alert parameter

20

• Of course CAS functionality is enabled

• Metadata Search is enables as well

• The main object use which corresponds to the source Centera Profile has Full Control ACL rights
• Additional user which corresponds the 2nd Centera profile has Read / Write and Delete rights
• Reconciliation user can read data only
3.5. Pre-check conflicts
There are several outcomes for Prechecks & Provision:

• Veto – ECS has discovered severe issue which prevents Transformation to proceed. Issues must be
resolved and prechecks to be retried before the Application switch over to ECS
• Error – ECS has discovered major issue which might impact some part of Transformation. It is unsafe
to perform the Application switch over to ECS while issue exists as it may cause service interruption.
21

Issues must be resolved and prechecks to be retried, however, in certain scenarios Error status can be
ignored
• Warning – Transformation is started. It is safe to perform the Application switch over to ECS. ECS has
discovered a minor problem which can be resolved afterwards
• Ok - Transformation is started without discovering any problem

Possible pre-check conflicts are:
Check Description
userExists automatically calculated ECS user name matches already existing ECS user
bucketExists automatically calculated ECS bucket name matches already existing ECS bucket
userOverlaps several automatically calculated ECS user names equals to each other

• Example of the possible pre-check conflict
<message>Unresolved ProfileMappings found: [NASforapp_T2/replication, ...]</message>
<name>PROFILE_MAPPING_CONFLICTS_RESOLVED_CHECK</name>
<status>VETO</status>

The most common reason for pre-check conflicts is the mapping conflicts. Is can be resolve as described
in the corresponding section above.

3.6. Switch application to ECS
Now we are ready to switch our application to ECS. Do this now.

Note: I tested that with CTA. It works just fine.

4. Run Enumeration
Important: Don’t start the enumeration before the application has been configured to communicate with
ECS instead of the Centera. Otherwise, new objects created after enumeration is started will not be
migrated.

At the Enumeration stage ECS creates a list of Centera content
o Centera Query is used to list clips
o ECS uses write time of the first written clip as a lower bound
o Current time as upper bound for the query
o ECS chops initially detected time interval into smaller pieces to parallelize processing
22

o Tasks to enumerate resulting time frames are spread across the ECS nodes randomly to
parallelize processing
o Enumeration of different time frames is performed without strict ordering
o Knowledge on how densely clips are located on a time line is not available for ECS – some
enumeration task may get more work than others

• Start Enumeration job
admin@host:/tmp> curl -s -k -XPOST ${HEADERS} ${TRANSFORMATION_BASE_URI}/enumeration |
<transformation>
</transformation>

• Check the status of the pre-checks
admin@host:/tmp> curl -s -k -XGET ${HEADERS} ${TRANSFORMATION_BASE_URI}/enumeration |
xmllint --format -
<enumeration_report>
<pool_result source_id="NASforapp_T2/NASforapp_T2">
<entry_count>5</entry_count>
</pool_result>
<processing_speed>4.9E-324</processing_speed>
<processing_speed_for_last_day>4.9E-324</processing_speed_for_last_day>
</enumeration_report>

• Call this request, ideally in a timed loop, until the API returns both "status": "Succeeded" and
"progress": 100.0.

5. Run data Indexing

During the Data Indexing phase, ECS will get all the metadata from Centera
o At this stage ECS processes the previously created list of clips/blobs and builds internal indexes
of Centera content
o Once Centera object is indexed it becomes native ECS object
o Indexed objects become available globally through all ECS VDCs in replication group
23

o Indexing also prepares for upcoming Migration planning how objects will be packed in chunks

• Run Indexing job
admin@host:/tmp> curl -s -k -XPOST ${HEADERS} ${TRANSFORMATION_BASE_URI}/indexing | xmllint
--format -
<transformation>
</transformation>

• Get Indexing status
admin@host:/tmp> curl -s -k -XGET ${HEADERS} ${TRANSFORMATION_BASE_URI}/indexing | xmllint -
-format -
<indexing_report>
<processed_bytes>18172960</processed_bytes>
<processed_objects>5</processed_objects>
</indexing_report>

• Call this request, ideally in a timed loop, until it returns both "status": "Succeeded" and "progress":
100.0.

Note: Quota is checked during Indexing for the quota enabled Namespace/Bucket
o May cause Indexing to fail if hard quota for Namespace/Bucket is exceeded
o No storage space pre allocated for objects content during indexing

In case any issue is detected, the list of C-clips that haven’t been migrated successfully can be obtained.
Any mismatched clips will not automatically retry, but require manual intervention to retry migration.

6. Run data migration

o Migration phase goes through all chunks created at indexing phase and fulfills it with objects
content Migration process includes erasure coding of migrated content on the fly
o Migration is preformed by all ECS nodes
o Several chunks are migrated at the same time without strict ordering
24

o Due to concurrent and orderless nature of migration some objects may be partially migrated at
particular point in time

Note: You must already have succeeded at the Data Indexing phase to begin the Data Migration phase.

• Run Migration job
admin@host:/tmp> curl -s -k -XPOST ${HEADERS} ${TRANSFORMATION_BASE_URI}/migration |
xmllint --format -
<transformation>
</transformation>

• Get Migration status
admin@host:/tmp> curl -s -k -XGET ${HEADERS} ${TRANSFORMATION_BASE_URI}/migration |
xmllint --format -
<migration_report>
<processing_speed>0.0</processing_speed>
<processing_speed_for_last_day>0.0</processing_speed_for_last_day>
</migration_report>

100.0.

7. Run a transformation data reconciliation

o During Reconciliation ECS goes through the list of objects collected during Enumeration phase
and verifies that those are readable from ECS
o As the result Reconciliation phase produces a list of objects which are not readable from ECS or
their checksum does not match
o Reconciliation uses special ECS data users which has no mapping to Centera profiles which
ensures that ECS logic won’t fallback to Centera during reading objects

25

• Run Reconciliation job

admin@host:/tmp> curl -s -k -XPOST ${HEADERS} ${TRANSFORMATION_BASE_URI}/reconciliation |
xmllint --format -
<transformation>
</transformation>

• Check the Reconciliation status
admin@host:/tmp> curl -s -k -XGET ${HEADERS} ${TRANSFORMATION_BASE_URI}/reconciliation |
xmllint --format -
<reconciliation_report>
<reconciled_objects>5</reconciled_objects>
<unread_objects>0</unread_objects>
</reconciliation_report>

100.0.

• Check if any mismatches exist
admin@host:/tmp> curl -s -k -XGET ${HEADERS}
${TRANSFORMATION_BASE_URI}/reconciliation/mismatches | xmllint --format -
<reconciliation_mismatches/>

o Re-Migration phase is needed to repeat Migration for objects which were identified as
unsuccessfully migrated
o Since ECS operates by erasure coded chunks during migration whole chunk re-migration is
initiated even though just a single object could be not migrated
o Reconciliation can be executed once more time as soon as Re-Migration is finished

26

8. Delete the previous Transformation job

• Check the status of the previous Transformation job
admin@host:/tmp> curl -s -k -XGET ${HEADERS} ${TRANSFORMATION_BASE_URI} | xmllint --format -
<transformation>
<phases>
<phase_id>Enumeration</phase_id>
</phases>
<phases>
<phase_id>Indexing</phase_id>
</phases>
<phases>
<phase_id>Migration</phase_id>
</phases>
<phases>
<phase_id>PreCheck</phase_id>
</phases>
<phases>
<phase_id>Reconciliation</phase_id>
</phases>
<source_ids>FMA_Site1_T/FMA_Site1_T</source_ids>
</transformation>

• Delete the Transformation job
admin@host:/tmp> curl -s -k -XDELETE ${HEADERS} ${TRANSFORMATION_BASE_URI} | xmllint --
format -
<transformation_phase>
<phase_id>Deletion</phase_id>
<status>Pending</status>
</transformation_phase>

• Check if the Transformation job is deleted
admin@host:/tmp> curl -s -k -XGET ${HEADERS} https://localhost:4443/object/transformation |
xmllint --format -
<transformationList/>

27

9. Appendix
9.1. Centera to ECS mappings
• Centera and ECS logical configuration mapping
9.2. Access control rules
• Access control mapping

ECS permission Centera mask Centera capabilities
Read r-qe---- Read, Query, Exist
Write ------wh Write, Litigation hold
FULL_CONTROL rdqe-cwh Read, Delete, Query, Exist, Clip copy, Write, Litigation hold
PRIVILEGED_WRITE ----D--- Privileged delete
DELETE -d------ Delete
• Centera Access Masks are collapsed into Effective Access Mask by following formula
• Effective Access Mask mapped to appropriate ECS Bucket Access rights for particular user
• ECS Bucket Access rights are automatically copied into Object Access Rights for each object
transformed

28

Centera To ECS 2.2.1 Native Migration (Transformation) : Vasily Pantyukhin

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Centera To ECS 2.2.1 Native Migration (Transformation) : Vasily Pantyukhin

Uploaded by

Copyright:

Available Formats

Centera to ECS 2.2.

1. The overall procedure .............................................................................................................. 2

1. The overall procedure

2. Preparation & Configuration

2.1. Centera preparation

• Start Centera Viewer and then access the CLI commands

Free Pool Capacity (Hard Stop): --

• Check details for the specific Pools

• Find the PEA file for CTA pool

• Check .pea file with JCenteraVerify

2.2. Authentication into ECS

• Authenticate with ECS keeping the token in the cookie file

2.3. Get the Replication Group ID

• Check all available RGs in ECS GUI

• Get the ID for the RG

2.4. Create transformation instance

• Create the config file

• Create the new Transformation job

3. Discovery & Precheck

3.1. Get profile mappings

3.1. Set profile mapping

3.2. Set transformation sources

3.3. Run transformation pre-check

List of the checks executed:

3.4. Check the ECS components created

All necessary ECS components have been created automatically on ECS.

• Centera Cluster is mapped to ECS Namespace

• User centera-<Centera ID>

• Buckets are 1:1 created corresponding to the Centera Pools

• Of course CAS functionality is enabled

3.5. Pre-check conflicts

There are several outcomes for Prechecks & Provision:

3.6. Switch application to ECS

Now we are ready to switch our application to ECS. Do this now.

5. Run data Indexing

6. Run data migration

7. Run a transformation data reconciliation

• Run Reconciliation job

8. Delete the previous Transformation job

• Centera and ECS logical configuration mapping

9.2. Access control rules

• Access control mapping

You might also like