EMC® Documentum® xPlore Version 1.3 Installation Guide EMC Corporation Conperate Hen Hopkin, MA. 01748-9103 wenn EMC comCopyright © 2010-2012 EMC Corporation. Altrights reserved [BMC believes the infomastion mths plication accurate a5 of ts publication dete. The formation is subject to ung witht ‘THE INFORMATION IN THIS PUBLICATION 1S PROVIDED “AS IS.” EMC CORPORATION MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT 10 THE INFORMATION IN THIS PUBLICATION, ‘AND SPECIFICALLY DISCLAIMS IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR Purpose. ‘Ure copying, and diction of my EMC softer decribed this publication resures an applicable sofas iene Forte most pto-date stg of EMC prodactnanes, see EMC Caperiton Tadnnats on EMC cm. Adobe wa Adobe PDF Livary ae ademas or registered ademas of Adobe Systems Ix inthe U5. and other comes. Alloter tademacks used Ihren ue the propery of Cex respecte emma Documnentatien Feedback ‘Yor ophin mates. Wie want to ear fram youregardng ou prodctdocamntati. I youhave feedback about how we can sma our documentation better or easier tows, pleae sends your feedback directly at 1CDocimentationFeedci ent cm.Table of Contents Chapter 1 Chapter 2 Chapter 3 Chapter 4 Chapter 5 Installation Overview. 9 Installation and upgrade workflow. 9 Planning the xPlore Environment. How xPlore works 1" Planning your installation 12 Determining your requirements 12 xPlore sizing calculation. 13 Sizing CPU 13 Sizing memory 14 Sizing storage. 15 Network and virtual machines 16 Hardware tradeoffs. 7 Planning High Availability and Disaster Recovery About high availability and disaster recovery 19 Planning for high availabilty a Two primary instances HA (active-active) process, 2 Planning for disaster recovery 24 Disaster recovery in a Documentum environment 2B Installing xPlore .. Before you install 29 Enabling SSL to Content Server 6.7 or higher 30 Enabling SSL to xPlore administrator only 3 Installing xPlore with Content Server 6.7 SPx and 7.0. 34 Installing xPlore with Content Server 6.6 36 Validating installation 38 Starting xPlore Administrator 38 Testing indexing 38 Testing search 39 Installing a secondary or spare instance 39 Adding CPS Instances. a Configuring CPS dedicated to indexing or search 42 Deleting an xPlore or CPS instance 43 Installing and Configuring index Agents. Preparing index agent installation. 45 [EMC Documentum xPlore Version 1.3 Instaation Guide 3Table of Contents Chapter 6 Chapter 7 Chapter 8 Chapter 9 Chapter 10 Configuring an index agent 46 Starting the index agent ar ‘Stopping the index agent ar Silent index agent startup, 48 Deleting an index agent 50 Performing Silent Installation.. Setting up silent installation. 5 Configuring sient installation 52 Running the silent install script 54 Verifying silent installation 56 Troubleshooting silent installation. 57 ‘Sample single-instance installation, 57 Setting up High Availability ‘Spare instance HA (standby). 61 Configuring active-active load balancer support. 63 Setting up primary instance HA (active-active) 66 Enabling queries ater active-active failover. er Restoring data after an instance fails, 68 Content Server full-text objects and initialization files. 69 Upgrading xPlore .. Upgrading xPlore n Upgrading xPlore HA (active-active) 4 Migrating from FAST to xPlore. Immediately replacing FAST with xPlore 15 Migrating indexes (reindexing) 76 Migrating large environments 7 Large migrations best practices. 7 Supporting a large number of ACLS. 7 Adding temporary instances. 78 Ensuring CPU, disk, and /0 capacity 78 Migrating data'to specific collections 78 Migrating content based on document age. 79 Custom routing 20 Targeting queries to specific collections. 80 Troubleshooting migration at Troubleshooting Installation. Debugging installation 83 Fixing indexserverconfig upgrade issues. 84 Verifying Content Server support in 6.6 85 Host name contains an underscore 86 Instance name falls 86 Cannot create a secondary instance. 86 EMC Documentum xPlore Vetsion 1.3 Installation GuideTable of Contents Cannot start a secondary instance. 86 CPS instance does not start 87 xPlore administrator problems with IES and IE9 browsers 87 Index agent errors 87 Global registry not available. 87 Index agent configuration fails. 88 Index agent version and OS. 88 Linux errors. 88 Chapter 11. Uninstalling xPlore. Uninstalling xPlore. ot Chapter 12 References. xPlore ports 93 HTTP status codes. 93 [EMC Documentum xPlore Version 1.3 Instaation Guide 5Preface ‘This document provides instructions for installing, upgrading, and uninstalling Documentum »Plore Intended audience ‘The reader Imows the business nature of the indexing data and the network and operating system for xPlore Revision history ‘The following changes have been made to this document Revision Date Description November 2012 Initiel publication Additional documentation ‘This guide provides installation information. For information on planning, administration and development, refer to + EMC Documentwn xPlore Administration and Development Guide + EMC Documentwn xPlore Release Notes + EMC Documentwn Environment and System Requirements Guide For additional information on Content Server installation and Documentum search client applications, refer to: + EMC Documentwn Content Server Installation Guide + EMC Documentwn Search Development Guide Path conventions in this document ‘The following conventions are used in this guide + $DOCUMENTUM (Linux) and %DOCUMENTUM% (Windows): Environment variables that specify the Documentum Content Server root installation directory. Windows default: C:\Documentun. Linus default: none + $DM_HOME (Linus) and %DM_HOME% (Windows): Environment variables that specify the Content Server installation directory and the product version subdirectory. Windows default: C:'\Documentum\productlversion_mumber, where verston_mumber is the version of Content Server. Linux default: $DOCUMENTUMIour_patih, where your_path is the path selected during installation, for example, prodtict/7 0 + xplore_home: Installation directory for xPlore [EMC Documentum xPlore Version 1.3 Instaation Guide 7Chapter 1 Installation Overview This chapter contains the following topics + Installation and upgrade workflow Installation and upgrade workflow ‘The following diagram shows the basic workflow for installing and upgrading »Plore. See related sections in this documentation for detailed instructions, eoeans erect Peery frees CUE Uy Teele gcc Cette files to the same location ore toy ee ay xPlore instances Pore ented Ce Cuey Coe cue acd ere instances Oe ey Install xPlore in a test environment first. Performance-test your »Plore installation with expected customer workloads, queries, and document profiles before moving your installation to a production environment. General installation steps: EMC Documentum xPlore Version 4.3 Installation GuideInstallation Overview 1. Run xPlore Installer (setup) to install »Plore program files 2. Run 2Plore Configuration Program (dsearchconfig) a. Create the primary xPlore instance ‘b. Optionally, create secondary xPlore instances. 3. Run Index Agent Configuration Program (configindexagent) to create index agents. General upgrade steps: 1. Run xPlore Installer (setup) and install the new 2Plore program files into the same location as the existing version. 2. Run 2Plore Configuration Program (dsearchconfig) to upgrade the primary xPlore instance first, then any existing secondary instances. 3. Run Index Agent Configuration Program (configindexagent) to delete all existing index agent, and then create new ones. 10 EMC Documentum xPlote Version 1.3 Installation GuideChapter 2 Planning the xPlore Environment This chapter contains the following topics + How xPlore works + Planning your installation + Determining your requirements + xPlore sizing calculation + Sizing CPU + Sizing memory + Sizing storage + Network and virtual machines + Hardware tradeoffs How xPlore works Documentum xPlore (#Plore) is a multi-instance, scalable, high-performance, full-text index server that can be configured for high availability and disaster recovery. For detailed information, see About high availability and disaster recovery, page 19. Installation locations ‘The xPlore index service and search service are installed as a WAR file to a JBoss application server that is included in the xPlore installer. The administration console and online help are installed as war files in the same JBoss application server The index is stored in the directory xplore_home/data/repository_name/defeultucene-index or another configured location, such as a SAN or NAS. xPlore instances An xPlore instance is one installation of the Plore WAR file in an application server container. The first instance that is installed is designated as the primary instance. You can have multiple instances on the same host (vertical scaling), although it is more common to have one xPlore instance per host (horizontal scaling). An instance can be configured to enable one or more of the following features + Content processing service (CPS) + Indexing service EMC Documentum xPlore Version 4.3 Installation Guide "Planning the xPlore Environment Search service + Administration console (includes analytics and instance and data management services) Spare -A spare instance can be manually activated to talce over fora disabled instance. Domains A domain isa separate, independent, logical grouping of collections within an xPlore installation. The Documentum index agent creates a domain and sends all indexed content to that domain in xPlore Planning your installation Determine your sizing and performance requirements Determine your hardware requirements including 2Plore server CPUs and memory, disk /O, storage, network, and virtual machine requirements. Determine your backup and recovery, high availability, and disaster recovery requirements. See Planning for high availability, page 21 and Planning for disaster recovery, page 24 Determining your requirements To size your full-text indexing system, use the xPlore sizing tool on EMC Community Network Determine the following requirements before using the sizing tool, 12 Indexing performance — Indexing rate. The document indexing rate or document throughput is the rate at which new objects are added to the system or submitted for indexing As the document complexity (for example, more words in a document or more words in a spreadsheet) increases, then the cost of creating the index increases. (More CPU is consumed and more disk /Os are generated ) Higher throughput requirements result in higher processing costs ‘You can increase throughput by adding more instances and by increasing the speed and capacity of the machines feeding the data source to xPlore — Indexing-to-query latency. Indexing-to-query latency is the time from when an object is saved to ‘when the object is searchable. If you require a low indexing-to-query latency, an object must ‘become searchable as fast as possible. Typically, this requirement is for environments in which. ‘many searches are performed and many objects are created and edited. Ina fixed data scenario, large quantities of unchanging business data are stored but rarely or never modified, and a longer latency period is acceptable A multi-instance installation provides a faster save-to-search time than a single instance. Query performance. Query performance is the speed at which results are retumed for a query. Query performance depends on the following factors: — Number of users — Rate at which the users issue queries — Complexity of the queries, which impact disk /O and CPU. A query is complex with wildcards, ‘many terms in the query, and unselective terms. EMC Documentum xPlote Version 1.3 Installation GuidePlanning the xPlore Environment — Number of results — Permissions of the user (few permissions cause slower queries) ~ Size of the index — Hardware capacity + Index size. The size ofan index depends on the size of the largest documents indexed and the amount of indexable content in the documents. A large file can contain a small amount of indexable text and a large amount of unindexable content, such as graphics. Asan index gets larger, the time it takes to merge the indexes grows because of the increase in disk VO and CPU demand. Use a multi-instance installation for large documents, to spread the index over multiple hosts To estimate your index size, you can index a subset of your most typical Content Server documents using an »Plore instance. Use the size of the sample index to estimate the size of the final index that would result from indexing all of your documents. For best results, use the xPlore sizing tool on EMC Community Network xPlore sizing calculation EMC provides tools and guides for system sizing to help you determine the appropriate number of instances and system capabilities that you require. EMC Documentum Professional Services (or qualified third-party integrators) can assist you in reviewing your needs and completing the »Plore sizing tool input (EMC Community Network), Note: Indexing stops if the indexing host nuns out of disk space or temp space Installations can range from simple to complex For example, a simple installation has single-term. searches and out-of-the-box latency for new documents that are medium-to-small size. A more complex installation has Boolean or wildcard searches, large result sets, low latency requirements, and large documents ‘An installation on under powered machines or not appropriately configured can result in poor performance. Indexing fails ifa disk fills up. To appropriately size your system, choose one of the following (the first is recommended) + Use the xPlore sizing tool available on EMC Community Network + Use the same hardware requirements that you established for your current FAST full-text indexing system. Budget for staff to perform validation testing (including performance) on production data with real user scenarios. Always test xPlore in multi-user mode before installing it for production use. Sizing CPU Plore can have periods of CPU-intensive activity To achieve higher indexing rates, some installations need additional CPUs for improved query response. Ina 64-bit operating system, 4-6GB of memory for each xPlore instance can be easily reached. However, CPU usually reaches maximum usage before the system runs out of memory. 64-bit JVMs EMC Documentum xPlore Version 4.3 Installation Guide 13Planning the xPlore Environment have these performance advantages over 32-bit JVMs: More memory to cache index structures for faster query access; Additional memory to index large documents, Service higher query and indexing rates, Each xPlore instance can use more than 2 GB of memory. Sizing memory Full-text indexing is a memory intensive operation, Itis best to have the most memory you can afford on an xPlore instance. xPlore instance memory usage is tuned out-of-the-box. If you have large documents, millions of documents, or a high simultaneous query rate, you can tune your xPlore instance memory usage ‘The following diagram illustrates the relationship between xPlore instance and operating system. ‘memory usage Total memory usage is divided between an xPlore instance and the operating system, Each xPlore instance has its own fixed size memory. This memory contains a Lucene cache and ‘working memory, an xDB page server cache, and optional virtual machine working memory. The operating system file buffer cache reserves memory space for temporary files. This cache is dynamic and expands to fill any remaining memory without interfering with xPlore processes. Figure 1 xPlore memory usage ello ate aT) ra xPlore | caches & | xDB buffer |B eauung working | caches | working eeics buffer cache ey pet (dynamically sized) xPlore instance (fixed size) Memory Plore out-of-box parameters are adequate for most applications. For further memory tuning, the following table provides a description of xPlore memory pools and recommendations for tuning. Configuration of these caches is described in the ZMC Documentum xPlore Administration amd Development Guide, “System sizing and tuning” section, Table 2__xPlore memory pools and tuning ‘Memory Pool Description Recommendations xPlore caches For example, temporary result None cache is used to buffer results to be sent to users 14 EMC Documentum xPlote Version 1.3 Installation GuidePlanning the xPlore Environment Memory Pool Description Recommendations Lucene caches and working Used in processing queries. Uses Higher query rates end langer memory application server JVM. numbers of documents require more memory. DB page server cache Used to store XIML fle blocks Increase size forhigher query rates. (ingest and query) Operating system file buffer cache Temporary file. Although this Its largest impact is on Lucene cache wall typically expand to fill (inverted index) performance any remaining memory, it will not interfere with xPlore processes, Other VM working memory Used to manage intemal virtual None machine operations Sizing storage + Multi-instance installations must store their data on the network in a Storage Area Network (SAN) or Network Attached Storage (NAS) device. At minimum, storage must be on a network share. ‘You can compensate for lower performing NAS by implementing a high performance networks throughout your 2Plore installation + When SAN remote mirrors are used, only one xPlore instance can run on the data (both indexing and querying at the same time). Two »Plore instances from two separate installations (#Plore federations), one local and one remote, cannot operate on the same index. + Determine the type of disk system using these constraints and recommendations: — Keep disk /0 latency as low as possible — Ifyour indexing storage requirements are small, use direct attached disks. For large indexes, use SAN devices and dist arrays for storing indexes and tokens. — Usea dist array in production environments, rather than a single disk, because a single disk creates a bottleneck for indexing + SANs typically represent virtualized storage. xPlore can be one of several applications attempting to leverage the virtual storage. As with virtual machines, ensure that the virtual storage is appropriately sized for all applications that access it ‘The following table describes storages areas that are used by »Plore EMC Documentum xPlore Version 4.3 Installation Guide 15Planning the xPlore Environment Table 3 Storage areas Files Description Indexing Queries DB segments Stores DFTXML, Lucene For batch XML files, the Random access retrieval indexes, and other xDB next fiee disk block is for specific elements and collections (metrics, consumed. summary auditing, ACLs, and groups) xDB federation logs* Stores transaction ‘Transactions axe logged When snapshot information before data is written to information is required. disk Lucene indexes (in xDB segments) Query Index information Inverted index lookups lockup and retrieval of updated through inserts and facet and security facets and security and merges retrieval Index agent content Temporarily stores ‘Temporarily stores NIA staging area content during indexing content during indexing process process * Old xDB federation logs are purged at each backup. For incremental backups, set the keep-Logtiles parameter to true and allocate more dist space Network and virtual machines Network + When using NAS, ensure that your network bandwidth is high and use jumbo frame support + Use Gigabit Ethemet for iSCSI in production environments. Do not use anything less than 1 GB (for example, 100BaseT or 10 GB Ethemet are good + Use high-capacity SAN Fibre Channel interconnects to the disk arrays. Virtual machines Because many applications are not busy all the time, you can install many of them on the same machine using virtual machines. In addition, virtual machines offer easier installation and recovery options than native installations. Potential drawbacks with virtual machines + Plore is very CPU- and disk I/O-intensive, and it can consume more resources than the average virtual machine + Other applications ona virtual machine host compete with xPlore and can cause xPlore performance to fiuctuate + The performance of virtual CPUs is not as good as the performance of native CPUs for xPlore CPU- intensive worldoads. 16 EMC Documentum xPlote Version 1.3 Installation GuidePlanning the xPlore Environment + The maximum number of CPUs supported by a virtual machine product is lower than a hardware environment. For example, VMWare ESX 35 supports up to eight cores. If you have fewer CPUs per operating system require more xPlore instances to perform the same work as in a native environment. Hardware tradeoffs ‘The following table shows how your environment and xPlore configuration options affect index size, indexing performance, and query performance. The table suggests which resources you can increase or improve to alleviate the reduction in service. Table 4 Hardware budgi erformance and size Environment or Storage Indexing Query Resources used configuration performance performance Documentsize 9 + + x Diskspace, memory Document + + x Disk space, CPU, complexity memory, network ‘bandwidth Number of ACLs + + + Diskspace, memory and groups Querycomplexity NIA x + CPU, memory, Disk 10 Total mumber of = + + + Disk space, disk documents 10, CPU, Ingestion rate NA + NIA Disk 110, CPU Query rate NA NIA + Disk 1/0, memory, CPU, more instances Number of NA + + collections Lemmatization = + + Disk space Diacritics + + + + + Hasa significant impact on index size, indexing performance, or query performance + x Does not havea significant impact. EMC Documentum xPlore Version 4.3 Installation Guide 7Chapter 3 Planning High Availability and Disaster Recovery This chapter contains the following topics + About high availability and disaster recovery + Planning for high availability + Two primary instances HA (active-active) process + Planning for disaster recovery + Disaster recovery in a Documentum environment About high availability and disaster recovery Business continuity is the umbrella term that covers all efforts to Keep critical data and applications running despite any type of interruption, Interruptions can be both planned and unplanned. Planned interruptions include regular maintenance or upgrades. Unplanned interruptions could include hardware or software failures, data corruption, natural or man-made disasters, viruses, or human error Enterprises can use the following solutions to meet business continuity requirements: + Backup and recovery is an essential part of both basic data disaster recovery and operational recovery strategies. Operational recovery is the ability to recover from errors that can occur on a regular basis but are not catastrophic, for example, data corruption or accidentally deleted files. In data disaster recovery, data backups are replicated but no target hardware/software installation exists, The installation hardware and software must be reinstalled, and then the data backups are reapplied. For information on the types of backup and restore, see EMC Documentum xPiore Administration and Development Guide. + High availability provides a high level of business continuity as well as high performance through, load balancing for mission critical systems. In a high availability configuration, a secondary system is maintained as an exact copy to which the primary system can fail over. + Disaster recovery responds to catastrophic failures. When your installation fails, you recover it by restoring it to a previously consistent state (that is, a particular point in time) from your backups. To restore your installation to a consistent state, use backup coordinator software like EMC® Networke®. This software ensures that all of your components and data source backups are synchronized. For more information about bacicup and recovery with Content Server, see the white paper Backup and Recovery of EMC Documentwn Content Server using the NetWorker Module for Documentun on EMC Online Support (https-//support.eme. com), Your business users and information technology department together can provide direction for your high availability and disaster recovery planning For instance, your information technology department can standardize on Cisco or Microsoft clustering. EMC Documentum xPlore Version 4.3 Installation Guide 19Planning High Availabilty and Disaster Recovery Recovery point and recovery time (RPO and RTO) RPO: Decide how much data loss you are willing to incur. Use this decision to calculate how often you need to perform backups. Backups should be performed at fixed intervals. The length of time between backups is called the Recovery Point Objective; that is, the masimum amount of data that you are willing to Lose. When your RPO is in units of minutes or hours, then you will probably need to make frequent incremental backups or have a high availability configuration B ecause the amount of data lost on the Content Server is reflected in the corresponding 2Plore indexes, the xPlore RPO is always the same as the Content Server RPO. Because any index data loss (because ofa failure or data corruption) can always be recovered by refeeding documents to xPlore, any backup of or high availability for xPlore shortens RTO (not the RPO) RTO: Decide how long you're willing to wait until the data is completely restored and business applications become available. The time it takes to completely restore data and for business applications to become available is called the Recovery Time Objective. Your RTO can be different from your RPO. For example, you might need an extremely short RPO to minimize potential data Joss, but you can have a long RTO in which you can tolerate 1 to 24 hours to recover data and get your business applications back online ‘After you determine your RPO and RTO, you can determine how much time you actually have to perform your backups (backup window). The backup window determines the type and level of your backups. For example, if you have a system that requires 24-hour, 7-days-a-week, 365-days-a-year availability, then there is no offiine backup window. So, you would have to perform an online backup (@lso known as a hot backup) in which the system is not taken offiine To determine your backup and recovery, high availability, and multi-site disaster recovery plan, you first determine the degree to which your xPlore installation is mission enitical-that is, if your organization considers your 2Plore installation to be down for a short period of time (1 hour for example) as beinga severe outage. Your Documentum repository could be mission cntical but xPlore full-text search functionality might not be. In general, the shorter your RTO and smaller your RPO, the ‘more mission critical your xPlore installation is Ifyou havea short RTO and small RPO, your mission critical installation be designed to recover from any kind of planned or unplanned interruption within the specified RTO and RPO. In general, a ‘mission critical configuration wall cost more (in terms of both cost and staffing) than a non-mission configuration because it will require more hardware, planning, and testing. Mission critical systems should not rely solely on backups: You should also consider investing in high availability solutions Sometimes a partial recovery that makes your most critical indexes available for querying or indexing new content may be suitable for your xPlore installation. Some examples of an acceptable partial recovery are + Tomake the previous 2 days of indexed data available for querying within 1 hour but make all of your indexes available within 24 hours + To make querying of all indexes available within 1 hour but resume indexing within 24 hours, Note: Partial refeeds and indexing are normally required to restore xPlore to a point-in-time that is synchronized with Documentum Content Server. There is usually a lag time between new content that was added to a Documentum repository and that content being crawled and indexed by xPlore 20 EMC Documentum xPlote Version 1.3 Installation GuidePlanning High Availabilty and Disaster Recovery Planning for high availability Ina high-availability (HA) deployment, ifan xPlore instance fails, then the secondary instance can replace it; CPS, indeang, and search operations continue on the replacement instance. An HA deployment provides the following benefits: + Increased query availability because support is provided for failover + Redundancy if host fails, If you require xPlore high availability, you wall typically also require high availability for your Content Server. Therefore, you should coordinate your Content Server and xPlore high availability deployments, Although many Content Server and xPlore high availability configurations are possible, a single repository served by two Content Servers is typical and can be used as a building block for more complex deployments. See the EMC Documentum Content Server Installation Guide for ‘more information about the different high-availability deployments and installation instructions for Content Server Plore supports these kinds of high availability deployments + Spare instance (standby) - A spare instance is a warm instance that can be manually brought online ‘when a running node fails. When the spare instance is started, it rebuilds the index from the previous instance's transaction log, which must be accessible in a shared directory (the data and index directories must also be accessible to the spare instance). This functionality is included with xPlore + Active Passive - Active Passive isa failover technology that enables an application and its data to automatically start up on altemative (passive) hardware after the primary application fails. Microsoft, Redhat, and Veritas cluster servers are examples of active-passive clusters. Although failover is automated, active-passive cluster requires a heavy investment in duplicate servers but not duplicate disk storage + Active-Active - Active-Active HA consists of two complete, running xPlore deployments. Two different index agents send the same content to each deployment. Queries are directed to one of the two active deployments or load balanced across both of them In Active-Active HA, you must manage two deployments. Thisis the most expensive option because it requires duplicate servers and disk storage. You could have as many as four times the mumber of hosts as your initial xPlore deployment or set up high availability in multiple data centers. You can also deploy a hybrid model: Active-Passive or Active-Active clusters for the indexes only For an example of HA using Cisco load balancer, see EMC Community Network: xPlore Documentation, High availability and disaster recovery cost ‘The cost of high availability and disaster recovery is determined as follows + Number of servers + Amount of disk storage — Storage for backups. For example, you could have 3 days of backups that require 6 times the amounts of database storage. Cost can be lowered if backups are not needed for the indexes: you could refeed, which would increase the RTO — Duplicate diste storage and /O for active-active high availability. Active-passive usually uses duplicate servers but shared diste storage EMC Documentum xPlore Version 4.3 Installation Guide aPlanning High Availabilty and Disaster Recovery + Cost of automation: Higher with an automatic failover system like Microsoft Failover Clustering than with manual failover (xPlore spare instance) ‘The following table illustrates how the cost increases as recovery time increases and automation becomes available Table 5 HA Strategy Comparison HA Strategy Cost Recovery Time | Failover Notes Automation Spare instance |S Slowest None Not as automated bout very inexpensive ActivePassive | $8 ‘Medium Yes Highly automated, standard and well-known technology. ‘Active-Active $85 Fastest Yes Has been in place the longest time Requires more time to administer the duplicate system, Two primary instances HA (active-active) process ‘The following diagram illustrates multi-instance HA using two xPlore primary instances on separate hosts. Note: The active-active HA configuration maintains indexing in case ofa failover. You must manually restart query services in the standby Content Server 2 EMC Documentum xPlote Version 1.3 Installation GuidePlanning High Availabilty and Disaster Recovery Figure 2 Dual xPlore primary instances HA before failover Before xPiore Failover Repository ‘Metadata | Content Fles (DB Cluster) | (WASISAN) Hoes} Hos? co CS FLEE*Y J index 4-198? J | index i t \ \ mm 7} Indo Index a ¥ Pin iinsz Les[ Sere] T Sel vod oad 7] on = Zz = 4 Conta Fen Ss lc = Te Se + Host: Primary Content Server host — Content Server (CS1) — Connection broker (CB 1) — Index agent! (1A1) + Host2: Secondary Content Server host — Content Server (CS2) — Connection broker (CB 2) — Index agent? (1A2) + Host3: First primary xPlore host — Index agent (1A1) that connects to CS1 ~ Primary instancel (Pri_Inst!) ~ Secondary instancel (Sec_Instl) + Hosta: Second primary xPlore host — Index agent (1A2) that connects to CS2 ~ Primary instance? (Pri_Inst2) ~ Secondary instance? (Sec_Inst2) EMC Documentum xPlore Version 4.3 Installation Guide 23Planning High Availabilty and Disaster Recovery + Hosts: First xPlore primary instance data, log, and configuration files (Contig_Files), which must be writable from both the running instances as well as the spare instances + NAS/SAN Full-text indexes for the first primary xPlore host + Hosté: Second »Plore primary instance data, log, and configuration files (Config Files), which must be writable from both the running instances as well as the spare instances + MAS/SAN2: Full-text indexes for the second primary xPlore host Figure 3 Two xPlore primary instances HA after failover ‘for xPlow Failover Repository \ Toisas] Cone Fos (Dechstey | WASISAN) NS Pe cet) | index 082} | index i ‘ { ( mH 7} Indo WZ LEM —¥ Wena! 1 Set || [Pt rsa nen Sees Vel EP Meek eas Fn ; . a-a/t p--e Z + {ssi Ur ‘stows | | [eamecron indore | "ross 1 Lonasisan’ | | Hoste NASSANZ Planning for disaster recovery Ina multi-site disaster recovery (DR) configuration, the primary installation—the set of machines, installed software, and data that comprise the system—is replicated to a remote site The remote site ‘must be located far enough away from the primary installation so that the same catastrophe will not damage it as well Furthermore, when the remote site is brought online, it must be configured like the primary site—The remote instance must mirror the primary instance with the same instance name, port mumber, directory structure, and usemame/password combination ‘The remote site does not have to recover to the exact same state as the primary site. For example, when a series of disk blocks are written on the primary site, they are sent to the remote site, but some of the blocks might be received out of order, some intervening blocks might be not be received, or the 28 EMC Documentum xPlote Version 1.3 Installation GuidePlanning High Availabilty and Disaster Recovery disk could be corrupt atthe remote site. In order to bring the remote system online, all the received blocks must be contiguous and in the same order as they were written on the primary site That is, bringing the remote system online is equivalent to auto-recovering xPlore at the primary site Although the remote site can nun on reduced capacity compared to the primary site, EMC strongly recommends, as a best practice, that the remote site run on hardware and software that has the same capacity as the primary one. For example, EMC Recoverpoint is a multi-site disaster recovery product For information about EMC disaster recovery technologies, see http: /Aarwnw eme. com/solutions/business-need/business-continuity-availability/affordable-disaster-recovery htm, To plan for disaster recovery, EMC strongly recommends performing regular backups, because bringing the remote system online is not guaranteed to succeed all the time (for example, the remote disk might have failed or data replication might have been unknowingly misconfigured). However, if no backup exists, the content can be re-fed from the original source system (for example, the Documentum repository) Disaster recovery in a Documentum environment When your full-text index installation fails, you recover it by restoring it to a previously consistent state (that is, a particular point in time) from your backups. Restoring to a particular point in time is also known as a point-in-time recovery. Content Server and indexing backup coordination If the associated Content Server has also failed, then you must coordinate the full-text index recovery with the Content Server recovery. Coordinated backups have the following characteristics 1. Content Server is usually deployed in a disaster recovery configuration: All data stores for the database, content, and full-text indexes are replicated from the primary site to a disaster recovery site. The disaster recovery site is backed up using a BCV-liee (snapshot) technology. 2. Data replication from the primary site to the disaster recovery site completes within the RPO ‘The amount of data being replicated differs between the database, content, and full-text indexes, so the overall amount of time it takes for each one to be completely replicated also differs Content generally takes the longest to replicate, and the database takes the shortest to replicate. Consequently, the replication time for content is the maximum replication time for the entire deployment. The data replication rate depends on the network bandwidth and the oncoming load as well asa host of other environment factors. To set the RPO correctly, determine the maximum, ‘amount of time it takes for data (new and modified) to be completely replicated from the primary site to the disaster recovery site. 3. The full-text index baclup interval is at least two times that of the database backup interval. In addition, you must wait for at least two times the entire deployment's maximum replication time before backing up the full-text index. Note: Since the full-text indexes can be restored point-in-time from the database and content, the full-text indexes could be backed up as infrequently as once a week whereas the database EMC Documentum xPlore Version 4.3 Installation Guide 25Planning High Availabilty and Disaster Recovery and content could be backed up many times a day. Full-text index backups must be performed frequently enough to meet the RTO. Thatis, the full-text index point-in-time recovery is determined by the amount of data received during the full-text index backup interval. Figure 4 Backup for object consistency Documentum Object Related Objects Content Metadata Ful-text Indexes Backup MUST be Can be re-indexed (but could coordinated take weeks for areally large index) In the following example, the recovery point objective is 8 hours. After replication of the database (content and metadata) and full-text indexes (hours 0-2), a point of failure occurs at 9.5 hours. The database is restored to the recent backup (0.5 hours before failure). The content is restored to the recent backup (7.5 hours before failure). The index is restored to 7.75 hours before failure. The Content Server administrator handles the content synchronization between database and metadata. For »Plore, you must refeed all content that was indexed since the backup was completed. Run ftmtegrity using a start and stop window to get the object IDs 6 EMC Documentum xPlote Version 1.3 Installation GuidePlanning High Availabilty and Disaster Recovery Figure5 RPO for disaster recovery site ate fe a me oc Ot | os oa" Wl op me c acup “s it r sop oom 1 Sa looren* | Sup iat) ie) I 1 ror hae aa fal Be ey ey a Note: + Italics: The operation is optional (Content Backup), + The time scale and line lengths of the operations indicate only a rough order of magnitude-that is, they are not typical nor do they represent a recommendation Restoring the xPlore indexes See the chapter Backup and Restore in EMC Documentum xPlore Administration and Development guide EMC Documentum xPlore Version 4.3 Installation Guide arChapter 4 Installing xPlore This chapter contains the following topics + Before you install + Enabling SSL to Content Server 6.7 or higher + Enabling SSL to xPlore administrator only + Installing xPlore with Content Server 6.7 SPx and 7.0 + Installing xPlore with Content Server 6.6 + Validating installation + Installing a secondary or spare instance + Adding CPS Instances + Configuring CPS dedicated to indexing or search + Deleting an xPlore or CPS instance Before you install + Install on the Content Server the latest cumulative service pack or hotfix, if any. Configure at least one Documentum repository and one connection broker. Note: If you do not install the latest hotfix for Content Server before you install xPlore, xPlore does not work properly. + Windows 2008 Server. For support of xDB command line utilities, install the Microsoft Visual. C++ redistribution package. + Limc Installation of xPlore in console mode is not supported. Set up your environment to support GUI installation. Installation on 64-bit Linux requires the following RPM packages before xPlore installation: ~ libXp.. i386 ~ libXi. 1386 — libXtst.. 1386 — libXt..i386 + All instances must be installed on the same operating system (Windows or Linus) + Toavoid permission-related errors, the same installation owmer (same username, same password) ‘must install all xPlore instances, index agents, and CPS instances. On Windows, the installation owner must also be in the same Windows domain. EMC Documentum xPlore Version 4.3 Installation Guide 29Instaling xPlore + Installation owmers of Content Server and xPlore (including CPS) must be identical if you want to ‘map content storage locations for indexing instead of streaming the file to the index agent temporary content location. (By default, the index agent streams the content ) + Install a primary instance before installing secondary instances. Secondary instances are optional. Enabling SSL to Content Server 6.7 or higher SSL is supported for connections from xPlore to Content Server 6.7 and higher Use the following procedure to enable SSL connections to »Plore Administrator and between xPlore and the Content Server. The xPlore host must be on the Windows platform. Prerequisites: xPlore and the index agent must be configured with fully qualified host names including the domain extension 1. Createa certificate keystore named my keystore using the Java keytool utility ‘You must first set the JAVA_HOME environment variable, for example xPlore\javaé4\1.6.0_31\bin. + Linwe $JAVA_HOME/bin/keytool -genkey -alias xplore -keyalg RSA “keystore ny.keystore + Windows: ‘AJAVA_HOME* \bin\keytool.exe -genkey -alias xplore -keyalg RSA -keystore my.keystore When the keytool utility asks for a Common Name (CN) value (first and last name), enter in the fully qualified host name of the application server as referenced by your browser or web service consumer. Use changeit as the password and tomcat as the alias. The keystore named my keystore is created and stored in the user home directory by default (user home system property) 2. Export the certificate with the following command: ‘JAVA_HOME* \bin\keytool.exe -export -alias xplore -Keystore my. “file mycerts.cer storepass changeit Output is Certificate stored in file 3. Import the certificate with the following command: import -trustcacerts -alias xplore ‘SJAVA_HOME* \bin\keytool. eystore -file mycerts.cer -storepass changeit Where is xplore_homeljava64/1.6.0_31/jrefib/security/cacerts 4, Enable an HTTPS port for the xPlore instance in JBoss. a Open servers! in xplore_home/jboss5. |O/serveriD ctmServer_PrimaryDsearch/deployh bossweb sar/. 'b. Locate the element 30 EMC Documentum xPlote Version 1.3 Installation GuideInstaling xPore Uncomment the element. Change the value of the keystoreFile attribute to match the path to your keystore file, for example, xplore_homel{boss5.1 O/server/DctmServer_PrimaryDsearch/confimy keystore €. Change the value of keystorePass to changett £ Change the port to your desired HTTPS port g Add the attribute keyAlias="xplore” For example: true” keyAlias="xplore” 5. Instruct JBoss to use the certificate and configure a new secure port. a Modify the file bindings-jboss-beans zm! located in xplore_home/jboss5. .O/server/D ctmServer_PrimaryDsearch/conffbindingservice beans/ME TA -INF ‘b. Locate the bean element: HttpsConnector Change the port to your desired HTTPS port For example jboss. web: service=WebServer HttpsConnector S302 JBoss Web HTTPS connector socket d. Locates xsl-when > element where and change the port 8443 to your desired HTTPS port For example €. Locate the element where the variable name is portHttps and change the select value to the configured HTTPS port. For example 6. Copy the file my keystore to xplore_homeljboss5. 1 O/server/DctmServer_PrimaryDsearch/cont 7. (Optional) To disable HTTP, comment out the following connector entry in the file xplore_homeljbossS. 1 O/server/DetmServer_PrimaryDsearch/deploy/jboss-web sat! server aml or add the redirect port to match your new HTTPS port 8. Update the watchdog heartbeat URL for the primary instance to show the HTTPS port To do so, edit the configuration file dsearch-watchdog-config xml. This file is located in xplore_homelwatchdog/config For example name="dsearch" PrimaryDsearcl 9. Change the dm_ftengine_config object in the Content Server to set HTTPS as the value of dsearch_grserver_protocol. Modify as well dsearch_qrserver_port and dsearch_config_port parameters a. Get the dm_ftengine_config object API>retrieve,c,dm_ftengine_config ‘b, Find the index value of dsearch_grserver_protocol (for example, 6), API>dump,c,1 ¢. Set the values for dsearch_grserver_protocol, dsearch_grserver_port, and dsearch_config_port ‘This example uses the index value retumed by the dump command: APDset,c,1,param_value[é] SETDETTPS 10. Stop all »Plore instances and edit indexserverconfig zanl in xplore_home/config Find the node element and change the value of the url attribute to match the HTTPS-enabled port. The host name must be the same host name you used to configure the index agent connection to »Plore. For eample Content Processing Service in the tree and then click Add. Enter the URL to the remote instance using the following syntax: http: //hostname:port/services To test the remote CPS service using the WSDL testing page, open a browser and specify the following syntax: hetp: //hostname:port/services/cps/ContentProcessingService?usdL ‘A page showing the CPS service configuration is displayed. Configuring CPS dedicated to indexing or search By default, every xPlore instance has a local CPS service. All CPS services receive processing requests on a round-robin basis. For high-volume environments with multiple xPlore instances, you can configure one or more CPS services to handle all processing requests for a specific xPlore instance. This reduces network overhead. You can configure additional CPS instances that are dedicated to indexing, for high ingestion requirements, or dedicated to search, for heavy search usage 1. Stop all xPlore instances. 2. Edit indexserverconfig xml in an XML editor. In the following example, two instances are shown inode elements. The content-processing-services element specifies the two CPS instances that are available to all xPlore instances. Set the value of the usage attribute to all, index, or search. appserver-instance-nane="PrimaryDsearch” ...primaryNode="true"... ://host1:8300/dsearch/"... hostname="host1” name="PrimaryDsearch"> 2 :20000/services" /> search" 21000/services" /> 3. Create a content-processing-services element under each node element, after the node/properties element, like the following 2 EMC Documentum xPlote Version 1.3 Installation GuideInstaling xPore ties> 4. Move the shared CPS instances to each node, where they become dedicated CPS instances for that node. Place their definitions within the content-processing.-services element that you created. For eample ‘properties> ‘http: //host1: 20000/services" /> ‘Make sure that you have a remaining global CPS instance after the last node element. (You have ‘moved one or more of the content-processing-service elements under a node.) For example Deleting an xPlore or CPS instance ‘rlp" context-characters: ()-#eeltzegtz/\E1()"> all” url="1ocal"/> Note: Do not delete the primary instance before you have deleted all secondary instances. The primary instance and the instance to be deleted must be running 1. Onthe machine on which the instance is installed, run xplore_home\setup\dsearch\configdsearch bat (Windows) or xpiove_home/setup/dsearch/configdsearch sh (Linux) and follow the instructions xplore_home is the xPlore root directory. On the Select Configuration Mode page, select the Delete Existing Server Instance option On the Delete Instance page, in the Instance field, select the name of the instance to delete On the Select instance binding for internal collections page, choose how to manage the instance intemal system data: + Delete - (Default) Deletes all of the instance internal collections EMC Documentum xPlore Version 4.3 Installation Guide 6Instaling xPlore + Change Binding to Other Instance - Select another instance to which to bind all of the instance intemal collections. That other instance must be running Make sure that you do not bind too ‘many intemal collections to a single instance. + Ignore (you have already cleaned it up) -No action is taken. Choose this option only if you have already deleted or bound this instance intemal collections to another instance, otherwise, an error occurs 44 EMC Documentum xPlote Version 1.3 Installation GuideChapter 5 Installing and Configuring Index Agents This chapter contains the following topics + Preparing index agent installation * Configuring an index agent + Starting the index agent + Stopping the index agent + Silent index agent startup + Deleting an index agent Preparing index agent installation Run the xPlore installer to install an index agent for each repository, Index agent requirements + Toavoid permission-related errors, the same installation owner (same username, same password, and same Windows domain) must install all xPlore instances, index agents, and CPS instances. + All instances are all installed on the same operating system (Windows or Linux) + Install »Plore ifit is not already installed on the index agent host. See Installing xPlore with Content Server 6.6, page 36 of Installing xPlore with Content Server 6.7 SPx and 7.0, page 34 + Verify that the xPlore primary instance, Documentum Content Server, and Documentum global registry are started. After successfully starting up Content Server and the connection broker, wait several minutes (typically 15 minutes) before configuring a Documentum index agent. + Each Documentum Content Server requires its own index agent Specify an alternate location for index agent storage You can create a domain and select its storage location before you configure the Documentum index agent. You can then select the alternate storage location during index agent configuration. The domain name must match that of the Content Server repository. EMC Documentum xPlore Version 4.3 Installation Guide 45Installing and Configuring Index Agents Configuring an index agent Run the xPlore installer to install an index agent for each repository. Then configure index agents using the following instructions, Requirements: + Toavoid permission-related errors, the same installation owner (same username, same password, and same Windows domain) must install all instances. + All instances are all installed on the same operating system (Windows or Linux) + Install »Plore ifit is not already installed on the index agent host. See Installing xPlore with Content Server 6.6, page 36 of Installing xPlore with Content Server 6.7 SPx and 7.0, page 34 + Verify that the xPlore primary instance, Documentum Content Server, and Documentum global registry are started. After successfully starting up Content Server and the connection broker, wait several minutes (typically 15 minutes) before configuring a Documentum index agent. + Each Documentum Content Server requires its own index agent. For information on using the index agent Ul after installation, refer to HMC’ Documentum xPlore Administration and Development Guide 1. Run xplore_home\setup\indexagent\configindexagent bat (Windows) or xplore_home/setup/indexagent/configindexagent sh (Linu), 2. (Windows only) Get Installation Owner Password: Specify the password and domain (default current host name), 3. Server Instance Information: + Server Name: Unique name for this index agent + Base Port: Port mumber (by default, 9200) for HTTP requests, The installer validates that the next 20 consecutive ports are available. Index agents on the same machine must have unique port numbers + Password for Adinin User: You can use the same password as the xPlore administrator. 4. Enter the fully qualified domain name of the index agent host. Primary Dsearch Server Information: Specify the DNS name and port (default 9300) of the »Plore primary instance 6. Enter the directory for local content area: Directory path to a directory in which content is temporarily stored during indexing The installation owner must have wnite permissions on this, directory. 7. Connection Broker Information: Click Connect after you enter the DNS name and port for the Documentum connection broker. 8. When the index agent configurator successfully connects to the connection broker, you see additional fields. Enter this information: + Select Docbase: Choose a Documentum repository to which the index agent connects + User Name: Enter the Content Server installation owner user name + Password: Enter the Content Server installation owmer password, 9. Global Registry Information: Select the global repository and enter login information. 10 Select Storage Location: Select the location to store indexes. (The storage location for indexes is different from the temporary content location that you previously specified for the index 46 EMC Documentum xPlote Version 1.3 Installation GuideInstaling and Configuring Index Agents agent) The same storage location is used for all index agents on the same machine. Use xPlore Administrator to create storage locations. 11. Ifyou install the index agent in a multi-instance xPlore environment, restart all xPlore instances, 12.Restart the Content Server. Note: This enables the index agent to respond appropriately to a change in the Content Server's dormancy status Starting the index agent Set up index agent filters before you start indexing or reindexing Otherwise, the filters cannot be applied to objects that already been indexed. You can install index agent filters that exclude cabinets, folders, or object types from being indexed. See "Using the index agent filters" in EMC Documentum xPlore Administration and Development Guide. 1. Start with a script or service. Jndex_agent is the value you specified for Server Name + Linus: Run xplore_homefjboss5.1 O/server/startindey_agent sh (Linux) + Windows: Start the index agent Windows service (or restat it, if it is already running) Documentum Index._agent , or sun xpiore_home\jboss5.1 O\server\startndex_agent cmd 2. Start the index agent UL Ina browser, entera URL that contains the index agent host DNS name and port (default 9200): http: //host:port/IndexAgent/login_dss.3sp 3. Loginasa valid user in the repository. The optional domain name is the name of an xPlore domain. Iftblank, the index agent creates a domain with the name of the repository. 4. Ifthe index agent is stopped, choose one of these modes: + Start Index Agent in Normal Mode: Normal made indexes content that is added to or modified in the repository from this point onwards. + Start new reindexing operation: (Migration mode) Starts the index agent and indexes all content in the repository (also called refeeding or recranwiing). Object types that are registered for full-text indexing in dmi_registry are indexed, then dm_acl and dm_group objects are indexed. Only one index agent can actively reindex the repository. + Continue: Reindex date time: Continue indexing date and ttme indicate when you stopped indexing. 5. Ifthe index agent is running, choose one of these actions: + Details: Statistics and status of indexing + Stop IA: Halts the index agent. This does not stop the Windows service or the JB oss instance + Index selected list of objects: Indexes a set of objects specified by eithera DQL statement or a file that contains a list of objects specified by r_object id. + Logoff Logs out of the index agent UL + Refresh: Updates statistics and status Stopping the index agent When you stop indexing using the index agent UI, the agent is still running You must stop the agent using a JBoss script or Windows service EMC Documentum xPlore Version 4.3 Installation Guide a7Installing and Configuring Index Agents In the following commands, Index_agent is the value you specified for Server Name during installation. + Windows: Stop the index agent service Documentum Index agent or run xplore_home\jboss5.1.O\server\stoplidex_agent cmd (Windows) + Linux: Run xplore_home/jboss5. 1. O/server/stopiidex_agent sh (Linux). Silent index agent startup You can start or shut down the index agent through the index agent web application. You can also script the start in normal mode or shutdown using Content Server, iAPI or DFC. Starting in migration mode cannot be scripted. Setting startup in Content Server Set the start_index_agents parameter in serverini to TRUE. ‘At Content Server startup, the Server checks whether the index agent associated with the repository is started. If not, and start_index_agents is TRUE, the Server starts the index agent using the dm_FTindexA gentB oot job Silent startup and shutdown with iAPI Use the retrieve and drop commands to get the index_name attribute of the dm_fulltext_index abject You use this attribute value in the start or stop script. For example: APID retrieve,c,dm_fulltext_index 3h0004a280000100 APT> dump,c,1 USER ATTRIBUTES index_name : Repo_ftindex_01 Now use the retrieve and dump commands to get the object_name attribute of the dm_ftindex _agent_config object. You use this attribute value in the start or stop script. For example retrieve,c,dm_ftindex_agent_config 0800277e80000e42 APT> dump,c,1 USER ATTRIBUTES object_name : Configl3éé8VMO_S200_IndexAgent Use the apply command to start or stop the index agent. Syntax 48 EMC Documentum xPlote Version 1.3 Installation GuideInstaling and Configuring Index Agents apply, c, ,FTINDEX AGENT ADMIN, NAME, S,, AGENT_INSTANCE_NAME,S,, ACTION, S,start|stop| status To start or stop all index agents, replace the index agent name with all. For example: apply, c,NULL, FTINDEX_AGENT_ADMIN, NAME, S,LH1_ftindex_O1, AGENT” INSTANCE NAME,3,al1,ACTION,S, start ‘The following example starts one index agent apply, c, NULL, FTINDEX_AGENT_ADMIN, NAME, $,LH1_ftindex_01,AGENT_INSTANCE_NAME, 5, Configl3é68VM0_9200_IndexAgent, ACTION,S, start Follow with these commands to get the results APID next,c, al ox API> dump,c,ql Status results + 0: The index agent is running. + 100: The index agent has shut down. + 200: The index agent has. problem Setting startup with a list of file IDs You can script startup to indexa list of documents like the list generated by fintegrity. 1. Create a text file with the object ID. Save as ids tat in the WEB-INF/classes directory of xplore_home/jbossS. |. O/server/DetmServer_IndexAgent/deploy/IndexA gent wat/. (Substitute the actual path to your index agent web application ) 2. Start the index agent servlet in normal mode. The objects in ids tat are automatically submitted for indexing A file ids tat done is created in the same directory as ids tat. This file lists the IDs of objects that were successfully indexed. Startup from the Java command line Use the following command: java. com. documentum. server. impl.utils. IndexAgentCtrl -docbase_nane ‘YepositorylName -user_name userName ~action actionName where action argument value is one of the following: start | shutdown | status | reset Silent startup and shutdown using DFC ‘The following method gets the da_fulltext_index object, the indexx_name attribute, and sets the DQL query public void shutdownIA(IDfSession sess) throws DfException EMC Documentum xPlore Version 4.3 Installation Guide 49Installing and Configuring Index Agents IfPersistent0b: IDfPersistent0b dm_fulltext_iné ect FTIndexob3 = ( t) sess. getObjectByQualification( where is_standby = false"); String indexWame = FTIndexob}.getString("index_name"); (query definition String query = "NULL,FTINDEX_AGENT_ADMIN,NAME,S," + indexNane + ",AGENT_INSTANCE_NAME, 5, a11,ACTION, S, shutdown" ; D£ClientX clientX = new D£ClientX(); IDfQuery gq = clientX. getQuery(); gq. setDOL (query): try ( IDfCollection col = q.execute(sess, IDfQuery.DF_APPLY); ) catch (DfException e) ‘ e.printstackTrac ) ) For startup, replace sinutdown with start in the query definition. Deleting an index agent When you migrate from FAST or when you upgrade 2Plore, you must delete and reinstall each index agent 1. Deleting a FAST index agent a Log on to the FAST index agent host as the user who installed the FAST index agent 'b. Stop the FAST index agent using Documentum administrator. ¢ Start the FAST Index Agent Configuration Program located in $DOCUMENTUM_SHARED/IndexA gents (Linux) or Start > Programs > Documentum > Index Agent Configuration Program (Windows). Use this program to delete the index agent 2. Deleting an 2Plore index agent a. Run xplore_home\setup\indexagent\configindexagent bat (Windows) or xplore_home/setup/indexagent/configindexagent sh (Linu) b. Select Delete Existing Index Agent and choose the instance to delete Ifyou are unable to connect to the repository, stop all JBoss processes in xplove_home, and then manually delete the JBoss index agent server instance. Note: Deleting index agent does not automatically remove the corresponding xPlore domain. If ‘you want ta remove the xPlore domain, you must do it manually in xPlore Administrator. 50 EMC Documentum xPlote Version 1.3 Installation GuideChapter 6 Performing Silent Installation This chapter contains the following topics + Setting up silent installation + Configuring silent installation + Running the silent install script + Verifying silent installation + Troubleshooting silent installation + Sample single-instance installation Setting up silent installation ‘The silent installation scripts install xPlore primary, secondary, and spare instances on local or remote hosts. You can also create a newindex agent for each repository on local or remote hosts. For remote installation, a third-party tool is required: + Windows: Pstools, downloaded from Microsoft + Linux: Expect, included in the silent installation archive. + Install the latest cumulative service pack or hotfix, ifany. Configure at least one Documentum, repository and one connection broker Note: If you do not install the latest hotfix or service pack, aPlore does not work properly + Lime Installation of xPlore in console mode is not supported. Set up your environment to support GUI installation. Installation on 64-bit Linux requires the following RPM packages to be installed before xPlore instalation: ~ libXp.. i386 ~ libXi. 1386 — libXtst.. 1386 — libXt..i386 ~ libXext 1386 + All instances must be installed on the same operating system (Windows or Linus) + Toavoid permission-related errors, the same installation owner (same username, same password, and same Windows domain) must install all instances. 1. Extract the silent installation archive SilentInstallerip to a directory. SilentInstaller ip is a separate download. The base directory is now SilentInstaller. EMC Documentum xPlore Version 4.3 Installation Guide stPerforming Silent Installation 2. Download a 32-bit version Java SE JDK 6 or higher (The 32-bit version is required by the installer, even on 64-bit hosts ) 3. Create a jak subdirectory of SilentInstaller and extract the JDK to this directory. 4, Download ant version 1.7 or higher and place it in a subdirectory of Silentlnstaller named art. ‘Make sure your ant library contains ant-contrib jar. Download it if it is missing, and place it in the antiib directory + Ifyouare installing to multiple hosts on Limux, follow the instructions in the tcl-expect-instal tat. ‘This file 1s Located in SilentInstaller/3rdParty/Expect. + Ifyou are installing to multiple hosts on Windows, add the pstools directory to SilentInstaller/3rdParty. Pstools download Your silent install directory must have the following content: Figure 6 Silent install directory Oe [F ~ comuter = toeai0sk(C) + Covmioads ~ sletin Organize ¥ Includeinibrary ¥ Sharewth y NewFolder bai Neve 1B arch sd 1.2.00 Oh Seritratller Wh ocraty bb reany Daw DB bont Di cesses Ban DB conta Dh cesses bik Bi config OD soript Bide plore Bs aloe 5. Download the xPlore installer to a temporary directory and note the location, Supply this path as the value of istailers location when you configure silent installation, Configuring silent installation You can configure one or more xPlore instances and zero or more index agents. Each Documentum repository can have only one index agent. You can configure an additional index agent to process ACLs and groups. For instructions, see EMC Documentum xPlove Administration and Development Guide. 1. Edit the file silent properties in StlentInstaller/config. You do not need to edit any other properties files in this directory, 2. Host details: + SMIP_HOST. DNS name of SMTP host (not Microsoft Outlook server). If blank, no watchdog notifications are sent to the administrator. + ADMINISTRATOR EMAIL_ADDRESS. Email address of xPlore administrator, to receive instance watchdog notifications If blank, no watchdog notifications are sent to the administrator. 52 EMC Documentum xPlote Version 1.3 Installation GuidePerforming Silent Installation + installers location: Location of the xPlore expanded installer archive containing setup exe or setup sh. + common mstallLocation: The path on each host machine for the xPlore home directory (xpiore_home in xPlore documentation), A subdirectory is created in this path to serve as the temporary staging location for content to be indexed 3. Storage location: Installs a data and config directory on the specified host. The index and configuration files do not have to be stored on the same host as the primary instance. The storage host and parent folder must be accessible by all xPlore instances. + storage fad’ Fully qualified domain name for storage host. If specified, storage.p is not used ‘Must contain only alphanumeric characters + storage tp: IP address of the host where the index data will be stored + storage parent folder: Path on the storage host that wall contain the data directory. 4. Edit the primary instance settings + primary mstance fad: Fully qualified domain name of xPlore primary instance. If specified, primary instance ip is not used. Must contain only alphanumeric characters. + primary mstance tp: IP address of the »Plore primary instance + primary tmstance port: Base port for the primary instance. Use this port to open »Plore administrator, for example, http: //server.emc.org:9300/dsearchadmin An additional 99 ports are used by 2Plore. + primary stance name: Name that identifies the instance. Use a name that tells you this is a primary instance. The instance name must contain alphanumeric characters + primary mstance password: Specify the password for the xPlore administrator Use the same password for all xPlore instances. The password is also used for xDB administration in xPlore ‘The user must have full permissions on the directory specified in installers location. The password is not encrypted. (It is encrypted in the GUI installer) 5. Edit the secondary instance settings. Each instance has the same keys as the primary instance, in the form nonPrimary instance key_name N where Nis a unique integer If you are installing only one instance, you wall supply this number Nas a command line argument + nonPrimany count. Number of secondary instances already installed in the xPlore federation. If you are not installing secondary instances, set to 0 + Ifthe secondary instance is on the same host as the primary, choose a port at least 100 units different from the primary port and any other secondary ports + Ifthe secondary instance is to be used as a spare for failover, set the value of nonPrimarylnstance instance isSpareNode N to 1 6. Edit the index agent instance settings Do not modify configlA properties: it wall be updated by the values that you enter in silent properties. Each instance has keys in the form indexagent instance key_namne N where Nis a unique integer. If you are installing only one index agent instance, you wall supply this number Nas a command line argument. Each indexagent docbase name must be unique. + indevagent count. Number of existing index agents already installed + indexagent mstance name: Unique name for the index agent. Configure only one index agent per repository. + indevagent mstance fadn: Fully qualified domain name of index agent host. If specified, indexagent instance ip is not used. Must contain only alphanumenc characters. + indevagent mstance tp: IP address of the index agent host EMC Documentum xPlore Version 4.3 Installation Guide 53Performing Silent Installation + indexagent mstance port. Unique port on the index agent host. The index agent uses 20 additional consecutive ports + indexagent mstance password: Password for the index agent administrator. Can be the same as the xPlore administrator password. + indevagent docbase name: Repository served by this index agent. Each repository must have its own index agent. The name must contain alphanumeric characters. + indevagent docbase user: Specify the Content Server instance owner + indexagent docbase password: Specify the password for the Content Server instance owner + indexagent connectionBroker host: IP address of the Content Server connection broker host. + indevagent connectionBroker port: Port number for the Content Server connection broker. Default: 1489. + indevagent global Registry Repository name: Name of a global registry repository that projects to the connection broker specified above + indexagent globalRegistry Repository user: Name of the global registry user. Default dm_bof registry + indevagent global Registry Repository password: Password for the global registry user. + indexagent localcontent area: Temporary location for index agent staging of content. This location is not the same as indexagent instance storage name Running the silent install script Run xplore bat or xplore sh in the SilentInstaller directory. Add an xPlore secondary instance. Configure all existing instances in silent properties. Add the nonPrimary instance settings and set nonPrimary count to the number of existing secondary instances. The following example runs the script to add a third secondary instance. That instance is configured in silent properties as nonPrimary instance key_na@me 3xplore.bat|sh config. one. instance -Dname=dsearch -Dnumber=3 ‘Add an index agent instance. Configure all existing instances in silent properties. Add the nonPrimary instance settings and set nonPrimary count to the number of existing secondary instances. The following example runs the script to add a third index agent. This index agent serves a different repository than the other two index agents. The new indexagent instance is configured in silent properties as indexagent key_name.3:xplore.bat|sh config. one. instance ~Dnane=indexagent -Dnumbe ‘The installation log install log is written to the directory specified in common installLocation. If you install to multiple instances, an install log is wnitten to the install location on each instance. The sui ce \ccessful script output is like the following downloads\SilentInstall>xplore Buildfile: .\script\build.xml. silent. install: setup: 54 [copy] Copying 1 file to C:\domnloads\vin [echo] begin == setup [echo] windows install [echo] end == setup EMC Documentum xPlote Version 1.3 Installation GuidePerforming Silent Installation [echo] waitfor [echo] end == waitfor [delete] Deleting: C:\downloads\win\silent. properties prmiary. config. updat [echo] begin to update primary configuration [propertyfile] Updating property file: C:\domloads\silent_..\config\config -Properties [echo] end to update primary configuration config. dsearci [move] Moving 1 file to C:\xplore\setup\dsearch [copy] Copying 1 file to C:\xplore\setup\dsearch [echo] begin == configure dsearch [exec] [exec] C:\xploi \dsearch>dsearchConfig. bin\java.exe" -£ config.properties [echo] end == configure dsearch [delete] Deleting: C: \xplore\setup\dsearch\ config. properties [move] Moving 1 file to C:\xplore\setup\dsearch LAX_YM "C: \xplore\jdk\jre start. primaryInstant [echo] begin start primary instance waitfor. primary. started: [echo] begin == vait for dsearch (instances) service started [echo] end waitfor [echo] end == start primary instance conf ig. nonPrimar [echo} there are 1 nonPrimary instances needed to be configured [echo] begin to configure nonPrimary instances [propertyfile] Updating property file: C:\downloads\silent_..\config\config -Properties config. dsearci [move] Moving 1 file to C:\xplore\setup\dsearch [copy] Copying 1 file to C:\xplore\setup\dsearch [echo] begin == configure dsearch [exec] [exec] C:\xploi \dsearch>dsearchConfig. bin\java.exe" -£ config.properties [echo] end == configure dsearch [delete] Deleting: C: \xplore\setup\dsearch\ config. properties [move] Moving 1 file to C:\xplore\setup\dsearch [echo] finish to configure non-primary instance: Dsearchllode2 LAX_YM "C: \xplore\jdk\jre start.nonPrimaryInstance: [echo] begin == start nonPrimary instance [echo] begin == vait for nonPrimary instance D: [echo] end vaitfor [echo] end == start nonPrimary instance Dsearchlode2 [echo] end to configure nonPrimary instance wchNode2 started EMC Documentum xPlore Version 4.3 Installation Guide 55Performing Silent Installation config. indexagent: vaitfor.primary. starte [echo] begin == wait for dsearch (instances) service started [echo] end vaittor [propertyfile] Updating property file: C:\downloads\silent_..\config\config TA. properties [copy] Copying 1 file to C:\xplore\setup\indexagent [copy] Copying 1 file to C:\xplore\setup \indexagent [echo] begin == config index agent [exec] [exec] C:\xplore\setup\indexagent>iaConfig. « a\java.exe" -£ config. properties [exec] Result: -1 [echo] end == config index agent [delete] Deleting: C: \xplore\setup \indexagent\config. properties [copy] Copying 1 file to C:\xplore\setup \indexagent LAX_VM "C: \xplore\jdk\jre\bi start. indexagent: [echo] begin [echo] end start index agent start index agent BUILD SUCCESSFUL Total time: 10 minutes 26 seconds Verifying silent installation After successful installation, the silent installation script starts the instances. 1. To test whether the primary instance is running, open your web browser and enter the following URL: http //host portidsearch. host is the DNS name of the primary instance hast and port is the Plore port (default 9300). These two settings are configured in your silent properties file. For eample http: //serve -emc. org:9300/dsearch Ifthe instance is running, you see a message like the following ‘The DSS instance PrimaryDsearch [version=1.3.0000.0159] normal 2. Start xPlore administrator with a URL to the primary instance. Substitute the primary instance name and port: http-//host port/dsearchadmin. For example: http: //server. emc. org: $300/dsearchadmin 3. To start indexing or migration, open the index agent UI with the following URL: hitp-Whost:IA_port/indexAgent/started_dss jsp. For example http: //server. emc. org: $200/IndexAgent/started_dss. jsp 56 EMC Documentum xPlote Version 1.3 Installation GuidePerforming Silent Installation Troubleshooting silent installation Secondary instance failure If you use different passwords for secondary instances in your silent installation properties, you see errors such as com xhive error XhiveException’ FEDERATION NOT_FOUND in xdb log or Logon failure: unknown user name or bad password in cpslog The log files are located in the logs subdirectory of the JBoss deployment directory Perform the following steps to rectify. 1. Stop the primary instance 2. Edit indexserver-bootstrap properties of the secondary instance xplore_homelboss5.1 O/server/DctmServer_PrimaryD search/deploy/dsearch war/WEB -INF/classes Change the adminuser-password to that of the primary instance, in clear text When you restart, that password is encrypted using a FIPS 140-2 validated encryption module 3. Restart the primary instance and then the secondary instance Sample single-instance installation ‘The following silent properties file configures a single secondary instance. The IP address of the primary and new instance hosts are the same Runa single-instance installation on the command line with the config one instance command. For the -Dnumber argument, specify the count index For example, the following command installs the instance configured as nonPrimary instance tp.1=10.8.46.133: xplore-bat config.one. instance -Dnane=dsearch -Dnumb: silent properties SMTP_HOST=-mailhub.emc.com ADMINISTRATOR EMATL_ADDRESS=aa@emc. com common. installLocation=C: \\xPlore storage. ip=10.8. 46.183 storage. parent. folde \xPlore xPlore. installation. ovner.domain= xPlore. installation. oyner.usernam: xPlore. installation. over. passwor -Aduinistrator assword primary. instance. ip=10.8. 46.133 primary. instance. hostname=Config8S18VM0 primary. instance. primary. instance. rimaryDsearch primary. instance.passvord=password ’ onfigure secondary instanc nonPrimary.count=0 # info for the first secondary instance nonPrimary. instance. ip.1=10.8. 46.133 EMC Documentum xPlore Version 4.3 Installation Guide STPerforming Silent Installation nonPrimary. instant nonPrimary. instant nonPrimary. instant # If this is a spare instance, nonPrimary. instance. isSpareNlode. nonPrimary. instant nonPrimary. instant nonPrimary. instant nonPrimary. instant # If this is a spare instance, nonPrimary. isSparellode. ¢.ip-2=10.37. 10.133 hostname. nonPrimary. 4p.3=10.37. 10.103 nonPrimary. hostname. nonPrimary. E00 nonPrimary. # If this is a spare instance, nonPrimary. instance. isSpareNlode. nonPrimary. instant nonPrimary. instant nonPrimary. instant nonPrimary. instant # If this is a spare instance, nonPrimary. instance. isSpareNlode. ip. 4-10.37. 10.103 hostname. + configure the index agent # Specify the number of index agents to # Tf you don’t need to install an in indexagent. count=1 # Info for the first index agent, in # To configure a second index agent for # copy these keys and change index 1 to ind: name. 1=Indexagent ind 4ip-1=10.37. 10.102 ind ind ind ind hostname. 1=HostA port. 2=S100 instance. password. 1=passvord indexagent. indexagent. 58 set following value searchInstance2 set following value searchInstance3 set folloving value searchInstanced set folloving value to 1 instead of 0 (not spare) to 1 instead of 0 (not spare) to 1 instead of 0 (not spare) to 1 instead of 0 (not spare) be configured. agent, set to 0 begins vith 1 a different repository, ind -docbase.user. ind -docbase. password. ind -connectionBroker. host. 1: ind . connectionBroker.port. ind jistryRepository-name. 1: ind istryRepository.user. ind: istryRepository. password. ind name. 2=Indexagent2 ind: 4p.2=10.37. 10.102 EMC Documentum xPlote Version 1.3 Installation GuidePerforming Silent Installation ind ind ind ind xagent. indexagent. xagent. indexagent. xagent. indexagent. xagent. indexagent. docbase.nane.2=repository docbase. user. 2=root docbase. password. 2=password connectionBroker.host.2=10.37.10.7 connectionBroker.port.2=1485 globalRegistryRepository.name. 2=repository globalRegistryRepository.user.2=dm_bof_registry globalRegistryRepository. password. 2-password EMC Documentum xPlore Version 4.3 Installation Guide 59Chapter 7 Setting up High Availability This chapter contains the following topics + Spare instance HA (standby) + Configuring active-active load balancer support + Setting up primary instance HA (active-active) + Enabling queries after active-active failover + Restoring data after an instance fails + Content Server full-text objects and initialization files Spare instance HA (standby) ‘The following diagram illustrates a Content Server HA deployment coupled with an xPlore spare instance HA. A single spare instance can support multiple xPlore instances. Your risk increases in proportion to the number of running xPlore instances that a single spare instance supports. You must perform more manual tasks when replacing a primary instance than when replacing a secondary instance. For instructions on configuring and activating an xPlore spare instance, see the EMC’ Docwnentua xPlore Administration and Development Guide EMC Documentum xPlore Version 4.3 Installation Guide etSetting up High Availabilty Figure 7 xPlore spare instance HA Before Failover After Failover Reposiory Repesiory Tetadata | Conentrins Netaiata | Contntries (08 Custer) | (NASISAN) (08 Custer) | INASISAN) Host vos Host Ye cst oT csi cs2 7 oS a AE ewe 45 oe . Ving eH . K H aah iar et art bras tt? Index dex Ind i i [PHS SP | Pri inst FiLinst dost | sec_inet Sp =x P= S05) | Tom Lee Loop leSedhg 7“ weary A i Contig Flies PA ncores Hosts NASISAN Key. + Host: Primary Content Server host — Content Server (CS1) — Connection broker (CB 1) — Index agent! (1A1) + Host2: Secondary Content Server host — Content Server (CS2) — Connection broker (CB2) — Index agent? (1A2) + Host3: Primary xPlore host — Primary instance (Pri_Inst) — Secondary instance (Sec_Inst) + Host: Spare Plore host — Spare primary instance (Pri_Inst_Sp) — Spare secondary instance (Sec_Inst_Sp) 62 EMC Documentum xPlote Version 1.3 Installation GuideSetting up High Availabilty + Hosts: 2Plore primary instance data, log, and configuration files, which must be writable from both the running instances as well as the spare instances. When you start the spare instances on Host to replace Host3, they must load the same configuration parameters as the instances on Host3. + MAS/SAW: Full-text indexes For indexing to be routed to the correct Content Server, IA1 must connect to CB1, and IA? must connect to CB2. You specify the connection broker when you configure the index agent. After failover from Host3, you manually start the Sec_Inst_Sp on Host 4, which reads the configuration data from Host. All indexing now goes through Sec_Inst_Sp on Hosta If the entire Host3 machine fails, switch to both Pri_Inst_Sp and Sec_Inst_Sp on Hosta. For instructions on how to switch, see EMC Documentum xPlore Administration and Development Guide Configuring active-active load balancer support An active-active load balancer configuration has the shortest down time of all high availability strategies. The load balancer automatically handles failover. This configuration places a greater load on the Content Server because duplicate full-text queue items are created and handled. ‘Two separate »Plore instances and two separate index agents are deployed against the same Content Server and repository. For indexing, the separate index agents keep indexes separately up to date ‘The load balancer distributes queries ‘This deployment gives both indexing and searching high availability. If indexing stops on one xPlore instance, the queue items for that instance accumulate. The corresponding queue items for the other instance continue to be processed. When the down instance resumes, it processes the accumulated queue items and catches up with the other instance. Note: The two indexes may not be identical if one xPlore instance is catching up due to a disruption in indexing. In this case, the same query can get different results when it is sent to one or the other Plore installation Figure 8 Load balancer active-active strategy Conent Sever whaed Reopositoy Indexing | | Tat v2 wow T Pie? instance instance Trdenes Trdexee Hostt Host2 EMC Documentum xPlore Version 4.3 Installation Guide 63Setting up High Availabilty In this diagram, separate index agents each handles the same indexing queue items. Separate xPlore primary instances handle the indexing, and the resulting indexes are approximately identical. The load balancer handles queries from Content Server and distributes the queries to an available xPlore instance ‘The generic procedure is as follows 1. Install and configure a The Content Server with a repository The load balancer The first xPlore primary instance and index agent 2. Stop the index agent for the first xPlore primary instance. 3. On the Content Server, run the create_fulltext_objects_ha ebs script in %@DM_HOME%/install/admin to update full-text index related objects in the repository: Gmbasic -f create_fulltext_objects_ha.ebs -eHAPreInstallstep == repository superuser password 4. Install and configure the second »Plore primary instance and index agent. Stop the index agent On the Content Server, run the post-install step in the create_fulltext_objects_ha ebs script dubasic -f create_fulltext_objects_ha.ebs ~eHAPostInstallstep == repository superuser password 6. Restart the Content Server and verify that two dm_fulltest_ index, two d_ftengine_config, and two din_ftindex_agent_config objects have been created APID ?,c,select r_object_id,index_name,ft_engine_id,is_standby from du_fulltext_index API> ?,c,select r_object_id,object_name from dm_ftengine_config API> ?,c,select r_object_id,object_name, index_name,queue_user from du_ftindex_agent_contig ‘The output of each command should retum two objects 7. Statt xPlore instances and index agents on both hosts Verify the heartbeat (Keepalive) URL for the load balancer. This URL detects failure of »Plore instances and retums true with a response code of 257 when the instance is searchable http: //xplore_host: xplore_port/dsearch/?action=CheckSearchable 9. Update dsearch_qrserver_host and dsearch_grserver_port values of both dm_ftengine_config objects in the repository with the VIP (Virtual IP) anid the port used by the load balancer. Both query plugins point to the same load balancer IP address and port. For example API> ?,c,select r_object_id,object_name from dm_ftengine_config robject_id object_name OB01béES8001aacb DSearch Fulltext Engine Configuration 00 0801bé€98001b110 DSearch Fulltext Engine Configuration (2 rows affected) 64 EMC Documentum xPlote Version 1.3 Installation GuideSetting up High Availabilty API>fetch,c, 0801bée98001aacb API>get,c,1, paran_nane[3] dsearch_grserver_host aPI>set,c,1, param_value[3] by load balancer for xPlore HA deployment> aPI>get,c,1, param_nane[5] dsearch_grserver_port aAPI>set,c,1,patam_value[5] API>fetch,c, 0801bée98001b110 aPI>get,c,1, param_nane[3] dsearch_grserver_host aPI>set,c,1, param _value[3] aPI>get,c,1, param_nane[5] dsearch_grserver_port aAPI>set,c,1,patam_value[5] 10 Enable load balancing by setting the load_balancer_enabled parameter to true for both the dm_ftengine_config objects. API> ?,c,select r_object_id,object_name from dm_ftengine_config robject_id object_name OB01beeSS001aach DS OB01béeS8001b110 Ds (2 rows affected) ch Fulltext Engine Configuration 00 rch Fulltext Engine Configuration API>fetch, c,O801beeS800Laach APDappend,c,1,param_nane SET>load_balancer_enabled id, c,1,param_value API>fetch,c, 0801béé98001b110 APDappend,c,1,param_nane SET>load_balancer_enabled id, c,1,param_value 11. Restart the index agents and the Content Server. ‘The following URL displays True when the host is searchable: hetp://:/dsearch/?actio EMC Documentum xPlore Version 4.3 Installation Guide 65Setting up High Availabilty ‘The white paper xPlore 13 High Availability in active-active setup with Load Balancer provides more information about setting up »Plore for load balancing and testing it Setting up primary instance HA (active-active) Ensure that no users are connected to the repository. Upgrade Content Servers to a supported version for xPlore For all scripts in the following procedure: + Scripts are located in: — Windows: %DM_HOME?4\installladmin ~ Limex: $DM_HOME/install/admin + Entera space before and after the double hyphen + No environment variables in paths. This procedure sets up a single repository with HA (two or more Content Server hosts). If you are setting up a single repository with a single Content Server host, follow these steps but specify the same connection broker for both index agents 1. Ifyou are upgrading an existing Content Server 5.3 SP? or 5.3 SP3 high-availability deployment, run the following scripts to create HA objects in the repository before you install xPlore and the Plore index agent: a. Delete existing index agents and index servers 'b, Execute the create_fulltext_objects_ha ebs script with the HACLeanupBeforeUpgradeStep ‘This script deletesall dm_ftengine_config and dm_fulltext_index objects Syntax dubasic -f create_fulltext_objects_ha.ebs HACLeanupBeforeUpgradeStep -- repository superuserpassword 2. Install and configure the primary xPlore deployment and primary xPlore index agent. Specify the piimary Content Server's connection broker as the connection broker for this index agent. The index agent connects to the repository and creates the full-text indexing objects that are required to run the scripts in the subsequent steps. 3. Shut down the primary xPlore index agent 4. On the primary Content Server host a. Log inas the installation owner. b, Execute create_filltext_objects_ha ebs as follows dubasic -f create_fulltext_objects_ha.ebs HaPreInstallstep -- repository superuserpassword 5. Install and configure the secondary xPlore deployment and secondary xPlore index agent. Specify the secondary Content Server's connection broker as the connection broker for this index agent. Do not start this index agent 6. On the primary Content Server host, 1og in as the installation owner and execute create_fulltext_objects_ha.ebs as follows 66 EMC Documentum xPlote Version 1.3 Installation GuideSetting up High Availabilty dubasic -f create_fulltext_objects_ha.ebs -e HAPostInstallstep =- repositorysuperuserpassword ‘This script creates a new dm_fulltext_index object anda new dm_ftindex_agent_config object for the secondary xPlore deployment 7. Restart both Content Servers 8. Use iAP! to confirm that the scripts created two of the following objects, one for each Content Server m_fulltext_index n_ftengine_config u_ftindex_agent_config a Each dm_fulltext_index object is associated with one xPlore index. Each has an associated du_ftengine_config object To verify dm_fulltext_index for the default Content Server APID ?,c,select e.r_object_id,e.object_name from dm_ftengine_config e, du_fulltext_index fi where e.r_object_id-fi.ft_engine_id and fi.is_standby-0 To verify on the standby Content Server APID ?,c,select e.r_object_id,e.object_name from dm_ftengine_config e, du_fulltext_index fi where e.r_object_id-fi.ft_engine_id and fi.is_standby-1 ‘b, To verify that there are dm_ftindex_agent_config objects for each index and that the queue user is correctly set for the second index, execute the following Documentum API quenes. For the default index agent API> 2,c,select ia.r_object_id,ia.object_name, ia. queue_user from du_ftindex_agent_config ia,dm_fulltext_index fi where iasindex_name=fi.index_namé and fi.is standby-0 For the standby index agent API> ?,c,select ia.r_object_id,ia.object_name,ia.queus_user from du_ftindex_agent_config ia,dm_fulltext_index fi where iasindex_name=fi.index_namé and fi.is standby=1 Note: The object_name is the name of the index agent and index_nane is the name of the index 9. Start both index agents, The indexes are created To verify that the indexes have been created correctly, nun ftintegrity or the State of th Index job on each Content Server. (The State of the Index job is available on Content Server 6.7 or higher) See the EMC Docwnentwn xPlore Administration and Development Guide Enabling queries after active-active failover When the primary xPlore deployment fails, indexing continues on the secondary xPlore deployment ‘You must manually enable queries to be serviced by the secondary 2Plore deployment 1. Retrieve the secondary dm_fulltext_index object ID with this DQL statement sel is_standby=1 t ft_engine_id from da_fulltext_index wh EMC Documentum xPlore Version 4.3 Installation Guide 67Setting up High Availabilty 2. On both the primary and secondary Content Server, edit the serverini file located in documenton_homeldba/configlservername. Under [SERVER_ STARTUP], set the ftengine_to_use value to the secondary dm_fulltext_index abject ID. 3. Restart both Content Servers. Restoring data after an instance fails In HA setup, you can restore data from one instance to another when one of the instances fails. ‘The following steps use two example instances: InstanceA and InstanceB. xPlore is set up on these two instances with mirrored information: same folder structures, same usemame/password, same instance name, same ports, etc. The example assumes that data corruption occurs on InstanceB 1. Stop xPlore and IndexA gent on InstanceA. Record the time the index agent is shut down, 2. Stop xPlore and IndexA gent on InstanceB 3. Bacup xplore_homelconfigindexserverconfig xml and xplore_home/config/XhiveDatabase bootstrap on InstanceB. You will use this backup to recover configuration information after you restore data 4. Copy the directories xplore_homeldata and xpiore_home/config from InstanceA to InstanceB. Do not change file permission 5. Optionally, start xPlore and the index agent in normal mode (not migration mode) on InstanceA to put InstanceA in service 6. Update xplore_home/configindexserverconfig xml on InstanceB with settings in the omginal backup copy Change the attributes “url” and “hostname” of the XML element index-server-configuration node to match those values in the backup file For Windows systems that use UNC paths, update the following XML elements or attributes to their original values in the backup copy: + fracing-configl tracing output[ air] + sstovage-locations/storage-locatton[@path] + Sadmin-confighoacinp-location[ path) 7. Update xpiore_home/config/XhiveDatabase bootstrap on InstanceB to match the original file Change attribute “host” of the element /server/node to match the value in the backup file For Windows systems that use UNC paths, update the following XML elements or attributes to their original values in the backup copy Update the path only and do not change the file name. + Iserverinodeflog/[@path] + Iserver/database/segmentfile/[@path] 8. If InstanceB failed some time ago, you must clear many duplicate indexing tasks that are queued as dimi_queue_items. Run DQL to truncate the queued items for IndexAgentB: In the following example, IA ser is the user for IndexAgentB,, date_value is a time just before the index agent sinutdown time recorded in step! 2,c,delete dmi_queus_item objects where name=/IA user’ and date_sent<=date ("date value’ ) For example: ?,c,delete dmi_queue_item objects where name=/du_fulltext_index_user’ 63 EMC Documentum xPlote Version 1.3 Installation GuideSetting up High Availabilty and date_sent<=date (‘5/22/2012 21:24:45") 9. Start xPlore and the index agent in normal mode on InstanceB Content Server full-text objects and initialization files HA requires more configuration of the full-text indexing system than a basic installation The following illustration shows the relationships between the configuration objects for the full-text indexes, index servers, and index agents and the Content Server and »Plore initialization files. Figure 9 Full-text index objects ‘am Sever contig Tater ocaTon, ‘am_Rangine._ contig ‘object name= "searcr ‘ecLnane= “dsearct" ‘ie_system_path (Sm_findex_agent_conig afuitextiny Ceancby Indexing Management > Index Agents and Index Servers, select the index server or agent, and choose Tools > Stop. 2. Remove FAST: Uninstall the FAST indexing server using the instructions in your version. Delete the Documentum index agent for FAST and stop the Content Server See Deleting an index agent, page 50. 3. Content Server 6.6 only: Run the script fulltext_setup_for_dss ebs to create objects for xPlore (Support is already installed with Content Server 6 7 and iugher) The scnpt is located in °%DM_HOME%\ \admin (Windows) or $DM_HOME /install/admin (Linus) on the Content Server machine Note: Run this script only once dubasic -f fulltext_setup_for_dss.ebs password fulltextHome EMC Documentum xPlore Version 4.3 Installation Guide 75Migrating ftom FAST to xPlore + repositoryName: Name of the repository to set up for »Plore. + username and password: Usemame and password for repository installation owner. + fiulltextHome: Fulltext directory on the Content Server, for example, Jexport/user| /Documentuny/fulltext (Linu) or C:\Documentum\fulltext (Windows). + Entera space before and after the double hyphen. Do not use environment variables in paths. For ewample dmbasic -f fulltext_setup_for_dss.ebs -eSetupForDss -- testenv tuserl mypwd C:\Documentum fulltext 4. Restart the Content Server. After successfully starting up Content Server and the connection ‘broker, wait several minutes (typically 15 minutes) before installing the Documentum index agent 5. Install the xPlore primary instance and start xPlore. See Installing xPlore with Content Server 6.7 SPxand 7.0, page 34 or Installing 2Plore wath Content Server 6.6, page 36 6. Install and configure a Documentum index agent for xPlore and start the index agent UI See Configuring an index agent, page 46 Set up index agent filters after you configure the index agent but before you start reindexing Filters can exclude cabinets, folders, or object types from being indexed. See EMC Documentum xPlore Administration and Development Guide. 7. Start the index agent UI and select Start new reindexing operation. All dm_sysobject, dm_acl, and dm_group objects are indexed unless they are excluded by a filter. ‘After indexing completes, you see Reindexing is completed. Please stop LA and start LA in Normal mode). Verify that the reindexing was completed successfully for all objects by using the ftintegnty tool. See EMC Documentum xPlore Administration and Development Guide Migrating indexes (reindexing) ‘To migrate indexes from FAST to »Plore, reindex your content using an xPlore index agent. See Immediately replacing FAST with »Plore, page 75, 1. Set up index agent filters after you configure the index agent but before you start reindexing Otherwise, the filters cannot be applied to objects that already been indexed. You can install index agent filters that exclude cabinets, folders, or object types from being indexed. See EMC Docwnenton xPiore Administration and Development Guide, “Using the index agent filters” section 2. Estimate the amount of time you can spend reindexing your content. xPlore provides guidelines for estimating your reindexing time as well as some best practices. For information on estimate indexing time and tuning indexing, refer to EMC Documentwn xPlore Administration and Development Guide. Some best practices that you can use to reduce reindexing time are: + Use multiple CPS instances, which reduces crawling time + Use multiple xPlore instances, which spreads ingestion across several JVMs or hosts. 3. Temporarily configure multiple CPS daemons for reindexing a. Stop the CPS instance in xPlore administrator. Choose Instances > Jnstarice_name > Content Processing Service and click Stop CPS. bo. Edit the CPS configuration file in the CPS host directory xplore_home/dsearch/cps/cps_daemon. Change the value of element daemon_cowt to 3 or more (default: 1). 76 EMC Documentum xPlote Version 1.3 Installation GuideMigrating ftom FAST to »Plore 4d. Change the value of connection_pool_stze to 2 e. Restart all xPlore instances £ Reset the CPS daemon_count to 1 and connection _pool_sice to 4 after reindexing is complete Migrating large environments + Large migrations best practices, page 77 + Supporting a large number of ACLs, page 77 + Adding temporary instances, page 78 + Ensuring CPU, disk, and VO capacity, page 78 + Migrating data to specific collections, page 78 + Migrating content based on document age, page 79 + Custom routing, page 80 + Targeting queries to specific collections, page 80 + Tuning parameters for large migrations, page 80 Large migrations best practices A large environment has a combination of some of the following characteristics: Tens of millions of documents, thousands of users, 8-15 TB of content spread over several file stores, and an existing FAST index smaller than 600 GB fora single node or smaller than 2 TB spread evenly over multiple nodes. A large migration typically requires more resources than day-forward indexing support. Migration best practices: + Plan indexing disk requirements with the xPlore sizing tool available from EMC Community Network. + Do full backups before migration and at points during migration. Reserve disk space for multiple backups + Test migration in a non-production environment + Change final merge interval. By default, changes are merged to the index every four hours to ensure low latency. Increase this interval during migration See information about the setting finalMerging/nterval in xdb properties, in EMC Documentum xPlove Administration cmd Development Guide Supporting a large number of ACLs Ifyour environment has more than 500,000 ACLs, tum off compression for ACLs Stop all »Plore instances. Edit indexserverconfig xml. Locate the element dmfimetadata /acl_name and dmftsecurity/acl_name. Set the value of the compress attnbute to fase 4. Save the config file and restart the primary instance, then restart all secondary instances EMC Documentum xPlore Version 4.3 Installation Guide 7Migrating ftom FAST to xPlore Adding temporary instances ‘The Content Processing Service (CPS) within »Plore is resource intensive Each xPlore instance can bbe configured to have its own CPS. Temporary instances of xPlore with CPS can manage ingestion without impacting ongoing ingestion and queries 1. Create an xPlore secondary instance for each backdfile collection you created. 2. Bind each bactdfile collection to a new temporary CPS instance 3. Migrate data, See Migrating data to specific collections, page 78 4 Move bactdile collections from the temporary instance to the primary instance. (Change binding using xPlore administrator.) 5. Delete the secondary (temporary) instances that you installed for migration. Ensuring CPU, disk, and I/O capacity Indexing is both CPU and 1/0 intensive 1. Temporarily increase CPU resources a. Bind extra collections to secondary instances, then move to primary instance when migration is finished bv. Add more cores per CPU instance 2. Improve disk /O if CPU is sufficient but indexing still has a bottleneck a. See EMC Documentum xPlore Administration and Development Guide for information on how to detect /O bottlenecks. 'b. Use striped dist allocations for xPlore data and log directories and for the index agent temporary staging area Migrating data to specific collections By default, xPlore routes documents in a round-robin fashion over multiple collections. When you create multiple collections, xPlore can ingest documents simultaneously to different sets of Lucene indexes. You can target queries to the appropriate collection and speeds up response time 1. Create at least three backdile collections using xPlore administrator and ingest documents to those collections. Add more collections if the hosts have good I/O capacity. If you use round-robin ingestion, you will not be able to perform bulk deletes of collection froma specific file store. Ingest data to backfile and day-forward collections using the index agent UL After data has been migrated, route new documents and updates to the day-forward collection, Optional: Merge collections for recent data to the day-forward (ongoing) collection Start ingestion to the backdile collections using the index agent UL After data has been migrated, direct new data and updates to a day-forward collection. 7. Optional: Merge collections for recent data to day-forward (ongoing) collection. You can determine a cutoff date for legacy data and then route your legacy data to backfile collections. ‘Add routing logic to a custom routing class to support the cutoff date. Use one of the following 78 EMC Documentum xPlote Version 1.3 Installation GuideMigrating ftom FAST to »Plore + Custom index agent BOF filter that implements IDfCustomlndexFilter. Base the filter on a date attribute. See ‘Filtering content from indexing" in EMC Documentwn xPlore Administration and Development Guide. + Custom xPlore routing class that tests for creation date, modification date, access date, or custom. date attnbute for routing to a specific collection. For a description of this routing customization, see Custom routing, page 80. For implementation details, see "Custom routing class" in EMC Documentun xPlore Administration and Development Guide ‘After ingestion, you can consolidate collections using xPlore administrator. You can also target queries to improve query performance and specificity. See Targeting queries to specific collections, page 80 Migrating content based on document age ‘When you separate content into age-based collections, you can query recent data separately. Merges are minimized, To determine whether your query load is primanly for recent content, perform the following steps. For large repositories with old data, separate old content from new content by collection. Old data is, not queried very often, as shown in the diagram Time basis of quentes: Figure 10 Time-based queries In typical customer environments most queries are focused on the most recent data g 2 23 re i : z 2 : | 33 38 BE 2 Newer (hotter) Older (colder) 1. Compare results from the DQL queries of your Content Server. First query: select count(*) from du_sysobject vhere datediff (year, r_creation_date, r_access_date) < 2 and datediff (year, r_creation date, r_modify date) < 2 Counts how many objects were modified and accessed within the past two years. 2. Second query: select count(*) from da_sysobject EMC Documentum xPlore Version 4.3 Installation Guide 79Migrating ftom FAST to xPlore Counts the number of objects in the repository. If these counts are close, a high percentage of documents have not been touched after two years. For example, ifthe first query counts 1 million documents modified and accessed out of 2.5 million objects, 40% of objects are recently touched, and 60% are not of interest to users. If your queries are time-based, you can create a custom routing class to route older documents to backfile collections. The class routes recent documents and new additions to a day-forward collection. Fora high-level description, see Custom routing, page 80. For detailed information, see the example code for this on the EMC Developer Network Custom routing ‘The EMC Documentum xPlore Administration and Development Gude fully describes routing customization. A custom routing class does the following 1. Uselogic to determine routing to a specific collection or set of collections. 2. Filter out content based on a date attribute 3. Create and test the routing or filter class before you create the day-forward (ongoing) collection. Targeting queries to specific collections Default queries can be targeted to the most recent collection. Targeted queries can examine all, collections in parallel Route queries with one of the following approaches: + DQL: IN COLLECTION clause. For example select r_object_id from dm document search document contains ‘benchmark’ in collection(’ default’ ) + DQL hints file: Requires disabling XQuery generation, and does not retum faceted results. For example select r_object_id from du _document search document contains ’ enable (fds_query_collection_default) + DFC QueryBuilder API addPartitionScopeQ) + DFC IDEXQuery API collection) + DFS ParitionScope object in a StructuredQuery implementation For details on creating queries with DFC and DFS APIs, see EMC Docwnentum Search Development Guide .chmark’ Tuning parameters for large migrations Table 6 Ingestion tuning parameters Parameter Where Value xPlore OS xPlore host Required: 64bit OS 80 EMC Documentum xPlote Version 1.3 Installation GuideMigrating ftom FAST to »Plore Parameter Where ‘Value Disk LO subsystem xPlore host Required: Supports at least 1500 UO per sec. Not required after migration xPlore instance VM Staat scxipt in Required: Atleast 4GB per xPlore splore_homeljposs5 1 Olserver instance DB page server cache WEB.INFiclasses/indexserver- Required: Stop all xPlore instances ‘bootstrap propeites in the primary and increase vave-cache-pages to instance war file 512 MB - 1 MB (inbytes). Restart xPlore instances. Number of open files nofile wliit parameter in Required for Linux: Set to 65,000+ lelclsecuntyfiits cont and reboot OS. Without i, the xPlore process can rum out of file descriptors. Index agent queve size indexagent zal in index agent Required: Set queue_sze in instance WEB-INFlclasses indexer_plugin_config indexer to 1500. Index agent callback queue size _indexagent xml in index agent Perit ae Requized: Set callback queue_aze in indexer_plugin_ config indexer to 1000, xdb lucene ramBufferSizeMB xb properties in primary Default is 3 MB. This canbe set to instance war file 64 MB or greater. Reset to default search war/WEB-INFiclasses afer migration xdb, lucene cleanMergelntervel xb properties Default = 300 sec. Set to 900 fore Jong migration, resulting in fewer ftushes to disk Reset to default after migration. xb, lucene finalMergingInterval xdb properties Required: Set to double or triple the default (14400 sec). Reset to default after migration Troubleshooting migration Ifyou installed Content Server 6.7 SP1 and enabled FAST, you must manually install subscription query support for xPlore. See MC Documentum xPlore Administration and Development Guide Check the full-text config objects in the Content Server to make sure you have installed xPlore support. ‘Then run ftintegrity. Instructions are in ZMC' Documentum xPlore Administration and Development Guide You have one full-text config object for xPlore, one for FAST, or both. 1. Get the object ID of the dm_ftengine_config object: retrieve,c,dm_ftengine_config EMC Documentum xPlore Version 4.3 Installation Guide atMigrating ftom FAST to xPlore 2. Use the object ID to get the full-text config object parameters, Substitute the object ID retumed by the retnieve API 2,c,select param name, param value from da_ftengine_config where r_object_id= Parameters that begin with dsearch indicate the xPlore config object. 2 EMC Documentum xPlote Version 1.3 Installation GuideChapter 10 Troubleshooting Installation This chapter contains the following topics + Debugging installation + Fixing indexserverconfig upgrade issues + Verifying Content Server support in 6.6 + Host name contains an underscore + Instance name fails + Cannot create a secondary instance + Cannot start a secondary instance + CPS instance does not start + xPlore administrator problems with IE8 and IE9 browsers + Index agent errors + Linux errors Debugging installation To log debug ouput from the installer, run the installer using the following options + Windows: Hold down the control key while you launch the installer (setup exe). Do not release the control key until the installer pops up a console window. The console windows looks like the following EMC Documentum xPlore Version 4.3 Installation Guide 3Troubleshooting installation Documents and Settings\Administrator, FAC-4CRBRZARFEDNLocal Settings\Ten| Risin $20 243295 Nndaws \Docunent sand Sett ings\Adninistrator, EMG~4CBBB2U8FBINLocal Set ings\rel pots 11326742229\Uindoworwosouroovjronjrocbinyjava exe” “Smo 6D77216 —smbOs3164| “classpath, "C:\Documents and. Settings \Aininistracor.ENC-4cBEB2O8FHD\Local Sece| Ings \Tenp\i\1132674222\ Instal lerbata\laClasses.2ips¢:\Docunent's and Sett ings\Ad| Hinistrator.ENC-4CBBBZ2OSFBD\Local Sett ings\Tenpv1\11326742229\Insta llerbata\exec jee -zipse:\Decumente and Sectinga\Adniniecwaton.ENC-4CBBB2OEFED\Local Sete ingest| Inpvi nti 320743225\Uiodoueringtal loxData\Exorute -2ipiG: MDacunents and, Sott inge tal Hinistrator .ENC-4CBBEZ08FBD\Local Set ings\Tenp\1 \11326742229\Insta llerbata\Reso Jreel-2ip;G:\Docunents and Sevtinys\Adminiotratur,ENG-4CBDD208FED\Local Seccinys| enp\i\ li 326742229\Uiindous\ instal lewData\Resounced -2ipsC-\Documents and Setting] Nidninistrator .FNC~4ChBB268FBD\Local Sect ings\Tenp\1\11326742229\InstallerDatay 2\Decunents and Settings\Administrator.EMC-4eDBBZOSFRD\Local Settings\lemp\INI4| 2e742229Windows\InstallerDatas"” com-zerog,lax.LAH "C=/Decuments and Settings “Al nin istnaton.EMC-4CBERAGSFED Local Sect ings /Temp/i/11326742229 Uindaus/setup. lay] ‘G:7Docunents and Sett ings /Adninistrator-EM¢-4CBBR208FBD/Local Sett ings/Tenp/| HaxD~ tmp" INo Argunents Jjavea.cless. path: G:\Docunents and Settings\Adninistrator.FMC-4CRRR208FBD\Local Settings\Temp| H\11326742229\InstallerData\lAClasses .2ip Gi\bocunents. and Sett:ings\Adninistrator. ENG-4GBBB208FBD\Local Sett ings\tenp| H\11326742229\InatallerData\Execute .= ip G:\Docunents and Sett ings\Adninistrator. FMC-4CRRR2A8FRD\Local Settings\Tenn| }1.\11326742229\Windows\InstallerData\Execute zip + Limur Set LAX_DEBUG=true in the environment variables and then launch the installer. To redirect output from the console to a file, set LAX DEBUG=file. After installation, a file named jalog is generated in the install directory. Checking installation versions All »Plore instances and index agents should have the same installation version. You can check the version in the version properties file in aplore_home/installinfo. Fixing indexserverconfig upgrade issues During upgrade to »Plore 1.3, a script upgrades the main configuration file, indexserverconfig xml, to support the new logging paradigm. If the script encounters errors, you must fix them before running sPlore. You can encounter the following errors: + The ile is updated. [error message and stack trace]. It's nota valid configuration file. The change isreverted. + Failed to read the file (/O exception). a4 EMC Documentum xPlote Version 1.3 Installation Guide