You are on page 1of 0

EMC Documentum

Kerberos SSO Authentication


A Detailed Review






































Abstract
This white paper introduces and describes a Kerberos-based EMC

Documentum

environment, and explains


how to deploy such a system with single sign-on (SSO) on the Documentum platform.

J une 2011




Copyright 2010, 2011 EMC Corporation. All rights reserved.
EMC believes the information in this publication is accurate as of its publication date. The information is
subject to change without notice.
THE INFORMATION IN THIS PUBLICATION IS PROVIDED AS IS. EMC CORPORATION
MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT TO THE
INFORMATION IN THIS PUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIED
WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Use, copying, and distribution of any EMC software described in this publication requires an applicable
software license.
For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com
All other trademarks used herein are the property of their respective owners.
Part Number h8031.2
EMC Documentum
Kerberos SSO Authentication
A Detailed Review 2



Table of Contents

Executive summary ............................................................................................ 5
Introduction......................................................................................................... 5
Audience...................................................................................................................................... 5
Kerberos authentication..................................................................................... 5
Kerberos architecture................................................................................................................... 5
Kerberos authentication flow in a DFS-based application........................................................... 6
Kerberos authentication flow in a Documentum Web Development Kit-based application......... 9
Configuring Kerberos authentication ............................................................... 9
Configuring Kerberos on a Content Server in a Windows/ UNIX environment ......................... 10
Modification to error message during Kerberos plug-in initialization......................................... 11
Replay cache filename change.................................................................................................. 11
Workaround for the Kerberos plug-in initialization error in UNIX............................................... 11
Prerequisites for WDK-based applications ................................................................................ 12
Determining the Service Principal Name (SPN) ............................................. 12
Specifying the SPN for repositories........................................................................................... 12
Specifying the SPN for DFS services ........................................................................................ 12
Specifying the SPN for WDK-based applications...................................................................... 13
Registering the SPN and generating the keytab file...................................... 13
Creating the keytab file for Content Server................................................................................ 13
Reinitializing Content Server...................................................................................................... 15
Configuring the SPN and keytab file for DFS services.............................................................. 15
Creating the keytab file for WDK-based applications ................................................................ 17
Creating Kerberos user accounts ................................................................... 18
Creating Kerberos users in a repository.................................................................................... 18
Configuring LDAP synchronization for Kerberos users.......................................................... 19
Creating a user account for a WDK-based application in the Active Directory......................... 19
Enabling Kerberos for DFS-based applications............................................. 23
Enabling Kerberos during DFS service deployment.................................................................. 23
Kerberos and J AAS configuration.............................................................................................. 25
Kerberos configuration........................................................................................................... 25
Kerberos keytab file................................................................................................................ 25
J AAS configuration................................................................................................................. 25
Using Kerberos authentication in DFS clients ........................................................................... 27
Kerberos authentication in a local DFS web application ................................................. 28
Kerberos authentication in a remote DFS client ............................................................... 30
Enabling Kerberos for WDK-based applications ........................................... 33
Prerequisites.............................................................................................................................. 33
Preparing the client machine and the browser to meet Kerberos SSO setup requirements..... 33
Creating the J AAS configuration file.......................................................................................... 36
Configuring the Tomcat application server............................................................................. 36
Configuring the WebLogic application server......................................................................... 37
Configuring the J Boss application server............................................................................... 39
EMC Documentum
Kerberos SSO Authentication
A Detailed Review 3



Configuring the WebSphere application server...................................................................... 40
Configuring the custom/app.xml file to enable Kerberos authentication ................................... 40
Enabling Kerberos SSO authentication in WDK-based applications..................................... 40
Configuring the Kerberos domain name ................................................................................ 41
Configuring Kerberos fallback................................................................................................ 41
Sample Kerberos configuration in custom/app.xml................................................................ 41
Configuring EMC CenterStage to enable Kerberos authentication........................................ 41
Enabling tracing......................................................................................................................... 42
Kerberos authentication use cases............................................................................................ 43
Client platform/Browser is not supported............................................................................... 43
All repositories are Kerberos-enabled and the user logs in to the Kerberos domain............. 43
Client machine is not part of the Kerberos domain................................................................ 44
Webtop is configured to work with mixed repositories........................................................... 44
The end user is registered in the KDC but is not part of the Kerberos-enabled repository... 45
Setting DES, AES128, and RC4 Kerberos encryption types ......................... 46
Conclusion ........................................................................................................ 47
References ........................................................................................................ 47
Glossary..................................................................................................................................... 47
Common issues with the configuration of Webtop or TaskSpace for Kerberos authentication. 48
Issue 1 ................................................................................................................................... 48
Issue 2 ................................................................................................................................... 48
Issue 3 ................................................................................................................................... 49
Issue 4 ................................................................................................................................... 49
Issue 5 ................................................................................................................................... 49
Issue 6 ................................................................................................................................... 50
Issue 7 ................................................................................................................................... 50
Issue 8 ................................................................................................................................... 50
Issue 9 ................................................................................................................................... 51
Issue 10 ................................................................................................................................. 51
Issue 11 ................................................................................................................................. 52
Issue 12 ................................................................................................................................. 52
Issue 13 ................................................................................................................................. 52
Issue 14 ................................................................................................................................. 52
Issue 15 ................................................................................................................................. 52
Issue 16 ................................................................................................................................. 52
Issue 17 ................................................................................................................................. 52
Issue 18 ................................................................................................................................. 52
Issue 19 ................................................................................................................................. 53
Issue 20 ................................................................................................................................. 53
Issue 21 ................................................................................................................................. 53
References for troubleshooting Kerberos-based SSO authentication....................................... 53

EMC Documentum
Kerberos SSO Authentication
A Detailed Review 4



Executive summary
Kerberos single sign-on (SSO) is a network authentication protocol designed to provide strong
authentication for client/server applications by using secret-key cryptography. The Kerberos protocol uses
strong cryptography so that a client can prove its identity to a server (and vice versa) across an insecure
network connection. After a client and the server have used Kerberos to prove their identities, they can also
encrypt all of their communications to ensure privacy and data integrity.
Kerberos provides secure and reliable authentication to multiple applications that use Kerberos for
authentication. In most distributed network systems, a password is used to prove a user's identity, and this
password is transmitted over the network from the client machine to the machine that the user wants to
access. So, a mechanism that prevents anyone from intercepting or eavesdropping on the transmitted plain
passwords is vital for security. In addition, another pain point while using passwords for authentication is
that the password must be supplied every time a connection is requested to the remote machine. Kerberos
helps users avoid this issue and solves the central problem of using passwords for authentication without
sending them over the network.
EMC

Documentum

supports the Kerberos SSO authentication feature using Microsoft Active Server
Domain Services for Kerberos Key Distribution Center (KDC) services. When using a thick client or a
Web-based client, users are automatically signed in and authenticated based on their Windows credentials.
For example, if Kerberos SSO is configured on a Documentum system, clients requesting services from the
Documentum repository will send a service ticket that the Documentum system uses to validate the clients
rather than prompting the user to provide login credentials.
Introduction
This white paper discusses end-to-end Kerberos SSO implementations on systems using Documentum 6.6
and later, including usage scenarios, code samples, and FAQs.
Audience
This white paper is intended for customers, partners, and consultants who are planning to set up and
configure SSO for Documentum using Kerberos in a Windows domain.
Kerberos authentication
Kerberos architecture
Kerberos operates by encrypting data with a symmetric key. A symmetric key is a type of authentication
where both the client and server use a single encryption/decryption key to send or receive data. When
working with the encryption key, the details are sent to a KDC, instead of being sent directly between each
computer.
Figure 1 describes how Kerberos authentication is used to verify the credentials of Documentum clients.
EMC Documentum
Kerberos SSO Authentication
A Detailed Review 5




Figure 1. Kerberos in a Documentum environment
On a Documentum system, the Kerberos authentication process involves the following steps:
1. A Documentum user logs in to a client computer that is in the Kerberos domain by specifying
Windows login credentials, such as a username and password, and accesses a Documentum application
from the client. The local computer/client sends the login credentials and the service name of the
application to the Key Distribution Center (KDC) for identification.
2. The Kerberos authentication service/server (AS) component at the KDC receives the request from the
client, verifies whether the client is the computer it claims to be, and generates a Ticket Granting
Ticket (TGT).
3. When the user wants to access a Documentum client in the domain, the client sends the TGT to the
KDC to obtain a Service Ticket (ST) for the service, using the Service Principal Name (SPN).
Note: It is mandatory that the SPN of all services are registered in the KDC. The KDC can provide the Service
Ticket only for a registered service.
4. The KDC Ticket Granting Service (TGS) authenticates the TGT and generates an ST.
5. The client running the Documentum application uses the relevant DFC-based API and provides the
username and ST as password.
6. The Documentum Foundation Classes (DFC) pass the login information to the Content Server service.
7. The Content Server service validates the ST and authenticates the user.
8. If authentication is enabled, the Content Server service sends the acknowledged ticket to DFC.
9. DFC sends the acknowledged ticket back to the client to validate the Content Server service. A session
is established and no further authentication is required.
Kerberos authentication flow in a DFS-based application
Documentum Foundation Services (DFS provides two distinct modes to satisfy Kerberos authentication
requirements: a local mode and a remote mode.
EMC Documentum
Kerberos SSO Authentication
A Detailed Review 6



A local mode assumes both the DFS client and the DFS service exist in the same J ava Virtual Machine
(J VM). The client can be part of a web application.
Figure 2 illustrates how Kerberos authentication is used for DFS local mode.

Figure 2. DFS Kerberos SSO flow with local mode
The authentication process shown in Figure 2 is as follows:
1. A remote user requires a Service Ticket (ST) with Ticket Granting Ticket (TGT) and the applications
Service Principal Name (SPN).
2. The KDC returns an ST for the application.
3. The web application client sends a web request with the Kerberos Simple and Protected GSSAPI
Negotiation Mechanism (SPNEGO) token in the HTTP header. Negotiation may occur between the
web browser and server.
4. The web server extracts the SPNEGO token from the HTTP header and accepts it with the Kerberos
utility; a new TGT is generated for the authentication to DFS. The DFS service gets a TGT from a
BinaryIdentity and passes it to DFC LoginInfo.
5. DFC initializes the Content Server ST with the TGT and logs in to the Content Server.
6. The Content Server validates the ST and returns a repository session.
7. The web server handles the web request and talks to the Content Server as a DFS local client. The web
page content is returned to the web client.

A remote invocation assumes the client and the server reside in separate JVMs. For DFS, communication
between client and server relies on Simple Object Access Protocol (SOAP), and login information will be
sent over the wire as SOAP WS-Security headers.
EMC Documentum
Kerberos SSO Authentication
A Detailed Review 7



Figure 3 illustrates how Kerberos authentication is used for DFS remote mode.
Content Server
DFS Server
KDC
DFS Remote Client
(1) Ticket-Granting Ticket
(TGT) +DFS SPN
(2) Service Ticket
(ST) for DFS SPN
(3) SOAP Request, passing ST through
J AX-WS handler | WCF Behavior
(4) Accept ST and
new TGT
KeyTab File
(5) DFC
LoginInfo with
Content Server
ST
(6) Repository
Session
(7) SOAP Response
* DFS extracts Kerberos ST from
SOAP Header, validates it with a
Kerberos Utility, and generates a
new TGT for DFC login.
* DFC initializes Content Server ST
and logins with it to get a repository
session.
* Keytab file can be used for
validation of long-term key.

Figure 3. DFS Kerberos SSO flow with remote mode
The authentication process in Figure 3 is as follows:
1. A remote user requires an ST with TGT and DFS SPN.
2. KDC returns an ST for a DFS service.
3. DFS remote client sends out a SOAP request, serializing ST as a binary token in the J ava API for XML
Web Services (J AX-WS) handler or Windows Communication Foundation (WCF) behavior;
negotiation is not supported.
4. DFS extracts Kerberos ST from the SOAP header, validates it with the Kerberos utility, and generates
a new TGT for DFC login.
5. DFC initializes the Content Server ST with the TGT and logs in to the Content Server.
6. The Content Server validates the ST and returns a repository session.
7. The DFS server handles the request with the repository session and returns the SOAP response.
EMC Documentum
Kerberos SSO Authentication
A Detailed Review 8



Kerberos authentication flow in a Documentum Web
Development Kit-based application

Figure 4. Documentum Webtop Kerberos SSO flow
Figure 4 illustrates a Kerberos single sign-on authentication process in Documentum Webtop.
The authentication steps are as follows:
1. The Webtop user logs in to a machine that is in the Kerberos domain.
2. A Ticket Granting Ticket (TGT) is generated.
3. The user opens a browser (IE/Firefox) and accesses the Webtop URL.
4. Webtop verifies whether the user has already been authenticated. If the user has not been
authenticated, Webtop displays Error 401 with WWW-Authenticate:Negotiate in the http header.
5. Using the TGT, the browser sends a request to obtain a Service Ticket (ST) for Webtop. The browser
wraps the ST in the SPNEGO token and sends it to Webtop in the http header.
6. Webtop passes the SPNEGO token to DFC to obtain the ST. DFC validates the ST. Webtop displays
the Repositories selection page. The Webtop user selects a repository. DFC then sends a request for an
ST for the repository by impersonating the user and passes the user information and the ST to the
Content Server.
7. The Content Server validates the ticket using the Kerberos plug-in and returns a repository session.
8. Webtop obtains a repository session and returns to the Webtop Main page.
Configuring Kerberos authentication
The Kerberos authentication plug-in that enables Kerberos support for KDC services is automatically
installed with Content Server. You must complete all additional steps to enable Kerberos SSO on Content
EMC Documentum
Kerberos SSO Authentication
A Detailed Review 9



Server, LDAP servers, DFS servers, DFS clients, and WDK-based applications subsequent to Content
Server installation.
Depending on the specific Documentum server/client environment that you use for Kerberos
authentication, you must configure the Kerberos authentication by:
Creating a Kerberos domain and adding all servers and clients that are participating in Kerberos
authentication to the Kerberos domain.
Choosing a Service Principal Name (SPN). A Kerberos SPN uniquely identifies a service that uses
Kerberos authentication.
Registering the SPN on the Active Directory and generating a keytab file. The keytab file contains
name/value pairs consisting of an SPN and a long-term key derived from a password. Both the service
and the KDC must have knowledge of the keytab file.
Creating Kerberos users, either manually or by synchronizing with an LDAP directory server.
Enabling Kerberos support on the DFS server and remote DFS clients, if applicable. You can configure
the DFS 6.6 web services to use the server-side JAX-WS handlers that interface with the Content
Server Kerberos implementation. The DFS 6.6 SDK includes new classes that support Kerberos
authentication for local J ava clients, remote J ava clients, and .NET clients. DFS SOAP clients that do
not use the support classes in the SDK can authenticate using WS-Security headers that comply with
the Kerberos Token Profile 1.1 specification.
Enabling Kerberos support in WDK-based applications, if applicable.
Configuring Kerberos on a Content Server in a Windows/ UNIX
environment
To configure Kerberos on a Windows/UNIX environment:
1. Install Content Server 6.7.
2. The $DOCUMENTUM/dba/auth directory is created and the Kerberos plug-in file is copied there. In
Windows, the plug-in filename is dm_kerberos.dll. In all UNIX platforms except HP-UX Risc, the
name of the plug-in file is libkerberos.so. In HP-UX Risc, the plug-in filename is libkerberos.sl. The
MIT Kerberos binaries are automatically copied in the $DM_HOME/bin directory.
3. To include the Windows/UNIX machine in the AD machine domain, include the following information
in the C:\windows\krb5.ini (windows) /etc/krb5.conf (or /usr/local/etc/krb5.conf in UNIX) file:
Under the [realms] tag, include the following:
QA2008. COM = {kdc = 10. 31. 107. 98
admi n_ser ver = 10. 31. 107. 98}
Under the [domain_realm] tag, include the following:
. qa2008. COM = QA2008. COM
Under the [libdefaults] tag:
comment t he f ol l owi ng l i ne:
def aul t _r eal m= EXAMPLE. COM
and i ncl ude t he f ol l owi ng:
def aul t _r eal m= QA2008. COM
4. Copy keytab files to the $DOCUMENTUM/dba/auth/Kerberos folder and restart the docbase using the
following command:
/ dm_st ar t ( docbasename) - ot r ace_aut hent i cat i on
This command enables an authentication trace in the dm_krb_docbasename.log file.
5. Set any supported encryption standard entries in the /etc/krb5.conf (or /usr/local/etc/krb5.conf) file.
EMC Documentum
Kerberos SSO Authentication
A Detailed Review 10



For example:
def aul t _t kt _enct ypes = aes128- ct s
def aul t _t gs_enct ypes = aes128- ct s

The supported encryption protocols are:
DES
rc4-hmac
AES 128

The above encryption types must be specified in the Kerberos configuration file for
default_tkt_enctypes and default_tgs_enctypes tags under the libdefaults section.

Modification to error message during Kerberos plug-in
initialization
The following updated error message is displayed during plug-in initialization when the keytab file is not
present.

Thu Mar 10 12: 51: 27 2011 064148 [ DM_SESSI ON_I _AUTH_PLUGI N_LOAD_I NI T] i nf o:
" Aut hent i cat i on pl ugi n ( ' dm_kr b' ) was di sabl ed. Thi s i s expect ed i f no keyt ab f i l e( s)
at l ocat i on / home/ dct m/ dba/ aut h/ ker ber os) . Pl ease r ef er t he cont ent ser ver i nst al l at i on
gui de.

This is valid for both Windows and UNIX.
Replay cache filename change
Replay cache is used to detect duplicate authentication requests. When the Kerberos protocol processes a
request, it makes an entry in the replay cache. If it processes a later request that matches an entry already in
the replay cache, it returns an error to the Content Server. The replay cache is periodically purged to
remove requests with expired lifetimes.

The replay cache should not be shared between processes since this could result in false replay errors
caused by different requests with the same timestamp.

The replay cache filename is the repository name. Its default location is pointed by %TEMP% or %TMP%
or C:\ for Windows, and by $TMPDIR for UNIX, or by the KRB5RCACHEDIR environment variable.

For example, if the repository name is testenv, the replay cache filename is testenv.

Workaround for the Kerberos plug-in initialization error in UNIX
If you see the following Kerberos plug-in initialization error message:
Er r or message:
[ DM_SESSI ON_E_AUTH_PLUGI N_LOAD_I NI T_ERROR] er r or : " Fai l ed t o l oad Aut hent i cat i on
Pl ugi n / expor t / space1/ document um/ dba/ aut h/ l i bker ber os. so. Pl ugi n i ni t i al i zat i on r et ur ned
er r or : ' 5
Ei t her al l ocat i on f ai l ed or Ker ber os cont ext cr eat i on f ai l ed' . "

Set the LD_PRELOAD environment variable to include the MIT Kerberos binaries so that
the loader preloads them before it loads the other binaries.

For Linux and Solaris, add libkrb5.so.dm.3 and libgssapi_krb5.so.dm.2 to LD_PRELOAD
For HP-UX Risc and HP Itanium, add libkrb5.dm.3 and libgssapi_krb5.dm.2 to LD_PRELOAD
For AIX, add libkrb5.so and libgssapi_krb5.so to LD_PRELOAD
EMC Documentum
Kerberos SSO Authentication
A Detailed Review 11




For example, in the C shell for Linux, the syntax would be:
set env LD_PRELOAD l i bkr b5. so. dm. 3: l i bgssapi _kr b5. so. dm. 2

Note: The MIT binaries are located in $DM_HOME/bin directory. If LD_PRELOAD contains values, you can
append new values to it.
Prerequisites for WDK-based applications
Active Directory machine:
Ensure that the Active Directory machine uses Windows Server 2003 or later to support the Kerberos 5
authentication protocol. Add the required computers and users to the Active Directory. Refer to the
Microsoft help for setting up an Active Directory and Kerberos domain.
Content Server machine:
Verify that the Content Server machine is running Windows Server 2003 or later. Add the machine to
the Kerberos domain and then install Documentum 6.6 Content Server.
Application server machine:
Ensure that the application server machine is running on a server with Windows 2003 or later. Install
the relevant application server after adding the machine to the Kerberos domain. You can deploy the
WDK-based application on the machine. For more information about deploying Webtop on an
application server, see the EMC Documentum Web Development Kit and Webtop 6.6 Deployment
Guide. Install a supported browser: IE6, IE7, IE8, Firefox 2.x, Firefox 3.0.x, or Firefox 3.5.x.
Client machine:
Ensure that the client machine is on a Windows XP or later platform. Install a supported browser: IE6,
IE7, IE8, Firefox 2.x, Firefox 3.0.x, or Firefox 3.5.x, which you can configure to access Webtop with
Kerberos SSO support..
Register Webtop as a Service Principal in the KDC. For more information, see theEMC Documentum
Web Development Kit and Webtop 6.6 Deployment Guide.
Determining the Service Principal Name (SPN)
A Kerberos Service Principal Name (SPN) uniquely identifies a service that uses Kerberos authentication.
The SPN format varies depending on the service that the Kerberos uses for authentication. At the very least,
you should configure the SPN for the repository on the Content Server. The following section describes
how to configure the SPN for repositories, DFS services, and WDK-based applications.
Specifying the SPN for repositories
EMC recommends the following SPN format for registering a repository on the KDC service:
SPN=ser vi ce- name/ docbase- name@domai n
Where service-name is the service that is constant for Content Server. For example:
CS/ REPO1@MYDOMAI N. COM
Where CS is the service, REPO1 is repository name, and MYDOMAIN.COM is the domain where
the Content Server SPN is registered. This convention uniquely identifies each SPN across all repositories.
Specifying the SPN for DFS services
The recommended SPN format for DFS core services is:
DFS/ host : por t @r eal m
EMC Documentum
Kerberos SSO Authentication
A Detailed Review 12



We recommend using a hostname rather than an IP address as the host string, such as
myhost.mydomain.com. The realm name, or the Kerberos configuration file, is the name of the Kerberos
realm (see Enabling Kerberos for DFS-based applications).
Unless you need to include a service name in the SPN, EMC recommends using the same format for
custom DFS services as well. This approach requires fewer configurations for the Active Directory since
you do not need separate user accounts for every module/service. It also means that you can access all
services on a host:port using the same ST. This approach is consistent with the way the SPNEGO protocol
builds an SPN for a web application.
Specifying the SPN for WDK-based applications
The recommended format for WDK-based applications, such as Webtop, is:
<ser vi ce>/ <f ul l y qual i f i ed host name>
Unlike Kerberos principal names, account names on Windows 2003 or Windows 2008 are not multipart
names. Therefore administrators cannot directly create an account name in the HTTP/hostname.dns.com
format. The only option for Windows 2003 or Windows 2008 systems is to create a principal instance using
Service Principal Name mappings. In this case, the administrator needs to create an account with a
meaningful name and hostname, and add Service Principal Name mapping for HTTP/hostname.dns.com.
Registering the SPN and generating the keytab file
To use Kerberos SSO you need to register a Service Principal Name (SPN) on an Active Directory using
the Windows Server Ktpass utility. The Ktpass utility enables an administrator to configure a service as a
security principal in the Active Directory. This procedure produces a keytab file, which contains the
name/value pairs consisting of an SPN and a long-term key derived from a password. Both the service and
the KDC service must recognize the keytab file. The length of the keytab filename depends on the OS
filename limitation. The name of the file uses the following format:
<r epo_name>. <xxxx>. keyt ab
Where <r epo_name> is the name of the repository for which the SPN is registered.
<xxxx>can be any string that makes the filename unique. This unique name is required because users can
register the Content Server in multiple trusted domains.
Creating the keytab file for Content Server
In the following procedures, a user is registered on the KDC service to act as the service principal for the
service. This user maps to the SPN and is distinct from users who need to be authenticated to use the
service.
There are two recommended procedures for registering the SPNs. The simplest approach is to create a
single SPN mapped to an Active Directory user, which sets a single password for the SPN.
To create a single SPN mapped to a user:
1. Log in to the machine that runs the KDC service.
2. Create a user in the Active Directory.
3. Using the user you created in the preceding step, set an SPN and create a keytab file using the ktpass
utility as follows:

ktpass /pass <password>
-out <keytab-file>
-princ <SPN>
EMC Documentum
Kerberos SSO Authentication
A Detailed Review 13



-crypto ALL +DumpSalt
-ptype KRB5_NT_PRINCIPAL /mapOp set /mapUser <user-name>

EMC recommends that you map the SPN service with all the supported encryption types when you
generate keytab files. This is valid for both Windows and UNIX.

If you want to generate a keytab file using a particular encryption protocol, for example DES-CBC-
MD5, use the ktpass utility as follows:

ktpass /pass <password>
-out <keytab-file>
-princ <SPN>
-crypto DES-CBC-MD5 +DumpSalt
-ptype KRB5_NT_PRINCIPAL +desOnly /mapOp set /mapUser <user-name>

For supported encryption protocols, refer to the section Configuring Kerberos on a Content Server in
a Windows/ UNIX environment.

In a Windows 2008 environment, you must run the set spn command in the following format to
provide Kerberos delegation privileges for the new user:
set spn - A CS/ <r eponame>@domai n. comdomai n. com\ <csuser name>

4. After you register the SPN for the user, the Delegation tab is displayed when you view properties of
the new user in the User Properties dialog box. Select the Trust this user for delegation to any service
(Kerberos only) option.
5. Copy the keytab file to the $DOCUMENTUM/dba/auth/Kerberos/ folder on the Content Server. This
folder is created during Content Server installation.
An alternate approach is to register multiple SPNs mapped to a single Active Directory user.
To create multiple SPNs mapped to a single user:
1. Log in to the machine that runs the KDC service.
2. Create a user in the Active Directory.
3. Set an SPN and create a keytab file using the ktpass utility with the user you created in the preceding
step, using the following syntax:
kt pass / pass <passwor d>
- out <keyt ab- f i l e>
- pr i nc <SPN>
- cr ypt o DES- CBC- MD5 +DumpSal t
- pt ype KRB5_NT_PRI NCI PAL +desOnl y / mapOp set / mapUser <user - name>
4. Set another SPN using the setspn utility and set <user-name>to the user you created in step 2.
set spn - A <SPN> <user - name>

In a Windows 2008 environment, you must run the set spn command in the following format to
provide Kerberos delegation privileges for the new user:
set spn - A CS/ <r eponame>@domai n. comdomai n. com\ <csuser name>

EMC Documentum
Kerberos SSO Authentication
A Detailed Review 14



After you register the user SPN, the Delegation tab is displayed when you view properties of the new
user in the User Properties dialog box. Select the Trust this user for delegation to any service
(Kerberos only) checkbox.
5. Run the ktpass utility for the second SPN. Use the salt and key version number (kvno) that were
created in step 3, using the following syntax:
kt pass / pass <passwor d>
- out <keyt ab- f i l e>
- pr i nc <SPN>
- cr ypt o DES- CBC- MD5 +DumpSal t
- pt ype KRB5_NT_PRI NCI PAL +desOnl y / mapOp set +RawSal t <sal t >
- i n <keyt ab- f i l e>
- kvno <number >
6. Copy the keytab file to the $DOCUMENTUM/dba/auth/Kerberos/ folder on the Content Server. This
folder is created during Content Server installation.
Windows 2008 account names are not multipart as Kerberos principal names. As a result, administrators
cannot directly create an account named HTTP/hostname.dns.com. Such a principal instance is created
using service principal name mappings. In this case, an account is created with a meaningful name and
hostname, and a service principal name mapping is added to HTTP/hostname.dns.com.
Reinitializing Content Server
The reinitialize server function initializes the Kerberos plug-in, which reloads the SPNs from the
repositorys keytab file. This command enables you to update the Kerberos system without restarting
Content Server. Reinitialization is required if a new SPN has been registered and copied to the repository.
However, this step is not required if you have changed the password on an existing SPN.
Administrators can invoke the reinitialize server function using Documentum Administrator by selecting
the Re-Initialize Server option on the Info tab of the Server Configuration Properties dialog box.
Configuring the SPN and keytab file for DFS services
To enable authentication of the DFS services on the KDC, you need to register the DFS SPN on the Active
Server KDC using the Microsoft ktpass utility. As you may also be registering an SPN for Content Server
on the same machine, you can choose to register the DFS SPN and other SPNs to the same user account. In
some cases it may also be useful to register multiple DFS SPNs to the same account. For example, you can
achieve load-balanced environments support for Kerberos by joining all load-balanced nodes into a single
account and assigning a single SPN to the cluster. If you need a different SPN to access the service (for
example, based on the service host IP rather than the load balancer name) you can also register this SPN
with the same account.
Windows 2008 account names are not multipart as Kerberos principal names. As a result, administrators
cannot directly create an account named HTTP/hostname.dns.com. Such a principal instance is created
using service principal name mappings. In this case, an account is created with a meaningful name and
hostname, and a service principal name mapping is added to HTTP/hostname.dns.com.
The following procedure explains how to register an SPN using one-to-one mapping between the SPN and
the user account, or a many-to-one mapping where multiple SPNs are registered to one user account.
Note: For reference information on the ktpass utility see http://technet.microsoft.com/en-
us/library/cc776746%28WS.10%29.aspx.
To configure the SPN and keytab file:
1. Create a user for the DFS service in Active Directory. This is optional, as you can use an existing
account.
EMC Documentum
Kerberos SSO Authentication
A Detailed Review 15



2. Do one of the following:
Map SPN(s) to usernames using a one-to-one mapping.
Map multiple SPNs to a username using many-to-one mapping.
3. Copy the keytab file to the host computer where the DFS services are installed.
To map an SPN to a username (one-to-one mapping):
Using the Microsoft ktpass utility, map the service SPN to the user account and generate a keytab
file for the service.
kt pass / pass <passwor d> - out <keyt ab- f i l e> - pr i nc <SPN>
- cr ypt o DES- CBC- MD5 +DumpSal t - pt ype KRB5_NT_PRI NCI PAL +desOnl y
/ mapOp set / mapUser <user - name>

In a Windows 2008 environment, you must run the setspn command in the following format to
ensure that the Delegation tab is included in the User Properties dialog box:
set spn A DFS/ myhost . mydomai n. com: 8080@SRV01. COM

After you register the user SPN, view the properties of the new user in the User Properties dialog
box. Click the Delegation tab and select the Trust this user for delegation to any service
(Kerberos only) checkbox to provide Kerberos delegation privileges to the new user.
Note: Make sure that you do not map the same SPN to more than one user account.
To map an SPN to a username (many-to-one mapping):
1. Using the Microsoft ktpass utility, map the first SPN to the user account and generate a keytab file,
using syntax like the following:
kt pass / pass <passwor d> - out <keyt ab- f i l e> - pr i nc <SPN>
- cr ypt o DES- CBC- MD5 +DumpSal t - pt ype KRB5_NT_PRI NCI PAL +desOnl y
/ mapOp set / mapUser <user - name>
Note the output to the console. Look for the salt string and the key version number (vno). You
need them in a succeeding step.
2. Using the setspn utility, set the next SPN, mapping to the same user account:
set spn - A <SPN> <user - name>
In a Windows 2008 environment, you must run the setspn command in the following format to
ensure that the Delegation tab is included in the User Properties dialog box:
set spn A DFS/ myhost . mydomai n. com: 8080@SRV01. COM

After you register the SPN for the user, view the properties of the new user in the User Properties
dialog box. Click the Delegation tab and select the Trust this user for delegation to any service
(Kerberos only) checkbox to provide Kerberos delegation privileges to the new user.
3. Run the ktpass utility for the second SPN without setting with the same user. Use the salt and key
version number (kvno) that were output in step 1 of this procedure.
kt pass / pass <passwor d> - out <keyt ab- f i l e> - pr i nc <SPN> - cr ypt o
DES- CBC- MD5
+DumpSal t - pt ype KRB5_NT_PRI NCI PAL +desOnl y / mapOp set +RawSal t
<sal t > - i n <keyt ab- f i l e> - kvno <vno>
4. Repeat steps 2 and 3 for each additional SPN.
EMC Documentum
Kerberos SSO Authentication
A Detailed Review 16



Creating the keytab file for WDK-based applications
After specifying that the SPN for the application server (on which Webtop is deployed) use Kerberos, the
administrator must create a keytab (key table) for the Webtop application. Webtop requires the keytab file
to authenticate itself to the Key Distribution Center (KDC).
The administrator must register the application as a Kerberos principal in the Active Directory to enable the
application to participate in Kerberos authentication. A Kerberos principal is a regular account on the
Active Directory. For example, you can name the principal, "name@YOUR.REALM". The realm name
follows the "@" character in the principal. The application server/web server that hosts the application does
not have to belong to the same Kerberos domain. Both the Active Directory and the application server can
belong to different network domains. However, the application server must have access to the Kerberos
domain network. For example, if the Kerberos domain REALM is WDKBLR.COM and the host machine
where the application server is running is WDKAPPS.WDKBLR.COM, then register the application
Kerberos principal in the wdkapps.wdkblr.com@wdkblr.COM Active Directory.
The administrator must use the ktpass command line tool to register the SPN service as a security principal
in the Windows Server Active Directory, and to create a keytab file on the KDC. This keytab file
(ktpass.exe) is bundled with the Windows 2008 Resource Toolkit package and must be installed
separately. Run ktpass.exe on the Active Directory Server machine and when the keytab file is generated,
move it to the <webt op_i nst al l at i on>/ WEB- I NF folder on the application server machine by
running the ktpass command in the following format:
kt pass / pass <passwor d> - out <keyt ab- f i l e> - pr i nc <SPN> - cr ypt o AES128-
CTS +DumpSal t
- pt ype KRB5_NT_PRI NCI PAL +aesOnl y / mapOp set / mapUser <user - name>

Where:
password - Password of the user
keytab-file - Location to save the keytab file
SPN - The SPN framed by the browser in the following format: HTTP/hostname.dns.com@REALM
(E.g. HTTP/wdkapps.wdkblr.com@WDKBLR.COM)
user-name - User name (E.g. wtuser)
For example, you can run the ktpass command using the following parameters:
kt pass / pass <passwor d> - out webt op. keyt ab pr i nc
HTTP/ wdkapps. wdkbl r . com@wdkbl r . COM cr ypt o AES128- CTS +DumpSal t
- pt ype KRB5_NT_PRI NCI PAL +aesOnl y / mapOp set / mapUser webt op_wdkapps
This command generates the webt op_wdkapps. keyt ab file on the Active Directory machine. Copy
this file to the <webt op_i nst al l at i on>/WEB-INF folder on the application server machine.
After generating the keytab file, view the Properties of the new user to verify the SPN registered to the user
as indicated in Figure 5.
EMC Documentum
Kerberos SSO Authentication
A Detailed Review 17




Figure 5. User logon name field displays the registered SPN
Creating Kerberos user accounts
Kerberos SSO can only authenticate users who are registered as Kerberos users in the Active Directory and
in a Content Server repository. WDK-based applications must have a user account in the Active Directory.
The following options are available for creating Kerberos users in a repository:
Create the user account manually, both in the Active Directory and in the repository.
Synchronize the users from the Active Directory using LDAPSynchronization, and then modify the
User Login Domain in the LDAP configuration object, as described in the Configuring LDAP
synchronization for Kerberos users section.
Creating Kerberos users in a repository
Only the installation owner, system administrator, or superuser can create users in a Content Server
repository. If an LDAP server authenticates a user, only a superuser can modify the users LDAP-mapped
attributes.
To create a Kerberos user account:
1. Start Documentum Administrator and connect to the repository where you want to create new users.
2. Navigate to Administration >User Management >Users.
3. Do one of the following:
To create a new user, select File >New >User. The New User page is displayed.
To modify an existing user, select the user, then select View >Properties >Info. The User
Properties page is displayed.
4. Enter the user information on the New User or modify the user information on the User Properties
page. For a detailed description of all fields, refer to the Documentum Administrator User Guide.
For Kerberos users, you should use different values for the User Login Domain and the User Source
fields rather than the ones you use for user accounts that do not use Kerberos authentication.
User Login Domain The domain in which the user is authenticated. This is typically a Windows
domain or the name of the LDAP server that you used for authentication. If you are using Kerberos
authentication with LDAP synchronization, you must set the user login domain to the short domain
name, as described in the section Configuring LDAP synchronization for Kerberos users.
User Source Specifies how to the server authenticates the username and password. For Kerberos
users, the value must be set to dm_krb.
EMC Documentum
Kerberos SSO Authentication
A Detailed Review 18



5. Click OK.
Configuring LDAP synchronization for Kerberos users
You can implement LDAP synchronization in conjunction with Kerberos SSO in two ways:
Using an existing LDAP configuration to authenticate Kerberos users.
Creating a new LDAP configuration to authenticate Kerberos users.
To use an existing LDAP configuration to authenticate Kerberos users:
1. Modify the user login domain attribute in the user object of all Kerberos users to use the short domain
name instead of the name of the LDAP server.
For example, if a Kerberos user is part of the wdkdomain.com domain, change the user login domain
attribute to wdkdomain.
2. Change the user source attribute in the user object to dm_krb for all Kerberos users who are
synchronized with LDAP, if the password is not in plug-in format. Changing the user source attribute
is optional.
3. Run the LDAP synchronization job.
To create a new LDAP configuration to authenticate Kerberos users:
1. Create an LDAP configuration object. Use the short domain name as the LDAP configuration object
name.
For example, if Kerberos users are part of the wdkdomain.com domain, create an LDAP configuration
object using wdkdomain as the LDAP configuration object name.
2. Change the user source attribute in the user object to dm_krb for all Kerberos users who are
synchronized with LDAP.
3. Run the LDAP synchronization job.
Creating a user account for a WDK-based application in the
Active Directory
You must create a user account for any WDK-based application in the Active Directory to map to the
Service Principal Name (SPN).
To create a user for Webtop in Active Directory:
1. On the Active Directory machine, navigate to Start >Programs >Administrative Tools >Active
Directory Users and Computers. The Active Directory Users and Computers console is displayed.
EMC Documentum
Kerberos SSO Authentication
A Detailed Review 19




Figure 6. The Active Directory Users and Computers console
2. Right-click the Users node, select New >User. The New Object - User wizard is displayed.
3. Specify the new users login name details, and click Next.

Figure 7. Specifying the name of the new user
4. Specify the new users password and click Next.
EMC Documentum
Kerberos SSO Authentication
A Detailed Review 20




Figure 8. Specifying the password for the new user
5. Click Finish.
The new user is created and listed in the Users node.

Figure 9. The new user entry is displayed in the list of users
6. Right-click the new user, and select Properties. The <User>Properties dialog box is displayed.
7. Click the Account tab.
8. Select one or both of the following encryption algorithms under Account options based on the
encryption algorithms you require:
Select the Use DES encryption types for this account checkbox.
EMC Documentum
Kerberos SSO Authentication
A Detailed Review 21




Figure 10. Properties set for the new Webtop user in the Account tab on a non-
Windows 2008 machine
On a Windows 2008 machine, select the This account supports Kerberos AES 128 bit
encryption account checkbox.

Figure 11. Property set for the new Webtop user in the Account tab on a Windows 2008
machine
In addition, on a Windows 2008 machine, click the Delegation tab and select the Trust this user for
delegation to any service (Kerberos only) checkbox.
EMC Documentum
Kerberos SSO Authentication
A Detailed Review 22






Figure 12. Property set for the new Webtop user in the Delegation tab on a Windows
2008 machine
9. Click OK.
Note: To specify that the user comply with the Kerberos protocol, the encryption type must be DES or AES.
Enabling Kerberos for DFS-based applications
The DFS 6.6 web services can be configured to use server-side JAX-WS handlers that interface with the
Content Server Kerberos implementation. The DFS 6.6 SDK includes new classes that support Kerberos
authentication for local J ava clients, remote J ava clients, and .NET clients. DFS SOAP clients that do not
use the support classes in the SDK can be authenticated using WS-Security headers that comply with the
Kerberos Token Profile 1.1 specification.
Enabling Kerberos during DFS service deployment
To enable Kerberos authentication in remote DFS services, you must add specific libraries to the DFS
archive prior to deploying and configuring server-side JAX-WS handlers in deployment descriptors. This
procedure applies to the core services delivered with the DFS product, and to custom services that are
upgraded to the DFS 6.6 (or later) runtime.
To enable Kerberos support in remote DFS services:
1. Open the services EAR file and locate APP-INF/classes/authorized-service-handler-chain.xml. If you
are deploying a WAR file, locate WEB-INF/classes/authorized-service-handler-chain.xml.
2. Insert a descriptor for the Kerberos Token Profile 1.1 support handler before the Context Local
Registry handler, as shown below, and then save the file.
<handl er - chai ns xml ns=" ht t p: / / j ava. sun. com/ xml / ns/ j avaee" >
<handl er - chai n>
EMC Documentum
Kerberos SSO Authentication
A Detailed Review 23



<handl er >
<handl er - name>Aut hor i zat i on</ handl er - name>
<handl er - cl ass>
com. emc. document um. f s. r t . i mpl . handl er . Aut hor i zat i onHandl er
</ handl er - cl ass>
</ handl er >
<handl er >
<handl er - name>Ker ber os Token Pr of i l e 1. 1 Suppor t </ handl er - name>
<handl er - cl ass>
com. emc. document um. f s. r t . handl er s. Ker ber osTokenSer ver Handl er
</ handl er - cl ass>
</ handl er >
<handl er >
<handl er - name>Cont ext Local Regi st r y</ handl er - name>
<handl er - cl ass>
com. emc. document um. f s. r t . i mpl . handl er . Ser ver Cont ext Handl er
</ handl er - cl ass>
</ handl er >
</ handl er - chai n>
</ handl er - chai ns>
EMC Documentum
3. In the web.xml deployment descriptor, specify env-entry settings (you can insert these anywhere inside
the <web-app>element). If you are deploying an EAR file, you need to modify the web.xml for each
module; for example emc-dfs.ear/services-core.war/WEB-INF/web.xml. The setting for the DFS
service SPN name is mandatory.
<env- ent r y>
<descr i pt i on>
Mandat or y pr oper t y def i ni ng t he SPN of t he DFS modul e ser vi ced by
t he handl er . The SPN i s def i ned at depl oyment t i me, when t he KDC
i s conf i gur ed.
The KDC r eal mi s r equi r ed as par t of t he SPN.
</ descr i pt i on>
<env- ent r y- name>df s. spn</ env- ent r y- name>
<env- ent r y- t ype>j ava. l ang. St r i ng</ env- ent r y- t ype>
<env- ent r y- val ue>DFS/ myhost . mydomai n. com: 8080@SRV01. COM</ env- ent r y-
val ue>
</ env- ent r y>
<env- ent r y>
<descr i pt i on>
Opt i onal , pat h and name of t he J AAS conf i g f i l e.
I f not speci f i ed her e, t hi s l ocat i on can be set i n a J VM command
l i ne par amet er :
Dj ava. secur i t y. aut h. l ogi n. conf i g=/ pat h/ t o/ J AAS. conf i g
</ descr i pt i on>
<env- ent r y- name>j aas. conf i g</ env- ent r y- name>
<env- ent r y- t ype>j ava. l ang. St r i ng</ env- ent r y- t ype>
<env- ent r y- val ue>c: / kr bCl i ent . conf </ env- ent r y- val ue>
</ env- ent r y>
<env- ent r y>
<descr i pt i on>
Opt i onal , pat h and name of t he Ker ber os conf i g f i l e. By def aul t ,
t he l ogi n modul e t r i es t o l ocat e i t i n:
1. The f i l e r ef er enced by t he J ava pr oper t y
j ava. secur i t y. kr b5. conf
2. $j ava. home/ l i b/ secur i t y/ kr b5. conf
3. c: \ wi nnt \ kr b5. i ni on Mi cr osof t Wi ndows pl at f or ms
4. / et c/ kr b5/ kr b5. conf on UNI X pl at f or ms
Kerberos SSO Authentication
A Detailed Review 24



5. / et c/ kr b5. conf on Li nux pl at f or ms.
</ descr i pt i on>
<env- ent r y- name>kr b5. conf i g</ env- ent r y- name>
<env- ent r y- t ype>j ava. l ang. St r i ng</ env- ent r y- t ype>
<env- ent r y- val ue>c: / wi nnt / kr b5. i ni </ env- ent r y- val ue>
</ env- ent r y>
4. Add krbutil.jar and jcifs-krb5-1.3.1.jar from the SDK to APP-INF/lib (in EAR files) or WEB-INF.lib
(in WAR files). Add commons-codec-1.3.jar and commons-lang-2.4.jar to the same location if they are
not already there.
5. Repackage and redeploy the archive.
Kerberos and JAAS configuration
To enable Kerberos support on the DFS server, the following files are required:
Kerberos configuration file
Kerberos keytab file
J AAS configuration
Kerberos configuration
The Kerberos configuration file must include Kerberos-specific settings such as the KDC address and
default realm name. The realm name, shown below, includes the KDC and administration server addresses.
[ l i bdef aul t s]
def aul t _r eal m= SRV01. COM
f or war dabl e = t r ue
t i cket _l i f et i me = 24h
cl ockskew = 72000

[ r eal ms]
SRV01. COM = {
kdc = myhost . mydomai n. com
admi n_ser ver = myhost . mydomai n. com
}
You can specify the location of this file in the web.xml deployment descriptors, or you can place it in a
location that you choose, as described in the sample web.xml shown in the section Configuring the SPN
and keytab file for DFS services.
Kerberos keytab file
The Kerberos keytab file is generated on the Active Directory using the Windows Server ktpass utility.
This file is then copied to the DFS server machine (the machine where the Kerberos ST will be validated).
You ,must specify the location of the keytab file in the JAAS configuration, as described in J AAS
configuration section.
J AAS configuration
The J AAS configuration file entry contains J AAS specific settings such as the LoginContext name (which
is also the name of the configuration entry), settings for the Kerberos login module, the DFS Service
Principal Name, and the location of the keytab file.
The location and format of the J AAS configuration settings vary depending on the application server
environment. Unless otherwise specified, you can specify a configuration file setting in a J VM command
EMC Documentum
Kerberos SSO Authentication
A Detailed Review 25



line parameter like -Djava.security.auth.login.config=/path/to/J AAS.config, or in a web.xml env-entry, as
described in the Enabling Kerberos during DFS service deployment section.
In each of the configuration files, the LoginContext corresponds with the DFS SPN by replacing separator
characters in the SPN with hyphen characters, and omitting the @realm segment. For example, with this
SPN:
DFS/ myhost . mydomai n. com: 8080@SRV01. COM
the J AAS configuration entry name (that is, the LoginContext name) will be:
DFS- myhost - mydomai n- com- 8080

Make sure that the SPN in the J AAS configuration matches the SPN defined in web.xml. For more
information, see the Enabling Kerberos during DFS service deployment section..
The following sections provide details specific to the application servers that you can use.
Tomcat and WebLogic
Tomcat and WebLogic will accept the following format for the J AAS configuration:
DFS- myhost - mydomai n- com- 8080
{
com. sun. secur i t y. aut h. modul e. Kr b5Logi nModul e r equi r ed
debug=t r ue
pr i nci pal =" DFS/ myhost . mydomai n. com: 8080@SRV01. COM"
r ef r eshKr b5Conf i g=t r ue
useKeyTab=t r ue
st or eKey=t r ue
doNot Pr ompt =t r ue
useTi cket Cache=f al se
i sI ni t i at or =f al se
keyTab=" C: / df suser . keyt ab" ;
};

JBoss
In the J Boss application server, J AAS configuration settings are provided in a file called login-config.xml
in a directory similar to J BOSS_HOME/server/DctmServer_DFS/. The settings have the following format:
<appl i cat i on- pol i cy name = " DFS- myhost - mydomai n- com- 8080" >
<aut hent i cat i on>
<l ogi n- modul e code = " com. sun. secur i t y. aut h. modul e. Kr b5Logi nModul e"
f l ag = " r equi r ed" >
<modul e- opt i on name =
" pr i nci pal " >DFS/ myhost . mydomai n. com: 8080@SRV01. COM</ modul e- opt i on>
<modul e- opt i on name = " useKeyTab" >t r ue</ modul e- opt i on>
<modul e- opt i on name = " st or eKey" >t r ue</ modul e- opt i on>
<modul e- opt i on name = " doNot Pr ompt " >t r ue</ modul e- opt i on>
<modul e- opt i on name = " useTi cket Cache" >f al se</ modul e- opt i on>
<modul e- opt i on name = " i sI ni t i at or " >f al se</ modul e- opt i on>
<modul e- opt i on name = " keyTab" >C: / df suser . keyt ab</ modul e- opt i on>
</ l ogi n- modul e>
</ aut hent i cat i on>
</ appl i cat i on- pol i cy>

WebSphere
The WebSphere J AAS configuration file is named wsjaas.conf and is located in a directory similar to
WEBSPHERE_HOME/AppServer1/profiles/AppSrv01/properties/. WebSphere does not use the
-Djava.security.auth.login.config system property to point to an alternative J AAS configuration file;
EMC Documentum
Kerberos SSO Authentication
A Detailed Review 26



however, you can add and modify a new JAAS login configuration using the administrative console, as
described in the IBM WebSphere documentation.
You can also perform a WebSphere J AAS configuration using the Administrative Console.
With WebSphere 7, you can navigate to the J AAS configuration dialog box using Security >Global
security >Java Authentication and Authorization Service.
With WebSphere 6, you can navigate using Security >Secure administration, applications, and
infrastructure >Java Authentication and Authorization Service.
The Kerberos entry has the following format:
DFS- myhost - mydomai n- com- 8080
{
com. i bm. secur i t y. aut h. modul e. Kr b5Logi nModul e r equi r ed
debug=t r ue
cr edsType=" bot h"
useKeyt ab=" FI LE: c: / ker ber os. keyt ab"
pr i nci pal =" DFS/ myhost . mydomai n. com: 8080@SRV01. COM" ;
};
With WebSphere you also need to add the following line to the Kerberos configuration file:
[ l i bdef aul t s]
def aul t _keyt ab_name = FI LE: c: / ker ber os. keyt ab
Using Kerberos authentication in DFS clients
The DFS Kerberos API deals specifically with transferring authentication information to the DFS service,
using either a remote web service call or a local J ava API call. The API does not obtain Kerberos tickets
from the Kerberos Key Distribution Center (KDC). Because DFS applications are multi-tiered, Kerberos
integration is based on delegated authentication. You must be able to forward all Kerberos tokens provided
to DFS through the API web services. The local J ava API accepts only Kerberos Ticket Granting Tickets
(TGTs).
This section focuses on using the DFS Kerberos API to integrate DFS-based local and remote DFS users
with DFS services that interact with Content Server instances that are enabled for Kerberos authentication.
General information about Kerberos, as well as details about obtaining service tickets from a Kerberos
KDC, are outside the scope of this documentation. For more information pertaining to Kerberos, see the
information on the following websites.
For general information on Kerberos, refer to:
http://web.mit.edu/Kerberos/
http://technet.microsoft.com/en-us/library/bb742516.aspx

For information on the J ava GSS API:
http://java.sun.com/products/jndi/tutorial/ldap/security/gssapi.html
http://java.sun.com/j2se/1.5.0/docs/guide/security/jgss/tutorials/index.html

For additional information on Kerberos single sign-on in J ava refer to
http://java.sun.com/j2se/1.5.0/docs/guide/security/jgss/single-signon.html

EMC Documentum
Kerberos SSO Authentication
A Detailed Review 27



Kerberos authentication in a local DFS web application
A local DFS application enables the DFS client and the DFS service to run within the same J ava virtual
machine. This type of consumer requires that you use the DFS client productivity layer. For this type of
application, DFS supports Kerberos authentication by providing a BinaryIdentity class that encapsulates a
Kerberos credential:
/ **
* Bi nar yI dent i t y i s not XML ser i al i zabl e and wi l l not be sent over t he
wi r e.
*/
publ i c cl ass Bi nar yI dent i t y ext ends I dent i t y
publ i c Bi nar yI dent i t y( Obj ect cr edent i al ,
Bi nar yI dent i t y. Cr edent i al Type cr edent i al Type)

A BinaryIdentity would typically be populated with a Ticket Granting Ticket and then added to a list of
identities in the service context:
I Obj ect Ser vi ce ser vi ce = Ser vi ceFact or y. get I nst ance( )
. get Local Ser vi ce( I Obj ect Ser vi ce. cl ass, cont ext ) ;

Obj ect t gt = . . . ;
ser vi ce. get Ser vi ceCont ext ( )
. set I dent i t i es( Ar r ays
. asLi st ( ( I dent i t y) new Bi nar yI dent i t y( t gt ,
Bi nar yI dent i t y. Cr edent i al Type. KERBEROS_TGT) ) ) ;
ser vi ce. cr eat e( . . . ) ;

The Kerberos utility package and the com.emc.documentum.kerberos.utility support users installing
Kerberos authentication in local DFS applications. To use this utility, add krbutil.jar and jcifs-krb5-1.3.1.jar
from the DFS SDK to your project classpath.
Figure 13 illustrates a mainstream scenario for using local DFS services and Kerberos authentication in a
web application.
EMC Documentum
Kerberos SSO Authentication
A Detailed Review 28




Figure 13. Web application using local DFS and Kerberos authentication
In steps 14 in Figure 13, a browser client obtains a service ticket from the KDC and passes it to the web
application as a SPNEGO token.
Steps 5 through 7 are critical steps for DFS support.
In step 5, the web application uses Kerberos utility static methods to extract the ST from the SPNEGO
token. In step 6, the web application calls the Kerberos utility again to accept the ST and obtain a Ticket
Granting Ticket (TGT). You can perform these steps with a helper method like the following:
publ i c St r i ng get DFSTGTf r omSPNEGOToken( St r i ng SPNEGO) t hr ows
GSSExcept i on
{
St r i ng df s_st =
Ker ber osUt i l i t y. get STFr omSpenegoToken( SPNEGO) ;
i f ( df s_st ! = nul l )
{
r et ur n Ker ber osUt i l i t y. accept ( m_sour ce_spn,
df s_st ) ;
}
r et ur n nul l ;
}
Here m_source_spn is a string containing the SPN of the service to be accepted (that is, the SPN of the web
application). The result is a java.lang.Object representing the TGT that you can pass to the DFS local API.
In step 7, the web client instantiates a BinaryIdentity using the result returned by the Kerberos utility and
sets the identity in serviceContext.
Note: Registration of the service context containing the Kerberos credentials is not currently supported.
I Obj ect Ser vi ce ser vi ce = Ser vi ceFact or y. get I nst ance( )
. get Local Ser vi ce( I Obj ect Ser vi ce. cl ass, cont ext ) ;

EMC Documentum
Obj ect t gt = . . . ;
Kerberos SSO Authentication
A Detailed Review 29



ser vi ce. get Ser vi ceCont ext ( ) .
set I dent i t i es( Ar r ays. asLi st ( ( I dent i t y)
new Bi nar yI dent i t y( t gt ,
Bi nar yI dent i t y. Cr edent i al Type. KERBEROS_TGT) ) ) ;

In steps 8 and later, DFC uses the TGT to obtain STs from the Kerberos utility for every repository
involved in the operation. These STs have the same login information as the original ST received from the
client, and use Kerberos delegation (provided by the Kerberos utility) to enable the Content Server to
authenticate the credentials. (Since the DFS runtime initiates these steps, you do not need to use any code.)
Kerberos keytab file, JAAS configuration, and Kerberos configuration
As described in the Kerberos keytab file section, the keytab file of the web application must be available
for the ST to be accepted. This is because the DFS service invocation happens inside a web application that
has an SPN defined. When the ST is accepted, the DFS local J ava can receive the resulting TGT in a
BinaryIdentity.
The location of the keytab file on the server is specified in the JAAS configuration file, which needs to be
configured in the application server; the configuration instructions for JAAS vary depending on the
application server. DFC also requires a Kerberos configuration file. For more information, refer to the
Documentum Foundation Services Deployment Guide.
Kerberos authentication in a remote DFS client
Remote DFS consumer communication between client and server relies on SOAP, and login information is
sent over the wire stored in SOAP WS-Security headers. As in the local case, we assume that the client has
already obtained a service ticket for the DFS service using user login credentials. This step is not shown in
the following figure. The service ticket is provided as handler-specific input to a J AX-WS handler or WCF
behavior.

Figure 14. Kerberos authentication in a remote DFS client application
EMC Documentum
Kerberos SSO Authentication
A Detailed Review 30



In Step 1 in Figure 14, you should assume that the DFS is already in possession of the ST obtained from the
KDC. If the DFS client is using the DFS SDK libraries, the DFS client sets the ST in a client-side J AX-WS
handler or WCF behavior. The J AX-WS handler or WCF behavior serializes the Kerberos service ticket in
the SOAP WS-Security header. On the server, server-side J AX-WS handlers validate the Kerberos service
ticket using the Kerberos utility (steps 3 and 4), and pass the ticket to the DFC layer for authentication on
the Content Server (steps 59).
From a DFS integration perspective, the main responsibility of the DFS consumer is to provide the ST that
it has obtained from the KDC for the DFS service to client-side J AX-WS handlers (J ava) or WCF
behaviors (.NET). The following sections describe the APIs that the DFS SDK for J ava and .NET
productivity layer clients to handle Kerberos tickets. AX-WS and WCF clients that do not use the
productivity layer can use the custom J AX-WS SOAP handler or WCF endpoint behavior provided in the
DFS SDK. Other types of SOAP clients will need to ensure that the Kerberos ticket is contained in the WS-
Security as defined in the Oasis Kerberos Token Profile 1.1 specification. A SOAP sample excerpted from
this specification is shown in the Kerberos Token 1.1 security header section.
DFS Kerberos remote Java API
To support Kerberos authentication in a J ava remote (SOAP) client, the DFS SDK provides a set of support
classes in the com.emc.documentum.fs.rt.handlers package (for details regarding these classes, refer to the
DFS javadocs). The classes enable a client to create an instance of a JAX-WS SOAP handler and add it to
the DFS client service object so that it can be invoked by the framework during client-side SOAP
processing, as shown in this example:
Ker ber osTokenHandl er handl er = new Ker ber osTokenHandl er ( ) ;
I Obj ect Ser vi ce ser vi ce = Ser vi ceFact or y
. get I nst ance( ) . get Remot eSer vi ce( . . . , cont ext Root ,
Ar r ays. asLi st ( ( Handl er ) handl er ) ) ;

byt e[ ] t i cket = . . . ;
handl er . set Bi nar ySecur i t yToken(
new Ker ber osBi nar ySecur i t yToken( t i cket ,
Ker ber osVal ueType. KERBEROSV5_AP_REQ) ) ;
ser vi ce. cr eat e( . . . )

The getRemoteService method is overloaded so that it can pass a list of JAX-WS handlers that the
framework will invoke when creating the SOAP message.
Note: A J AX-WS client that does not use the full DFS productivity layer could also use the
KerberosTokenHandler to add serialization of the Kerberos token to J AX-WS SOAP processing, by adding it to
the handler chain without using the getRemoteService productivity-layer method.

DFS Kerberos remote .NET API
To support Kerberos authentication in a Windows Communication Framework (WCF) client, the DFS SDK
provides a set of support classes in the Emc.Documentum.FS.Runtime.Behaviors namespace. (For details
on these classes refer to the CHM documents in the SDK.) The classes enable a client to create an instance
of the WCF behavior and add it to the DFS client service object so that it can be invoked by the framework
during client-side SOAP processing, as shown in the following example:
Ker ber osTokenHandl er handl er = new Ker ber osTokenHandl er ( ) ;
Li st <I Endpoi nt Behavi or > handl er s = new Li st <I Endpoi nt Behavi or >( ) ;
handl er s. Add( handl er ) ;
I Obj ect Ser vi ce ser vi ce = Ser vi ceFact or y
. I nst ance. Get Remot eSer vi ce<I Obj ect Ser vi ce>( . . . , cont ext Root ,
handl er s) ;

byt e[ ] t i cket = . . . ;
handl er . Set Bi nar ySecur i t yToken(
EMC Documentum
Kerberos SSO Authentication
A Detailed Review 31



new Ker ber osBi nar ySecur i t yToken( t i cket ,
Ker ber osVal ueType. GSS_KERBEROSV5_AP_REQ) ) ;
ser vi ce. cr eat e( . . . ) ;

The getRemoteService method has been overloaded so that it can pass a list of custom behaviors that the
framework will invoke when creating the SOAP message.
Note: A WCF client that does not use the full DFS productivity layer could also use the KerberosTokenHandler to
add serialization of the Kerberos token to WCF SOAP processing, by adding the custom endpoint behavior
without using the getRemoteService productivity-layer method.

Kerberos Token 1.1 security header
The following example illustrates how a Kerberos ticket is serialized in the SOAP WS-Security header as
described in the Oasis Kerberos Token Profile 1.1 specification. The ticket must be Base64 encoded.
<S11: Envel ope xml ns: S11=" . . . " xml ns: wsu=" . . . " >
<S11: Header >
<wsse: Secur i t y xml ns: wsse=" . . . " >
<wsse: Bi nar ySecur i t yToken
Encodi ngType=" ht t p: / / docs. oasi s- open. or g/ wss/ 2004/ 01/
oasi s- 200401- wss- soap- message178secur i t y- 1. 0#Base64Bi nar y"
Val ueType=" ht t p: / / docs. oasi s179open. or g/ wss/
oasi s- wss- ker ber os- t oken- pr of i l e- 1. 1#Ker ber osv5_AP_REQ"
wsu: I d=" MyToken" >boI BxDCCAcCgAwI BBaEDAgEOogcD. . .
</ wsse: Bi nar ySecur i t yToken>
. . .
</ wsse: Secur i t y>
</ S11: Header >
<S11: Body>
. . .
</ S11: Body>
</ S11: Envel ope>

Enabling DFS JAX-WS handlers for Kerberos
A remote DFS consumer that uses Kerberos requires that the J AX-WS Kerberos handlers be enabled in the
DFS web application. You must do this during DFS deployment by modifying the authorized-service-
handler-chain.xml deployment descriptor and by adding some required jar files to the DFS web application.
For instructions on how to do this, refer to the Documentum Foundation Services Deployment Guide.
Other server configuration requirements
In the application illustrated in the preceding diagram, the server-side J AX-WS handlers call the Kerberos
utility to accept the ST provided by the client. In order to accept the ST, the Kerberos utility requires that
the keytab file containing the DFS SPN be available.
The location of the keytab file on the server is specified in the J AAS configuration file. This file needs to
be configured in the application server; the configuration instructions for JAAS vary depending on the
application server you are using.
In addition, you also need a Kerberos configuration file (krb5.ini); this file is either placed in a well-known
location or referenced in an application deployment descriptor.
For more detailed information on the Kerberos keytab file, J AAS configuration, and krb5.ini, refer to the
Documentum Foundation Services Deployment Guide.

EMC Documentum
Kerberos SSO Authentication
A Detailed Review 32



Enabling Kerberos for WDK-based applications
The Kerberos SSO authentication scheme is used to authenticate the user who wants to log in to a WDK-
based web application from a computer that is in the Kerberos domain.
When Kerberos-based single sign-on authentication is enabled on Webtop or any custom WDK application,
users of Documentum Webtop or custom WDK-based applications are automatically authenticated and
logged in to the repository using their credentials stored in the users privatecredential area on the Windows
platform. Unlike other SSO solutions where users must specify usernames and passwords to validate their
credentials on the policy server, the Kerberos-based single sign-on authentication does not pose any
authentication challenge to the user. The only time the users credentials are authenticated is when the user
logs in to the local machine using the Windows domain credentials. In this manner, the user can log in to
any WDK-based application having logged in to the local computer.
To enable Kerberos authentication for WDK-based applications, such as Webtop, you must create or
modify configuration files on the application server. The client machine should already be configured to
use Kerberos authentication before the system has enabled Kerberos-based authentication.
Prerequisites
Deploy Webtop on the application server machine, and connect to a Content Server that has been
configured for Kerberos SSO authentication. For more information about deploying Webtop on an
application server see the Documentum Web Development Kit and Webtop Deployment Guide. For
more information about configuring Content Server for Kerberos SSO authentication, see the Content
Server Installation Guide, Content Server Administration Guide, and the Documentum Administrator
User Guide.
Install IE6 or IE7 or IE8 or Firefox 2.x or Firefox 3.0.x or Firefox 3.5.x on the client machine.
Register Webtop as a Service Principal in the Key Distribution Center (KDC). Refer to the Create
user account for Webtop in the Active Directory section for more information on registering Webtop
as a Service Principal in the KDC.
Preparing the client machine and the browser to meet Kerberos
SSO setup requirements
You must ensure that the client machine is already configured to use Kerberos authentication before you
prepare the system for enabling Kerberos-based authentication.
On the client machine, if the DNS is not configured to resolve the fully qualified hostname to the
application server IP address, you must edit the %WI NDI R%/ syst em32/ dr i ver s/ et c/ host s file
and map the application server IP to its fully qualified domain name.
Webtop uses the browsers SPNEGO support to implement Kerberos SSO. In this case, the browser
requests a service ticket from the KDC for a specific application server. The browser prepares the SPN in
the following format: HTTP/fully qualified URL@REALM. For example, if the application server URL is
wdkapps.wdkblr.com and the realm is WDKBLR.COM, then the browser-framed SPN is formed as
follows: HTTP/wdkapps.wdkblr.com@WDKBLR.COM.
You must configure the browser on the client machine from where you will access the WDK-based
applications to use the SPNEGO protocol.
To configure Internet Explorer:
1. Log in to the Windows active directory domain.
2. Start Internet Explorer (IE).
3. Select Tools >Internet Options. The Internet Options dialog box is displayed.
EMC Documentum
Kerberos SSO Authentication
A Detailed Review 33



4. Click the Security tab.
5. For IE7 and IE8, select the Uncheck Enable Protected Mode checkbox for the Internet and Local
Intranet Web content zones.
6. Select the Local intranet icon and click Sites. The Local intranet dialog box is displayed.

Figure 15. Selecting the option to include all local sites in the local intranet zone
7. Ensure that the Include all local (intranet) sites not listed in other zones checkbox is selected, and
then click Advanced. The Local intranet dialog box is displayed.
8. Specify the Webtop URL in the Add this Web site to the zone field, and click Add. The Webtop
URL is added to the list of Web sites (for example, http://wdkapps.wdkblr.com).

Figure 16. Adding the Webtop URL to the list of websites
9. Click OK twice.
10. In the Internet Options dialog box, select the Advanced tab, and scroll to Security settings.
EMC Documentum
Kerberos SSO Authentication
A Detailed Review 34




Figure 17. Security tab in the Advanced tab of the Internet Options dialog box
11. Select the Enable Integrated Windows Authentication (requires restart) checkbox.

Figure 18. Enabling Integrated Windows Authentication
12. Click OK.
13. Restart the IE browser to activate the configuration.
To configure Firefox:
1. Start Mozilla Firefox.
EMC Documentum
Kerberos SSO Authentication
A Detailed Review 35



2. Type about:config in the Address field. The about:config screen is displayed with a list of
Preferences.

Figure 19. Configuring Firefox
3. Type network.n in the Filter field.
4. Double-click and edit the value of the network.negotiate-auth.trusted-uris string by
adding the Webtop URL (for example, http://wdkapps.wdkblr.com).
5. Double-click and edit the value of the network.negotiate-auth.delegation-uris string
by adding the Webtop URL (for example, http://wdkapps.wdkblr.com).
6. Restart the Firefox browser to activate the configuration.
Creating the JAAS configuration file
The new KerberosSSOAuthenticationScheme class uses the J ava JAAS and GSS-API to perform Kerberos
authentication. The administrator must create the JAAS configuration file in the <webtop_app_root>/WEB-
INF folder for the J AAS configuration.
The JAAS configuration process varies across application servers.
Configuring the Tomcat application server
To configure the Tomcat application server:
1. Install the Tomcat application server.
2. Deploy the webt op. war file and configure the df c. pr oper t i es file.
3. Copy the webt op. keyt ab file generated using the ktpass command to the <web- app-
r oot >/ WEB- I NF folder.
4. Create the kr b5Logi n. conf file in the <web- app- r oot >/ WEB- I NF folder for J AAS with the
following contents:
HTTP- wdkapps- wdkbl r - com
{
com. sun. secur i t y. aut h. modul e. Kr b5Logi nModul e r equi r ed
debug=t r ue
pr i nci pal =" HTTP/ wdkapps. wdkbl r . com@WDKBLR. COM"
r ef r eshKr b5Conf i g=t r ue
useKeyTab=t r ue
EMC Documentum
Kerberos SSO Authentication
A Detailed Review 36



st or eKey=t r ue
doNot Pr ompt =t r ue
useTi cket Cache=f al se
i sI ni t i at or =f al se
keyTab=" <web- app- r oot >/ WEB- I NF/ webt op. keyt ab" ;
};
5. Modify the kr b5Logi n. conf file with the following details:
The login context name (HTTP-wdkapps-wdkblr-com) must match the registered SPN. Replace / and
. with - in the SPN to get the login context name.
Set the principal to SPN.
The keytab file must point to the location of the webtop.keytab in the WEB- I NF folder of Tomcat.
6. Create the kr b5. i ni file in the %WI NDI R%folder with the following contents:
[ l i bdef aul t s]
def aul t _r eal m= WDKBLR. COM
f or war dabl e = t r ue
t i cket _l i f et i me = 24h
cl ockskew = 72000
def aul t _t kt _enct ypes = des- cbc- md5 des- cbc- cr c des3- cbc- sha1
def aul t _t gs_enct ypes = des- cbc- md5 des- cbc- cr c des3- cbc- sha1
per mi t t ed_enct ypes = des- cbc- md5 des- cbc- cr c des3- cbc- sha1

[ r eal ms]
WDKBLR. COM= {
kdc = WDKWI N5175. WDKBLR. COM
admi n_ser ver = WDKWI N5175. WDKBLR. COM
}
7. Modify the kr b5. i ni file with the following details:
default_realm - Specify the Kerberos domain name
The realms section must point to the KDC server
8. Edit the cat al i na. bat file of Tomcat and specify the following J ava options:
set J AVA_OPTS=%J AVA_OPTS%- Dj ava. secur i t y. kr b5. conf =%WI NDI R%/ kr b5. i ni
- Dj ava. secur i t y. aut h. l ogi n. conf i g=<web- app- r oot >/ WEB-
I NF/ kr b5Logi n. conf
- Dj avax. secur i t y. aut h. useSubj ect Cr edsOnl y=f al se
Configuring the WebLogic application server
To configure the WebLogic application server:
1. Install the WebLogic application server.
2. Deploy the webt op. war file and configure the df c. pr oper t i es file.
3. Copy the webt op. keyt ab file generated using the ktpass command to the <web- app-
r oot >/ WEB- I NF folder.
4. Navigate to the <WebLogic_Home>\user_projects\domains\<Your_domain>\bin\setDomainEnv.cmd
file or the setDomainEnv.sh file of the WebLogic application server, and add the following
configuration:
EMC Documentum
Kerberos SSO Authentication
A Detailed Review 37



- Dj ava. secur i t y. kr b5. conf =<l ocat i on of kr b5. i ni > -
Dj ava. secur i t y. aut h. l ogi n. conf i g=<l ocat i on of kr b5Logi n. conf > -
Dj avax. secur i t y. aut h. useSubj ect Cr edsOnl y=f al se
5. Create the kr b5Logi n. conf file in the <web- app- r oot >/ WEB- I NF folder for J AAS with the
following contents:
HTTP- wdkapps- wdkbl r - com
{
com. sun. secur i t y. aut h. modul e. Kr b5Logi nModul e r equi r ed
debug=t r ue
pr i nci pal =" HTTP/ wdkapps. wdkbl r . com@WDKBLR. COM"
r ef r eshKr b5Conf i g=t r ue
useKeyTab=t r ue
st or eKey=t r ue
doNot Pr ompt =t r ue
useTi cket Cache=f al se
i sI ni t i at or =f al se
keyTab=" <web- app- r oot >/ WEB- I NF/ webt op. keyt ab" ;
};
6. Modify the kr b5Logi n. conf file with the following details:
The login context name (HTTP-wdkapps-wdkblr-com) must match the registered SPN. Replace / and
. with - in the SPN to get the login context name.
Set the principal to SPN.
The keytab file must point to the location of the webt op. keyt ab file in the WEB- I NF folder of
Tomcat.
7. Create the kr b5. i ni file in the %WI NDI R%folder with the following contents:
[ l i bdef aul t s]
def aul t _r eal m= WDKBLR. COM
f or war dabl e = t r ue
t i cket _l i f et i me = 24h
cl ockskew = 72000
def aul t _t kt _enct ypes = des- cbc- md5 des- cbc- cr c des3- cbc- sha1
def aul t _t gs_enct ypes = des- cbc- md5 des- cbc- cr c des3- cbc- sha1
per mi t t ed_enct ypes = des- cbc- md5 des- cbc- cr c des3- cbc- sha1

[ r eal ms]
WDKBLR. COM= {
kdc = WDKWI N5175. WDKBLR. COM
admi n_ser ver = WDKWI N5175. WDKBLR. COM
}
8. Modify the kr b5. i ni file with the following details:
default_realm - Specify the Kerberos domain name
The realms section must point to the KDC server
9. Navigate to the
<WebLogi c_Home>\ user _pr oj ect s\ domai ns\ <Your _domai n>\ bi n\ set Domai nEnv.
cmd file of the WebLogic application server, and add the following configuration:
- Dj ava. secur i t y. kr b5. conf =<l ocat i on of kr b5. i ni > -
Dj ava. secur i t y. aut h. l ogi n. conf i g=
<l ocat i on of kr b5Logi n. conf > -
Dj avax. secur i t y. aut h. useSubj ect Cr edsOnl y=f al se
EMC Documentum
Kerberos SSO Authentication
A Detailed Review 38



Configuring the J Boss application server
To configure the J Boss application server:
1. Install the J Boss application server.
2. Deploy the webt op. war file on the production server and configure the df c. pr oper t i es file.
3. Copy the webt op. keyt ab file generated using the ktpass command to the <web-app-root>/WEB-
INF folder.
4. Edit the l ogi n- conf i g. xml file in the %J BOSS_HOME%\ ser ver \ pr oduct i on\ conf
folder and add the following contents:
<appl i cat i on- pol i cy name=" HTTP- wdkapps- wdkbl r - com" >
<aut hent i cat i on>
<l ogi n- modul e code=" com. sun. secur i t y. aut h. modul e. Kr b5Logi nModul e"
f l ag=" r equi r ed" >
<modul e- opt i on name=" debug" >t r ue</ modul e- opt i on>
<modul e- opt i on name=" st or eKey" >t r ue</ modul e- opt i on>
<modul e- opt i on name=" useKeyTab" >t r ue</ modul e- opt i on>
<modul e- opt i on name=" keyTab" >C: / j boss- eap- 4. 3/ j boss-
as/ ser ver / pr oduct i on/ depl oy
/ webt op. war / WEB- I NF/ ker ber osas. keyt ab</ modul e- opt i on>
<modul e- opt i on
name=" pr i nci pal " >HTTP/ wdkapps. wdkbl r . com@WDKBLR. COM>HTTP
/ wdkapps. wdkbl r . com@WDKBLR. COM</ modul e- opt i on>
</ l ogi n- modul e>
</ aut hent i cat i on>
</ appl i cat i on- pol i cy>
5. Modify the following details:
The login context name (HTTP-webtop-wdkblr-com) must match the registered SPN. Replace / and
. with - in the SPN to get the login context name.
Set the principal to SPN.
The keytab file must point to the location of the webt op. keyt ab file in the WEB- I NF folder.
6. Create the kr b5. i ni file in the %WI NDI R%folder with the following contents:
[ l i bdef aul t s]
def aul t _r eal m= WDKBLR. COM
f or war dabl e = t r ue
t i cket _l i f et i me = 24h
cl ockskew = 72000
def aul t _t kt _enct ypes = des- cbc- md5 des- cbc- cr c des3- cbc- sha1
def aul t _t gs_enct ypes = des- cbc- md5 des- cbc- cr c des3- cbc- sha1
per mi t t ed_enct ypes = des- cbc- md5 des- cbc- cr c des3- cbc- sha1

[ r eal ms]
WDKBLR. COM= {
kdc = WDKWI N5175. WDKBLR. COM
admi n_ser ver = WDKWI N5175. WDKBLR. COM
}
7. Modify the kr b5. i ni file with the following details:
default_realm - Specify the Kerberos domain name
The realms section must point to the KDC server
EMC Documentum
Kerberos SSO Authentication
A Detailed Review 39



Configuring the WebSphere application server
To configure the WebSphere application server:
1. Install the WebSphere application server.
2. Deploy the webt op. war file and configure the df c. pr oper t i es file.
3. Copy the webt op. keyt ab file generated using the ktpass command to the <web- app-
r oot >/ WEB- I NF folder.
4. Navigate to the
<WebSpher e_Home>\ AppSer ver \ pr of i l es\ AppSr v01\ pr oper t i es\ wsj aas. conf
file and add the following configuration:
HTTP- host Name- r eal m_Name { com. i bm. secur i t y. aut h. modul e. Kr b5Logi nModul e
r equi r ed debug=t r ue cr edsType=" bot h" useKeyt ab=" f i l e: f ul l Pat hToKeyt abf i l e"
pr i nci pal =" HTTP/ host Name. r eal mName" ; };
5. Create a configuration file to specify the KDC server to which the application server should connect, in
the %WINDIR% folder for the Windows operating system, or in the /etc/krb5 folder for the AIX
operating system. The names of the configuration files in the Windows operating system and the AIX
operating system are krb5.ini, and krb5.conf respectively. To support Advanced Encryption Standard
(AES) in the WebSphere application server, specify aes128-cts-hmac-sha1-96 as a permitted
encryption type, as shown below:
[ l i bdef aul t s]
def aul t _r eal m= WDKBLR. COM
f or war dabl e = t r ue
t i cket _l i f et i me = 24h
cl ockskew = 72000
def aul t _t kt _enct ypes = aes128- ct s aes128- ct s- hmac- sha1- 96 des3- cbc-
sha1
des- cbc- md5 des- cbc- cr c
def aul t _t gs_enct ypes = aes128- ct s aes128- ct s- hmac- sha1- 96 des3- cbc-
sha1
des- cbc- md5 des- cbc- cr c
per mi t t ed_enct ypes = aes128- ct s aes128- ct s- hmac- sha1- 96 des3- cbc- sha1
des- cbc- md5 des- cbc- cr c
[ r eal ms]
WDKBLR. COM= {
kdc = WDKWI N5175. WDKBLR. COM
admi n_ser ver = WDKWI N5175. WDKBLR. COM
}
Configuring the custom/app.xml file to enable Kerberos
authentication
Copy the <aut hent i cat i on>tag from the wdk/ app. xml file into the cust om/ app. xml file and
perform the configurations discussed in this section in the <enabl ed>and <domai n>tags within the
<aut hent i cat i on>tag, to enable Kerberos authentication in WDK-based applications.
Enabling Kerberos SSO authentication in WDK-based applications
An application-level setting is provided in cust om/ app. xml within the <aut hent i cat i on>tag to
enable or disable Kerberos-based SSO authentication. The default value defined for the <enabl ed> tag
EMC Documentum
Kerberos SSO Authentication
A Detailed Review 40



in the <kerberos_sso>element is false. Set the <enabled>tag to true to enable Kerberos SSO
authentication in WDK-based applications.
<ker ber os_sso>
<enabl ed>t r ue</ enabl ed>
</ ker ber os_sso>
Configuring the Kerberos domain name
An application-level tag is provided to specify the Kerberos domain, within the <aut hent i cat i on>
tag. Enter the domain name in the <domai n>tag.
<ker ber os_sso>
<domai n><domai n_name></ domai n>
</ ker ber os_sso>
Configuring Kerberos fallback
The Kerberos SSO Authentication Scheme provides the option for falling back to the default login
mechanism for the web application when a failure occurs. Set the <docbase_l ogi n_f al l back>tag
in the <ker ber os_sso>tag in wdk/ app. xml , to support the default login to the web application, as
follows:
<docbase_l ogi n_f al l back>t r ue</ docbase_l ogi n_f al l back>

The default value of the <docbase_l ogi n_f al l back>tag is false.
Copy the <docbase_l ogi n_f al l back>element into the <ker ber os_sso>tag in
cust om/ app. xml .
Sample Kerberos configuration in custom/app.xml
The following code snippet is an example of the final configuration for Kerberos in cust om/ app. xml .
<aut hent i cat i on>
<! - - Ker ber os SSO aut hent i cat i on scheme conf i gur at i on - - >
<ker ber os_sso>
<enabl ed>t r ue</ enabl ed>
<br owser s>
<wi ndows>
<i ever si ons>6. 0, 7. 0, 8. 0</ i ever si ons>
<f i r ef oxver si ons>2. 0, 3. 0, 3. 5</ f i r ef oxver si ons>
</ wi ndows>
</ br owser s>
<! - - Enabl e l ogi n f al l back t o DocbaseLogi n scheme - - >
<docbase_l ogi n_f al l back>f al se</ docbase_l ogi n_f al l back>
<! - - Mandat or y conf i gur at i on: Pr ovi de t he ker ber os r eal m
/ domai n name. - - >
<domai n>WDKBLR. COM</ domai n>
</ ker ber os_sso>
</ aut hent i cat i on>

Copy t he <aut hent i cat i on> t ag f r omt he wdk/ app. xml f i l e i nt o t he
cust om/ app. xml f i l e.
Configuring EMC CenterStage to enable Kerberos authentication
EMC CenterStage

supports Kerberos single sign-on authentication. Complete instructions for configuring


Kerberos for use with Content Server are provided in the relevant sections of this white paper. This section
EMC Documentum
Kerberos SSO Authentication
A Detailed Review 41



discusses specific extensions that must be made to the CenterStage app.xml to enable Kerberos SSO
authentication.

An extensible app.xml configuration file can be located and modified using Documentum Administrator.
The file path is /Cabinets/system/applications/CenterStage Pro/app.xml. You can copy the <authentication>
element from /Cabinets/system/applications/CenterStage/app.xml, paste it into
/Cabinets/system/applications/CenterStage Pro/app.xml, and then modify the settings, or you can retype the
information as shown in <xreflinkend="appXmlExample"/>.

Note: In a distributed system, only the app.xml file in the global repository needs to be updated. The other
repositories in the system will inherit these settings from the global repository.

Update the authentication/kerberos_sso elements as follows:
Set the <enabled>element to true.
Set the <domain>element to your Kerberos domain.
Set the <docbase_login_fallback>element to true. Should there be a failure in the Kerberos
mechanism, the fallback behavior will be to ask the user to log in to the repository.

Consider the following sample authentication code that shows an element from an app.xml file:
<aut hent i cat i on>
<! - - Ker ber os SSO aut hent i cat i on pr ot ocol conf i gur at i on - - >
<ker ber os_sso>

<enabl ed>t r ue&l t ; / enabl ed>

<domai n>DCTMLABS. COM&l t ; / domai n>

<docbase_l ogi n_f al l back>t r ue&l t ; / docbase_l ogi n_f al l back>

</ ker ber os_sso>
</ aut hent i cat i on>
Enabling tracing
You can enable trace messages to troubleshoot Kerberos authentication in WDK-based applications.
To enable tracing:
EMC Documentum
1. Edit the following entries in <web-app-root>/WEB-INF/classes/log4j.properties:
l og4j . r oot Cat egor y=ERROR, f i l e
l og4j . cat egor y. MUTE=OFF
#Enable trace messages from WDK:
l og4j . l ogger . com. document um. web=DEBUG
#stdout is a ConsoleAppender that uses a PatternLayout:
l og4j . appender . st dout . t hr eshol d=ERROR
l og4j . appender . st dout =or g. apache. l og4j . Consol eAppender
l og4j . appender . st dout . l ayout =or g. apache. l og4j . Pat t er nLayout
l og4j . appender . st dout . l ayout . Conver si onPat t er n=%d{ABSOLUTE} %5p
[ %t ] %c - %m%n
#file is a FileAppender that uses a PatternLayout:
l og4j . appender . f i l e=or g. apache. l og4j . Rol l i ngFi l eAppender
l og4j . appender . f i l e. Fi l e=c: / t emp/ wdkt r ace. l og
l og4j . appender . f i l e. MaxFi l eSi ze=10500KB
l og4j . appender . f i l e. MaxBackupI ndex=10
l og4j . appender . f i l e. l ayout =or g. apache. l og4j . Pat t er nLayout
Kerberos SSO Authentication
A Detailed Review 42



l og4j . appender . f i l e. l ayout . Conver si onPat t er n=[ %d{I SO8601}| %- 5p| %-
22t | %C| %M| %- 4L] %m%n
This will send the trace to the "c:/temp/wdktrace.log" file.
2. Edit the <web- app- r oot >/ WEB-
I NF/ cl asses/ com/ document um/ debug/ Tr acePr op. pr oper t i es file by modifying the
following parameters:
Set SESSIONENABLEDBYDEFAULT to true
Set SESSION to true
3. Restart the application server.
4. Clear the browser cache.
5. Start the browser and log in to the WDK-based application.
Note:
- Ensure that the Domain Name and Host Name values do not contain the underscore (_) character.
- By default, Windows 7 does not support DES encryption. You must enable DES encryption explicitly. For
instructions on how to enable DES encryption, see Setting DES, AES128, and RC4 Kerberos encryption types.
Kerberos authentication use cases
There are multiple use cases for Kerberos authentication in WDK-based applications. This section
describes the flow in Webtop use cases.
Client platform/Browser is not supported
The KerberosSSOAuthenticationScheme validates the client platform and browser requirements. If the
client does not match the requirement, then it returns nul l to AuthenticationService, which delegates the
call to the next authentication scheme.
Note: The KerberosSSOAuthenticationScheme does not return the 401:Unauthorized error to the browser. The
authentication is handled by the next appropriate authentication scheme.
All repositories are Kerberos-enabled and the user logs in to the Kerberos
domain
This is the ideal case where all the repositories are Kerberos-enabled and end-user accounts exist in all the
repositories. The following steps describe the Kerberos authentication in such a setup.
1. The user logs in to a machine in the Kerberos domain and the TGT is generated.
2. The user opens the supported browser and accesses the Webtop URL.
3. The KerberosSSOAuthenticationScheme verifies whether or not the user has been authenticated thus
far, and returns the Error 401 with WWW-Authenticate:Negotiate in the http header if necessary.
4. The browser receives the 401 error code and the Negotiate authorization header in the Http Response.
The browser then requests a Service Ticket for Webtop. The browser wraps the Service Ticket in the
SPNEGO token and sends it to Webtop in the http header.
5. Webtop parses the SPNEGO token to get the Service Ticket. Webtop validates the Service Ticket.
EMC Documentum
Kerberos SSO Authentication
A Detailed Review 43



6. Webtop displays the repositories selection page.
7. The user selects the required repository and submits the Login page.
8. The KerberosSSOAuthenticationScheme requests the Service Ticket from KDC using client
credentials for the selected repository.
9. The KerberosSSOAuthenticationScheme passes the Service Ticket of the selected repository to
Content Server using the DFC Session API.
10. The Content Server Kerberos plug-in accepts the Service Ticket and upon successful acceptance, it
returns the repository session to Webtop. Webtop displays the Main listing page.
Client machine is not part of the Kerberos domain
In this use case, all the repositories are Kerberos-enabled. Here, either the client machine is not part of
Kerberos domain or end user is not logged in to the Kerberos domain repositories. The steps below
describe the Kerberos authentication in this setup.
1. The user logs in to a machine that is not in the Kerberos domain.
2. The user opens a supported browser and accesses the Webtop URL.
3. The KerberosSSOAuthenticationScheme verifies whether or not the user has been authenticated thus
far, and returns the Error 401 with WWW-Authenticate:Negotiate in the http header if necessary.
4. The browser receives the 401 error code and the Negotiate authorization header in the http response.
Since the client machine is not logged in to the Kerberos domain, the browser fails to get the Service
Ticket from the KDC.
5. Webtop displays the warning message to the user indicating that Kerberos authentication is not
supported, and provides a link to fall back to the default repository authentication.
6. The user clicks the link and Webtop displays the Webtop Login page.
7. The user specifies the login name and password and then selects the repository.
Note: The username refers to the name of the repository user that is not part of the KDC.
8. The Docbase Authentication Scheme passes the username, password, and repository information to
Content Server using the DFC Session API.
9. Content Server validates the username and password details, and upon successful validation, returns
the repository session to Webtop. Webtop displays the Main page.
Webtop is configured to work with mixed repositories
In this use case, Webtop is configured to work with mixed repositories, some of which are Kerberos-
enabled while the rest are not Kerberos-enabled. In this case, the user should assume that the client machine
is logged in to the Kerberos domain and the end-user account exists in the KDC.
1. The user logs in to a machine in the Kerberos domain, and a TGT is generated.
2. The user opens a supported browser and accesses the Webtop URL.
EMC Documentum
Kerberos SSO Authentication
A Detailed Review 44



3. The KerberosSSOAuthenticationScheme verifies whether or not the user authenticated thus far, and
returns the Error 401 with WWW-Authenticate:Negotiate in the http header if necessary.
4. The browser receives the 401 error code and the Negotiate authorization header in the http response. It
then requests a Service Ticket for Webtop. The browser wraps the Service Ticket in the SPNEGO
token and sends it to Webtop in the http header.
5. Webtop parses the SPNEGO token to obtain the Service Ticket. Webtop validates the Service Ticket.
6. Webtop displays the Repositories selection page where the user can select a Kerberos-enabled
repository or a repository that is not Kerberos-enabled.
7. If the user selects a Kerberos-enabled repository, then the user may observe the following behavior:
The KerberosSSOAuthenticationScheme requests the Service Ticket from KDC using client
credentials for the selected repository.
The KerberosSSOAuthenticationScheme passes the Service Ticket of the selected repository to
Content Server using the DFC Session API.
The Content Server Kerberos plug-in accepts the Service Ticket and on successful acceptance it returns
the repository session to Webtop.
Webtop displays the Main listing page.
8. If the user selects a repository that is not Kerberos-enabled , then the following behavior may be
observed:
Webtop displays a warning message to the user and provides a link to fall back to default repository
authentication.
The user clicks the link to view the default repository login page.
The user specifies the login name and password and then selects the repository.
Note: The username refers to the name of the repository user that is not part of the KDC.
The Docbase Authentication Scheme passes the username, password, and repository information to
Content Server using the DFC Session API.
Content Server validates the username and password details, and upon successful validation, returns
the repository session to Webtop.
Webtop displays the Main page.
The end user is registered in the KDC but is not part of the Kerberos-enabled
repository
In this use case, the client machine is part of the Kerberos domain and the end user is registered in the
KDC. However, the end user is not a valid user in the Kerberos-enabled repository.
1. The user logs in to a machine in the Kerberos domain, and a TGT is generated.
2. The user opens a supported browser and accesses the Webtop URL.
3. The KerberosSSOAuthenticationScheme verifies whether or not the user has been authenticated thus
far, and returns the Error 401 with WWW-Authenticate:Negotiate in the http header if necessary.
4. The browser receives the 401 error code and the Negotiate authorization header in the http response. It
then requests a Service Ticket for Webtop. The browser wraps the Service Ticket in the SPNEGO
token and sends it to Webtop in the http header.
5. Webtop parses the SPNEGO token to obtain the Service Ticket. Webtop validates the Service Ticket.
EMC Documentum
Kerberos SSO Authentication
A Detailed Review 45



6. Webtop displays the Repositories selection page where the user can select a Kerberos-enabled
repository or a repository that is not Kerberos-enabled.
7. The user selects the required repository and submits the login page.
8. The KerberosSSOAuthenticationScheme requests the Service Ticket from KDC using client
credentials for the selected repository.
9. The KerberosSSOAuthenticationScheme passes the Service Ticket of the selected repository to
Content Server using the DFC Session API.
10. Content Server fails to validate the Service Ticket since the user is not a valid user in the repository.
11. Webtop displays an error message to the end user and provides a link to fall back to the default
repository authentication.
12. The user clicks the link and Webtop displays the Webtop Login page.
13. The user specifies the login name and password and then selects the repository.
Note: The username refers to the name of the repository user that is not part of the KDC.
14. The Docbase Authentication Scheme passes the username, password, and repository information to
Content Server using the DFC Session API.
15. Content Server validates the username and password details, and upon successful validation, returns
the repository session to Webtop.
16. Webtop displays the Main page.
Setting DES, AES128, and RC4 Kerberos encryption
types
In Windows 7 and in Windows Server 2008 R2, the Data Encryption Standard (DES), RC4, and AES128
Kerberos encryption types (security settings) for Kerberos are disabled by default. If you log in to Webtop
or any WDK-based applications from a client computer where the Windows 7 or Windows Server 2008 R2
operating system is installed, you must perform the configuration provided in this section.
To select the DES, AES128, and RC4 Kerberos encryption types:
1. Open gpedi t . msc to configure the machine policy by navigating in the Windows Start menu and
selecting Run.
2. Type gpedit.msc. The Group Policy Management Console (GPMC) is displayed.
3. In the navigation pane, double-click theLocal Computer Policy node. The Local Computer Policy
options are listed.
4. Double-click the Computer Configuration node. The Computer Configuration Settings options are
listed.
5. Double-click the Windows Settings node. The Windows Settings options are listed.
6. Double-click the Security Settings node. The Security Settings policies are listed.
EMC Documentum
Kerberos SSO Authentication
A Detailed Review 46



7. Double-click the Local Policies node. The Local Policies options are listed.
8. Double-click the Security Options node. The Security options are listed.
9. Double-click the Network security: Configure encryption types allowed for Kerberos option.
The Network security: Configure encryption types allowed for Kerberos dialog box is displayed.
10. Select the DES_CBC_CRC, DES_CBC_MD5, RC4_HMAC_MD5, and AES128_HMAC_SHA1
encryption types.
11. Click OK.
12. Select File >Exit.
Conclusion
In summary, Kerberos is a solution to your network security problems. It provides the tools of
authentication and strong cryptography over the network to help secure your information systems across
the entire enterprise. Kerberos is intended to centralize authentication, simplify the user experience, and
reduce system administrators account management process.
References
The following documents provide additional, relevant information. Access to the following Documentum
documents is based on your Powerlink login credentials. If you do not have access to the following content,
contact your EMC representative:
EMC Documentum Content Server Deployment Guide
EMC Documentum Foundation Classes Deployment Guide
EMC Documentum Web Development Kit and Webtop Deployment Guide
Other DCTM WPs on Kerberos

The following are third-party references:
Windows 2000 Kerberos Authentication
http://technet.microsoft.com/en-us/library/bb742431.aspx#XSLTsection123121120120
Service Principal Names
http://www.microsoft.com/technet/prodtechnol/windows2000serv/reskit/distrib/dsbd_int_brkw.mspx?
mfr=true
Single Sign-on Using Kerberos in J ava
http://java.sun.com/j2se/1.5.0/docs/guide/security/jgss/single-signon.html
MIT Kerberos Distribution Page
http://web.mit.edu/kerberos/dist
Kerberos V5 Installation Guide
http://web.mit.edu/kerberos/krb5-1.6/krb5-1.6.3/doc/krb5-install.html#Doing-the-Build
Glossary
Authentication server (AS): A server that issues tickets for a desired service. These tickets are then
given to users for access to the service. The authentication server is composed of the authentication
server and the Ticket Granting Server. The AS responds to requests from clients who either do not
have credentials, or do not send their credentials with a request. The server issues a Ticket Granting
EMC Documentum
Kerberos SSO Authentication
A Detailed Review 47



Ticket (TGT) that is used to gain access to the Ticket Granting Server (TGS) service. The AS usually
runs on the same host as the KDC.
Key Distribution Center (KDC): A service that issues Kerberos tickets. This service usually runs on
the same host as the Ticket Granting Server (TGS).
Client: An entity on the network (a user, host, or application) that can receive a ticket from Kerberos.
Credentials: A temporary set of electronic credentials that verify the identity of a client for a
particular service. Credentials are also called a ticket.
Keytab (or key table): A file that includes an unencrypted list of principals and their keys. Servers
retrieve the keys they need from keytab files instead of using kinit.
Service Principal Name: An SPN is the name by which a client uniquely identifies an instance of a
service. The Kerberos authentication service can use an SPN to authenticate a service. When a client
wants to connect to a service, it locates an instance of the service, composes an SPN for that instance,
connects to the service, and presents the SPN for the service to authenticate. An SPN follows the form
root[/instance]@REALM. For a typical user, the root is the same as the login ID. The instance is
optional. If the SPN has an instance, it is separated from the root with a forward slash ("/").
Ticket: A ticket is a temporary set of electronic credentials that verifies the identity of a client for a
particular service. A ticket is also called credentials.
Ticket Granting Server (TGS): A Ticket Granting Server issues tickets for a desired service. These
tickets are given to users for access to the service. The TGS usually runs on the same host as the KDC.
Ticket Granting Ticket (TGT): A Ticket Granting Ticket is a special ticket that enables the client to
obtain additional tickets without applying for them from the KDC.
Simple and Protected GSSAPI Negotiation Mechanism (SPNEGO): SPNEGO is a GSSAPI
"pseudo mechanism" that is used to negotiate one of a number of possible real mechanisms.
Common issues with the configuration of Webtop or TaskSpace
for Kerberos authentication
This section contains a list of common problems that you may encounter when configuring Webtop or
TaskSpace for Kerberos authentication.
Issue 1
To generate logs, enable the following attributes in the <root>\WEB-
INF\classes\com\documentum\debug\TraceProp.properties file:
com.documentum.web.common.Trace.SESSIONENABLEDBYDEFAULT=true
com.documentum.web.formext.Trace.SESSION=true
Issue 2
The following error occurs while processing the J AAS login configuration file:
j ava. l ang. Secur i t yExcept i on at
j avax. secur i t y. aut h. l ogi n. Conf i gur at i on. get Conf i gur at i on.
Cause
This error may occur if an issue such as a syntax error exists in the file.
Solution
Verify the configuration file for errors carefully. Refer to the J AAS Login Configuration File for
information about the syntax that is required in the login configuration file.
EMC Documentum
Kerberos SSO Authentication
A Detailed Review 48



Issue 3
The following exception occurs when the password is incorrect:
j avax. secur i t y. aut h. l ogi n. Logi nExcept i on: Kr bExcept i on: Pr e- aut hent i cat i on
i nf or mat i on was i nval i d ( 24) - Pr eaut hent i cat i on f ai l ed
This error can occur due to one of the following causes:
Cause 1
The password is incorrect.
Solution
Verify the password.
Cause 2
If you use the keytab to obtain the key (for example, by setting the useKeyTab option to TRUE in the
Krb5LoginModule entry in the J AAS login configuration file), then the key may have changed when you
updated the keytab.
Solution
Consult the Kerberos documentation to generate a new keytab and use that keytab.
Cause 3
Clock skew This error can occur if the time on the KDC and the client differ significantly (typically 5
minutes).
Solution
Synchronize the clocks (or have a system administrator to do so).
Cause 4
The Kerberos realm name is not in uppercase.
Solution
Change the case of the Kerberos realm name to uppercase.
Note: It is recommended that all realm names be in uppercase.
Issue 4
The following exception occurs when the client/server machines date and time is not synchronized with
the date and time of the KDC machine:
GSSExcept i on: No val i d cr edent i al s pr ovi ded ( Mechani sml evel : At t empt t o obt ai n
new I NI TI ATE cr edent i al s f ai l ed! ( nul l ) ) . . . Caused by:
j avax. secur i t y. aut h. l ogi n. Logi nExcept i on: Cl ock skew t oo gr eat

Cause
Kerberos requires that the time on the KDC and the client be loosely synchronized (default is less than 5
minutes). If that is not the case, this error will occur.
Solution
Synchronize the clocks (or have a system administrator do so).
Issue 5
The following error occurs when the default realm name is not specified in krb5.conf or krb5.ini:
j avax. secur i t y. aut h. l ogi n. Logi nExcept i on: Kr bExcept i on: Nul l r eal mname ( 601) -
def aul t r eal mnot speci f i ed

Cause
The default realm is not specified in the krb5.conf Kerberos configuration file (if used), provided as a part
of the username, or specified through the java.security.krb5.realm system property.
EMC Documentum
Kerberos SSO Authentication
A Detailed Review 49




Solution
Verify if your Kerberos configuration file contains an entry specifying the default realm. You could also
directly specify it by setting the value of the java.security.krb5.realm system property and/or including it in
your username during authentication using Kerberos.
Issue 6
The following exception occurs when the KDC machine is down:
j avax. secur i t y. aut h. l ogi n. Logi nExcept i on: j ava. net . Socket Ti meout Except i on: Recei ve
t i med out

Solution
Verify whether the Kerberos KDC is up and running.
Issue 7
The following exception occurs when javax.security.auth.useSubjectCredsOnly=true:
GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos Ticket)

Cause
This error may occur if no valid Kerberos credentials are obtained. In particular, this error occurs if you
want the underlying mechanism to obtain the credentials but you have not indicated this by setting the
javax.security.auth.useSubjectCredsOnly system property value to FALSE (for example,
with -Djavax.security.auth.useSubjectCredsOnly=false in your execution command).

Solution
Ensure that you set the javax.security.auth.useSubjectCredsOnly system property value to FALSE if you
require that the underlying mechanism obtain the credentials, rather than your application or a wrapper
program (such as the Login utility used by some of the tutorials) by performing authentication using J AAS.
Issue 8
The following error exception occurs when krb5.conf or krb5.ini is not available:
j avax. secur i t y. aut h. l ogi n. Logi nExcept i on: Coul d not l oad conf i gur at i on f i l e
<kr b5. conf > ( No such f i l e or di r ect or y)

Cause
The sample execution commands specify the default Kerberos realm and KDC by setting values for the
java.security.krb5.realm and java.security.krb5.kdc system properties. If required, you can use a krb5.conf
Kerberos configuration file instead. Such a file includes information about the default realm and KDC. To
use a krb5.conf file, you must set the java.security.krb5.conf system property to specify the location of the
file, instead of the realm and KDC properties. You could also choose not to set any of these properties and
therefore invoke an attempt to locate the krb5.conf file in a default location. An error, "Could not load
configuration file <krb5.conf>(No such file or directory)" appears if the file cannot be found.

Solution
Verify whether the krb5.conf Kerberos configuration file is available and readable.
EMC Documentum
Kerberos SSO Authentication
A Detailed Review 50



Issue 9
The following exception occurs when an unsupported encryption type is used:
j avax. secur i t y. aut h. l ogi n. Logi nExcept i on: Kr bExcept i on: KDC has no suppor t f or
encr ypt i on t ype ( 14) - KDC has no suppor t f or encr ypt i on t ype

Cause
The KDC does not support the encryption type requested.
Solution
The implementation of Kerberos by Sun supports the following encryption types: des-cbc-md5, des-cbc-
crc, and des3-cbc-sha1.
Issue 10
Applications can select the desired encryption type by specifying the following tags in the krb5.conf
Kerberos Configuration file:
[ l i bdef aul t s]
def aul t _t kt _enct ypes = des- cbc- md5 des- cbc- cr c des3- cbc- sha1
def aul t _t gs_enct ypes = des- cbc- md5 des- cbc- cr c des3- cbc- sha1
per mi t t ed_enct ypes = des- cbc- md5 des- cbc- cr c des3- cbc- sha1
If not specified, the default value is:
des- cbc- md5 des- cbc- cr c des3- cbc- sha1
When an unsupported encryption type is used, the following exception occurs:
j avax. secur i t y. aut h. l ogi n. Logi nExcept i on: Kr bExcept i on: KDC has no suppor t f or
encr ypt i on t ype ( 14) - KDC has no suppor t f or encr ypt i on t ype


Cause
This exception occurs when using the native ticket cache on some Windows platforms. Microsoft has
added a new feature in which the session keys for TGTs are no longer exported. As a result, the native TGT
obtained on Windows contains an empty session key and null EType. The affected platforms include
Windows Server 2003, Windows 2000 Server Service Pack 4 (SP4), and Windows XP SP2.

Solution
You must update the Windows registry to disable this feature. The allowtgtsessionkey registry key should
be added and set correctly to allow session keys to be sent in the Kerberos TGTs.
The registry settings must be set to the following values on Windows Server 2003 and Windows 2000 SP4:
HKEY_LOCAL_MACHI NE\ Syst em\ Cur r ent Cont r ol Set \ Cont r ol \ Lsa\ Ker ber os\ Par amet er s
Val ue Name: al l owt gt sessi onkey
Val ue Type: REG_DWORD
Val ue: 0x01 ( def aul t i s 0 )

By default, the value is 0. Set it to "0x01" to allow a session key to be included in the TGT.
The registry settings must be set to the following values on Windows XP SP2:
HKEY_LOCAL_MACHI NE\ Syst em\ Cur r ent Cont r ol Set \ Cont r ol \ Lsa\ Ker ber os\
Val ue Name: al l owt gt sessi onkey
Val ue Type: REG_DWORD
Val ue: 0x01
EMC Documentum
Kerberos SSO Authentication
A Detailed Review 51



Issue 11
The KDC reply does not match expectations.

Cause
The client is unable to understand the KDC response.

Solution
Verify whether you have set all the krb5.conf file configuration parameters appropriately and consult your
KDC vendor's guide.
Issue 12
This issue is specific to Windows 7 or Windows 2008 R2.
If you go to gpedit.msc the required encryption types are enabled in the following file:
Local Computer Policy/Computer Configuration/Windows Settings/Security Settings/Local
Policies/Security Options/Network Security: Configure encryption types allowed for Kerberos
Issue 13
This issue is specific to WebSphere application server.
If aes128 is used, the encryption type is: aes128-cts-hmac-sha1-96
Issue 14
This issue is specific to aes128 encryption type on Windows and non-WebSphere application servers.
If aes128 is used, add the encryption type aes128-cts rc4-hmac
Issue 15
This issue is applicable if Windows 2008 is used as an Active Directory.
Delegation can be enabled by selecting Trust this user for delegation to any service (Kerberos only) in
the User properties/Delegation tab.
Issue 16
The default path for krb5.conf on AIX is as follows:
/etc/krb5/krb5.conf
Issue 17
The default path for krb5.ini on Windows is as follows:
%windir%\krb5.ini
Issue 18
Verify whether Kerberos-related configuration files are loaded as applicable, when the application server
starts up.
EMC Documentum
Kerberos SSO Authentication
A Detailed Review 52



EMC Documentum
Kerberos SSO Authentication
A Detailed Review 53
Issue 19
The keytab format for Content Server must be as follows: <docbasename>.<3digitnumber>.keytab
This is not applicable to application servers.
Issue 20
Ensure that the Kerberos authentication is enabled in wdk/app.xml or custom/app.xml and the correct
domain name is specified.
Issue 21
Kerberos configuration does not work on a Windows 2008 R2 64-bit Kerberos Distribution Center (KDC)
machine after setting kdcUseRequestedEtypesForTickets to 1.
Solution
Do not set "kdcUseRequestedEtypesForTickets" in the
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Kdc registry path, on the KDC
machine for Windows 2008 R2 64-bit.
References for troubleshooting Kerberos-based SSO
authentication
Use the following references to access articles for troubleshooting Kerberos-based SSO authentication:
Oracle documentation for J ava GSS/Kerberos Troubleshooting
How to Enable Kerberos Event Logging on Windows

You might also like