You are on page 1of 16

Best Practices in Siebel CRM Performance Management

:
Monitor, Measure and Manage the End User Experience

executive white paper

measure and manage how end users are utilizing the application to optimize business process execution. 3 Executive Summary Why End User Performance Management is needed to optimize Siebel CRM apps 5 ROI Isn’t Real Until It’s Realized The ROI impact of issues with adoption and efficient and effective use 6 Traditional Performance Management Approaches Aren’t Enough Monitoring system and application health is no longer enough for service of business apps 7 Overview of Knoa Experience and Performance Manager A review of the architecture and functionality of Knoa EPM 9 Knoa EPM Use Cases Knoa EPM delivers results to the entire application team.Best Practices in Siebel CRM Performance Management: Monitor. This section details use case examples compiled from Knoa’s customer base 14 You Can Manage What You Measure The impact that real end user experience and performance metrics can have on the application management team 15 Could You Benefit From Knoa EPM? Four questions to help you evaluate the potential value of Knoa EPM for your Siebel deployment . Measure and Manage the End User Experience 2 Contents Knoa® Software is leading the performance management industry with end user experience monitoring solutions that reach beyond infrastructure and application performance to the monitor.

if application users are executing key processes effectively and efficiently – the key to achieving business value and ROI. It’s important to optimize the Siebel application for your own unique business environment. but also. user needs and usage patterns. Knoa’s Experience and Performance Manage-ment suite lets IT departments measure response times. Knoa® Software is leading the performance management industry with end user experience management solutions that reach beyond infrastructure and application performance to monitor. networks and applications. “An end user performance management package from Knoa Software is capturing the CIO’s attention for its ability to boost worker productivity and optimize application performance. How can you tell how if your Siebel solution performs for each person who uses it? How can you identify which users have difficulty executing which transactions? What does it mean when your end users say the system is “slow”? How can you pinpoint the actual issues with the real data you need to take corrective action? Knoa can give you the answers. Knoa customers not only know if core enterprise applications are delivering an acceptable user experience. measure and manage how end users are utilizing the application. Without these measurements. Yet many frustrated execs find that despite the small fortune they have spent on products to manage servers. The result is significantly reduced ROI. it’s all for naught if you do not build and sustain adoption and efficient and effective use of the Siebel application by your end users. But. and effective use of the application by the end users. distinguish between system and user error and monitor work flows – all in real time…” – Brian Gillooly Optimize IT . efficient use. What is often overlooked is that application performance is just one of three key drivers of return on investment (ROI) from your Siebel application. companies are unable to identify and resolve critical issues that impact the user’s ability to optimize business process execution. Measure and Manage the End User Experience 3 Executive Summary CRM solutions. like Siebel. Enterprises with mission critical Siebel deployments do not have accurate measurements of adoption.Best Practices in Siebel CRM Performance Management: Monitor. the business constituencies still complain about the performance. automate complex business processes where high levels of application performance are crucial. usability and availability of the Siebel CRM application. and significantly increased cost of application support. An efficient infrastructure is critical.

Knoa EPM also analyzes and aggregates information to provide actionable insight into process compliance. at all times. In a recent study conducted by Forrester Consulting an overwhelming 87 percent of IT executives either strongly agreed (50%) or agreed (37%) that the best way to manage the service of business critical applications is by measuring performance at the end user level and rapidly resolving significant performance problems. Knoa EPM provides a complete picture of end user experience by focusing on both the experience the end user receives from the application and the performance the end user achieves working with the application. application execution across that infrastructure is monitored and measured. Sixty seven percent of the surveyed organizations are either much more focused (27%) or more focused (40%) on the end user experience than they were two years earlier. You can think of Knoa EPM as the “third leg” to the performance management stool – infrastructure. the Forrester Consulting report found that change is imminent. Knoa EPM captures transaction response times as well as ecosystem and application errors – along with a time-synchronized workflow – from all users. Forrester Consulting Knoa Experience and Performance Manager (EPM) monitors enterprise applications from the perspective of the end user to capture the actual end user experience. and now. end user metrics. .Best Practices in Siebel CRM Performance Management: Monitor. adoption issues and a number of other management dimensions. Measure and Manage the End User Experience 4 For all the investment in application management solutions. training requirements. In fact. applications. at all locations. Forrester concluded that best practices companies are intent on collecting metrics on the end user experience. there’s a need for a change. This information is used for alerting and proactive remediation of end user issues (whether they are system errors. Think about it! The infrastructure is monitored and measured and optimized. nearly two-thirds of companies (66 percent) rely upon help desk calls as the main indicator that end users are experiencing response time and/or availability problems. user errors or performance problems). across the entire application. Clearly.

companies are unable to identify. In the absence of comprehensive and representative data. Ongoing costs are high. and lead to decisions that are not in the best interest of the business. and resolve critical issues that impact the user’s ability to effectively execute. Flying Blind Here’s how one of Knoa’s customers described the problem that led them to deploy Knoa EPM.Best Practices in Siebel CRM Performance Management: Monitor. End users complain that the application is “slow” and “doesn’t work”. ROI is only realized from this investment if the end users adopt the application and use it efficiently and effectively to improve the execution of core business processes – which is very difficult to achieve. Consider this all too common scenario that afflicts many CRM deployments. and effective use of the application by the end users. Software maintenance averages 20%. as well. efficient use. And across the board there are complaints that things take “twice as long” with the new system. One Version of the Truth. Software licensing costs can range from $1. and hamper ROI realization. and it’s not clear if there are system problems or if they are just using the wrong procedure. annually.000 to $2. and occasional stopwatch timings. Gartner reports that 60% of ongoing costs of a typical deployment are for support staff. limited visual observations. Implementation services are 2-3 times the software costs and there is investment in back-end infrastructure. Measure and Manage the End User Experience 5 ROI Isn’t Real Until It’s Realized The initial investment in CRM applications is high. companies are able to identify and resolve critical issues that impact the users’ ability to effectively execute. remote users can’t access the system. Employee turnover slowly erodes the user base proficiency. many discussions regarding application performance are based on reported user impressions. Without these measurements. Months after deployment a high percentage of the functionality is not being used. The result is significantly reduced ROI. This has resulted in extrapolations about the performance of the whole system that are inaccurate. efficient and effective use of the application. There are complaints about poor response time. With end user metrics. “User adoption of our CRM application has been stalled by a number of factors affecting field productivity.800 per end user. . Knoa EPM gives the enterprise real metrics for adoption. and significantly increased cost of application support. Some users find the software cumbersome to adopt into their routines and create their own non-compliant work-arounds.” Enterprises with mission critical CRM deployments do not have accurate measurements of adoption.

adoption and efficient and effective use are the keys to ROI from the application investment. Measure and Manage the End User Experience 6 Traditional Approaches to Performance Management Aren’t Enough Anymore CRM applications like Siebel are complex. Each Siebel customer has their unique business environment – user needs. In a recent study conducted by Forrester Consulting an overwhelming 87 percent of IT executives either strongly agreed (50%) or agreed (37%) that the best way to manage the service of business critical applications is by measuring performance at the end user level and rapidly resolving significant performance problems. network or transaction management) to provide a true perspective of the end user experience with the application. Knoa believes that the application is “not performing” – from the end user’s perspective – whenever the end user is unable to effectively execute the key processes required for their role. . In the face of this complexity. In our vision. usage patterns. It’s the flip-side of the flexibility they provide to meet a wide variety of customer needs. customizations.Best Practices in Siebel CRM Performance Management: Monitor.g. architectures and platform environments vary widely. application performance. Is the application enabling the end users to efficiently and effectively? After all. application availability). it is becoming increasingly difficult for measurement and metrics taken at the infrastructure level (system. Received Error Messages Site Navigation Issues Transaction Execution Issues Login Issues Content Issues 31% 31% 31% 40% 37% Snapshot of the End User Experience with online transactions (Source: USA Today. aggregate transaction response times. from the end user perspective includes: • Slow transactions • Cumbersome Navigation • System Errors • Non-intuitive user interface which causes excessive user errors Knoa EPM goes beyond the response-time dimension to monitor and capture metrics on the entire end user experience – including system and user errors as well as the user navigation and interaction with the application (workflow). Has the application been adopted? Are the end users using the application proficiently. Knoa EPM extends performance management to a whole new level by monitoring and managing the “health” of the end-user population. 2006) The monitoring capabilities of traditional performance management products provide a significant amount of insight about application health (e. transactions that are failing. system components.

and the sequence of events in the user’s interaction with the application. Knoa EPM monitors the Siebel application from the perspective of the end user to capture the actual end user experience. etc. including system and application errors and user created errors • Comprehensive perspective of application utilization – which transactions are used. is transferred to the Knoa EPM analysis server which prepares a series of targeted analyses and reports. This information is viewed through the Knoa EPM Console. Knoa EPM uniquely captures a complete picture of the end user experience and behavior: • Actual User-Experienced Response Time for key system transactions. which presents a series of management dashboards and drill downs for analyzing and querying the performance data. and a time-synchronized workflow – from all users. as well as the experience that they receive. etc. Knoa EPM selectively captures the actual steps. called track data. This client engine monitors end users’ interaction with the application.” – Bernd Harzog CEO. such as navigating between application screens. as well as ecosystem and application errors. for how long. Knoa EPM captures a number of metrics around both the application’s performance and the end user’s performance: • All user errors • All system errors • Transaction response times This information.Best Practices in Siebel CRM Performance Management: Monitor. Who's using it? What's the response time? Who's getting errors? How many transactions are being abandoned? Knoa EPM is the only end user experience management solution that monitors the entire end user experience from the perspective of the end user. at all locations. companies can manage the performance achieved by end users. In addition to this timesynchronized workflow. Measure and Manage the End User Experience 7 Knoa Experience and Performance Manager Knoa EPM gives the Siebel support team a comprehensive "fact-based" perspective about the end user's experience with the application. • Complete Quality of Experience. execute operations. at all times. in what sequence. “The combination of the user-base coverage and application depth provided by this approach is tough to beat. APM Experts . With this information. Knoa EPM is based upon a passive monitoring engine that is deployed to the client workstation along with a template specifically designed for the target application. save operations.

Director. define transactions to the system or train the system to recognize new transactions. Knoa EPM’s self-discovery capabilities eliminate. CRM Performance. . • Application usage • Transaction usage • Screen usage • Function usage • Active/idle time • User behavior Non-Intrusive to Siebel Application The Knoa EPM system does not insert tags or instrumentation into the Siebel application. Measure and Manage the End User Experience 8 A Unique Architecture for End User Experience Management “The insight into actual end user experience gives us benefits across the board. British Telecom Knoa EPM is based upon a unique architecture that was designed. Automated Discovery means: • No up-front requirements definition or project scoping • Faster implementation using out-of-the-box templates • lower overall cost of ownership • Adaptable technology to easily support the most aggressive release/update schedules Global Coverage The ‘out of box’ Knoa EPM solution gives you global coverage of the entire Siebel application. whether frontend or back-end.Best Practices in Siebel CRM Performance Management: Monitor. from the ground up for user-centric monitoring and experience management. Key capabilities the architecture delivers are: Automated Discovery The ‘out of box’ Knoa EPM solution automatically ‘sees’ every Siebel transaction and the steps that the user is taking before. without any discernable system overhead or impact. helping target response time issues. during and after the transaction. The Knoa components do not require integration with any components in the application landscape. completely.” – Stuart Smith. identify usage and adoption issues and even make priority decisions on ongoing application investments. across all instances and systems in the field. This global coverage also gives you a comprehensive perspective on adoption and usage across the entire user population. Small Footprint/Low Impact Knoa EPM uses a Seibel-specific template to selectively record the significant user events that will produce actionable insight. or require any back-end processes that impact application performance. create any system events. It is this light operational footprint that enables you to monitor and measure the user experience across the entire application landscape. solve end user problems faster. the need for customers to design a data collection strategy.

The number one complaint from users was long response times for key functions. for every user. Knoa EPM delivers results to the entire Siebel support team. Here are some real use case examples submitted by Knoa customers: Knoa Use Case: Performance Management Improve Siebel Application Performance Excessive application response time is one of the most critical performance issues encountered by the users of today’s complex mission critical applications. A leading consumer packaged goods company had put millions of dollars over 3 years in a new Sales Force automation solution. With Knoa EPM you know the response time that was actually experienced by the end-user – for every transaction. by being able to isolate performance problems to specific areas of the application. such as creating and saving CRM records. . It’s also a huge support problem. The QA/Performance team was only able to act on these performance issues once they escalated to critical levels. Using an alert system that notified the IT team whenever response times in the field exceeded set tolerance thresholds. In addition. the team was able to preempt the escalation of performance problems in many cases. The company implemented the Knoa EPM system to detect performance fluctuations for key functions in real-time. Measure and Manage the End User Experience 9 See the forest and the trees. every time.Best Practices in Siebel CRM Performance Management: Monitor. the team also reduced the time to resolve critical issues.

Eliminating the module from production saved the organization tens of thousands of dollars in development time and resources. they made a decision to discontinue support. using the Knoa application usage data. and a detailed review of Business Operations based on Knoa data. the IT Team discovered that few employees actually used the new functionality. The IT Support and Development team of a leading global financial services company received requests from their field users to implement a costly and extensive functionality enhancement for their enterprise sales application. After the enhancement was implemented.Best Practices in Siebel CRM Performance Management: Monitor. . Based on this fact. Measure and Manage the End User Experience 10 Knoa Use Case: Application Utilization Link investment and effort to business impact With Knoa EPM you gain valuable. actionable insights into application utilization: Who is using the application and for how long? Which areas and functions are most used? Which areas and functions are least used? What usage patterns are creating strains on IT infrastructure? Knowing the usage profile for your Siebel application and your user base is the most fundamental step towards application and user performance management. The IT team dedicated considerable resources to implement this request and launch the new capability.

in the long run.Best Practices in Siebel CRM Performance Management: Monitor. Measure and Manage the End User Experience 11 Knoa Use Case: End User Proficiency Drive efficient and effective use by managing errors With Knoa EPM solution you can identify 100% of all user-experienced errors. the IT department of a leading computer services company realized that a large percentage of the errors encountered by users were never reported to the Support Desk. the IT department promptly identified the issue. Based on the Knoa data. Knoa EPM also gives you the context in which each error was encountered and the path that the user took before encountering the error. and 2) re-engineer the search functionality of the application to eliminate the error altogether. a single error type. The productivity losses and infrastructure impact of repeated and failed queries executed by users amounted to thousands of dollars each day. in the short run. . which was then addressed with a two-fold strategy: 1) provide targeted user training in order to avoid the error. In particular. generated while executing searches. whether they are caused by user actions or by any of the application layers. accounted for 25% of all errors generated by the company’s CRM application. By implementing Knoa EPM. This information is available in real-time for the end user support team.

The information being communicated to the support team from the help desk was very sketchy and incomplete. . they found she was using an incorrect process to validate the customer information (a common process step in many different transaction types). However. In the past the CRM support team would spend hours on the phone with the inside rep. before. Investing further. A major electronics company was well into their deployment of a CRM solution. Measure and Manage the End User Experience 12 Knoa Use Case: End User Support Accelerate Problem Resolution with end user metrics and workflow Knoa EPM captures a complete record of end user behavior. her supervisor was able to instruct her in the proper customer validation process and “the problems with the system” were quickly resolved.Best Practices in Siebel CRM Performance Management: Monitor. and often had to make a costly on-site support call to deal with issues like this. by looking at a recorded record of her actual interaction with the application. Knoa EPM enabled a dramatically different approach. they reviewed this ISRs’ recorded experience with the application. Through the management console. With this insight. First the team looked at the metrics and found that the response time she was experiencing was fine – well within the sub-3 second goal. An inside sales reps in the Raleigh NC call center was experiencing problems. This end user workflow data transforms the end user support process. Next they found that she was generating four times more user errors than her peers. during and after the transaction.

Best Practices in Siebel CRM Performance Management: Monitor. The fix was simple. There were grumblings in the outside sales force that the opportunity management screen was really difficult. The first instinct was to schedule an e-learning retraining session. . No retraining was needed. The end-to-end transaction time improved by 25% as well. The application team removed those fields from the screen and the errors were eliminated. slow and cumbersome. When the KPI information for errors was reviewed. However. a closer review of the recorded work flow for a representative sample of users revealed that the errors were being caused when the sales reps were trying to put data in extraneous fields that were on the screen but not required for the transaction to execute. the application team noticed that virtually every end user was making the same errors on these screens. Measure and Manage the End User Experience 13 Knoa Use Case: Application Support/Engineering Application Support/Engineering: Key Performance Indicators (KPI) that report user errors on an aggregate level were useful in resolving another vexing issue for that electronics provider.

Knoa also measures and reports critical information about how your users experience the application’s actual performance. adjust business processes.Best Practices in Siebel CRM Performance Management: Monitor. Questions Knoa Answers for British Telecom Is the application performing effectively? Are end users able to effectively and efficiently execute critical business processes – in other words. As an added benefit. and without a strategy to gather insight about end user’s experience and actual performance with the application. Knoa Experience and Performance Manager accurately determines how each employee actually uses the application and provides input to comprehensive reports that you can use to pinpoint training needs. real-time data on the user experience to allow the help desk and key users to resolve user’s questions rapidly? Can we maintain an electronic audit trail that tracks the completion of key transactions? What is the impact of ineffective procedures on operations? It’s rule #1 in the management handbook. so they can quickly resolve issues with minimal disruption to the end users • Training organizations can pinpoint which specific application areas (and which specific users) require additional training • Application and Process Engineering can spot cumbersome work flows and errors that are affecting end user satisfaction and efficiency • The CIO has comprehensive metrics from the entire application landscape to support reporting to the business on SLAs and insight into application usage and performance to make intelligent decisions about where to deploy resources to improve the application performance and effectiveness • The business executives have actionable insight to drive the highest possible levels of adoption. Measure and Manage the End User Experience 14 You Can’t Manage What You Can’t Measure. isn’t it? And the rule that is the driving force behind the investment in CRM applications in the first place. and ensure that your end users are using the application to deliver all the value you expect from the application. has the training program led to real results? Can we provide objective. many IT organizations are left with the reality that they can’t manage and improve end user adoption and build end user proficiency. efficient and effective use of the application which is the key to a business return on the investment . • Application Support teams have real-time metrics to proactively address response time issues problems • Help Desk personnel have immediate visibility user into actual user interaction with the application. But. without a strategy to measure end user adoption.

When your end users do report problems to the help desk. An EPM system will make sure you have visibility into all errors – for all users. Sixty seven percent of surveyed organizations are either much more focused (27%) or more focused (40%) on the end user experience than they were two years earlier. Who is using which transactions? How long are they active? How long are they idle? Knoa EPM gives you the ability to pinpoint and analyze user adoption issues that are hindering the achievement of business results. Employee productivity impact is kept to a minimum when there is an issue (users spend less time on the phone with help desk and problems are resolved faster). you will now know if the users were experiencing long transaction response times or if there were a lot of system errors presented to the end user. you’re probably right. Knoa EPM system captures and reports real metrics about the end users experience with the application. in all locations. Perhaps it's a user training issue. or perhaps the errors are being caused by a cumbersome or confusing user interface that can be corrected. abandon the transaction and/or find workarounds.Best Practices in Siebel CRM Performance Management: Monitor. Do your end users ever use system performance as a key reason why critical objectives are not met on time and on budget? One of the benefits of Knoa EPM is that it gives you the ability to correlate business outcomes with efficient. This information is immediately available to help desk personnel so there is minimal business disruption from issues with the application. You will also know if the end users themselves were causing a lot of errors by their own behavior. Forrester Consulting recently reported that an end user typically experiences a problem 6 times before they call the help desk. In fact. Measure and Manage the End User Experience 15 Can Knoa Experience And Performance Management System Help You? Are you certain that all of your end users are actually using the functions of the application required to execute their role? Knoa EPM captures and reports real metrics about application usage. It is often reported in surveys of the service management process that 70-80% of all end user problems are not reported because the end users ignore the errors. effective application usage. Forrester Consulting 2006 . do you find they are wasting a lot of time trying to answer questions and document or recreate the problems? Knoa EPM captures the actual work flow that the user experienced – before. all the time. Knoa EPM gives you the ability to pinpoint and proactively remediate problems which impact the efficiency or the effectiveness of the end user population using the application to accomplish business goals. Do you suspect that there are a lot of problems and issues that are never surfaced to the help desk? Well. You will then know where to apply corrective action. during and after the transaction.

measure and manage how end users are utilizing the application to optimize business process execution. Training IT Support Business 5 Union Square West 4th Floor New York. please visit www. Only with the insight into end user metrics that an EPM system can provide can you really know if your core enterprise applications are delivering an acceptable user experience and if application users have adopted the applications and are executing key processes effectively and efficiently – the key to achieving business value and ROI from the application investment.com. NY.knoa. For more information about Knoa. Expertise and Performance Management systems are becoming a priority for the business executive and CIO who need to drive the highest possible levels of adoption and efficient and effective use of core enterprise applications to drive a business return on investment. Knoa customers not only know if core enterprise applications are delivering an acceptable user experience. if application users are executing key processes effectively and efficiently – the key to achieving business value and ROI. NY 10003 212-807-9608 www. Knoa is headquartered in New York. but also.knoa.Best Practices in Siebel CRM Performance Management: Monitor.com . Measure and Manage the End User Experience 16 About Knoa Knoa® Software is leading the performance management industry with end user experience monitoring solutions that reach beyond infrastructure and application performance to the monitor.