mySAP CRM provides a wealth of information and resources for your call center team. It becomes even more powerful when you use the computer-telephony interface (CTI) to take advantage of a direct link between your customer interactions with voice self-service channels and mySAP CRM Interaction Center (IC). This integration makes your team more efficient, productive, and customer-focused.

Maintaining the critical connection between mySAP CRM and the CTI is not without its challenges, however. Unlike mySAP CRM, which SAP itself supports, CTIs are the domain of various telephony vendors who specialize in the complex messaging required to turn voice interactions into application interactions. Still, as far as your callers and agents are concerned, the health of your mySAP CRM is irrevocably linked to the health of your CTI integration.

The most significant problem with a mySAP CRM IC-CTI failure is that everyone notices immediately. Inbound callers notice that your call center team doesn’t know who they are. Each team member has to run queries to look up customer information on the fly, so callers still in queue notice that it takes longer to get a response.

Team members, who may be incentivized on call volume, become frustrated because they don’t have the information they need to serve customers adequately in a timely manner. Plus, team members outside your call center may experience poor performance because extra queries in the call center can slow down the rest of the mySAP CRM system.

Examples of the pain caused by botched contact center implementations abound. In one well-publicized case, a large-scale call center/CTI system crash lasted six months and cost the company thousands of potential new customers and an estimated USD$100 million in lost revenue.

The best way to ensure the integrity of your mySAP IC-CTI link is to test thoroughly. Unfortunately, some companies assume that desktop load testing also tests this link. This is not the case, and I will explain why and describe the types of CTI tests that you need to perform. First, let’s look at the major causes of mySAP IC-CTI link failure.

Link Failure Causes

The symptoms of CTI problems are easy to see: dropped calls, slow or missing screen pops, multimedia toolbar instability, incorrect routing. Because CTI is fundamentally an integration, a variety of root causes may contribute to these symptoms, including:

• Memory leaks in CTI drivers, particularly with customized versions deployed in customer configurations
  
  
• Incorrect configuration in single sign-on (SSO) architectures, which causes problems with agents’ connectivity to mySAP CRM and their telesets
  
  
• Insufficient application servers to handle the load, or application server configurations that do not scale properly
  
  
• Attempts to use software optimized for the wrong operating system to conform to an operating system standard across the installation
  
  
• Poorly written queries, particularly on screen-pop iViews
  
  
• Additional overhead due to unnecessary mySAP CRM logging
  
  
• Errors in private branch exchange (PBX) configuration, which may route callers inaccurately, resulting in no screen pops
  
  
• Memory leaks due to mySAP IC misconfigurations

Why Desktop Performance Testing Isn't Good Enough 

Desktop performance testing emulates the load experienced by the system under test-in-production conditions. In most cases, desktop emulators are used to scale up the protocol-level traffic experienced by the server from many simultaneous client connections.

In mySAP CRM, this could mean replicating the Web Application Server (Web AS) traffic that occurs when thousands of users log in, view partner information, create opportunities, and perform queries. Desktop emulators recreate these complex transactions with user-defined data suited for client-server or Web-based environments in which each user maintains a single channel of connection (stateful in client-server, stateless in Web) to the server infrastructure. This is important because to make test scalability possible, desktop emulators rely on the existence of a single communication channel to ensure that users are unique.

However, in a CTI environment, that's not good enough.

In a CTI environment, a call center team member creates at least two connection channels to the mySAP CRM server environment: one for the desktop (which “talks” to Web AS) and one for the multimedia toolbar (which talks to mySAP CRM IC and ultimately the CTI, automatic call distribution [ACD], and PBX through Web AS).

So, when the call center team member logs into mySAP CRM and then into a phone queue through the toolbar, two sets of communications begin. The call center team member may be adding partners, performing queries, and interacting with the majority of the mySAP CRM desktop — all of which takes place in a single-threaded HTTP connection to Web AS. At the same time, the call center team member also operates a softphone through the multimedia toolbar, and this communication (new call event, transfer, release call, polling, etc.) takes place in a separate, parallel channel that is related but asynchronous to the rest of the desktop activities.

To make matters more complicated, the softphone responds to events triggered by pieces of the infrastructure (PBX, CTI, or ACD, for example) that are completely outside the SAP core infrastructure, making this traffic even more asynchronous.

These two channels don’t stay separate forever; they join in the screen pop. When a call center team member accepts a call, the multimedia toolbar (from its unique channel) issues an instruction to the desktop to get the partner’s information (from its unique channel). At this moment, both channels come together — and then they separate and move on.

So here's the problem: For a performance test regimen to be valid, it needs to accurately reproduce traffic. In a CTI environment, that means it must emulate many users, each generating asynchronous, multi- threaded traffic that is synchronized at specific events (screen pops). Plus, it has to emulate many users scalably, with minimal load-generation requirements.

However, a desktop testing regimen isn’t designed to do multichannel communication in the context of a single user; it assumes that each user gets his or her own channel. So you’ve got a performance test that typically creates a maximum of only half as many connection threads as real-life traffic — also known an invalid load test, or as a recipe for disaster.

But that's not all. The mySAP CRM IC is a broker. It brokers inbound traffic to mySAP CRM users, and brokers their responses back to the sources of inbound traffic. This means the mySAP CRM IC is designed to deal with inbound traffic such as customers and team members. It doesn’t do its work unless both sets of stimulus exist. So, you must load your inputs to the mySAP CRM IC with both customers and team members. You also must load them in a specific way because the mySAP CRM IC manages the relationship between customers and team members.

Launching thousands of unconnected calls in an environment that is simulating thousands of unrelated agents is not ideal. You need simulated “team members” actually serving simulated “customers” to make the mySAP CRM IC do its work. Putting the mySAP CRM IC through its paces is the whole point of a test regimen.

So, why is your desktop load testing regimen not mySAP CRM-CTI testing? For real CTI testing, you need:

• Accurate, coordinated, production-level load consisting of inbound callers and team members serving those callers
• Accurate, asynchronous multichannel communication for team members interacting with mySAP CRM desktops and the multimedia toolbar in a single transaction

Manual Voice Testing 

It's simple; testing with a single call (or a small number of manual calls) is a functional test, and should be performed before any load testing. However, it is not a system test or a load test. Many issues with CTI integration do not appear until the system is under load. For example, memory leaks in the CTI driver and CTI-mySAP CRM IC link, which can result from customization, can slowly degrade system performance until the system crashes. Issues like this are not visible during a functional test.

Similarly, the performance of the mySAP CRM IC and Web AS under load is an extremely important metric; crashes under load are visible to callers and team members throughout your organization. Other problems include call variable delivery and resulting routing decisions, which often revert to default routing under load conditions in production. While any manual test call may be routed correctly, how will 10,000 calls per hour be routed by your deployment?

What About a Switch Simulator? Switch simulators are useful development tools, but they were designed for developer-level component testing of CTI APIs and work by directly calling CTI APIs to generate CTI events. The advantages of switch simulators are that they are generally inexpensive (included with CTI development kits), and they are software-only. They do not require the presence of a production PBX, and they can generate a number of simultaneous events to validate the API throughput.

However, that is all they can do. They cannot measure the caller impact of CTI API capacity (e.g., caller response time). They cannot test the integration of interactive voice response (IVR) and other methods of call variable creation with CTI and routing strategies, and they cannot intelligently work with agent testing to test skills- based routing down to the agent desktop. Ultimately, they serve as a component test, not a system test.

The best solution for a CTI integration test is to pick up where switch simulators leave off — by measuring the impact of CTI API configuration on caller experience, by measuring IVR/CTI/CRM integration performance and data validity, and by stressing production- level infrastructure rather than development environments, using a system that will scale to support the experience of real users rather than just APIs.

A Comprehensive Approach 

Perceived logistics and technical barriers have led many to believe that testing some of a system's components — through techniques like switch simulation, manual voice testing, or desktop performance testing — is enough. This simply isn't the case.

The focus must be on comprehensive performance testing of the entire mySAP CRM infrastructure (all telephony, CTI, and mySAP CRM application components), as well as full system tests, with accurate emulation of production- level caller and team member usage (Figure 1). Only full system testing and ongoing performance management will ensure the high performance and high availability of this critical and complex system.

Figure 1

A true CTI test involves a production-level load consisting of virtual callers, virtual agents, and diagnostics synchronized in real time to ensure the scalability of call center infrastructure and intelligent routing strategies

The main goals of a CTI integration test are:

• System performance: Validate the performance of the system at capacity, as seen by the primary users of the system (inbound callers, call center agents)
  
  
• Routing validity: Validate the integrity of the system’s call variable delivery (from variable creation in the IVR through the agent desktop)
  
  
• Diagnostics of problems with either of the above

Given the above, to perform a full system test of a CRM-enabled call center, you need the following:

• Automated virtual callers traversing relevant telephony applications
  
  
• Automated virtual agents traversing relevant SAP applications, accepting and serving callers
  
  
• Automated non-agent business users creating background load (desktop load testing is sufficient here)
  
  
• Instrumentation on diagnostics for all supporting systems
  
  
• Synchronization of the data collection and automation events for all of the above, to provide a single, unified test data set.

Dan Koloski

Dan Koloski, a 10-year veteran of the software industry, has extensive experience designing, building, deploying, and testing Web and voice applications. He heads the CRM service assurance business for Empirix, where he has driven the introduction of several innovative products including the first integrated quality solution for complex, multichannel CRM implementations. He has also authored numerous papers and articles on application quality. Dan is a graduate of Yale University and holds an MBA from Harvard Business School. Empirix delivers a range of testing and monitoring solutions for next-generation networks, contact centers, and Web-based applications. Hammer Service Assurance for mySAP CRM is being developed in partnership with SAP. For more information, visit www.empirix.com.

You may contact the author at dkoloski@empirix.com.

If you have comments about this article or publication, or would like to submit an article idea, please contact the editor.