Recently, my company helped one of our clients integrate SAP's Supply Chain Event Management (SAP EM) with SAP BW. SAP EM is a component of SAP SCM 4.0 that offers a broad range of functions for tracking, notification, and alert creation.

SAP EM can connect to other parts of mySAP Business Suite including SAP R/3, SAP BW, and SAP CRM, as well as third-party systems. It was first released with limited availability as Supply Chain Event Manager (SCEM). SAP EM is now available as part of the SAP Supply Chain Management 4.0 solution. SAP EM is typically used for tracking and controlling business processes along the entire supply chain, integrating supply chain planning, supply chain execution, customer relationship management, and product lifecycle management in both SAP and non-SAP applications.

Our client decided to use SAP EM as part of the overall plan for revamping its core system landscape with SAP products, promoting standardization, and reducing costs related to interface maintenance. SAP BW was already a component of this landscape and the tool of choice for providing reporting to SAP EM data.

Unlike some other SAP source systems for which SAP provides extractors that are fully mapped to the source tables, SAP EM tables need to be manually mapped by BW profiles to allow data extraction to BW. This can be a difficult process because not all SAP EM tables are easy to extract manually. I'll describe how to simplify this process by explaining the main SAP EM features and architecture. I will also discuss the benefits of SAP EM and the steps necessary for a successful SAP EM integration with BW.

SAP EM Terminology

Understanding SAP EM terms will help you grasp its major functions and how its components interact. The main logical entities are messages and handlers. Other important terms include info parameters, control parameters, and expected events. Refer to Table 1 for a glossary of SAP EM definitions.

Event messages are the relevant steps of a supply chain process stored by SAP EM. Event messages that belong to the same process are grouped under a unique event handler. In other words, event handlers are the logical structures that hold all the event messages of a particular supply chain process. You can think of the event handler as a briefcase and the event messages as the papers inside.

For instance, an event handler may be related to a shipment or container (a group of shipments). An event message is a specific occurrence that applies only to a certain shipment or container and thus logged by SAP EM within this related event handler. An event handler acts as the placeholder of all shipment-related event messages, capturing, for example, their date, time, location, and type. An event handler has two well-defined sections, a header section and a message section (Figure 1).

Figure 1

The two parts of an event handler

The header section contains general information about a shipment, such as creation date, last change on date, or destination. The tracking ID number, event handler-specific information used by SAP EM for sorting, querying, and most importantly, message-to-handler matching date, are continued in the header section. Every message received by SAP EM carries data about its own occurrence (e.g., date, location, and type) and the tracking ID number related to the corresponding shipping process. It is this crucial information that drives the handler-message matching process.

When receiving an event message, SAP EM first identifies its type and related tracking ID number. Then it scans the rule set configuration logic to determine what tasks and procedures should be executed. A rule set includes several different rules that evaluate and process event messages for a particular event handler.

Each event message may be processed differently. For example, an Attempted delivery rule message updates an event handler differently than a Successful Delivery (Figure 2). Both are logged in an event handler message section, but the Successful Delivery may carry important data relevant for quality service management and control such as delivery time and location. This kind of information can be extracted from the message log section and registered under the header section as information or control parameters. These parameters can then be used for direct database querying or as selection fields on specific BW reports.

Figure 2

The two parts of an event handler

Event messages can also update expected events on an event handler. Expected events are process milestones used for monitoring, notification, and alert creation throughout the supply chain process. They help to pinpoint possible or existing overdue steps that can affect the overall shipping time.

The use of expected events is optional and should be configured. This configuration can be applied to specific event handler types. Let's say all express deliveries within the US should be completed in two business days. The system can be configured in such a way that an expected event is set every time an express delivery is created in SCM. The expected event would specify the expected delivery date and time based on the event handler creation date and time. Rule set configuration dictates that the actual date and time of this expected event should be updated as soon as the system receives a successful delivery message related to this event handler.

The example in Figure 3 shows an event handler created on March 15, 2004. An expected event is based on creation date plus two days, so the planned delivery data is set to be due on March 17, 2004. Several event messages can be received by this event handler, which does not affect the information within the expected delivery data. As per the configuration, as soon as a message type Delivery is received, the system stores its date as the actual delivery date within the expected event. This result can now be used for reporting to show whether a particular shipment was delivered according to service standards.

Figure 3

An example of an expected event

Relate “Extraction-Friendly” SAP EM Tables to BW Profiles

Information related to the elements — event handlers, event messages, info parameters, control parameters, and expected events — must be brought into BW via interfaces (extractors) built into the tables that store such data. Table 2 presents each EM element and its related table.

These SAP EM tables are BW extraction-friendly designed by SAP to be automatically available for use on the BW profile configuration. Others can be included only via user exits and custom development.

The data mapping between SAP EM table fields and BW-SAP EM extractor fields can be set up easily. This is done in the implementation guide via the menu path SAP Customizing>Supply Chain Event Management > SAP Business Information Warehouse Interface > Define SAP BW Profiles (transaction SPRO). As shown in Table 3, BW profiles are subdivided into four main configurable sections.

Fields from the event handler's Header sections are mapped to fields of SAP-delivered extractor 0SCEM_1: Header, Info, and Control parameters including header, info parameter, and control parameter tables. Note that although SAP EM is now available as part of SAP Supply Chain Management 4.0, the extractors' technical names have been retained from when the technology was available in a limited release as Supply Chain Event Manager (SCEM). The Events section's event message and expected event tables are mapped to the fields in the extractor 0SCEM_2: Event and Message Headers.

The Groups section is used when calculating expected events of the same event handler. For example, suppose an event handler has two expected events: One event, Delivered to Third Party, is due one day after the order creation and the Subsequent Successful Delivery event is due two days after the first expected event. These expected events may be needed to measure the service efficiency of shipping partners per region. To accomplish this, the actual date of the expected event Subsequent Successful Delivery could be subtracted from the Delivered to Third Party actual date in order to measure how long it took the delivery to be completed after handing the shipment to a third party. This can be accomplished under the Groups section by defining a new event group comprised by these two expected events. You also can define how the duration should be calculated under this event group. The results are then mapped into the third available extractor, 0SCEM_3: Groups.

The Function PlugIn section allows BW-SAP EM extractors to reach data beyond the five extraction-friendly tables via custom ABAP code. User exits can enhance the data in the five tables with data from other EM tables. Figure 4 shows what these four BW profile subdivisions look like in SAP EM.

Figure 4

SAP EM’s four BW profile subdivisions

All sections except Function PlugIn have a time stamp that determines sub-section as well as value mapping and duration calculation. This is necessary because SAP EM stores message-related date and time data under a time-stamp format — YYYYMMDDHHMMSS. To map this data into specific extractor date and time fields, a "splitting" procedure is required. A sub-section of the BW profile that describes how the time stamp should be split to populate the respective date and time fields is configured under Header Time Stamp Resolving.

Direct and Queued Update Modes

After setting up the mapping, it is necessary to determine the update model to use. The two possible options are direct mode and queued update mode. The rule of thumb that SAP suggests is to choose queued update mode if serious performance problems are expected due to an extremely high data volume during delta upload. In all other cases, choose direct as the update mode.

The main difference between the methods is that under direct update, the data is immediately queued and prepared for extraction using a dedicated BW queue table within SAP EM. This can degrade system performance if high data volumes require a preparation step.

A solution for the high-volume scenario could be to queue the new data first and then, during a down time period, execute the data preparation for BW extraction. This is exactly what the queued update mode does. The new data arriving in SAP EM is first queued into an EM queue table. Later, a batch program can be executed. It moves and prepares the data from the EM queue table into the BW queue table. Figure 5 illustrates both modes.

Figure 5

Direct and queued update modes

Initialization and Delta Triggering Options

After building the bridge between SAP EM and BW, it is necessary to trigger and maintain the data flow between them. Performing the initialization, or the extraction of historical data triggers this process. After a successful initialization, a delta mechanism setup is required in order to keep new or changed records flowing into BW. Initialization and delta triggering procedures use rule sets, the same mechanism applicable to messages. An SAP-delivered event — BWUPLOAD — is key to both procedures. A rule related to this event should exist within the rule set. This rule is configured based on event BWUPLOAD and associated with the BWUPLOAD function module.

The main idea is that as soon as a BWUPLOAD event is applied to a handler (step 1), the function module BWUPLOAD is performed for this handler (step 2). Then, the handler header and message fields are queued for further extraction (step 3). Figure 6 illustrates this concept.

Figure 6

The steps of the function module BWUPLOAD

Next, the BWUPLOAD event is applied to the handlers to trigger the queuing. The SAP-delivered program R_BW_EXTRACTION_INIT is used for initialization (Figure 7). This program provides several selection options for identifying existing event handlers within the SAP EM database and can be accessed through transaction BWIU.

Figure 7

SAP-delivered program R_BW_EXTRACTION_INIT

When the initialization program selects an event handler, a BWUPLOAD event is automatically applied to the event handler. The BWUPLOAD event then triggers the function module BWUPLOAD and consequently all relevant event messages and event handler header fields are queued.

After the initialization is complete, it is necessary to configure the delta process, or capture new event handlers and event messages. This task follows the same steps as the initialization, the BWUPLOAD event, and rule set. The difference is the triggering mechanism. You have three possible ways to trigger a delta process within SAP EM: based on a specific final message, an expected event, or every message.

Triggering based on the last message means the message should be associated with a BWUPLOAD function module other than the BWUPLOAD event itself. Let's say that a normal shipping process should end with a successful delivery message type. Having this message type related to the BWUPLOAD function module on the rule set triggers SAP EM to queue a handler as soon as it is complete. The downside of this method is that incomplete shipping processes would not normally be captured, possibly requiring manual intervention to be migrated to BW (i.e., selective initializations). However, companies that require an end message to close a delivery, even if the delivery is unsuccessful, should be able to use this first delta approach without problems.

A delta process can also be based on an expected event. At the time an event handler is created, an expected event due on a predefined date is also set. Let's say that an event handler and its expected event are due 45 days after the creation date. As soon as this date is reached, the expected event is considered due and the BWUPLOAD function module is executed. The data is queued and the process ends. Unfortunately, a fixed timeline presents the risk of uploading incomplete processes. Again, manual intervention may be required. For some companies, any shipments that take more than 45 days are considered an anomaly and discarded for analysis.

The third option to trigger the delta process is to relate every event message to the BWUPLOAD function module by associating the BWUPLOAD rule to the last rule on the rule set as demonstrated in Figure 8. By default, as soon as an event message is received and its related rule executed, SAP EM automatically jumps to the End rule. If the End rule is associated to the function module BWUPLOAD, then this function will be executed every time, independently of message type.

Figure 8

BW upload rule decision process

However, this method also carries disadvantages. The BWUPLOAD function module is designed to extract all event messages and event handler header fields relevant to BW. This can send a message to be queued several times, depending on how many times an event handler is updated. Figure 9 shows a possible scenario.

Figure 9

Every message triggers BWUPLOAD

In this scenario, an order message type creates a new event handler. Consequently, BWUPLOAD is executed because it is associated to the End rule on the rule set. Order message and event handler header data is sent to the queue tables. Later, a transfer message related to the same event handler is received. BWUPLOAD again is applied to the event handler. This sends all messages and event handler header data into the queue tables, therefore sending order message data again. The subsequent messages Info and Delivery would cause the same thing again and again.

It is important to take data volume into consideration if this method is selected. Remember to be very selective on messages and parameters to be extracted. Make sure data load volumes are administrable to avoid performance issues due to an extremely large set of delta volumes.

Overall, the experience of implementing SAP BW on top of SAP EM was positive. Working together with SAP, we were able to find solutions and execute new developments that guaranteed a stable integration environment. BW definitely improved the reporting capabilities of SAP EM data, bringing more flexibility and power to analytical approaches without affecting the transactional performance of the source system.

Christian Savelli

Chris Savelli, senior manager at COMERIT, has been dedicated to SAP BI and Analytics projects since 1998.  He holds multiple SAP certifications covering HANA, BW and ECC applications and has expertise in managing all aspects of the information creation process, utilizing SAP BI technologies to satisfy strategic, analytical and reporting needs. Chris Savelli started his career at SAP and subsequently held senior level positions at consulting companies Deloitte and Accenture. His education background includes a bachelor of science degree in robotics and a master of science degree in engineering both from the University of Sao Paulo, as well as a post-graduate diploma in business administration from the University of California at Berkeley.

You may contact the author at csavelli@comerit.com.

If you have comments about this article or publication, or would like to submit an article idea, please contact the editor.