SAP has a powerful tool for scheduling and monitoring jobs (transaction code SM36—Define Background Job). It offers a lot of functionality, including periodic runs, starting one job when another finishes, and starting after an event. A key limitation of this tool is that it works only with SAP software. Companies that run SAP software often use third-party applications that have to communicate with SAP products. These companies usually prefer to use one scheduling and monitoring tool across all applications—SAP and non-SAP. This way they don’t have to teach the operations staff different tools, and they can synchronize job execution in systems running different software.

One popular scheduling product is Control-M from BMC Software, Inc. It interfaces well with SAP R/3 and with other applications. Control-M now works with BW by adding functionality to SM36. However, the product has limitations when dealing with data loads. Specifically, Control-M cannot “see” individual data loads, so error reporting is a problem. Also, it can show the status of a job, but not the statuses of all the loads in the job. You can’t get a “big picture” view of all data loads.

I’ll describe how to enhance BW as well as Control-M to overcome these limitations. On the BW side, you need to make configuration changes, implement a Business Add-in (BADI), and create an ABAP program to send information to Control-M. For Control-M, you need to write a PERL script that picks up information sent from BW and delivers it to the Control-M console.

My example uses data loads in BW 2.1 done by InfoPackage groups (Figure 1). You can use InfoPackage groups for BW 3.0, but it offers a new scheduling tool, a process chain, which is more powerful than InfoPackage groups. Nonetheless, scheduling data loads with BW 3.0 and process chains still requires development similar to what I describe below.

Figure 1

Loading data using InfoPackage groups in BW 2.1

BW’s InfoPackage groups allow multiple InfoPackages (data loads) to run at the same time or sequentially. Using Control-M scripts, you can schedule and monitor InfoPackage groups in the same way as R/3 jobs. You could then monitor those jobs through Control-M’s Console GUI. Figure 2 shows how multiple loads could be organized in BW as an InfoPackage group.

Figure 2

How multiple data loads are organized by an InfoPackage group in BW

To schedule an InfoPackage group, use transaction code SM36 or click on the Selection options button of the Maintain InfoPackage screen (Figure 3). When executed, each InfoPackage group generates an SAP job with parameters taken from Selection options of the InfoPackage group (Figure 4). One issue here is that individual loads are not visible to Control-M, so it recognizes an error only if that InfoPackage group produces an ABAP dump—a rare event during BW loads. Most of the errors are reported internally, such as master data inconsistency or non-permitted characters as shown in Figure 5. These errors are invisible to Control-M.

Figure 3

Click on Selection options to schedule an InfoPackage group

Figure 4

Enter the InfoPackage group scheduling information on this screen

Figure 5

Control-M does not recognize data-load errors like these

The other issue is that Control-M can show only one status per InfoPackage group. Ideally, operations would like to see a status for each data load in that InfoPackage group.

To resolve these issues, you must make a few enhancements to BW as well as to Control-M. These enhancements primarily create placeholders in Control-M for each InfoPackage in the InfoPackage group (Control-M enhancement) and deliver the status of each BW load to the Control-M (BW enhancement). Figure 6 outlines the interaction between Control-M and BW once these enhancements are made.

Figure 6

Control-M/BW interaction flowchart

The file system of the operating system on which SAP runs (e.g., UNIX or OS/390) serves as an environment for the BW and Control-M interface. This way, BW creates files with information regarding InfoPackage group content and settings, as well as the status of completed loads. Control-M reads those files and deletes them after the InfoPackage group load is finished.

Control-M doesn’t have any information about how many InfoPackages are in the submitted job. Providing this information to Control-M requires an ABAP program. This program:

• Uses the InfoPackage group name as a parameter.

• Selects all records (InfoPackages) from the InfoPackage group definition table (RSPAKPOS).

• Has a path to a directory as a parameter.

• Creates a file with the InfoPackage group name in the given directory. This file has one record for each InfoPackage in the given group. When that InfoPackage (data load) runs, it creates a file with the name written in this record.

The ABAP code should be a stand-alone program and called by a master job, as shown in Figure 7. In this screen, I’ve named the program ZBWSCHEDULE_MSTRPGM. Figure 8 demonstrates an example of a file (BI_BTCHMSTRDATATTR_D.txt) that this ABAP program generated for the InfoPackage group shown in Figure 3. Figure 9 illustrates the contents of that file.

Figure 7

An ABAP program, ZBWSCHEDULE_MSTRPGM in this case, provides information about the InfoPackages to Control-M

Figure 8

File created by the ABAP enhancement to BW

Figure 9

The contents of file BI_BTCHMSTRDATATTR_D.txt

In addition to the ABAP code, you need a PERL script that picks up the information that the ABAP program generates and sends it to Control-M. This PERL script:

• Has the same parameters as described for the master job (job name and path where file is located) in addition to the regular Control-M parameters.

• Submits the job to SAP for execution.

• Creates a “file watcher” task for every record of the file discussed above. These file watchers are waiting for files to be created with the name listed in the file created by the master job. Once a file is created, the file watcher reads the status from the file. The file watcher returns the status found in that file.

• Removes all the files described above once its task is complete.

This PERL script should be placed on the server with Control-M. Control-M invokes it every time a BW InfoPackage group starts. Figure 10 demonstrates how the master job (QBWMSPAD01) and file watchers appear in the Control-M Console.

Figure 11

The Control-M console showing the master job and file watchers

You need additional configuration and programming to create the files for which the file watcher looks. The configuration is done in BW for each InfoPackage of the InfoPackage group. It enables the use of a BADI, a new type of SAP-delivered enhancement similar to a user exit.³ The BADI executes at the end of the load and triggers the status-creation program.

In this case, you want to use the BADI BW_SCHEDULER, accessible through transaction code SE18. You can implement BW_SCHEDULER using the IF_EX_BW_SCHEDULER interface (Figure 11). To configure BW so that the BADI executes at the end of load, check off the Execute Business Add-in (se18 or se19/BW Scheduler) option as shown in Figure 12.

Figure 11

Implement the BW_SCHEDULER BADI using the IF_EX_BW_SCHEDULER interface

Figure 12

Configure BW so that the BADI executes once the data loads

Figure 13 demonstrates the parameters for the USER_EXIT method of the IF_EX_BW_SCHEDULER interface. You implement this method, which I’ve labeled ZTESTSCHEDULE_BMC, through transaction code SE19 in BW. Figure 14 illustrates how this implementation links to the interface.

Figure 13

Parameters for the IF_EX_BW_SCHEDULER interface

Figure 14

Use transaction code SE19 to link the USER_EXIT to the interface

An ABAP program, the beginning of which is shown in Figure 15, enables this implementation. This program:

• Uses the parameters of the user exit listed in Figure 14

• Creates a unique file name in the same way as was done for the master program, based, for example, on the DataSource name, Source System name, and so on.

• Creates a record using the status parameters OK or NOTOK and text that’s used by Control-M for error reporting.

Figure 15

The ABAP program that implements the USER_EXIT

Figure 16 shows a list of files created by such BADI for the InfoPackage group in my example. Figure 17 shows the contents of the first file created by the BADI when the load of InfoSource 0COORDER has a “green” status.

Figure 16

The files created by the user exit

Figure 17

Contents of file BI_BTCH0COORDER_ATTRR3QCLNT260.OUT

The file watcher picks up this file, delivers the status to the Control-M monitor, and then deletes this file. The Control-M Console displays completion statuses for the InfoPackage group (Figure 18).

Figure 18

Completion statuses shown in the Control-M Console

On a final note, I’d like to make you aware of three cases where files with statuses should not be created, but they are anyway. First, refer back to Figure 2. Note that all InfoPackages are checked. In real configurations, some of the InfoPackages could be unchecked in the InfoPackage group. Those unchecked InfoPackages should not have statuses associated with them.

Second, no status is created if you don’t check off the option to execute the BADI at the end of a data load (Figure 12). Finally, InfoPackages and InfoPackage Groups sometimes could be run manually (not by Control-M). In this case, Control-M’s PERL script is not running and files created by BADI aren’t deleted. So, the next time Control-M runs the job in all three cases, it incorrectly picks up the status from the old file. To ensure that no files are created in these three circumstances, you should extend specifications to your master program and USER_EXIT implementation program specifications.

For the first two cases, the master program should go through the appropriate SAP tables to check if the check boxes are on or off. If at least one of them is off, no reference to the InfoPackage should be included in the file that this program creates. In the third case, the BADI should check the user name that runs the load (usually it’s specific for jobs scheduled by Control-M and is set up by the Basis team). If this name is different from the default name, the BADI shouldn’t create a status file.