In the not-so-distant past, SAP systems were fairly isolated from the outside world. In those days, if you wanted to get data into or out of an SAP system, you basically had one of two options: You could import or export a file, or you could implement a point-to-point interface using SAP’s proprietary Remote Function Call (RFC) protocol. However, since the advent of SAP NetWeaver technology in the early 2000s, there have been significant advances in the interfacing capabilities of SAP systems. While this increase in supported interface technologies is a good problem to have, the changes have come at such a rapid pace that many ABAP developers have had a difficult time determining when and where to apply specific interface technologies. This article attempts to provide some clarity in this arena by surveying the various ABAP-based interface technologies that are supported in modern SAP NetWeaver Application Server ABAP (AS ABAP) systems. Along the way, I’ll identify the relative strengths and weaknesses of each technology, illustrating specific integration scenarios for which the use of a particular interface technology makes sense. The end goal is to derive a decision matrix that ABAP developers can use as a guide for selecting the right interface technology to suit a particular integration scenario.  

Batch File Processing with ETL

  Despite the many advances in modern middleware technology, the most common method of integration today is batch file processing using the extract, transform, and load (ETL) technique. As the name suggests, the ETL process extracts data from a source system, transforms it into a (typically proprietary) file format that can be accepted by a target system, and then loads it into a target system. Since almost all systems support file processing, the ETL technique often represents a least common denominator approach to integration.

Overview

From an ABAP development perspective, batch file processing requires little in the way of specialized skill sets. On the outbound side, you usually have an ABAP report program that reads some data from the database and outputs it to a file. On the inbound side, data is read from a file and used to post transactions using API functions (e.g., BAPIs). Often, the most difficult requirement is in figuring out what to do if something goes wrong. Here, developers must decide whether to halt the process, dump the error records to a reprocess log, and so on.

Pros and Cons

Table 1 describes some basic pros and cons for using the ETL technique to implement integration scenarios.

When to Use

In spite of all the problems with the ETL approach outlined above, batch interfaces still have their place in the ABAP world. In particular, the ETL approach is a natural choice for implementing the following integration scenarios:

• Initial data loads or conversions
• Implementing large-scale extracts with thousands or millions of records and large record sizes
• Integrating with legacy systems that may already have ETL-based integration scenarios in place
• Throw-away interfaces and prototypes for which speed of development is a crucial design factor

Aside from the basic scenarios outlined above, some serious consideration should be given towards whether ETL makes sense in the long run. Here, it is easy to get into trouble by letting short-term gains overshadow long-term software investments.

RFCs and BAPIs

Aside from the file-based ETL approach, RFCs and BAPIs represent the second-longest-tenured interfacing technology in the ABAP world. Based on SAP’s proprietary RFC protocol, BAPIs make it possible for external systems to communicate directly with an SAP system. For example, you can use RFC-based technology to interact with an SAP system from a Java or Microsoft .NET-based system.

Overview

Conceptually speaking, RFCs represent an SAP proprietary mechanism for implementing remote procedure calls (RPCs). Put simply, RPCs allow a program running on one machine to invoke a procedure in another system or address space with minimal setup on the part of the programmer. Figure 1 illustrates this exchange between SAP NetWeaver AS ABAP systems as well as third-party systems using RFC-based connectors. These days, there are several different variants of the RFC protocol that can be used to build more advanced integration scenarios using RFC technology. Table 2 describes each of these protocol variants. For the most part, the extended protocol variants described in Table 2 are primarily geared toward internal SAP-based communication (e.g., from an SAP CRM system to an SAP ERP system using CRM middleware and qRFC). Typically, external applications use the sRFC protocol variant to perform real-time lookups or operations against business objects in the system. More often than not, these functions are implemented as BAPIs. Conceptually speaking, you can think of BAPIs as RFC-enabled function modules provided by SAP that conform to a set of design standards that are consistent across modules. So, whether you’re changing a customer master record using BAPI_CUSTOMER_CHANGE or creating a purchase order using BAPI_PO_CREATE1, the API has a similar look and feel. These days, you can find BAPIs to perform basic create, remove, update, and display (CRUD) operations on most of the major business objects within the system (see transaction BAPI for a complete list).

Pros and Cons

As is the case with most technology decisions, there are some tradeoffs to consider when thinking of implementing RFC-based interface solutions. On one hand, RFCs are highly efficient for implementing real-time calls against an SAP system because there’s minimal overhead at the protocol layer. On the other hand, such performance boosts are offset by complexity of development, lack of openness, and so on. Table 3 illustrates these tradeoffs. When to Use Up until just recently, if you were looking for an API for interfacing with an SAP system in real time, BAPIs were the only game in town. However, now that SAP has provided ABAP-based support for Web services, developers have several other options to choose from, most notably Enterprise Services. The major advantage these Web service-based options have over RFCs and BAPIs is openness based on industry standard technologies such as XML, SOAP, and WSDL. Given this, you should not implement new integration scenarios based on RFCs and BAPIs if you can avoid it — especially if there’s a corresponding Enterprise Service that could be used instead. In the event that you can’t find an existing Web service to leverage, another option is to use tools within the ABAP Workbench to wrap up an RFC or BAPI call as a Web service operation. One of the major benefits of using these tools is that they allow you to simplify the external interface by renaming obscure parameters with intuitive names that are more easily interpreted by developers existing outside the SAP space (e.g., renaming LIFNR to VendorNumber).

ALE and IDocs

Application Link Enabling (ALE) is a framework that provides for the reliable transfer of data from one system to another. This includes the transfer of messages from one SAP system to another, or the transfer between an SAP system and some external system. In either case, messages are transmitted asynchronously in the form of an Intermediate Document (IDoc), which is a SAP-proprietary document format used to transmit business messages.

Overview

When developing integration scenarios, one of the most fundamental questions to answer up front is whether messages should be transmitted synchronously or asynchronously. When it comes to reliable messaging, many interface architects prefer to develop asynchronous solutions in which some sort of message-oriented middleware (MOM) assumes the responsibility of reliably delivering a message to its intended recipients. This eases the burden on the sender system by decoupling it from the receiver systems that may or may not be available whenever a message is initially transmitted. As I noted earlier, IDoc messages are transmitted asynchronously, with the ALE framework playing the role of MOM. Thus, IDocs are well suited to implementing reliable messaging scenarios in which one system wants to notify another system of an event or milestone that occurred, such as the creation of a material master record or a G/L account posting. Here, for example, you can leverage the SAP standard IDoc types MATMAS05 or ACC_GL_POSTING01, respectively. From a technical point of view, IDocs can be transmitted to external systems in several different ways. Most of the time, IDocs are transmitted using the tRFC and qRFC protocol variants described in Table 2. However, you can also transmit IDoc messages via flat files, XML/HTTP, and so on.

Pros and Cons

Table 4 describes the basic pros and cons for working with ALE and IDoc technology.

When to Use

When developing new integration scenarios, it’s hard to ignore the wealth of software assets that exist in the ALE and IDoc space (especially for SAP ERP systems). Here, I’m talking about more than just the definition of some basic IDoc message types:

• You can use built-in function modules to generate outbound IDoc messages or process inbound IDoc messages.
• You can use many hooks to generate IDoc messages automatically whenever certain events occur within the system (e.g., the creation of a material master record or the modification of a sales order). Most of these scenarios can be implemented with little to no custom development efforts.
• You can use many built-in transactions or programs to search for messages, reprocess failed messages, and so on.

Rather than reinventing the wheel, it often makes sense to leverage this core functionality instead of creating new integration scenarios from scratch. This is particularly the case whenever you have a middleware tool such as SAP NetWeaver PI sitting between the ALE/IDoc layer and the downstream system that shields the external system from the complexities of IDoc processing. For new interface scenarios in which there is not an existing IDoc type, you should look to more modernized interface solutions as IDoc technology is slowly becoming deprecated. This is evidenced by the fact that new dimension systems within the SAP Business Suite are no longer delivering out-of-the-box interfaces based on IDoc technology. Instead, they’re standardizing around ABAP Proxies, Enterprise Services, and so on.

ABAP Proxies

Since the SAP NetWeaver AS ABAP 6.40 release, ABAP developers have had the option of generating proxy objects that can be used to interface directly with SAP NetWeaver PI. These proxies are generated in reference to service interfaces defined within the SAP NetWeaver PI Enterprise Services Repository (ESR) tool (which is based on Web standards such as WSDL and XSD).

Overview

As the term “proxy” suggests, ABAP proxies define a simplified interface that brokers the exchange of messages between ABAP programs on one hand and service interfaces defined in SAP NetWeaver PI on the other. This exchange is made possible via the ABAP Proxy Runtime that is included with every SAP NetWeaver AS ABAP system from version 6.40 onwards. Figure 2 shows the positioning of the ABAP proxies in relation to the ABAP Proxy Runtime and the SAP NetWeaver PI Integration Server used to broker managed communications. Unlike other ABAP development objects you might be accustomed to working with, ABAP proxies are generated objects that are created in reference to service interfaces defined in the SAP NetWeaver PI ESR tool. This generation process takes place within the ABAP Workbench in the Enterprise Service Browser perspective. From a technical perspective, ABAP proxies are implemented as ABAP Objects classes that define methods to either send or receive messages. Behind the scenes, these proxy objects take care of low-level protocol handling details and so on. As an ABAP developer, if you wish to send a proxy message, all you have to do is instantiate the proxy class and call the generated method used to represent the target service operation. In the other direction, the ABAP Proxy Runtime takes care of all this setup, so all you have to do is define an implementation for the server proxy method.

Pros and Cons

Table 5 describes some of the pros and cons for working with ABAP proxies.

When to Use

Given their tight coupling with SAP NetWeaver PI, the first thing you have to ask yourself when considering the use of ABAP proxies is whether you want to lock yourself in to the use of this middleware platform. For many companies, it is an easy decision because they already have a PI instance stood up. For those without SAP NetWeaver PI, the decision process is more involved. In general, I recommend using ABAP proxies only in situations in which you know that SAP NetWeaver PI is going to be in the interfacing landscape for the foreseeable future. Assuming that’s the case, ABAP proxies represent a robust, viable option for developing synchronous and asynchronous integration scenarios and should be preferred over legacy technologies, such as RFCs or IDocs.

Enterprise Services and SOAP Web Services

In the past several years, SOAP Web services have emerged as the standard for new interface development efforts. This is due in no small part to their openness based on well-known Internet standards such as HTTP and XML. This openness has leveled the playing field, allowing all manners of disparate systems to engage in SOAP-based communication scenarios. Since the 6.40 release of SAP NetWeaver AS ABAP, this list has been expanded to include SAP systems.

Overview

When you get past all the hype, message exchange with SOAP basically consists of wrapping up an XML message in a SOAP envelope and sending that message over the wire using HTTP (Figure 3). This base-level simplicity makes it possible for all manners of systems to participate in SOAP-based Web service scenarios. Here, you don’t need to worry about figuring out how disparate programming languages and systems communicate with one another because almost any system these days supports XML and Unicode out of the box. Indeed, all it takes to participate in SOAP message exchange is an XML parser (or even a string library) and an HTTP client and server. These days, ABAP developers have numerous options for creating SOAP-based Web services within the ABAP Workbench. You can create these services from scratch, generate them in reference to service interfaces defined in the ESR or a WSDL document, or even generate them off RFC-enabled function modules. What’s more, SAP NetWeaver AS ABAP provides a robust SOAP runtime environment that makes it easy to host these services. In addition to home grown Web services, SAP also offers a vast catalog of pre-delivered Web services that are intended to replace and expand the API defined via BAPIs. SAP refers to these services as Enterprise Services because they were designed from the ground up using SOA best practices for developing Web services within the enterprise. You can access the current catalog of available Enterprise Services at the Enterprise Services Workplace at https://www.sdn.sap.com/irj/bpx/esworkplace. If you take a closer look at an Enterprise Service definition, one of the things you’ll notice is that SAP has gone to great lengths to define the service interface in a vendor-agnostic manner. For example, you won’t find many SAP-centric terms within the message types. One of the primary goals here is to deliver a set of services that is intuitive and easy to use for any type of consumer. The upshot of this is that less time has to be spent haggling over low-level field mapping issues and the like. The more transparent the service interface, the quicker and cheaper the integration.

Pros and Cons

Table 6 describes some of the pros and cons for working with Enterprise Services and SOAP-based Web services.

When to Use

Web services represent the future of interfacing both within the SAP landscape and beyond. Therefore, if you’re developing a brand-new integration scenario, SOAP-based Web services should be one of the first places you look. Here, you have the option of developing new service consumers and providers within the ABAP Workbench or leveraging from the ever-increasing catalog of Enterprise Services delivered by SAP.

REST-Based Services

These days, there’s a lot of excitement around Representational State Transfer (REST)-based services and the simplicity they offer over SOAP-based Web services. Up until recently, these developments were of casual interest to ABAP developers because there was not a whole lot going on with REST-based services within the SAP landscape. However, all that changed with SAP NetWeaver Gateway, which you can use to quickly and easily generate REST-based services based on the Open Data Protocol (OData).

Overview

REST is an architectural philosophy set forth by Roy Fielding in a doctoral dissertation back in 2000. At its core, REST is focused on mapping resources (e.g., purchase orders, material master records) onto a URL-based addressing scheme that you can use to perform basic CRUD operations using methods defined in the HTTP protocol (e.g., GET, PUT, POST, and DELETE). From a technical perspective, this basically amounts to sending messages back and forth using XML and HTTP. However, unlike SOAP-based Web services, there’s not all the additional overhead of a SOAP envelope, headers, and so on. As a result, REST-based services are considered to be somewhat more lightweight and easy to consume. To a large degree, REST-based services are not used in many industrial-strength enterprise applications. However, they are quite useful in integrating simple clients, such as smart phones and tablets. While you wouldn’t want to use REST-based services to perform your next data conversion, you might run into situations in which it makes sense to stand up some REST-based services for mobile applications, for example. Technically speaking, ABAP developers have had the ability to create REST-based services since SAP NetWeaver AS ABAP 6.40. Here, because of the simplicity of REST, you can simply create an Internet Communication Framework (ICF) handler module to create custom services. For those that are not so well versed in the intricacies of HTTP programming, SAP NetWeaver Gateway offers a robust alternative. Here, you can generate OData-based interfaces (OData is based on REST principles) in a matter of minutes using the provided workbench tools.

Pros and Cons

Table 7 describes some of the pros and cons for working with REST-based services.

When to Use

Unlike some of the other technologies considered, REST-based services have a very finite set of use cases in an SAP environment. In general, the most common usage scenario for REST-based services is in situations in which external clients need to perform simple operations involving minimal data exchange. For example, you might use REST-based services to provide status updates from the system to mobile clients. For everything else, I recommend sticking with industry trends and standardizing around SOAP-based services.

Decision Matrix for Choosing the Right Interface Technology

At this point, you should have a clear understanding of the different interfacing technologies supported within the ABAP landscape. Now, the question is, how do you determine which technology should be used to implement a particular integration scenario? To guide you in this decision-making process, I put together a decision matrix to identify those interfacing technologies that best match the integration requirements (Table 8). Once you narrow down your choices, it should be easier to find the solution that works best for the problem at hand.