Fivetran SAP AppConnect Simplifies Data Integration to Cloud

SAP ERP data is central to many companies’ business processes that span across finance, sales, distribution, production planning and more. In fact, 99 out of 100 largest companies run SAP, and being able to access, combine and analyze data within SAP systems can help businesses drive more informed decision-making. But oftentimes, high-quality enterprise data is locked away due to SAP license restrictions and complex data integration solutions that risk business operations.

To address these concerns and limitations, Fivetran has developed SAP AppConnect. With this new capability, table selection and initial load is handled through the SAP Netweaver application layer. Direct access to the database is not required, and customers can still take advantage of Fivetran’s log-based change data capture (CDC) with the HANA Runtime license edition.

SAP AppConnect optimizes the initial load during the data replication process and simplifies data integration to ensure comprehensive visibility and access without any business disruptions. The ability to extract changes from transaction logs of various databases has the least impact on SAP applications because of the asynchronous nature of CDC. Changes are captured independent of the source application performing the changes.

The initial load of data is an important process in the setup of real-time replication. Many organizations have tables with multiple terabytes of data to transfer, and the goal should be to reduce risk to business operation and minimize initial load time.

‍Prerequisites

The following is required to implement SAP AppConnect for data replication:

• SAP ABAP version 7.5 and above
• SAP ECC 6.0 or S/4 application
• Oracle or SAP HANA database

Two main components of Fivetran SAP AppConnect 

The first is enabling Fivetran to communicate with the SAP application using the RFC (Remote Function Protocol). This connection can be made from the capture agent or the Fivetran Hub. SAP’s RFC libraries must be installed wherever Fivetran is connected. The connection to the SAP source can then be configured like any other connection in the Fivetran Hub, as displayed in Figure 1. Note that the user will be an SAP (background) user.

The second component of AppConnect resides on the SAP source system and is offered through SAP transports (*.kbr). Customers can upload, check and activate these transports through the standard SAP transport procedures. The transport will generate all the ABAP functions in the Fivetran namespace (/Fivetran/) in SAP. Since it doesn’t connect directly to the database, AppConnect doesn’t require a specific database user or schema.

After the location has been set up, operations are identical for users to other (SAP) sources and targets. In addition, features like refresh, compare and unpack (the ability to decode cluster, pool, and long text data) work with connections through AppConnect as they do with direct connections.

The most important part — capturing changes from the database log files — which leverages proprietary log-parsing capabilities for Oracle and HANA has remained unchanged.

Housekeeping queries around log reading will go through AppConnect and the SAP application layer. The log files can be online redo log files on the source machine, log files on a physical standby configuration, and/or backup/archived log files that reside on the source environment or a remote location. Fivetran processes changes with the lowest possible latency and impact on the SAP source system.

In short, this capability allows enterprises to perform log-based change data capture despite common license restrictions for databases managed by SAP applications, such as SAP HANA Runtime.