With the constant changes to business needs and the introduction of new technologies, companies are looking at solutions that reduce infrastructure costs, but still are able to leverage the latest products and technologic innovations. As a result, more companies are moving in the direction of a cloud model. Current trends indicate that more and more companies are investing in cloud-based platforms. In response, SAP has introduced its data migration cloud solution, SAP Data Services on Demand.
Key Concept
Previously, SAP offered BusinessObjects Data Services (now rebranded simply as SAP Data Services), which was a data-migration and data-validation tool. Now, SAP has released the cloud version of SAP Data Services for migrating data from on-premise systems to SAP cloud systems based on the old BusinessObjects Data Services on-premise tool. This article focuses on the features offered by the cloud SAP Data Services on Demand solution.
SAP Data Services on Demand is a data-migration tool based on the on-premise SAP Data Services 4.2 application. It is a cloud solution used to migrate data from on-premise systems to SAP cloud applications, such as SAP Integrated Business Planning and SuccessFactors, just to name a few.
Prerequisites The following prerequisites are necessary to access and work on SAP Data Services on Demand.
- Prior knowledge about data services data integration.
- Access to the SAP Data Services on Demand web link, which allows users to work on SAP Data Services on Demand.
An Overview of SAP Data Services on Demand
SAP Data Services on Demand gives users extraction, transformation, and loading (ETL) capabilities to migrate data from on-premise systems, such as SAP ERP Central Component (ECC) and SAP BW systems, non-SAP systems such as Oracle, DB2, and SQL Server, and flat files to the cloud. To use the SAP Data Services on Demand tool, companies have to have the SAP Data Services on Demand agent software installed on their on-premise systems. This agent acts as a connector to this web-based tool and does the tunneling work. It also has the following capabilities:
- It is a bi-directional process and it offers data integration. The data flow can be bi-directional in nature (i.e., not only can data be migrated from on premise to the cloud, but data can be pulled from the cloud and pushed on premise on an as-needed basis).
- It offers graphical flows and mappings.
- It has centralized monitoring and administration capabilities.
- It has pre-built adapters built in.
- It offers lightweight installations.
The Architecture of SAP Data Services on Demand
Figure 1 shows the architecture diagram for the SAP Data Services on Demand landscape. As represented in the diagram, the company landscape and the SAP landscape are separated by a firewall. All communication between the two landscapes is done via HTTPS protocol.
Figure 1
The SAP Data Services on Demand landscape architecture
To use SAP Data Services on Demand, you do not need a VPN, reverse proxy, or any other firewall exception. The data is streamed from the on-premise source to the cloud target securely via HTTPS and is never persisted, which removes the possibility of any data leakage.
The SAP Data Services on Demand agent communicates with the SAP applications via Remote Function Calls, which are encryptable via a secure network connection (SNC).
As pointed out earlier, the data flow is bi-directional in nature. SAP Data Services on Demand Agent uses long polling (e.g., it places requests to the server and waits for a response when the task is ready to execute). The communication direction is always from the SAP Data Services on Demand agent to the cloud.
How to Install and Configure SAP Data Services on Demand
Before using SAP Data Services on Demand for migration activities, there are two steps that need to be taken.
- Create an organization
- Install and configure the SAP Data Services on Demand agent
Create an Organization
To use the SAP Data Services on Demand services you first need to have an organization created for your use. Without an organization name assigned, you are not able to log on to the SAP Data Services on Demand services. To create an organization, you need to request one from the SAP HANA Cloud Integration operations team. Once the organization is created, an administrator is created for that organization who will do the administration jobs, such as user creation and monitoring.
SAP has multiple data centers, organized by region; therefore, the SAP Data Services on Demand URL is data-center specific. For example: https://<Data center Specific Address>/DSoD/session/logon.
The administrator logs on to the region-specific URL, and the screen shown in Figure 2 opens and asks for the name of the organization. Enter the name of the organization and click the Log On button.
Figure 2
Log on to the region-specific data center
This opens the screen in Figure 3, where you’re prompted to enter the user ID and password. Enter them and click the Log On button again.
Figure 3
Enter log-on credentials
Next, you need to finish the installation and configuration of the SAP Data Services on Demand agent on the client premises.
Install and Configure the SAP Data Services on Demand Agent
In the next screen that opens (Figure 4), select the Agents tab at the top of the screen. This is because you need to install and configure the agent first. Click the Download Agent Package link and it directs you to the SAP Service Market Place form (not shown) where you can download the appropriate agent, depending on your operating system (the SAP Data Services on Demand installation package is available both for Windows and Linux).
Figure 4
Download the agent package for SAP Data Services on Demand
The next step is to create a new agent instance. Click the New Agent link on the right side of the page (Figure 4). Once you click the New Agent link the screen in Figure 5 opens. Here you give the new agent instance a name and a description and assign it to a group (either new or existing). By default the group name is the organization name. You can create a new group if you wish. When you’ve completed your entries, click the Next button.
Figure 5
Create a name and description for the new agent, and assign it to a group
In the next screen that opens (Figure 6) you can see two steps. You have already completed Step 1—downloading the agent. Now you have to finish Step 2—download the configuration file to the local server system. Download the file now and keep it—this configuration file is needed later.
Figure 6
Download the configuration file and store it to use later
Next, you have to install the SAP Data Services on Demand agent you downloaded earlier from the SAP Service Market Place.
It is recommended by SAP that the SAP Data Services on Demand agent should be installed in a separate machine that doesn’t contain any other SAP application. As it is a lightweight program and does not use system resources, in our experience we have found that is OK to install it on a shared system. However, it is very important to note here that the agent should not be installed on a system on which another instance of SAP Data Services is already installed. This is because both the agent and SAP Data Services use the same ports and other resources, and they can interfere in each other’s activities.
If you are installing the agent on a Windows machine, simply run the .exe file you downloaded earlier from the SAP Service Market Place. (Conversely, if you’re using a Unix or Linux operating system, download the appropriate version to use.) During the .exe run the first screen that opens is shown in Figure 7.
Figure 7
Log on to SAP HANA Cloud Integration to configure the Data Services agent
In the Administrator user name field, enter the system administrator’s user ID (on which the agent has been installed), and enter the Administrator password (which is the system administrator’s password). You also need to provide the configuration file path (which you downloaded earlier). When the agent is installed for the first time, instead of the Re-specify the agent configuration file option, you see a field where you can select a file to upload. Once you browse to the directory where you have saved the configuration file and selected it, the Upload button becomes active and you can use it to upload the appropriate file.
Although the communication is via HTTPS protocol, if the company’s landscape is very secure and needs a proxy, you can give the Proxy host and port during configuration, so that all the communication is done via proxy. If the organization is using a proxy server for secure connection then you need to give the credentials of the proxy server to authenticate the agent to communicate.
After you’ve made your entries to the screen in Figure 7, click the Configure Directories option on the left. This opens the screen in Figure 8 where you define and add the directories that will be accessible to SAP Data Services on Demand during run time. Any text files kept in these white-listed directories are readable. You can also create files in these directories when doing a write-back from SAP cloud applications. To add white-listed directories, click the Add button and a pop-up screen opens where you can give the path of the directory. Once you close the pop-up screen you can see the directory you just added.
Figure 8
Register the directories that are accessible to SAP HANA Cloud Integration
Next you want to configure the adapters. Click the Configure Adapters on the left and the screen in Figure 9 opens. As of now, there are two adapters available in the Adapter Type field —SuccessFactors (shown) and another drop-down option, not shown, OData. You can change the configuration of these respective adapters. If you make any changes in this screen, click the Save button; otherwise, just click the Exit button.
Figure 9
Configure the adapters
Select the Configure PGP (Pretty Good Privacy) option on the left, and the screen in Figure 10 opens. Here you leave the entries as they default as, in this case, you are not using a PGP environment. After you’ve checked the entries, click the Exit button.
Figure 10
Check the configuration
Once the configuration is complete, log on to the URL used earlier (https://<Data center Specific Address>/DSoD/session/logon) and, in the screen that opens (Figure 11), check to make sure your agent is properly configured.
Figure 11
Log on to SAP HANA Cloud Integration and check the agent’s configuration
If, under the Status column you see a green square 
then the agent is properly configured and ready for the job. If the status is a red triangle 
then the agent has not been properly configured or the agent is down and you have to repeat the above steps.
An Overview of the Functionalities of SAP Data Services on Demand
Once the installation is complete, you are now ready to work on SAP Data Services on Demand.
First, let’s review the different tabs available on the tool. The first tab is GET STARTED (Figure 12). This tab explains the flow of development and execution of the jobs. It also contains help documentation for the tool. If at any point in time developers need any reference for development they can go here to find more information.
Figure 12
The GET STARTED tab details
The next tab is the DASHBOARD tab (shown in Figure 13). It gives a comprehensive picture of the number of jobs that were executed during a particular time period. You can change the time period parameters on the left in the Time Period field.
Figure 13
The DASHBOARD tab details
Next is the PROJECTS tab (shown in Figure 14). This is one of the most important areas in the tool. All the projects and the jobs are created in this tab. Also, you promote the jobs from development to production here.
Figure 14
The PROJECTS tab details
The DATASTORES tab is where data stores are created (shown in Figure 15). Data stores are the logical connections to the source and the targets to the Data Services tool. The data transform is the same in the DATASTORES tab as in the PROJECTS tab of the tool.
Figure 15
The DATASTORES tab details
The AGENTS tab (shown in Figure 16) contains the information about all the SAP HANA Cloud Integration agents that are installed and registered from the company landscape. Also, as mentioned earlier, from this tab you get the link to download the agent and the configuration file.
Figure 16
The AGENTS tab details
Under the ADMINISTRATION tab (shown in Figure 17) you find the functionalities that are used by the administrator. Here the administrator can do administrative activities such as user creation and deletion. The monitoring activities can be done here as well.
Figure 17
The ADMINISTRATION tab details
And, finally, the SETTINGS tab (shown in Figure 18) contains the user’s information (name, email, and so on) and his/her settings, such as language.
Figure 18
The SETTINGS tab details
Biswajit Biswas
Biswajit Biswas works at SAP GD and is a subject matter expert in SAP analytics. He has five years’ experience. He is proficient in the SAP BusinessObjects suite of reporting tools and SAP Data Services. He has been associated with development of Rapid Deployment Solutions for analytics on SAP HANA, focusing on the utilities industry.
You may contact the author at biswajit.biswas@sap.com.
If you have comments about this article or publication, or would like to submit an article idea, please contact the editor.
Virendra K. Soni
Virendra K. Soni is a Certified SAP Data Services Consultant and is currently associated with SAP. He has 6.5 years of industry experience in Data Migration, Data Conversion, and Data ware Housing with the Retail and Self-insurance industries. Prior to SAP he has worked with Capgemini, CSC and HCL Technologies.
You may contact the author at Virendra.kumar.soni@sap.com.
If you have comments about this article or publication, or would like to submit an article idea, please contact the editor.
Sunil Mehta
Sunil Mehta is a solution architect at SAP. He received his master’s degree in Computer Management from Symbiosis in Pune, India. He is a certified SAP FI/CO/BOBJ consultant, working in analytics. During his career he has been associated with Accenture, IBM, Capgemini, and KPMG, and has worked in various roles, including as a consultant solution architect and a project manager.
You may contact the author at Sunil.Mehta@sap.com.
If you have comments about this article or publication, or would like to submit an article idea, please contact the editor.
Premium
Premium
