The jewel of any organization is its data. As such, the review of backup and restore operations is one of the major controls addressed during an IT audit. This control element — backup and restore — is usually reviewed for effectiveness, relevance, and currency.

Furthermore, compliance with legal regulations such as the Sarbanes-Oxley Act, Health Insurance Portability and Accountability Act (HIPAA), ISO 27001:2005 (Information Security Management System), and the Gramm-Leach-Bliley Act (GLBA) are some of the drivers compelling organizations to take the issue of backup and restore operations seriously. Even though these regulations might not explicitly mention backup and restore as legal requirements, a common denominator for all these regulations is the emphasis on protecting and safeguarding corporate, business, and personal data against loss. More importantly, you need to be able to make them available when needed by defined authorities, especially auditors. Although a number of organizations have put in place infrastructures and technologies (such as clustering, disk redundancy [RAID], and replication) to safeguard data loss, the role of database backup cannot be overlooked in the quest for ensuring complete data security and system availability.

Consider a scenario in which an organization is required by law to produce its yearly financial reports from its enterprise system on a particular day. You find out that there is a media failure (e.g., hard disk crash) of the main production server and there is no backup from which to restore data. This could lead to serious litigation and fines, which consequently affect the profitability of the business. In extreme cases, it could even threaten the very existence of the company. An effective backup and restore procedure is an integral part of your business continuity or disaster recovery plan.

The management of backup and restore operations in an SAP environment can be cumbersome considering the different components and technologies of the numerous systems that typically make up an SAP system landscape. Therefore, it is expedient that an organization has well-formulated and documented procedures that govern the implementation of backup and restore operations. The backup policy must be comprehensive and address pertinent areas of the process — documentation, strategies, systems, and people.

This article is best suited for system administrators who are responsible for performing backup and restore operations in an SAP environment. It is invaluable for developing a new backup and restore procedure from scratch, or for improving an existing one. Also, IT auditors can gain knowledge of controls to check when carrying out disaster recovery (as it relates to backup and restore) checks. I’ll show you the recipe for an effective backup and restore procedure for your SAP system landscape, including:

• SAP system landscape
• Roles and responsibilities
• Data consistency management
• System uptime requirement
• Equipment and systems
• Change management
• Storage of backup tapes
• Restore test

However, before addressing these points, I’d like to provide a concise description of backup, restore, and recovery operations.

Backup, Restore, and Recovery in Context 

Backup involves the processes that permit data restoration in the event of a system crash or errors via previously copied data. Backup types include:

• Complete backup: A full backup of all files that you intend to back up
• Differential backup: A backup of changes to the database since the last complete backup
• Incremental backup: A backup of changes to the database since the last complete or differential backup
• Online backup: A backup taken while the system is operational
• Offline backup: A backup taken when the system is not operational

SAP recommends certain best practices related to backup operations in the SAP system. Your backup policy should make provisions to comply with these recommendations, including:

• Perform a complete online backup daily
• Perform a complete offline backup at least once every four weeks
• Perform a backup of offline redo log files daily and after every online and offline backup
• Perform backup verification once a week
• Maintain a backup cycle (i.e., a backup tape retention period) of 28 days 

SAP recommends that a complete backup be performed for all databases. However, you can perform an incremental backup to reduce the backup window, especially in a large database environment or where parallel processing of backup operation is an issue. It is also important to back up system files such as SAP system files, database system files, and operating system files.

The restoration of a database usually requires two closely related operations: restore and recovery. Restore implies the act of restoring data from a backup. Recovery deals with the act of applying the changes that were done on the database since the last backup. It is common to use the term “restore” to jointly refer to restore and recovery operations. Broadly speaking, recovery operations can be complete or incomplete. A complete recovery recovers the database to the last committed transactions. An incomplete recovery, on the other hand, resets the database to a specific point in time such as time stamp or log sequence number.

Now let’s go through each of the key aspects of an effective backup and restore procedure. All these areas are crucial in developing an effective policy.

SAP System Landscape

The system landscape is made up of systems that exchange data that is logically related from an application perspective. The system landscape directory is an invaluable functionality in SAP NetWeaver for providing detailed information about the SAP system landscape. The system landscape directory acts as a repository for all information about the SAP system landscape such as product and software components (e.g., Java, ABAP, database, and operating system), system roles (e.g., DEV, PROD, and QAS), and third-party components.

The system landscape directory enables you to plan for the necessary tools and infrastructure needed to support the software life cycle activities of your SAP system landscape, including backup and restore operations. Before you set up the backup policy for your organization, you need to thoroughly analyze your system landscape. This enables you to appreciate the specific requirements of the different components of your system landscape as it relates to backup and restore.

A system landscape can consist of different SAP systems, databases (such as Oracle, Microsoft SQL Server, and MaxDB) and operating systems (such as Solaris, Linux, and Windows) each with their specific requirements for backup and restore operations. You can use the data source (e.g., database or operating system file) information to drive the kind of backup (e.g., complete or incremental). A typical system landscape can consist of the SAP NetWeaver application server (such as SAP ERP and SAP CRM), standalone engines (for example, LiveCache, SAP gateway, and Internet Pricing and Configurator [IPC]) and client systems (e.g., mobile infrastructure client, SAP GUI, and business process explorer).

The backup requirements (e.g., files, tools, and technology) and type of backup of the various components can differ. For example, the data source of application servers and some standalone engines (e.g., SAP LiveCache) include databases and file systems. It is important to perform a full or incremental backup of business data and operating system files to ensure that you can recover completely in the event of a disaster while optimizing available resources. However, for some standalone engines (e.g., SAP gateway and SAP GUI) without business data, only file systems might necessarily be backed up. When implementing a backup and restore strategy, you should comprehensively analyze all connected systems and their corresponding processes.

Roles and Responsibilities

The definition of roles and responsibilities as it relates to who does what in the backup and restore operation is a key ingredient for a backup policy. The different tasks that different personnel can perform as it relates to backup and restore operations include backup scheduling, changing backup tapes, monitoring the success or otherwise of backup runs, update of backup logs, safe keeping of backup tapes, and restore test of backup tapes among others. People involved in a backup process must be aware of what they are expected to do in the entire process.

You need to ensure that some of these activities are segregated where possible to avoid compromises. For example, the person performing the backup (or backup monitoring) should not necessarily be responsible for performing the restore test. If a staff member is absent, the backup policy must be clear on delegation management. A team (or preferably a designate) must be responsible for making critical decisions about database backup and restore, especially as it relates to incident management in the event of a failed backup or restore test.  

Data Consistency Management 

An incomplete recovery is based on a point-in-time restore operation in which the database is reset to a previous point. Incomplete recovery is not advisable and should always be avoided as much as possible. This is because data is always lost since all logs are not fully applied during recovery. Incomplete recovery can lead to data inconsistencies especially when data is exchanged (via data communication technologies such as Remote Function Call [RFC]) frequently between multiple systems such as between SAP ERP and SAP CRM. SAP recommends that you do not perform an incomplete recovery of a production system.

Nevertheless, some scenarios require you to perform an incomplete recovery. Possible scenarios include when log files or database backup tapes are destroyed or cases of database corruption. Resolving business data inconsistencies is not an easy task and requires the use of data consistency check tools such as data integrity manager, transaction LX23 (stock comparison between inventory management and warehouse management), and transaction MB5K (stock consistency check). Furthermore, sound understanding of the affected business data is essential because resolving data inconsistency is beyond the technical team. The backup policy should clearly state how situations necessitating incomplete recovery are handled, especially related to follow-up activities aimed at eliminating inconsistencies after the restore operation.

System Uptime Requirement

Respect for service level agreements is a critical consideration when setting up a backup policy for your organization. The availability of the system should not be traded for backup and restore operations except in rare situations where that requirement is unavoidable. Depending on the type of backup, some backup options might require certain services to be down for the operation to be successful.

Backup and restore operations should not be scheduled at a time that might affect the optimal performance of the systems in the SAP system landscape. It might be necessary to consider the order in which systems in the SAP landscape should be backed up or restored. When scheduling, take into consideration the duration to complete specific tasks. This is important because of possible dependencies of activities. There might be a need to review previously defined schedules periodically because of the continuous growth of the database. The backup and restore procedure should clearly spell out the availability of the different systems and components in the SAP landscape with reference to the corresponding business process.

Equipment and Systems 

Aside from the standard functionality provided by SAP and the underlying database system, companies often implement third-party solutions to perform backup and restore operations. This is often because of the need for the provisioning of extended backup and restore functionalities. Different equipment (such as tape cartridges and tape drives) and systems play an important role in the backup and restore operation of an organization. The choice of tools to use for backup and restore can affect data throughout and consequently performance makes it an important consideration. The backup and restore procedure needs to be clear on permitted equipment and systems as it relates to vendor, product, and product version. The specification of the backup tools to be used must be stated explicitly in the backup policy. Also, you need to define the preventive maintenance schedule of associated backup tools to ensure that the equipment performs optimally at all times.

Change Management 

Generally, SAP recommends at least a three-system landscape consisting of the development (DEV), quality assurance (QAS), and production (PRD) systems. An important reason for this requirement is to make changes to the production system transparent while enforcing data integrity across all systems. The backup and restore procedure should make a provision for the backup and restore of production and non-production systems. Users are often quick to focus only on the backup and restore of the production system, while exposing non-production systems to avoidable risk of data loss. As far as backup and restore operations are concerned, all systems are important. What might differ is the frequency and approach. Non-production systems are where developmental activities (including customization) occur. If they are not backed up and data is lost, the implication can be adverse. It might entail a complete reconfiguration and distortion of the project plan.

SAP recommends that you perform a complete backup of your SAP system after every successful installation. Over a period of productive use of the system, you need to perform certain maintenance activities on the system, including the application of Support Packages, implementation of SAP Notes, kernel upgrade, database patching, upgrade, and migration. It is necessary to perform a backup of the SAP system before and after carrying out such maintenance activities. The backup policy should clearly state how these operations are handled for specific maintenance activities.

Storage of Backup Tapes 

SAP recommends performing backup on certain media, especially tapes. However, it is not enough to just comply with this requirement and store tapes in house. You need to make efforts to guarantee the safety of the backup tapes. A backup policy should address issues relating to the storage of backup tapes at off-site locations. In this context, an off-site location refers to sites outside (geographically apart) where the backup is performed. This practice is necessary because of events (such as wildfire, flood, and earthquake) that might adversely affect the data center where productive systems reside.

The backup policy must clearly define where the tapes must be stored and the attributes of the off-site locations. You should store backup tapes in a locked fireproof safe. The passcode to the safe must be kept secure. You also need to be careful regarding the temperature and humidity level of the location. You also need to define the expectation of physical security access. Furthermore, a system or backup log should be in place to manage the movement of backup tapes to and from the off-site location. The functions involved in taking backup tapes from one location to another and the consequent documentation should be segregated to enforce control.

Restore Test 

Often times, restore tests are not performed as expected. This is because there might not have been a need to perform a backup in the production environment over a long period of time. There is no point in performing a backup that cannot be restored. Although, the restore operation is done less frequently, it remains the most sensitive test of the veracity of a backup and restore procedure. The backup and restore procedure should clearly address how a restore test is to be performed in terms of frequency and approach. Restore testing should be done when it does not affect normal and optimal productive operation of the system.

It is important that you perform a restore test periodically (the more often, the better) to ensure that you can return to business in the event of a disaster. The result of the restore test must be reviewed and signed off by defined authorities in the organization. You also need to define and document the procedure for performing the restore test. Remember not to perform the restore test in the production environment. This is to avoid possible occurrence of data inconsistencies or corruption.

Kehinde Eseyin

Kehinde Eseyin is a security architect. He holds a bachelor’s degree in computer science. He has about 12 years of IT security, governance framework, IS risk, and compliance experience gained by working in numerous global organizations. Over the years, he has demonstrated competencies in security design, information assurance, cyber security, data privacy, threat and vulnerability management, penetration testing, business architecture, project management, IT audit, IS controls framework, and identity and access management.
 

You may contact the author at eseyinok@gmail.com.

If you have comments about this article or publication, or would like to submit an article idea, please contact the editor.