The ever increasing volume of valuable CRM data offers a challenge to delivering consistent system performance over time. By taking advantage of mySAP CRM data archiving, you can achieve benefits more quickly than with traditional R/3 data because CRM archiving objects have fewer complexities. These benefits include improved performance, compliance with policies and regulations, and synchronizing your archiving efforts with R/3 or SAP ECC. Archiving, which is a standard component of R/3 and mySAP CRM Basis, removes qualifying data from online tables and stores the information in archive files (Figure 1).

Figure 1

Archiving process

Archiving mySAP CRM data differs in several ways from archiving in other SAP applications:

• The volume of valuable but short-lived master data and transactions is high.
  
  
• The level of data interdependency among mySAP CRM, R/3, and Business Information Warehouse (BW) is high.
  
  
• SAP standard retrieval options for archived data in mySAP CRM is largely transparent and integrated in many common transactions.
  
  
• CRM data is less likely to have complex business retention rules.

I’ll highlight the most important steps to develop a successful archiving strategy for your mySAP CRM environment, explore considerations specific to CRM, and focus on standard CRM archiving options. Then I’ll discuss storage and retrieval methods.

Archiving Strategy

Before planning your CRM data archiving strategy, you should understand your organization’s motivations for archiving. Clearly understanding the goals will narrow or broaden your focus for each phase of the archiving process. Motivations for archiving include:

• Controlling CRM database size and growth rates
  
  
• Improving performance in high-volume CRM transactions
  
  
• Synchronizing R/3 archiving processes with mySAP CRM
  
  
• Complying with internal retention policies by destroying CRM data after a defined period of time
  
  
• Complying with industry regulations governing electronic records such as Sarbanes-Oxley and IRS Rev Proc 98-25
  
  
• Cleaning up errant or obsolete master data

The best archiving strategies incorporate as many of these motivations as possible to provide the best foundation for your organization’s future archiving efforts. The seven steps shown in the example CRM archiving project overview in Figure 2 are defined by seven time-independent milestone activities. Many phases can overlap or can be performed in parallel.

Figure 2

Overview of CRM archiving tasks click here for a larger version of this image

Durations are only one example of CRM archive project phases and do not illustrate typical efficiencies of performing certain tasks in parallel or with overlap. My example is based on a large global company with average resource scheduling difficulties.

The following seven steps should be part of your planning process for implementing mySAP CRM archiving:

Step 1. Perform a Technical Analysis

The foundation of your archiving project rests on a comprehensive technical analysis of table size, table growth, performance, technical dependencies, and data issues. Findings should include historical database growth, a projection of database growth into the future, and identification of the largest and fastest-growing tables.

Transaction DB02 often can provide the needed statistics (Figure 3). However, in rare circumstances in DB2, Informix, MSSQL, or other databases, functionality may be limited. In such cases it is best to contact your DBA for needed database, table, and index statistics. Go to the database system section and click on the Space statistics button at the top right.

Figure 3

Check on space statistics under the Database system tab via transaction DB02 click here for a larger version of this image

After downloading the results in Excel, you can project database growth (Figure 4). Database projections are usually based on historical growth rates (percentage of monthly growth). Consider spikes in your projections for developments such as upgrades, new functionality, and new company codes. The projections create awareness, but more importantly are used as a baseline for setting expectations in step 4 of Figure 2, “Develop archive strategy,” after archiving savings can be properly estimated.

Figure 4

Spikes represent new functionality in June 2005 and a planned upgrade in June 2006 click here for a larger version of this image

Launch DB02 again to identify your largest and fastest-growing tables. Go to the Table and Indexes tab shown in Figure 3 and click on the Space statistics button.

The resulting list displays the size and growth rate for each table in your mySAP CRM system. If you suspect your statistics are not complete because you are running Informix, MSSQL, or DB2, ask your DBA for a list of each table size and growth rate. Otherwise, download the statistics to Excel using System>List>Save>Local File. The initial display may require the use of the menu option all objects on/off. This ensures your results include all table entries.

Map the Tables to Archiving Objects

Since it is impractical to map the more than 30,000 tables that may be active in your mySAP CRM system to archiving objects, I recommend focusing on the largest 100 to 200 tables. In addition, it is critical to consider any additional fast-growing tables that may not make the 100 to 200 largest tables. These tables may represent new functionality that could quickly become one of your largest tables. Consider including any other tables that may not make the top 200 list but might be suffering from performance problems. You can usually find these via transactions ST03(N), ST05, and ST10. The remaining tables are most likely statistically irrelevant.

Some common CRM high-growth tables are:

• CRMD_ORDER_INDEX: index for CRM business transactions

• CRMD_SCHEDLIN: schedule lines of CRM business transaction items

• CRM_JEST: status information for the CRM business object

• PRCD_COND: conditions for a CRM business transaction (CRM Enterprise)

• SMOKONV: conditions for CRM business transactions (Middleware)

Now that you have identified the top tables you may want to archive, you need to map them to archiving objects. Go to transaction DB15 to map the tables to the archiving objects (Figure 5).

Figure 5

Transaction DB15 is split into two sections. The top half maps tables to archiving objects while the bottom half displays all tables that will be archived by a particular archiving object. click here for a larger version of this image

Some tables in Figure 6 require additional analysis. Upon mapping some CRM-specific or standard SAP tables, you will find that a single table may map to more than one archiving object. To map these tables correctly, you need to better understand the composition of the table in question. You can query many of these tables using SE16 or TAANA to determine distribution based on order type or time, for example. You can then assign percentages to multiple archiving objects depending on the outcome.

Figure 6

Detail of tables mapped to archiving objects click here for a larger version of this image

In some cases, a table may not map to an archiving object. You can usually find information about the management of the data in those standard tables through the SAP notes system. In the case of custom (Z) tables, although rare, custom archiving objects can be developed. However, other data management techniques should be considered.

The most common CRM-specific archiving objects are:

• CRM_ACT_ON: CRM activities

• CRM_CLM_CRM: CRM call lists

• CRM_COMP: CRM complaints

• CRM_LEADS: lead management

• CRM_LEAS: leasing contracts

• CRM_OPPT: opportunities

• CRM_SACONT: sales contracts

• CRM_SALDOC: CRM sales document

• CRM_SERORD: CRM service orders

• CRM_SRVCOM: CRM confirmations

• CRMD_BUS2000108: CRM lead management

• CRM_MKTPL: marketing planner

• CRM_SUR: product change statements

In addition to the above CRM-specific archiving objects, more than 75 common archiving objects may also be mapped to tables in your mySAP CRM system. For example, standard EDI and workflow archiving is possible. The final technical analysis results should show the data volume by archiving object (Figure 7). A complete ranking of archiving objects includes data management techniques where applicable (for example, SAP note 536414 shown in Figure 6).

Figure 7

Mapping data volume by archiving object/SAP note

Detect Data Dependencies

Data dependencies can prevent the archiving of particular objects, limit the amount of data archived, or disrupt business processes. Using transaction SARA, click on the network graphic icon, which is located on the top left corner of the menu bar, for the technical dependencies of each archiving object (Figure 8). Most CRM archiving objects are independent. Technical dependencies discovered using the network graphic icon might force your strategy to include a smaller archiving object first before you can archive one of your larger archiving objects. Step 3 reveals functional dependencies such as archiving sales contracts prior to archiving sales orders. Later in step 5, business complete and status issues may surface. All these dependencies not only contribute to the overall strategy, but also may increase your scope of objects.

Figure 8

Check for data dependencies click here for a larger version of this image

Fortunately for CRM archiving, one usual dependency involving BW does not always apply. R/3 environments present limited opportunities to load archived data directly into BW. Therefore, you must either wait until BW is loaded or create a custom solution for extracting the archived data. The rule of thumb for archiving R/3 data has always been: Load BW first then archive R/3 data. However, CRM to BW extractors are the only extractors that support the extraction of archived CRM data. They include:

• 0CRM_SALES_ORDER_I: CRM sales orders item

• 0CRM_SALES_CONTR_I: CRM sales contract item

• 0CRM_COMPLAINTS_I: CRM complaint item

• 0CRM_OPPT_I: CRM opportunity item data

• 0CRM_OPPT_H: CRM opportunity header data

• 0CRM_QUOTA_ORDER_I: quotations for sales orders item

• 0CRM_QUOTATION_I: quotations for contracts item

• 0CRM_SRV_PROCESS_H: service orders header data

• 0CRM_SRV_PROCESS_I: service orders item data

• 0CRM_SRV_CONFIRM_H: confirmations header data

• 0CRM_SRV_CONFIRM_I: confirmations item data

• 0CRM_SRV_CODES: service codes

• 0CRM_SRV_CONTRACT_H: service contracts header data

• 0CRM_SRV_IBASE_TRAN: IBase transaction data

Therefore, BW dependencies in CRM should not hold up your archiving efforts in your mySAP CRM environment in these key areas. For more information on extracting archived CRM data, see SAP note 643541. This note is updated frequently.

Step 2. Build Your Archiving Project Team

When you are building your archive project team, involve the following people: Basis/ DBAs, IS business functional process experts, tax experts and auditors, end users, the archive administrator, project managers, and realistically an external data management (archiving) consultant to guide the project. The project manager’s role is to keep the project on course and ensure appropriate consideration is given to each phase.

Step 3. Perform a Functional Analysis

To build on your technical findings, you need to conduct a functional analysis of legal and business considerations for each object. For example, you need to identify the necessary reports and transaction codes required to access archived CRM data, discover business process dependencies such as warranty/returns processing, and determine acceptable access times to archived data.

After compiling your list of top archiving objects, you need to talk to the business about each object. A mini-workshop is the best way to introduce the archiving project as well as the objects.Residence, retention, legal requirements, and business requirements should be discussed. Ultimately, the functional analysis reveals each archiving object’s specific residency, legal, business, and retention requirements.

An example of a business process that could be disrupted by archiving is return order processing. If warranty periods on particular products are three years and the return process for such warranty claims involves creating return orders with reference to the original order, archiving orders before the three-year warranty expires may not be a good idea. Return orders or credit memos that reference the original order do not work if the original order has been archived. The order must be online to “create with reference.” Therefore, the business process for processing returns under warranty can be changed, the archiving online residency period must coincide with the warranty period, or a custom solution can be developed to create return orders with reference to archived orders. These are the types of scenarios that should be discussed with the business in the mini-workshops.

Often, legal requirements may dictate that data must be destroyed after a certain period of time (usually seven years). Since archiving moves the data to structured archive files, access to the archived data is still provided through standard supported CRM transactions. However, destroying these archive files is the only way to destroy the CRM data. Data cannot be directly purged from your mySAP CRM environment without using standard CRM archiving objects first. Archiving is the first step in compliance with legal and corporate retention policies. These findings have a major impact on how much data can be archived and the development of your archiving strategy.

Step 4. Craft an Initial Archiving Strategy

Based on your technical and functional analyses, start with three to five archiving objects that are most in line with your original motivations. It is best to pick a mix of objects that address each of your motivations as a foundation for future archiving objects.

Based on your legal requirements, retention of data in mySAP CRM environments may be capped at seven years. That means that as your environment approaches seven years of age, all data must be archived. It would not be unusual to implement all of the CRM archiving objects over the course of a few years. Residency rules identified in the functional analysis reveal how much data is actually “archivable” based upon initial residency discussions. Look for objects with the highest returns in combination with the least business impact to tackle first. Residence and retention times should also be documented.

Step 5. Prototype Your Objects

Create a prototype to show to the user community. Depending on the quality and quantity of data available, this may be initially developed in a development environment. The selected archiving objects must be configured via menu path SARA>Customizing. The development system must also be prepared with an exchange directory using transaction FILE. It is best to search SAP notes for each archiving object before your first archiving run. Depending on your release and Support Package, many notes may need to be reviewed to identify particular issues.Ultimately, your prototype should display the impact of archiving in the online database and provide visibility to all available access methods. These findings should be presented to the user community initially involved in the functional workshop in step 3 for feedback and guidance in developing robust test plans.

Step 6. Test Your Prototype

After you complete your initial prototype in a development environment and share your results with the user community, it is time to transport your archiving configuration to a QA environment. QA environments are usually a copy of production environments and therefore contain production-like data that provides fertile grounds for production simulation.

Conduct three key tests:

• Initial test: Validate archivable data in the QA environment. Archiving objects should be run in test mode to see what percentage of data qualifies for archiving based on residency rules and business requirements. Issues regarding technical or business complete status interfering with archiving of target data should be presented to the business community as needed. Data cleanup may be necessary to continue archiving with desired results.
  
  
• Mass test: Based on run times in your initial test, create variants that run in an acceptable amount of time as you plan for production archiving.
  
  
  Tip!
  Always split your archive variants by legal entity and fiscal year where applicable. This allows you to create distinct archive files for subsequent storage and easily identify expired data upon its expiration in accordance with the defined retention period.
• Unit test: Test end-user business scenarios, transaction codes for access to archived data, reports, and other related business processes and transactions. Requests for transactions that are not currently archived enabled can be directed to SAP via the SAP notes system. Custom solutions are possible.
  
  
  Tip!
  Functional analysts should create test plans and scripts for related business process scenarios. A group of end users should perform the testing and identify any data inconsistencies, business process disruption, or additional transactions that should be archive enabled.

Upon completion of testing it is important to obtain user acceptance of the proposed archiving schedule and retrieval options before any production archiving takes place. Bear in mind that archived data cannot be reloaded into the online database.

Step 7. Finalize Your Implementation

When you prepare the production archiving schedule, ensure that all archiving object customizing and configuration are transported and correct. This also applies to any SAP notes applied in your development and QA environments. Ensure you have the appropriate authorizations to execute archive jobs in your production environment.

Storage of Archive Files

Production archive files contain business data that may not only be vital to conducting routine business, but are also required to provide necessary access to archived data. Access is mandated by internal and external auditors as well as judicial and governmental requests. Therefore, these files must be managed much like “live” SAP data and require backup procedures as well as disaster recovery scenarios. Furthermore, it is critical to be able to prove the files remain unaltered over time. You have many storage options available to manage archive files, from simple file system archiving to more robust third-party ArchiveLink-certified systems.

You can find an official list of SAP ArchiveLink-certified partners at www.sap.com/partners/directories/SearchSolution.epx. Most third-party storage systems that may already be in place at your organization for R/3 archiving and imaging can be extended to your mySAP CRM system. SAP makes it even easier by providing the familiar archive transactions found in R/3 such as SARA, DB15, and FILE.

You can test communications with third-party archive systems simply by using archiving object EXAMPLE. This object can identify any authorization issues at the exchange directory level as well as storage to the third-party archive system.

Ask these questions about storage:

• Do you have enough room in the exchange directory?
  
  
• Is a backup procedure in place?
  
  
• If you are using WORM (optical) jukeboxes, do you have enough platters, or open slots?

Next you need to define variants for your runs, execute and monitor your archiving runs, and plan the future archiving schedule.

Retrieving Archived CRM Data

You may have to convince your CRM team that it is a myth that you cannot retrieve archived CRM data. Many people associate archiving with deleting data. Although this is technically true, that is only half of the process. After archiving the data, SAP provides standard retrieval options to access the archived data in CRM that in many respects are much better than R/3.

See Figure 9 to see how to search for archived activities. Figure 10 is an example of a search for service orders. Figure 11 shows the display of data from the archive. You can also develop custom retrieval methods to provide precise reporting on archived information. For example, you can develop any report in SAP to consider both online and archive data. This may be an excellent way to provide visibility to archived data using familiar reports.

Figure 9

Example search for archived activities click here for a larger version of this image

Figure 10

Search for service orders click here for a larger version of this image

Figure 11

Figure 11 Display from archive click here for a larger version of this image