Recently, I completed a data archiving project for a client, and I want to share some knowledge gleaned during my experience. I highlight important decisions that you need to make during the life cycle of an archiving project. You need to make these decisions at stages ranging from requirement analysis (RA) to actual implementation. I also discuss a few important transaction codes and their relevance in this process.

What Should You Archive?

During the RA phase of a project in which I was involved, the company had project system, FI- CO, materials management (MM), and sales and distribution (SD) consultants who compiled a list of archiving objects that cater to tables growing at an alarming rate. The company’s Basis team provided all the year’s records along with each table’s growth rate data using transaction code DB02. This information sheet acts as a basis for all future scope decisions.

Because data from modules such as MM, SD, FI, Asset Accounting (AA), and SAP ERP Human Capital Management flows to the controlling (managerial accounting [CO]) module, it’s difficult to archive CO data. For example, CO tables might contain certain data for the Work Breakdown Structure (WBS), cost center, profit center, or internal order depending on landscape configuration. To find out which objects should be used to archive CO data, SAP has provided two analysis programs: RARCCOA1 and RARCCOA2. When my client’s consultants executed these programs, they were able to understand the company’s CO objects selection.

The SAP system archives a set of tables rather than individual tables at a time. This method of archiving is achieved through an archiving object. For example, if tables BKPF and BSEG need to be archived, object FI_DOCUMNT is the recommended object. For archiving ANLA and ANLE tables, the AM_ASSET object is used; for COEP and COEPB tables, the CO_ITEM object is used. For the GLPCA table, the EC_PCA_ITEM object is used. For the GLPCT table, the EC_PCA_TOTAL object is used. (To view a list of all archiving objects available in the SAP system use  transaction code SARA.)

After you finish selecting the list of objects, the rest of the data that needs to be archived needs to be finalized in agreement with business and federal laws. For example, if an online system contains the past 12 years of data, and all data, excluding the current fiscal year plus five years, needs to be archived, the residence period in the system should be set as 60 months (12 months times five years). The SAP system does not archive running year data in most cases after you finalize the list of objects. Therefore, the next step is to create business rules for each document type.

Companies become deeply involved in this process because they need to understand their business processes related to document type posting and clearing scenarios. For example, in document type SA (related to the general ledger) both line items should be cleared. In document type KR (related to a vendor), a vendor payment should be completed. For document type ZA (a custom document type), clearing is not required. Document archival should be done only if the predefined criteria are met. Therefore, you should define rules carefully.

How Data Is Displayed After Archiving

During the archival write operation, the SAP system writes data in flat files that need to be archived. SAP has provided an info structure for each object, also known as an InfoSet. The required InfoSet for each object needs to be activated, preferably before the deletion of data from the online database. The InfoSet helps you retrieve data faster from archive files using most of the standard transactions and reports. Activation and data (an activated InfoSet is filled with data during a delete operation) in an InfoSet can be operated from transaction code SARI.

For a particular object, the SAP system often provides more than one InfoSet to work with various primary field combinations. InfoSet filter fields can be customized depending on individual requirements. An InfoSet can also be deleted and refilled from archived files.

An InfoSet stores primary fields of each row in online database tables and references the related data location in archive files. For example, an FI document number 1000001 contains 780 line items that are archived and stored in session 402, file 1, at location row 450, column 1. Therefore, the InfoSet contains the primary key (document number, plus year, plus company code) and a file name along with the coordinates (X, Y) of data in files. Whenever you execute transaction codes such as FB03, the system internally accesses the InfoSet, retrieves its file name and coordinates, points to the relevant archive file, and displays data immediately. This concept is known as the Archive Information System (AIS).

Another option is for a user to directly select the archiving object and file name under Database > Archive. However, this search takes time as the SAP system performs a search on the entire file, going through each record in a file one by one. Therefore, I do not recommend this approach.

For each object, SAP has recommended specific transactions or reports to pull up data from an archived database. For example, for FI documents the transaction codes are FB03, FBL1N, FBL3N, and FBL5N. For CO data, the transaction code is KSB5. In my example I use FBO3.

In the screen that appears (Figure 1) select the Database and Archive indicators. You also select the Archive Information System radio button. Press Enter to save your selections.

Figure 1

Select a data source

Reports related to the AM_ASSET module are capable of fetching data from archive files internally and therefore do not require any selection in the database. During execution if the report finds any data that is archived, a pop-up is displayed with the following question: Do you want to read archive data? You can select yes or no.

In SAP ERP Central Component (ECC) 6.0, if the AIS check box needs to be selected by default, this default selection of the check box can be achieved by maintaining the report or table transaction or program name in table ASDATASRC1 with an X indicator. Mapping of transaction codes and archiving objects can be done by maintaining an entry in table ARCH_REPOW.

Demystifying FI_DOCUMNT Archival Logic

FI_DOCUMNT is the object that is most widely used across any SAP archiving project. However, the trickiest part of this object is the logic coded inside this object for archiving. You can find out whether a document is archived by using transaction code FB99. If the document is not archived, reasons for each line item are displayed.

The common assumption is that if the document is reversed or cleared, or cleared before the residence period, it is archived. However, in my experience I find more cases in which the document is archived when it’s been neither reversed nor cleared.

After analyzing several hundred documents with various document types, I conclude that documents that fall within a residence period are archived if they meet the following criteria:

• All line items are cleared.
• All line items are profit-and-loss (P/L) type general ledger (G/L) accounts (with open item not enabled) even if none of them is cleared.
• One line item is a P/L account, and another is a customer, vendor, or reconciliation account.
• The customer, vendor, or reconciliation account is cleared and the P/L account is not cleared.
• Both line items are balance sheet or reconciliation accounts, and the customer or vendor type is cleared.
• Both line items are balance sheet types, and the document is not cleared.
• The G/L used is the same, and the sum is zero.
• If the documents are reversed.

Documents that fall within a residence period are not archived if they fit any of the following criteria:

• Both line items are different balance sheet/recon/customer/vendor types, and no clearing is performed.
• One vendor line item is cleared, but another item, goods receipt or invoice receipt (GR/IR) account, is not cleared. Because another item is a recon account without clearing, archival does not happen.

Explanation

The SAP system considers the reconciliation account, vendor account, customer account, and balance sheet account for archiving only when line items are cleared. The system archives (P/L) line items even if they are not cleared.

The SAP system does not check any relation of documents with their source data. For example, the SAP system archives AA document types related to asset acquisition and AF (depreciation posting) document types related to asset depreciation even if these documents’ related assets are deactivated. Document type TR, which is related to treasury instruments such as swaps, options, fixed deposit, and mutual funds, is archived by the SAP system even if its instrument is not closed.

Demystifying AM_ASSET Archival Logic

The AM_ASSET object is unique in many ways. This object archives master data and transactional data automatically. Therefore, if you want to archive only master data, make sure to put this object in the exclusion list.

For other objects you can execute a write job numerous times for testing. However, in the AM_ASSET object, if the write job is carried out by selecting a production radio button, no other write job can be carried out further for that fiscal year. The SAP system allows the delete operation only after the production mode write job is complete.

Whenever you run a write job in production mode, the SAP system makes table entries in table T093R. An SAP write program checks these T093R tables to determine whether an additional write job is allowed or not. To write again for the same variant and year, delete the session numbers from this table or enter an X in the row under the Test run column. For example, in Figure 2 you delete the numbers 232, 238, and 240 that are listed in the Run columns.

Figure 2

ECC 6.0 table T093R

Usually, when you execute write jobs, only a single session is generated. However, in this case, three sessions are created:

• One under AM_STEUER object (which are the customizing tables)
• Master data and transactions (mainly all asset tables)
• Posted values (mainly ANLP table)

A write program usually writes only as much data in archived files as is deleted by a deletion program. However in this object, the scenario is different. An asset is always archived when an asset has ANLC values or ANEP values for the given fiscal year (when the fiscal year meets the retention period). In such cases, values such as ANLA (contains an asset master record) and ANLB (contains asset depreciation details) are archived, but not deleted. Table 1 lists five types of asset tables.

The delete job again reads the written data into a flat file and deletes data that fits into residence criteria.  (The write job is followed by the delete job for data deletion.) If you are not planning to archive the PS_PROJECT object (i.e., WBS master data), I recommend that you not archive assets under construction (AUC) assets. However, the SAP system archives AUC assets also. To rectify this behavior, implement SAP Note 1893289.

Transaction values archiving does not mean archiving of AA and AF document types from BKPF and BSEG values. Transaction values pertain to the table ANEP. Note that the retention period can be configured only in years and not in months.

Result Analysis and Settlement Breakdown Post CO_ITEM Object Archival

Problem: After archiving a CO_ITEM object during result analysis, my team found that the system started taking almost three times as long for result analysis and settlement background jobs. The system even posted negative values for archived CO documents (FY 2004) in the current fiscal year (FY 2013).

Reasons: If a CO_ITEM is archived, it causes records from database table COVP (which contains documents data posted by result analysis) to be archived and deleted. Therefore, these records can't be selected by the Dynamic Item Processor (DIP) — which was used in my team’s result analysis method — anymore. In result analysis, the DIP always selects all cost records to the objects. These cost records are compared with the document flow of the dynamic items.

In my example, the SAP system selected a WBS element to which no cost records from database table COVP have been posted because my cost was booked on WBS and result analysis works on a WBS system that was not able to retrieve archived historical data. Also, because no COVP records were selected to these dynamic items, the system assumed that the costs were billed by mistake and thus have to be credited. Therefore, the system posted negative values of historical archived year data in the current fiscal year.

This error occurs because:

• The DIP created negative amounts
• More items need to be simulated, resulting in an error message
• The runtime was higher compared with the runtime before data archival

Resolution: Maintain the residence time of the DIP sources in the customizing (SPRO) node Project System > Revenues and Earnings > Integration with SD Documents > Creating quotations and Project Billing > Residence Time of the DIP Source, or execute transaction code ODP5.

Maintain a period of fewer than XX (for example, 81) periods.

Checklist for a Successful Go-Live

1. During the result analysis phase create a business rule sheet that acts as a basis for all future discussions. Try to maintain a hold on operation and time consumption for write and delete operations for all required archival objects. Work on all possible time reduction strategies, such as operations that can be carried out in a series or in parallel.

2. Break data of the FI_DOCUMNT archiving into fiscal year and periods as much as possible. Because the FI_DOCUMNT works in parallel, you can schedule multiple variants in parallel for write and delete jobs.

3. CO_ITEM archival cannot be carried out in parallel, as only one variant operation is allowed by the system at a time. CO_TOTAL object write can only be started for a year if data deletion for that year is completed by CO_ITEM object.

4. Using transaction code AOBJ, you can maintain transaction archival configuration for each object. I recommend that each object be configured with a file size of 10,000 objects per 100 MB. Therefore, the system creates a new archival file for whichever object completes first. However, be careful. If a 10,000-record file size is just 3 MB to 10 MB, you should put only 100 MB in the setting for that object. This reduces the number of archival delete jobs, thus reducing the monitoring and tracking effort.

5. Archival objects such as TRTM_FTR, used for archiving of treasury instruments, do not have a read transaction provided by the SAP system. Therefore, reloading of data and reading these objects are the only options for this object. Select this object only if you are comfortable with this behavior.

6. Try to work on environments that have production data replication as much as possible before actual production data archiving. By production data replication, I am referring to the system having the same hardware configuration as the production system and being refreshed with data from the production client. Production data replication provides you with enough confidence in estimating the time required for each year’s data archival and helps you come up with a solid go-live strategy.

7. Prepare a sheet for each object and each year’s data archival write and delete job’s duration, as you would for the production replica environment. This helps in deciding on a cutover plan; for example, to determine the time required for each object and each operation per year and per variant.

8. As part of customization almost all companies add Z custom fields to standard tables to save custom fields data on screen. Archival objects archive added custom fields from tables as well. However, the situation may vary for specific scenarios. Therefore, testing of such fields post-archiving is recommended in a sandbox environment during the result analysis phase.

9. Do not promise that all custom or standard transactions and reports will work after archiving. Only reports that support AIS are capable of reading archived data.

10. SAP has a list of transactions and reports that support the retrieval of data from an archived database for each object. You can get this Excel sheet by visiting the ILM section on SAP’s Web site.

11. Before archiving, use the Data Retention Tool (DaRT) provided by SAP. Using this tool, you can create data extracts that are archived and deleted from the online database after archiving. DaRT and archiving have no such association; however, DaRT extracts can be used later on for legal and compliance reporting. DaRT is used to meet legal data retention and reporting requirements. DaRT configuration allows you to configure what data needs to be extracted and where the files need to be placed.

12. When you use the transaction code SARA, prerequisites objects for that object can be found in network graph form (Figure 3). During planning this sequence needs to be taken into consideration. For example, before the CO_TOTAL object, the CO_ITEM object data deletion for a particular year needs to be completed.

Figure 3

Prerequisite objects to use when executing transaction code SARA

13. If you are planning to archive, for example, five years of data, don’t archive all the data at once. Archive just three years of data, test it thoroughly, and then move ahead. This testing should include not only read transactions and reports testing but also end-to-end business process testing — for example, time and expense posting, invoice posting, bill creation, result analysis, settlement, and execution of BW reports on data for reconciliation of data between the R3 and HANA reporting tool.

14. Plan for system load testing by carrying out all modules’ write and delete jobs in parallel. This helps in understanding how many jobs can be executed during actual production archiving.

15. Transaction code AS_AFB can be used to view all files created during a write job. It can display all data contained in the file and can search for a specific file with a specific field (just like an SE16 transaction code). If a foreground search takes too much time, it can be scheduled in background mode also.

16. Transaction code TAANA is very helpful for finding the number of records in a specific table and can segregate records depending on fiscal year, document type, or period. This information is helpful in a variant creation exercise. Virtual field search mode also plays a key role in finding the data count for a specific object type.

Gagan Pareek

Gagan Pareek has six years of overall IT expertise related to SAP ERP configuration and support. He has also been part of multiple custom solutions deliveries that involved integrating technology platforms such as .Net, Java, and Enterprise Portal with SAP systems. Gagan has worked on various domains such as capital market, portfolio management, and insurance. He holds an MBA in finance and has completed engineering studies at Mumbai University.

You may contact the author at gaganpareek@gmail.com.

If you have comments about this article or publication, or would like to submit an article idea, please contact the editor.