When you manage a global SAP implementation, you can automate multi-location regression testing to reduce the overall project execution cycle. The following tips are a result of my experience as the project lead for the testing of a global rollout project. It included multiple locations across the globe and involved major SAP functionality such as sales and distribution, materials management, warehouse management, SAP ERP Financials, SAP Controlling, and production planning. The following are the four phases in a global template implementation project I found relevant for planning and executing tests:

• Global Program Preparation Phase
• Global Business Blueprint Phase
• Global Realization Phase
• System Integration Testing Phase

The steps in this article were performed in SAP Solution Manager 7.0, but you can use this method with SAP Solution Manager 3.2 and 4.0 as well.

Global Program Preparation Phase

In the preparation phase, you plan your testing strategy. You can use SAP Solution Manager to reference Roadmap best practices when you are planning your tests. You create testing models for automation depending on your organization’s goal. For example, the goal could be a reduction in the project implementation cycle by using one of two approaches. You could automate all your test cases except the ones that do not lend themselves to automation. You would still have a few tests that need to be done manually, which I’ll cover in the “Global Realization Phase” section. Note that not all of these automated test cases would be used in every rollout site. The other choice is a partially automated testing phase. With this method, you automate only those test cases that are either repetitive over different locations or comprised of 70% repetitive test cases steps. A fully automated test phase approach may help with an earlier go-live because rollout sites would not have to manually create test cases for the functions that are repetitive. Therefore, many project managers recommend it for a multi-location rollout due to the short testing execution time. Once you select your testing model, you can select test automation tools and testing standards through the use of the Global ASAP Template Roadmap in SAP Solution Manager. The use of Roadmap transaction RMMAIN helps you understand the details of the Global Template Roadmap, as well as aspects such as the automation of test tool planning and the overall testing strategy (Figure 1). Along with Roadmap-provided strategies for test tool selection, the scope of the project plays a major role when you’re considering regression testing. This may involve SAP GUI-based transactions, Web-based transactions, or both. For SAP GUI-based transactions, extended Computer-Aided Test Tool (eCATT) is the automation tool recommended by SAP. eCATT is available with SAP Web Application Server 6.20 and later. The recorded script lies in SAP Solution Manager and executes in the quality server of the satellite system that is being tested. After execution, the test result is stored in SAP Solution Manager (Figure 2). For Web-based transactions, SAP recommends use of HP QTP as the automation tool because there is a standard SAP interface available between Solution Manager and HP QTP. For both kinds of transactions, the HP QTP and SAP eCATT combination is recommended along with SAP Solution Manager because they can handle various business testing requirements in the SAP landscape. Similar to eCATT, the execution flow appears as shown in Figure 3. From a testing execution perspective, the extent to which SAP Solution Manager is used is also finalized in this phase based on the overall project execution planning. The testing will either be managed using ASAP phase-based project management features or by independent test catalogs in SAP Solution Manager. If the complete SAP project is managed using SAP Solution Manager project management, use the ASAP-phase-based features. You should use the independent test catalogs if you are not using any project management features of SAP Solution Manager.

Global Business Blueprint Phase

The goal of the Global Business Blueprint phase is to prepare the Business Processes Hierarchy, which includes a complete list of business scenarios and processes in the scope of the implementation. You can create the Business Process Hierarchy with transaction SOLAR01 or by following the quality standards of the organization. These quality standards generally include predefined Microsoft Excel or Word templates. Logically, the Business Process Hierarchy in these templates should be the same as the hierarchy in SOLAR01. The business and IT teams work on the scope, which is then established based on project management’s expectations and the project’s allotted budget. The testing plan is based on those SAP implementation plans. I encourage you to use transaction SOLAR01 so that you can leverage the complete project management features of SAP Solution Manager, such as configuration, testing, training, and reporting. These system-based features cannot be leveraged if the traditional quality process (i.e., using the templates I have just described) is followed. From a testing perspective, this phase is crucial because test cases are prepared in the Global Realization Phase based on the business requirements. If the requirements change in the Global Realization Phase, then the test cases must also change, which can hamper your company’s testing efforts with change requests.

Global Realization Phase

Identifying and finalizing the automation scope and carrying out automation (including mock execution) are the key testing steps in the realization phase. You carry out configuration, development, testing, and training-related activities using configuration transaction SOLAR02 in SAP Solution Manager. The following steps are realization phase-specific testing activities. Step 1. Identify and finalize the automation scope. As the business configuration is carried out in the satellite development server, the unit testing of those activities is also carried out in the same server manually. The system integration test is carried out in the satellite quality server after you transport configuration with related data and development from the development server to the quality server for the integration test. Now that the test cases are ready, you can finalize the automation scope. As a rule of thumb, every unit test case for individual configuration steps should contribute to end-to-end business scenarios. For example, transaction VA01 is a unit transaction test case while the order-to-cash cycle is an integration test scenario. You do not have to execute unit test cases in the satellite quality server separately if they are covered in any of the integration test cases. Test case automation and final execution should happen only on the satellite quality server. Before starting the automation, you identify the automation-relevant business cases. Step 2. Automate your test. In the case of eCATT, test cases are automated using the eCATT transaction of SAP Solution Manager. After the automation of all test cases, a pool of eCATT scripts is available in SAP Solution Manager. If you use HP QTP to automate the test script, you can transfer it to SAP Solution Manager using the standard adapter connection mentioned earlier. In SAP Solution Manager, this HP QTP script is wrapped with eCATT test script using transaction SECATT, which is an option enabled and installed with the standard adapter. With either of the tools, the eCATT scripts pool will be available at the end of the automation phase. An automated eCATT test case is called eCATT Test Configuration, which is a combination of eCATT Test Script along with its eCATT Test Data. eCATT Test Script contains the end-to-end business scenario, and eCATT Test Data contains the parameters listed from this Test Script, to which the input values are given at runtime. The eCATT Test Configuration also stores the information about the satellite system where this test case is executed. You use transaction SECATT in SAP Solution Manager (Figure 4) to create all four components (eCATT Test configuration, Test Script, Test Data, and System Data). Step 3. Handle manual test cases. Even in the case of a completely automated test model, certain test cases may involve dynamic behavior or interface testing that is beyond the scope of automation and thus needs manual testing. An example is any transaction that in execution results in a table or grid control without a search icon. This table or grid contains multiple entries at runtime, and you won’t know where the required values are in the table until then. You can maintain manual test cases in two ways: upload the manual test document or create the manual test case in SAP Solution Manager’s SAP script format. The manual test document is a Microsoft Word document with a predefined test case template. Business analysts provide the details, such as the end-to-end testing scenario, transaction codes, and input data. The most common way is to upload the test cases, which is possible with the project management feature of SAP Solution Manager. Using configuration transaction SOLAR02, click the Test Cases tab shown in Figure 5. You then can upload manual test documents (Figures 6 and 7) and maintain them with test results after execution.   Step 4. Perform a mock test execution. As the test case of one business scenario is automated, you should carry out its mock execution periodically until the final testing phase. This ensures that the quality system behavior is uniform through the final testing phase. A failure of a test case means a change has taken place in the system either in configuration or in master data. You can then take necessary corrective actions, such as changes to configuration settings, to make sure the business process is running smoothly. At least two mock executions, including all test cases, should be planned before the final testing phase. This greatly helps overall test execution management and preparation of test data for the testing phase. You can carry out the overall mock execution with SAP Solution Manager using SOLAR02 test cases, test plans, and test packages with transaction STWB_2. The Project/Plan/Package combination is shown in Figure 8. If an organization is not using the project management capabilities of SAP Solution Manager, then it can group the pool of automated eCATT scripts under the Test Catalog using transaction STWB_1. The mock execution can also be carried out for test catalogs using transaction STWB_1, test plans, and test packages in STWB_2 transactions (i.e., the Catalog/Plan/Package combination in Figure 8). Details on the creation of the test catalog, test plan, and test packages are covered next in the “System Integration Testing Phase” section.

System Integration Testing Phase

The primary activities in this phase in SAP Solution Manager are the preparation of test plans and test packages for real testing. Some of the activities include test package allocation to testing teams, test data creation, execution, results verification, defect tracking, and reporting. Two ways of creating test plans are explained below. You select the method based on whether the project management or test catalog method is in place. If project management is carried out in transaction SOLAR02, then the test plan is created with transactions SOLAR02 and STWB_2. If no project management is in place, the test plan is created from the test catalog using transactions STWB_1 and STWB_2. I’ll show you how to use both. Step 1 (using project management). Create a test plan using transactions SOLAR02 and STWB_2. All the test cases should be attached in the Test Cases tab of transaction SOLAR02 of a template SAP Solution Manager project (Figure 9). You can copy the template from the SAP Solution Manager template project to the local implementation project. The template includes components that will be uniformly executed for all locations. You create a test plan using transaction STWB_2. Select the template project by clicking the TEMPLT_PRJ button in Figure 10. Provide a suitable test plan name in the Title field. I entered Asset Liability Management in my example. The user logged in to SAP Solution Manager is listed in the Person Responsible field. Click the green checkmark (execute) icon in Figure 10. Then select the appropriate test cases for this test plan by selecting or deselecting the check boxes in the Business Processes hierarchy on the left in Figure 11. Generate it by clicking the Test Plan button highlighted in Figure 11, and then you see the screenprint in Figure 12. Alternative Step 1 (using test catalog). Create a test plan using transactions STWB_1 and STWB_2. From transaction STWB_1, create the test catalog for all the automated test cases by adding an automated eCATT scripts pool to this catalog (Figure 13). When the automated scripts are ready, you add them in the test catalog. Follow a strict discipline of naming conventions for automated scripts to make the selection of the scripts easy. Along with these naming convention standards, you need to maintain an Excel-based scorecard that contains details about the business scenarios along with the automated script name that are in the scope of the current project. With the help of this Excel scorecard reference, you select automated scripts while creating the catalog, as shown in Figure 14. After you select the test scripts in Figure 14, you go back to Figure 13 by clicking the OK icon in Figure 14. Click the continue icon in Figure 13. A screen similar to Figure 15 appears and displays the test catalog you generated. Place this catalog at a suitable position in the SAP component hierarchy (Figure 16). In this case, this position is available if you navigate to Test Workbench > Test Organizer > Test Catalog. This helps you retrieve the catalog contents. Once the test catalog is created, use transaction STWB_2 to create a test plan. Provide the title and select the Test Catalog from the component hierarchy. With this information, create the test plan (Figure 17). Return to the main screen (Figure 12) of transaction STWB_2 after you save the plan. Select the test plan again and go into edit mode by clicking the Edit icon at the top of the screen. Select the appropriate test cases of the test catalog from the component hierarchy (Figure 18). Generate the test plan by clicking the Generate button. Step 2. Create test packages from transaction STWB_2 using Test Plans. Once the test plan is created, you can logically group the test cases under the test plan using transaction STWB_2 (Figure 19).These groups are called test packages. The creation of a test package is shown in Figure 20. All packages for the given test plan are as shown in Figure 21. You create the test plans from either of the methods in step 1. These test plans contain test scripts based on the selection criteria mentioned above. The option of dividing test plans into test packages is present in the test plan itself, as shown in Figure 19. You can see in Figure 21 that the sum of all the scripts from all the packages (Dynamic ALM, Static ALM) matches that of the test plan (Asset Liability Management) shown above them. In addition, the sum of the scripts from all the plans matches that of the test catalog (i.e., the entire project’s test scripts). Every script is part of a test package. Step 3. Assign test packages to a tester using transaction STWB_2. Once the test packages are created, their assignment to the tester team (Figure 22) can be carried out in transaction STWB_2. Then the system shows the test plan with test packages (Figure 23). Every tester can see his or her worklist (i.e., test packages for execution) using transaction STWB_WORK (Figure 24). Step 4. Prepare the test data. Once the worklist is allocated to a tester team, the team can proceed to create the test data, which is completed based on the test data of the mock executions of test cases. If the mock results were successful, then that data can be referred to for final execution. Step 5. Execute the test using transaction STWB_ WORK (Figure 25). The assigned testers execute the automated scripts in a group (as found in packages). Do this by clicking the Automatic Test button. A screen appears that shows all the scripts that can be deselected if needed. By default, all the scripts are selected for execution. The scripts execute one after the other, automatically logging the results in SAP Solution Manager and showing you the execution status (Figure 26) and a summary of the status (Figure 27). There is no need to maintain the test results separately. Note that a tester must execute a manual test case in the quality server and then manually record the results in the test document if there are manual testing scenarios that cannot be automated. The tester manually uploads the results in the test case using the attachment option and changes the execution status in transaction STWB_WORK. You should store these test cases with execution results either in transaction STWB_WORK or on a central file server for future reference. The test results of a global template project become a reference for the rest of the rollouts to other locations. You can change the expiration date of eCATT logs of successful automated test cases so you can use them as references in rollouts. Step 6. Handle test defects with transaction STWB_WORK through the error log. If you find a defect in a test case, you can log it using the standard support desk ticket feature of SAP Solution Manager. This feature is integrated with test management in the test case. You can register an issue with the support or help desk team by double-clicking the error log (Figure 28). In these cases, a business analyst looks into the issues. If an issue is related to input data or changes in configuration settings or master data, the system may provide clear error details. However, in the case of an error due to different screen sequences of the executed transaction with respect to the recorded one, the system provides more technical data (e.g., the screen number and program name) rather than the clearly stated error. In such a scenario, try to analyze the transaction manually and rerecord if possible. Otherwise mark such scripts for manual execution only. To see the support desk ticket creation popup, click the Create Message icon (Figure 29). You can route the tickets to the support team, which in turn can resolve issues using transaction CRM_DNO_MONITOR. The prerequisite for this functionality is configuration of support desk settings using transaction SPRO. Alternatively, you can use a Microsoft Excel-based template for tracking defects. Step 7. Run the test reports. During the system integration test execution, daily reports of results are required for tracking and reporting purposes. Different types of reports are available in transaction SOLAR_EVAL (Figure 30) that you can use to cater to these requirements, such as total number of test cases, number of testers with module or scenario assignment details, overall execution status (i.e., % completion, pass, and failure statuses), and major issues, if any (Figure 31).