Keeping an SAP project on time is a difficult endeavor, typically with last-minute enhancements, resource constraints, and technical issues threatening to delay completion. In many situations, testing becomes an afterthought — completed hastily and without sufficient rigor.

One of the dangers of haphazard testing is that issues escape notice long enough to cause larger problems down the road. Without an extensive formal testing strategy, your project can end up costing much more than initially thought, says Nathan Sharp, project management specialist at Siemens. “The earlier an issue is found, the cheaper it is to fix it,” he says.

Siemens relies on a highly formalized testing process to ensure that projects are ready for go-live. As part of an internally-developed IT methodology called PM4Energy, the Siemens testing strategy is now firmly entrenched in all SAP initiatives (Figure 1).

Figure 1

The PM4Energy methodology

“It has become a way of life,” says Sharp. “The company experienced problems in the past when project managers didn’t take this approach so it’s not a hard sell anymore — they know what can happen if they don’t go through this rigor.”

For example, one of the fundamental tenets of the testing methodology is to test entire applications and processes, even if some parts aren’t being altered. Similarly, although your changes may only involve SAP systems, testing must include external applications as well. Siemens has also divided testing into five distinct categories to ensure consistency. See the sidebar “Five Types of Testing” for details.

Siemens also ensures that everyone involved in testing is aware of his or her responsibilities. One of the key tasks of the project manager is to define and communicate roles and responsibilities so that role owners clearly understand expectations and timelines, says Sharp. “If people are not completely sure what their responsibilities are then there’s a good chance the tests are not going to get done. It’s a two-way street to make sure both the project management and project team are on the same page in terms of expectations.” 

See Figure 2 for an example of a clearly defined chart of roles and responsibilities.

Figure 2

Communicating roles and responsibilities

10 Elements of a Successful Testing Strategy

When it comes to executing testing, the Siemens approach is to unite business and IT groups into one testing experience, says Sharp. He has identified 10 key elements of a successful testing strategy.

1. Test Approach Document

The test approach document covers the overall approach of testing: everything that will be going on, when, and by whom. It describes how issue tracking, change management, and progress reporting will be performed, and details what tools will be used.

Sharp enters all documentation into Microsoft SharePoint, allowing each member to have access to the same information. This way, process owners can then take the information from SharePoint and present individual test lists to each member each day to ensure everyone is clear on their responsibilities for the day. “All that prep may be weeks or months of effort,” says Sharp, “but then the test itself is a well-oiled machine.”

2. Test Plan

Having a solid test plan written and approved before the end of the build phase is important to the success of the testing phase. The test plan should explain relevant procedures for all test activities, describe the types of testing, and include all terms and abbreviations. It should detail prerequisites and testing goals, and illustrate what is in scope and what is out of scope.

Sharp underscores that the test plan is different from a project plan. “All the calculations regarding when you should do the test and how many people are on it have already been done separately,” he says. “This is a document that anyone could pick up, including the testers, and understand what they’re supposed to be doing and the process that they’re following, in what is a concentrated one or two week period of time.”

3. Test Schedule

Properly organized test schedules (Figure 3) ensure that resources are available well ahead of time. If resources are requested too close to test time, you run the risk that your top choices will be unavailable for testing. Therefore, create a test schedule months in advance to request staff members, time blocks, and other necessary resources, such as conference rooms.

Figure 3

An example test schedule

4. Test Script

To clearly communicate test requirements, create test scripts to be distributed to team members. The test script should contain preconditions, test cases and steps, and test data. Be sure that team members include both positive and negative results in order to get a complete picture of the process, according to Sharp.

5. Kick-Off Presentation

A kick-off presentation is held to get everyone on the project team on the same page. In this meeting, the test lead explains the reason for the testing and defines the desired results so members can see what success looks like for the project. He or she explains the process, documentation, and tools in detail so everyone is clear on what needs to be done and how, says Sharp.

The project sponsor may run this meeting, but the role can also fall to a business person or manager. Approval from the role owner’s manager must be granted before documenting the role owner in the test approach document. Once the process is underway, sub leads may be needed to help the test lead process the daily activity of testing, which will increase as scripts are run and issues begin to be logged.

6. Test Issue

Generally, a test issue is a bug in the software — something that isn’t performing as described in the functional specifications. Bugs are documented in SharePoint and receive priority treatment to resolve the matter. Encourage testers to log issues as soon as they are found so immediate action can be taken, says Sharp

However, issues can also be classified as an enhancement — something that wasn’t in the original specifications but could be improved upon. Enhancements don’t receive the same level of attention as bugs, but testers should be encouraged to document these as well and if time permits the issue will be addressed. “There are probably more enhancements coming out than bugs,” Sharp says, “but we can’t keep adding scope for obvious reasons — we’re going to run out of time and budget.”

Test issues should be handled via a clearly defined process that identifies the responsibilities of each role in the test and ensures that each issue is resolved or accounted for, says Sharp. A sample test issue process can be seen in Figure 4.

Figure 4

A clearly-defined process ensures issue resolution

7. Beginning of Day Presentation

A team meeting at the beginning of each testing day not only presents the agenda for the day, but also allows the team to go over the project’s progress and major issues together. This way, everyone involved can regroup daily to discuss problems and best practices. The beginning of day meeting also clearly presents each member with what is expected of them that day so that testing stays on track, according to Sharp.

8. End of Day Status Report

At the end of each day, data from SharePoint can be extracted and imported into a program such as Microsoft Excel or PowerPoint to create easy-to-understand charts and graphs, says Sharp. These visual aids help the team and interested sponsors stay updated on the project’s progress. In some cases, it may be necessary to highlight testing backlogs and identify the teams or people responsible for them (see sidebar).

The end of day status report can be turned into an end of day meeting if face-to-face meeting time is required. However, this should only be done if absolutely necessary, Sharp says, as the more meetings that are held during the day the less time there will be for testing.

9. Testing Closure Presentation

On the last day of testing, a test closure presentation should be held to present what the team has achieved. This should be more formal than the beginning of day meetings or end of day reports, Sharp says. During this meeting, present the data collected over the course of testing and discuss whether testing was successful as compared to the initial goals.

10. Test Summary Document

Compile conclusions from the test closure presentation to create a test summary document. The document should include the results of testing and necessary next steps. Include sign-off approval of the testing and distribute the document to team members and project sponsors.

Other Resources

“Implement Proven Testing Practices and Techniques for Large-Scale Global SAP Rollouts,” by Jose Fajardo, SAP Professional Journal

"Build Bullet-Proof Testing Strategies to Comply with Legal and Industry Regulations,” by Mitresh Kundalia, GRC Expert

“Best Practices for Speeding Up Your Application Testing,” by Davin Wilfrid, ERP Expert

Laura Casasanto

Laura Casasanto is a technical editor who served as the managing editor of SCM Expert and Project Expert.

You may contact the author at lauracasasanto@gmail.com.

If you have comments about this article or publication, or would like to submit an article idea, please contact the editor.