With cybersecurity an ever-present topic across every technology platform, it’s understandable that 476 million individuals and more than 75,900 companies are customers of McAfee — a privately held organization that sells businesses and consumers software licenses to cybersecurity solutions that protect their computers, smartphones, and tablets from viruses, identity theft, and privacy invasions. Headquartered in Santa Clara, California, McAfee protects and manages its own software and systems with shared services run from an IT Center of Excellence (CoE) in Bengaluru, India.

McAfee has one global SAP ERP 5.0 instance running all its back-end finance, controlling, accounting, materials management, and order fulfillment processes. While other non-SAP systems are integrated for processes involving sales, licensing, and customer service, from an order-to-cash perspective, SAP ERP is the single system of truth for the company’s revenues and bookings. As a primarily partner-based sales organization that connects with many resellers and distributors, McAfee needs to ensure that these resellers and distributors have a positive user experience with its SAP landscape, and that they can quickly and easily punch their orders into the SAP system.

To set itself up for success in providing a streamlined experience for resellers and distributors, customers, and internal business users, McAfee is in the planning phase for an SAP S/4HANA migration, which will commence later in 2019 with a small test migration. The migration project was held back due to a more pressing implementation of SAP Revenue Accounting and Reporting, which took the majority of shared services’ resources during 2018. This implementation required upgrading McAfee’s accounting codes to ensure the business met the compliance deadline of the Financial Accounting Standards Board’s new revenue recognition standards, which was the close of January 2019 for private companies.

There was a bit of a perfect storm happening at McAfee during the implementation project. In addition to the looming compliance deadline to completely shift its financial reporting, the business was recently spun off from its parent company in a “clone and go” separation, which resulted in about 30% of its IT strength remaining. “When a company separates and sets up as a standalone, issues undoubtedly are going to occur, and updates continuously have to be released, tested, and confirmed to be working properly,” says Mouli Subrahamanayan, IT Director at McAfee India. “With this happening and the SAP Revenue Accounting and Reporting and other projects coming in, we still had to maintain our headcount at what it was.”

At the same time, the business was adopting an agile methodology for all its IT projects. In moving away from a waterfall methodology — in which a project manager oversees individuals who are each dedicated to quality assurance, testing the functionalities, and then user-acceptance testing cycles — McAfee adopted an agile sprint cycle strategy with all resources on the scrum team expected to collaborate on incremental development of stories released each sprint cycle, every two weeks. “Moving to an agile methodology, stories are consistently getting completed, tested, demoed for feedback, and revised, and you don’t have that hour of dedicated time that you’d have on a waterfall to test all your functionalities,” Subrahamanayan says. “With McAfee’s environment of approximately 40 systems for handling lead-to-order, order-to-cash, and source-to-pay processes that are actively worked on, enhanced, and enriched for the customer experience, there are moving parts all over.”

With all of that going on simultaneously, the CoE at McAfee needed to come up with some creative strategies to do more, faster. “It was up to the automation team to find ways to supplement that gap in human resources and achieve the high quality of delivery we wanted, especially in the testing landscape of the McAfee environment,” says Subrahamanayan.

Adopting Automated Testing

The testing landscape at McAfee historically operated with manual processes, which posed some difficulties when it came to the increased amount of testing during this time. “When changes happen in the system or new features are added, we have to ensure the system performs the same as before the changes were applied and fix any issues before those changes go into production,” says Vindhya Renukaprasad, Automation Manager at McAfee India. “Our biggest challenge was using manual processes to keep up with the testing because a lot of changes were coming in that we were expected to test in a very short period of time.”

Manually creating different test scenarios for end-to-end processes — such as order to cash — not only takes a long time, but also can be a costly endeavor. “By industry standards, a defect detected in development could cost about $100,” Subrahamanayan says. “But if the defect slips all the way to production, the cost of fixing it is almost 50 times greater.”

And it wasn’t just the SAP Revenue Accounting and Reporting deployment that was driving the increase in testing — there were a variety of other application changes McAfee put into place that required testing as well. For example, the business replaced its custom-developed system for configure, price, and quote (CPQ) processes with a third-party non-SAP system. With that change, the CoE needed to test the end-to-end scenario to ensure the quotes that were created for sales were properly converted into orders in SAP ERP and then fulfilled seamlessly. “This involved simulating end-of-quarter volumes for all end-to-end testing scenarios — that is, creating and testing 60,000 orders and validating the loads in the supporting non-SAP applications, such as the invoicing, licensing, and analytics software — so we knew that when we went to production, we could ensure that everything in that chain of events stood up to the volume and the systems behaved as they should,” says Renukaprasad.

Faced with the challenge of completing a large amount of testing in a short period of time, McAfee decided to pursue an automated testing approach. Implementing automated testing would allow time for agile-based scrum teams to focus on each end-to-end scenario and ensure that back-end systems are working as expected to complete customer orders and register license keys for McAfee software in a timely fashion. Automation would also set up the CoE to be much better prepared for the SAP S/4HANA migration on the horizon.

Implementing Automation Software

The automation team within the CoE was tasked with finding a solution that could automate testing for functional use cases that needed to be validated as well as automate business process execution. This team consists of around 13 members — a combination of automation engineers, manual testers, and business analysts. In April 2017, after evaluating the available software that could handle SAP testing and business process automation, McAfee called in a handful of vendors to provide hands-on demonstrations and showcase the software’s ease of use, which was a major decision factor. “If the solution was very cumbersome and sensitive to changes in the environment, then we would need to have a full-time automation engineer dedicated to doing only that,” says Subrahamanayan. “We gave each vendor a use case to build — for an end-to-end scenario involving a data stream from a custom application into SAP ERP then into a third-party sales system and back into a custom application — so that we could understand how easy or complicated it would be to use the automation software to build out the scenario. It also gave our engineers time to get their hands dirty with the software, take it for a test drive, and gauge their comfort level.”

After the demonstrations, McAfee decided that Worksoft Certify performed well above the rest of the competition. What tipped the scales for decision makers, according to Subrahamanayan, was the framework for SAP testing that came prebuilt within the software, which was one of the core requirements for the solution. As an added bonus, McAfee could apply the testing framework effectively to areas beyond the on-premise SAP system, such as the CPQ, workflow, and sales cloud applications, as well as new custom applications to be onboarded. “A Worksoft architect even extended the framework to build a special capability that could handle the level of scroll back we had in a multi-layered .NET page,” he says. “The interactions we had with everyone at Worksoft — from sales to system architects and technical support to customer success — and the amount of investment and dedication they showed was another huge deciding factor for us.”

By the third quarter of 2017, McAfee had finalized the test build and onboarded the automation, and by early in the fourth quarter, it had its first successful delivery into production. A Worksoft expert came to the office to help with the initial setup and execution and provided classroom training and mentoring sessions on how to work with Worksoft Certify. “In fact, they will continue to come onsite twice a year to make sure we are following the framework guidelines and not creating problematic situations,” says Subrahamanayan.

McAfee, however, doesn’t envision many problems in the future and expects user adoption to be wide and easy. “A big advantage of Worksoft Certify is that it is code-free, which is a real asset for people without that particular skillset of knowing how to write code, and it was exciting to realize that it is not only for SAP and supporting applications, but for web applications too,” says Renukaprasad.

Automation in Action

Currently, McAfee uses Worksoft Certify in its SAP environment for both testing and business process automation. Manual tasks, such as performing regular health checks on the SAP system, are now automated, and changes to the SAP system are tested through automation. “For every SAP patch or package that needs to be applied, we apply the patch and then execute Worksoft in the test case scenarios,” Subrahamanayan says. “There is no need for human manual testing effort to be involved in it at all, and the automated testing happens before the changes get to the production environment so any issues that we face can be worked on, escalated, and resolved at a much lower cost.”

The team is also extending the value of Worksoft automation for robotic process automation (RPA) with non-SAP systems and web applications. For instance, the organization has completely automated its “call off orders” business process, which is used when customers have a recurring need for expendable goods and invoice items are created as needed until the blanker order contract is fulfilled, the end of the order period is reached, or a predetermined maximum order value is reached. This process previously required 300-500 hours of manual, error-prone processes at the end of each quarter for salespeople to enter these call off orders into the non-SAP sales system, according to Subrahamanayan. “Now, there is no human intervention required — rather, a systematic automated process flow is followed, and salespeople can make better use of their time performing value-added activities, such as making new sales.”

McAfee is also using Worksoft Certify in ways that it didn’t anticipate. For example, the CoE uses a popular open-source DevOps tool to build the transports for software deployments, and it has integrated Worksoft Certify with this tool to automate the end-to-end build testing that needs to occur before the deployment moves from development to quality assurance. “We wanted to automate the automation so that our engineers are not strapped with the job of executing test cases, so we let our business analysts, scrum managers, and delivery teams trigger the build testing on their own from the open-source tool to the SAP environment,” says Subrahamanayan. “They then receive a complete report on the number of test cases executed and how many passed or failed, including screenshots of the failures with the error messages the system provided. This way, they can appropriately address and fix those issues without waiting for an automation team member to do it and share the findings.”

Additionally, the CoE used the solution as a performance tester to test bulk loads on downstream systems. “For example, we tested how the SAP system processes a volume of 60,000 orders with 250,000 line items. Then, we performed test calculations to make sure the SAP system was performing as expected before opening the flood gates on delivery to see how the entire environment handled the load,” says Subrahamanayan. “These were use cases that originally were not in the context of what Worksoft Certify was intended for.”  

As a result, the ROI was much greater than expected. “When building a business case, we had to project what our ROI would be, and we blew our original projection out of the water and gave back much more than we initially promised,” says Subrahamanayan. “Because of that software and the support from Worksoft, the automation team is getting the attention it deserves and has proven itself as capable of delivering on promises.”

Quantifiable Results

Through its automation journey, the CoE created the capacity for the agile teams to perform more valuable work, according to Renukaprasad. “With automation, the coverage and the quality of the deliverable has increased to much more than what manual testing could do,” she says. “And it has been helping us to take the deliverables much faster into production.”

Subrahamanayan adds that it’s more than just the velocity of delivery that is impressive — it’s also the number of additional stories getting delivered. “Regression testing previously was a two-day effort during our two-week sprint cycles,” he says. “But today, the regression is completely removed from the sprints’ scope of story delivery. Because those test cases are automated, the number of stories being delivered into production has increased more than 25%. Effectively, the same number of team members can accomplish much more than they could before.”

According to Subrahamanayan, McAfee saved close to 2,500 hours of manual effort through a combination of test and business process automation delivered to its production environment by using Worksoft Certify, which equates to roughly $200,000. And in addition to cost savings and efficiency gains, application maintenance is at an all-time low. “Once we started using our test case build and automation, we had zero major incidents caused by any of the applications on which automation was running, which is huge because it’s the first time in the history of McAfee that we have had a zero result like that,” he says.

An Automated Future

McAfee will continue to use Worksoft Certify as its preferred automation tool of choice as it moves to adopt SAP S/4HANA. “We want to make sure that we continue that quality of delivery for our business,” Subrahamanayan says, “and automation is going to be key to ensuring that the time to market is accelerated and assured.”