How can you prepare now for optimal SAP GRC 10.0 performance, and what steps can you take to prevent problems post-rollout for your GRC 10.0 landscape? GRC 2013 speaker and Deloitte expert Kurt Hollis took questions on March 7 in our Compliance Forum discussion thread, moderated by Matt Moore, conference producer, GRC 2013.  You can view the full discussion in the Forum archives here, or read our edited transcript. Matt Moore, GRC 2013: Welcome to today's forum with GRC expert and GRC 2013 speaker Kurt Hollis for his technical advice on SAP solutions for Governance, Risk, and Compliance (GRC) 10.0. This is a great opportunity to ask Kurt your technical questions on implementing GRC 10.0 and maintaining the system after go-live. Kurt is a compliance expert, a featured speaker at our GRC conferences and a manager at Deloitte Consulting, LLP in the SAP Solutions Network. Welcome, Kurt, and thanks very much for joining us today! Kurt Hollis: Hello to everyone joining today discussion.  Thanks, Matt, for having me here today.  I will be answering your questions posting here during this session about the SAP GRC systems. Pedro Vaz: Kurt, What are the main step s to follow during an upgrade from 5.3 to 10? What are the usual errors to take in consideration? Kurt Hollis: The main steps to upgrade from 5.3 to 10 for Access Control are listed below: SAP provides a migration tool with capability to export data from existing Access Control 4/5.3 to Access Control 10.0 • Java-based versions 5.2/5.3 cannot be upgraded directly to ABAP-based version 10.0 due to the difference in platform • The migration tool provides following functionalities:

- Automated process with limited manual effort

- Ability to transfer configuration data, which includes SAP connectors, RAR configuration parameters, ERM configuration parameters, as well as CUP configuration parameters

- Export master data, including user roles and rule sets

- Ability to export data onto a spreadsheet and customize it before import

- Flexibility to filter and modify data before export

• Ensure SAP NetWeaver is at least 7.0 SP11 and AC 5.3 SP13 • Access Control 5.2 will first need to be upgraded to version 5.3 • Complete the configuration in IMG for the SAP connectors and new components

- Refer to the GRC 10.0 Configuration Guide for more information

• Create appropriate owners/users within Access Control 10.0 • Create the appropriate Org. Unit to import the RAR data • Before migrating CUP and ERM data, manually create all AC 5.3 custom fields in GRC 10.0 using identical names • Activate Business Configuration Sets (BCSETS) for 10.0 based on details in the migration guide • Upgrade the RTA in the ERP back end to the new GRC 10.0 plug-ins For Process Control, it is a direct upgrade to the NW 7.02/7.31 and then the GRC 3.0 --> 10 upgrade.  There is a guide from SAP with all the details for PC 3.0 --> 10 upgrade. Be sure to do the pre- and post-steps in the guide -- very important. (I cover this at the GRC 2013 conference this year.) Matt Moore: Kurt, thanks again for taking questions today. I also want to ask you about GRC sizing, which you cover in great detail in your GRC 2013 session. I think you mention SAP Access Control’s SoD risk tables as one big consideration. What do you advise about keeping SoD in mind for accurate sizing? Kurt Hollis: Hi Matt: Thanks for this very important key consideration.  The sizing and performance considerations are covered in four main points. 1.  The hardware sizing guides from SAP are OK for the individual pieces but do not take into consideration the entire needs of the running system.  Should be at least 16GB of memory (WebDynpro likes lots of memory and so does the newer NW releases) and at least 2 CPU, 4 cores are preferred for production, and the disk space for the DB is about 120-200GB.  Surprised? It is an analytics platform and the risk tables need space. 2. Correct rule sets help minimize the risks detected and loaded into the risk tables in the DB.  Make sure to spend time to get the rule set correct. 3. Risk analysis jobs are intensive. Set up parallel processing and make sure you filter the roles and profiles that are known to have excessive risks like SAP_ALL or roles with many auth that cause huge violations.  The job is to clean up these risks. 4.  Monitor the system and handle the performance issues according to best practices.  (My presentation at the GRC 2013 conference covers these in more detail.) KimberlyEdwards: We have just moved from 5.3 to the GRC 10.0 within the last two months. We are having performance issues in the Firefighter Controller area. The controllers are receiving their e-mail notification to approve Firefighter access, however when they go to approve it takes several minutes for each window to open up during the approval process. Do you have any suggestions for resolving this issue? We don't see this problem in any other areas. Kurt Hollis: Hi Kimberly, I have seen this problem and it is many times related to the memory and performance tuning of the system. It has to open up the WebDynpro screens and these consume more resources then normal.  I have seen the systems not tuned for this activity. I will post more details on this issue later so check back.  Also, some fixes to this are in later support packages from SAP for GRC and NetWeaver.  What release are you on and what SP levels? KimberlyEdwards: We are on SP8 at this time on 10.0. Kurt Hollis: Hi Kimberly:  I do not see an issue with that release level SP8, so that is good.  It has many of the performance enhancements. While this is running long time, I would have one of the Basis and DBA look at the system and trace the activity.  I was thinking a long search is happening and possible table issues. Also, I have noticed some DBA not tuning the GRC database with same attention as other SAP ERP systems databases.  This needs to be checked. See if the memory buffers are OK -- the ABAP program buffer should be 500,000 at least.  Instance parameters should be reviewed by your Basis team. I would suggest, after all this is checked, and if you are still having issues, to raise the issue with SAP using OSS message. malinirao: Hello Kurt, Can there be any performance issues if we create multiple clients in SAP GRC 10.0 system? Suppose we have one SAP GRC 10.0 Application hosted on two system landscapes but we want to create multiple clients for purpose of training. Will this create any performance issues? Kurt Hollis: Hello Malinrao: This is perfectly fine. What I have suggested to many is to set up one client as your source for the other future client copies. I have set up a client strategy like this, below: Initial copy of client 000 --> 100.  Then set up 100 to base settings and post-implementation completed. Use this to copy to other clients including your training client. Client 100 - master clean copy to use as source and golden config DEV, source of transports. No risk analysis data or jobs run here to keep clean.  Rules loaded is OK. Client 120 - unit test client for DEV Client 140 - sandbox client (useful for testing rule sets and changes) Client 160 - your training client Only warning: If you have large risk analysis tables in the system, client copy can duplicate this and make the DB table very large.  I filter this table during client copy and this help s.  That is the reason for the clean Client 100. I also would like to add that perfomance is generally not an issue; however, if running the sync and analysis jobs on multiple clients, you need to space these out on the 24-hour clock so as not to overload the system processing. I set up a new client in the DEV system to get a jump start on production risk analysis and ran the analysis in the DEV client. This is OK if you manage it carefully.  Jobs, data and rule sets are the focus of this effort. Kurt MohammedTouseefuddin: What is the best way of sizing to overcome any future performance issue? Kurt Hollis: Hi Mohammed, I can summarize the best way of sizing here: - SAP Web Dynpro requires more memory then typical apps; therefore 16GB memory minimum suggested for good performance - Batch jobs for risk analysis require more CPU and memory suggest two or four CPU cores minimum and 16GB memory - Database space: Initial 100-200GB plus yearly growth Additional space required for AC risk analysis data, which can start at 20GB and extend up to 200GB for the main tables! - Keep a correct Rule set, this pays off for performance. - Set up the filters for profiles and roles that are already known to have excessive risks from the running jobs. - Filter the users/roles/and profiles to those only needed for GRC analysis and applications, example, do not bring in the system all the SAP deliver roles. - Set up for parallel job running - Performance tune the system and monitor the performance, be prepared to make changes to the system if needed during first few months of operation. kumar05siva: Hello Kurt, I'd appreciate if you please shed some light o n "Maintain Connectors and Connection Types" under Common Component Settings. My RFC is working and the system user has all authorizations. Still, I get PFCG Authorization error. Thank you. Kurt Hollis: Hello Kumar05siva: Regarding the connectors and PFCG authorization error: - In the GRC security guide, we find the list of exact authorizations for the RFC connections.  Check the Plugin is matching version SP level to what you have here. - Re-check the authorizations again - try with SAP_ALL. The user must be system user. - The configuration for SPM requires the FF roles to be in the configuration on both the GRC system and the plugin configuration on the remote system.  Make sure the plugin is installed at the correct levels. Note that what you are seeing is not common. When do you see the error - during what application test? There are more steps needed for the connectors and the sync jobs need to be run before using the applications. The GRC 2013 conference covers this in detail, especially the Lab at the event. malinirao: Hello Kurt, Thanks for the response, I would like to know: If I am not migrating from AC 5.3 to 10.0 but going with a new implementation, can I still use the AC 5.3 rule set in AC 10.0? What are the options to use the same rule set as per AC 10.0 requirement? Do I face any performance issues due to this Rule set upload or do you advise creating new rule set in 10.0? Kurt Hollis:  Good question. The AC 5.3 rule set is perfect to use for the GRC 10 system.  This is loaded using import rule set. ZijadDzanovic: Hi Kurt, Can you please let me know if you have experienced any issues with EAM controllers not receiving logs into their work inbox? Kurt Hollis: This sounds like a workflow issue and for that I make sure all the workflow jobs are setup correctly and running first.  Then I check the workflow setup for determination agent assignments to verify correct.  Then look at the logs to evaluate the issues. This is hard work to fix -- have to analyze and find what is causing this.  I will ask my colleague about this one.  Look for a response later today. MarcNoergaard:  Hi Kurt Can you please share some more detailed information regarding:  How can monitoring help detect performance problems before they occur? Kurt Hollis: Hi Marc: Several challenging areas with monitoring: 1.  Monitoring the overall system and performance --  This is done using the standard Basis checklists for daily tasks.  Looking at memory, buffers, OS level performance, database, ST22 dumps, ST21 system logs, and SMICM for Web monitoring. 2. Monitoring the jobs running, especially the risk analysis and sync jobs -- Using SM37 to monitor the jobs. 3. Monitoring the workflows using workflow administration. Also, be sure to check the system DB tables for growth for the GRAC application. (More on this covered at the GRC conference in the performance presentation.) freddycortes: Hello Kurt, Can GRC 10.0 SAP systems connect to APO, BI, BO, PCM, BPC? Kurt Hollis:Hi: For Access Control, the SAP GRC Plugin allows connections to both ERP and any other NetWeaver ABAP-based system plus the SAP Portal. You can connect to APO, BI, BPC (10 version). Business Objects and PCM would need to be done through third-party connector or using Indentity Management.  I will try to get more information on this and post another reply. Kurt Marko Suswanto: How do you assess and optimize the GRC environment to perform background job synchorization (authorization, action usage, etc)? Since in my case, the date is very huge. Kurt Hollis: Hello Marko: Another great question.  How you assess and optimize the GRC environment for backgroup job sync involves several steps: 1. The auth sync, user/role/profile (object) sync, and the usage sync jobs are scheduled like below: GRAC_ACTION_USAGE_SYNC Daily Action Usage Job GRAC_PFCG_AUTHORIZATION_SYNC Weekly Profile Generator (PFCG) roles authorization      synchronization GRAC_ROLE_USAGE_SYNC  Daily Role usage synchronization GRAC_ROLEREP_PROFILE_SYNC Daily Role repository profile synchronization GRAC_ROLEREP_ROLE_SYNC Daily Role repository role synchronization GRAC_ROLEREP_USER_SYNC Daily Role repository user synchronization GRAC_SPM_AUDIT_LOG_SYNC Weekly Emergency Access Management (EAM) audit      log synchronization GRAC_SPM_LOG_SYNC_UPDATE Weekly Emergency Access Management (EAM) log      synchronization GRAC_SPM_WORKFLOW_SYNC Weekly Emergency Access Management (EAM)      workflow synchronization Batch Risk Analysis Job  Daily Risk Analysis Job Schedule the jobs so they are running at separate times.  Be sure the database is sized sufficiently. Run the jobs for one connector at a time, create variants to run the jobs, use incremental when possible. 2. The Batch risk analysis job is the most intensive job.  I run this using parallel processes and make sure the rule set is cleaned up for only the needed risks, and the filter for the roles and profiles is properly setup.  Monitoring of this job is necessary. 3. Action Usage Sync: This job will take a long time to finish when the job is scheduled for the first time or it has been a while since the last job was run. Sometimes it may take up to a few hours to complete. This job may be doing a full scan of table GRACACTUSAGE and get a large amount of data, which may take a very long time to complete. The best practice for scheduling this job is after the very first job is completed -- immediately schedule a next period job that runs ;every 4 hours, so that each job will select a much smaller amount of data and complete much quicker. malinirao: Is there any performance tuning guide created by SAP for GRC 10.0 as it was available for SAP GRC AC 5.3? Also, soon SAP GRC 10.1 version is expected to be released. Will you advise customers to go for 10.0, or wait until 10.1comes in? Kurt Hollis: Hi Malinirao: No performance tuning guide from SAP like before, but some notes about it: Notes 1583640, 1580877,  and 1584623. A performance guide is a great idea.  I have presentation for this topic at the GRC 2013 conference. The driver to 10.1 would be mobility and HANA supportability and other new capabilities, setting up for future enhancements.  But from what I heard, no major changes to existing capabilities. I would not wait unless I had a strong need for those things.  I would definitely install 10.0 on the new NetWeaver 7.31 EHP1 platform though, to be ready for 10.1. Kurt malinirao: Hello Kurt, Thanks for the response, I would like to know if I am not migrating from AC 5.3 to 10.0 but going with new Implementation, can I still use the AC 5.3 rule set in AC 10.0? What are the options to use the same rule set as per AC 10.0 requirement? Do I face any performance issues due to this Rule set upload or do you advise creating new rule set in 10.0?

MarcNoergaard: Hi Kurt, Another question from me.

Access Control: Have you experienced problems with the migration of rulesets from v5.3 to v10?

Kurt Hollis: Hi; Good questions. The AC 5.3 rule set is perfect to use for the GRC 10 system.  This is loaded using import rule set. No performance issues with this rule set from AC 5.3 in the GRC 10 system. Consider this: • Access Risk Management (ARA) comes delivered with SAP’s best practice standard delivered Rule Set (used as a starting point for new implementations).  The Rule Set is delivered in BCSETS and loaded using transaction SCPR20. • For those who already have implemented Risk Analysis and Remediation (RAR) (such as previous version 5.3) should have a customized Rule Set to meet the business requirements.  Rather then activate the new Rule Set delivered by SAP in the BCSETS in GRC 10 it is best to consider importing the customized Rule Set into GRC 10 using the SPRO Rule Set import function. • It is a good idea to evaluate the changes incorporated into the most recent SAP Rule Set to determine if the changes should be added to custom Rule Set.  Any modifications desired can be manually made using the Rule Architect feature of access risk management. • Administrators may decide it is easier to implement this new rule set (which will overwrite existing custom Rule Set) and just make the customization changes again to the newly imported default Rule Set.  This works best when minimal changes to the default Rule Set have been made. • See SAP note 986996 for an explanation of the Rule Set (has an attachment to the note). Some good info: -  A new Rule Set has been delivered Q4 of 2012. See SAP Note 1809810 - GRC - Access Control - Access Risk Management Rule Update Q4, 2012 - GRC 10.0 support pack 12 includes the latest Rule Set included in the delivered BC_SETS.  (Access Control 5.3 Support Pack 21 includes the latest Rule Set) - The note has an attached zip file called 2012_Q4_Rule_Update_Document.zip - The attachment summarizes all of the delta changes made to SAP's Rule Set for Q4 2012. Q4, 2012 Rule Set changes have been made in below 3 BCSETS: GRAC_RA_Rule Set_SAP_BASIS BC Set GRAC_RA_Rule Set_SAP_R3 BC Set GRAC_RA_Rule Set_SAP_NHR BC Set kumar05siva: Thank you Kurt. Daniel Uribe Santos: Hello Kurt, Is it possible install SAP GRC Access Control 10 on SAP NetWeaver 7.0? Thanks! Kurt Hollis: Hello Daniel: SAP GRC Access Control 10.0 must be installed only on NW 7.02, EHP2, SP06 or higher (currently SP12 is out). Or now you can install it on NW 7.31 EHP1, SP06 (preferred). NW 7.0 does not have the support needed to run this advanced WebDynpro application. One of the key ones is integrated NWBC. Also, you should only consider running GRC AC 10 in a new system and not on top of some other system you may have running like combining systems, not supported. Kurt ZijadDzanovic: Hi Kurt, Can you please let me know if you have experienced any issues with EAM controllers not receiving logs into their work inbox? Kurt Hollis: Hi Zijad: There is a job which must run to get this to happen. - Check the required job - the configuration m ust have YES for “send emails”. - All users must have email addresses maintained in SU01 and including the workflow user WF-BATCH. - Transaction SCOT is good place to check the SMTP email setup, monitoring and jobs. Kurt Matt Moore: Kurt, I know that you have several other questions to get to, but I'd like to ask about the hands-on lab (A practical guide to post-installation configuration of SAP Access Control) you'll be presenting at GRC 2013. Can you talk a little bit about the format and content of the lab? Kurt Hollis: Hi Matt: Thanks for having me at this forum. Yes, this year at the GRC 2013 conference, I have put together a very detailed hands-on lab that walks you through performing the entire setup of the SAP GRC Access Control 10 system, where you end up with a working system for Risk Analysis, Access Management, and Super User Management. Very excited about this -- we have GRC running on every laptop! Matt Moore: A full summary of all the questions will be available here in the Compliance Forum on Monday. And of course, I invite you to our annual GRC 2013 conference in Las Vegas, March 19-22. Kurt will be presenting a number of technical sessions on GRC 10.0, including a hands-on GRC 10.0 lab on March 19^th and 20^th that we’re very excited about! We hope you have the chance to see on of Kurt’s sessions in person in a little over a week. You can get updates on the conference by following me on Twitter @mattmoorewis, and you can discuss the event using the hashtag #grc2013 And finally, thank you to Deloitte’s Kurt Hollis for taking the time to respond to these questions.