Data volume continues to increase among organizations of all sizes and verticals, and many have turned to business intelligence (BI) tools to make the best use of their data. These BI technologies are key to integrating, analyzing, and leveraging all of this data, but for those just starting this process, the question remains — what kind of BI approach should we take to reap the most rewards? An agile BI approach can reduce the time it takes for traditional BI to deliver value to an organization and also helps to quickly adapt to changing business needs. SAP’s Neil McGovern recently took readers' questions on how to discover the benefits of agile BI and how to incorporate an agile data mart directly into your SAP HANA platform to meet your organization’s specific data and analytics needs. We welcome you to view the chat replay or read the full, edited transcript below. Meet the panelist:      Neil McGovern, Senior Director, Marketing, SAP Neil has more than 25 years of experience in the software industry. He is responsible for product marketing for SAP HANA analytics, including SAP HANA, SAP Business Warehouse, SAP Event Stream Processor, and SAP HANA Advanced Analytics. Neil also has led product marketing for the mobility and financial services groups for SAP/Sybase. Prior to Sybase, Neil was VP of Engineering at New Era of Networks, and CTO of Convoy Corporation, where he was a pioneer in the Enterprise Application Integration market. He has a degree in Computer Science and an MBA.   Transcript Neil McGovern, SAP: Hi. I’m Neil McGovern from SAP. Today we are going to focus on agile data marts. The key is agile. This session will address the concepts of analytic agility, the delegation of business intelligence practices from the IT department to the rest of the organization. IT has become overloaded with demand for answers, and allowing the rest of the organization to access and analyze data as independently from IT as possible will speed up decision making and free up IT resources to focus on strategic rather than tactical goals. The agile process is not the same as agile development. They are frequently confused because some of the goals are similar — such as shorter times to market, fail-fast, etc. — however, the culture change required in IT to adopt agile development is similar to the culture change required by line of business (LoB) — and IT — to adopt agile analytics. This is not a “How do I do this in SAP HANA?” session. We have an SAP HANA expert online for some of the time to address these questions, but most will be part of the follow up after the session. I’m interested in your questions about agile analytics and how in-memory databases such as SAP HANA can help deliver self-service analytics to business users so they can iterate quickly (i.e., fail-fast) and improve business decision making.   Natalie Miller, SAPinsider: Hi Neil, thank you so much for joining us today. Welcome to you all and thank you for joining today’s live Q&A. I’m Natalie Miller, features editor of SAPinsider and insiderPROFILES, and I’m thrilled to have today’s panelist, Neil McGovern, Senior Director of Marketing at SAP, joining us today to answer questions on the benefits of agile data marts. Comment from Venkat: Could you please help me to understand the difference between SAP HANA operational and agile data marts? I know that SLT (SAP Landscape Transformation Replication Server) is possible in operational data marts, but not in agile.   Neil McGovern: Operational analytics is prescriptive — it answers questions such as, “How many days of inventory do I have based on the last three months of sales?” or “Where am I vis-à-vis my budget for spending in such-and-such a category?” Agile analytics is focused on answering questions that have not been asked before, such as “What can I do to increase productivity?” and so on. The key is that an organization has to grant access to atomic data to business analysts and allow them to sandbox with the data, integrate new data sources, etc. Comment from Anthony: With analytic agility, isn’t there still a need for IT or a graphic modeler to create entity relationships or with the SAP HANA views?   Neil McGovern: The goal of agile data marts is to allow ad hoc binding of data sets, and as such, ERDs are a valuable resource — tools such as SAP PowerDesigner are very good for developing physical and logical data models to describe the data. In addition, the key to getting results via an agile approach may be more of a network relationship between data sets as well as a more hierarchical approach. Comment from Venkat: Is it best practice to develop ODS (operational data source) in SAP HANA?   Neil McGovern: It is a current best practice to develop ODSs with SAP HANA. The potential for real-time reporting via faster queries is a key value. However, the latency reduced via faster queries pales in significance with the latency reduction achieved by removing replication. As we will see from our early SAP S/4HANA customer successes, the ability to perform operational analytics on the application data, rather than the warehoused data, has a much greater impact. Comment from Ted: It has proven very difficult to get good performance from models that join replicated ECC tables at run-time. How do you address complex joins and performance of models without transforming data into a star schema or other optimized format?   Neil McGovern: The current ECC schema was designed for transactional throughput, not analytics performance, and thus you are correct — a lot of transformation needs to be performed to get it ready for reporting purposes. One of the goals of SAP S/4HANA is to eliminate this issue. In the interim, BW plays the role of “making sense of the data,” of course. Comment from Venkat: Can we use SAP HANA for OLTP database for a Java application? Can we develop reports on top of OLTP?   Neil McGovern: Yes, SAP HANA can be used for Java applications. In fact, I’d go as far as to say that SAP HANA is great for Java applications — all that in-memory performance is a great boost. Comment from Ted: So are you prescribing the creation of a library of data marts or models that users can then join together to build their own agile view?   Neil McGovern: Yes. One of the disadvantages of the current approach LoB is taking to overcome the issues addressed by an agile data mart approach is that they are using Microsoft Excel to download and crunch their data. Data in Microsoft Excel is independent from all other sources and invisible to IT. Even the “chaos” of multiple, often redundant, data marts in an IT visible container is dramatically preferable. There is a strong recommendation from Gartner to aggregate an organization’s business analysts into a coherent group so they can share dispersed data marts. Comment from Ted: So how do you see users using SAP HANA in an agile data mart? In my experience, they lack the skills to add or change data models, so all they can do is consume already built models.   Neil McGovern: SAP HANA is an underlying technology that can deliver answers to ad hoc questions in real time. It is only part of the total agile data mart solution, as you correctly pointed out. However, the ability of business analysts with strong meta data understanding to slice and dice data, create interim tables, etc., is a key component to agile analytics. Agile analytics incorporates agile data marts, sophisticated analytics (spatial, text processing, etc.) as well as graphic tools to visualize the data. IT cannot be the only source of data-literate business users. Comment from smallavolu: How is agile process different in SAP HANA versus other traditional databases?   Neil McGovern: There are several key advantages to SAP HANA (this applies to all in-memory databases). First is the ability to “play” with the data in real-time, regardless of the amount of data required to answer a given query. This helps during the discovery process. Frequently, this results in views that are unique to the data mart (i.e., views that are not currently in the EDW). The second advantage is the publication of these views, which again may require considerable processing that needs to be performed in real time (i.e., for a dashboard). Thus, the key is performance and access to sophisticated analytics techniques (such as spatial, graph, etc.), coupled with the business knowledge that only LoB users have. Comment from Guest: What are the key benefits of agile data marts?   Neil McGovern: The top benefit is speeding up the answers to questions that have not been asked (or have been asked but not answered) before. The key is to reduce the pipeline of IT requests for new reports, dashboards, etc., by allowing users to manipulate the data themselves. Too many times business users get frustrated because they ask the wrong question, but only find out days or weeks later when they get the answer back from IT. The ability to sandbox is key to faster and better answers. From a technical standpoint, traditional EDWs struggle to quickly incorporate new data sources and mash up those sources with existing data — data marts are excellent at this. Comment from Mark: What are the strategic SAP tools most suitable to analyze/explore the data in a way that truly can return the answers that can be classified as agile analytics — for example, answering questions that reach into discovery of new insights? I mean from the business user perspective, not the back end.   Neil McGovern: I’m not an expert with SAP Lumira, however it is designed to allow users to play with the data and generate visual results. Comment from Guest: Can you give an overview of the basic concepts of analytic agility?   Neil McGovern: Sure. The key is to deliver fast, adaptable, responsive, and flexible business intelligence to business users. Part of this is reducing and ultimately eliminating the pipeline of data query requests that swamp IT without losing the key values that IT delivers, such as data consistency, security, etc. Agile data marts underpin agile businesses, resulting in the ability of a business to change faster and more profitably. Comment from Guest: With the agility to create data marts, there is potential to have numerous narrow or highly-customized data marts. What strategy works to mitigate this risk of creating hundreds or more data marts that can become difficult to manage and maintain in the long run?   Neil McGovern: This is a key question. Agile data marts are a more controlled version of the current chaos, which is data snapshots downloaded to Microsoft Excel then emailed around the organization — warts and all. The issue of fragmentation, while not fully addressed by data marts, is at least mitigated. SAP HANA allows for smart data access, so an SAP HANA data mart can use another SAP HANA data source and still maintain near real-time speed (essential for LoB to sandbox) while ensuring that the data accessed is fresh and correct. There will come a time when we only have one copy of all data in an organization, and that single copy will be used by applications, reports, etc. We have to remember that our current practice of replicating transactional data into an EDW then manipulating it to yield results is the result of performance issues with old technology. Comment from Ethan: Do I need a data mart or a data warehouse? What’s the difference between the two?   Neil McGovern: I think the best answer to this is that the schema of a data warehouse is controlled by IT, whereas the schema for data marts is controlled by business analysts. I predict this answer may get me into hot water. Comment from Guest: How do we start with an agile approach?   Neil McGovern: I’d start by identifying the most sophisticated business analysts and teaching them how to use SAP Lumira and SAP PowerDesigner. I’d then give them a high-performance database and set up access (without updating, creating, or deleting capability) to centralized data sets or views, and show them how to create their own interim data sets. Then I’d offer to show them how to use advanced analytics toolsets (SAP HANA has some great ones for spatial, text, etc.) and let them play around with the data. They are already doing most of this in Microsoft Excel already — so it isn’t a brainpower issue: it is a tooling issue. Give them access and training to a new set of tools that will make them more productive and allow IT to control the process to some degree. Comment from Tamas: What tools support the creation of such data marts today, and how do they make the creation of data models simple enough for end users to utilize?   Neil McGovern: At SAP we have SAP Lumira for data visualization, SAP PowerDesigner for meta data visualization, and, of course, SAP HANA for in-memory performance, so complex queries can be answered quickly. One of the key roles that IT plays is query optimization, which is less of a problem with an in-memory database — if a badly written query takes five minutes but can be optimized to one minute, it usually needs to be; if a badly written query takes 0.5 seconds instead of 0.1 seconds, nobody cares. Comment from Anthony: Is this approach tool agnostic? In other words, is it a best practice to use BOBJ or SAP tools versus a BI tool like CliqView? Neil McGovern: This approach is tool and even vendor agnostic. If you have a Gartner account, query “agile” and you will see dozens of articles describing the agile business approach (not to be confused with agile development methodology, although they have some similarities). We recommend investing in SAP tools because we work for SAP, but also because we know that most of our customers will be moving to SAP S/4HANA and the move towards a single data set for both analytics and transactions will be based on SAP application data and the tools used as part of these applications (SAP HANA, SAP Lumira, etc.). Natalie Miller: As we come to the end of today’s Q&A, I’d like to thank you all for joining us. And a big thank-you to Neil for all these insightful answers! Neil McGovern: Thanks for all the great questions. I’m not an SAP HANA Live expert, and also have not dived into the SAP S/4HANA schema (but I’m about to), so I’ll find an expert to answer these questions offline. Cheers, Neil.