Thanks Ravi, for your reply - which does sound a lot like the kind of reply I would get from our IT department, This confirms to me that our IT attitude to what I am doing is not unusual, and they are likely following standard business IT practices in what they try to tell me.
Having said that, it also reinforces to me that there is a wide gap between what IT thinks the business needs and what business analysts really need. This gap is certainly industry wide and not limited to our company. I have tried many times to give our IT examples of why we need such huge amounts of data. Sometimes I am not sure they still understand. Maybe I will try with a few examples here - to see if the IT folks here can grasp it.
Why I need tens of thousands of records and not just 500?
Example 1: List Price Change: We have dozens of Price Lists globally (varying by currency) with about tens or even hundreds of thousands of skus on each Price List. When we review prices we must look at every record. We must download multiple attributes about each material, such as material type, cataolg codes, PLCM status, Inventory, Sales volume, costs and margin information, etc (all found in various SAP tables). Once in Access (or Excel) we will filter on various columns or subdivide to pass to different Product Managers for review. But this kind of exercise cannot be done without looking at every single sku, to determine whether we want to change the price and by how much. We have an ABAP program that can do parts of this, but we still must download and then merge with other tables to get us to what HANA can give us in a fraction of the time. HANA is at least 100 X faster in getting us the data needed just to even start these types of analytics.
Example 2: Sku level alignment: Recent decision to align certain aspects of the business based on certain material criteria required us to pull many columns of data at the sku level for over 1 million skus, then review with other teams in the company. Again filtering was needed to properly identify categories, but "summaries" or less than 500 rows at a time would never get us where we need to be. Also - as in most cases - predetermining the categories based on a handful of fields would not cut it. We really needed to review at the sku level.
Example 3: Price Waterfall charts are typically summarized from transactional details. However, the analytics to design this really requires a bunch of detailed pre-analysis at a transactional level requiring us to analyze lots of data in an easy to manupulate form. It is true, once we have completed the analysis we could then use some other means to pull small sets of summarized data as you have suggested. But proper anlayitics for projects like this often needs the ability to analyze large amounts of data easily and ability to "look" at the data to make decisions on the final format.
There are many business applications where business just need access to large amounts of data downloads (which include joining of multiple SAP tables) to make sense of the data (as well as for data governance - which I haven't even touched on here). HANA is the perfect tool - except we are often told by IT not to do things this way. I find this frustrating because we really cannot do our job properly without large amounts of data. We have looked at a couple of other tools, but so far found nothing that can manipulate large amounts of data like Access. I also feel frustrated when IT tells us they don't like how we do things, but can never give us a solution to support us. If Access works, then why are we always trying to fix something that is not broken.
I almost feel like all IT HANA report type people should work in a Pricing Department for a couple of years before moving to IT so they have a better idea of some real business needs.
But thanks for the conversation: Bottom line to me seems to be that there is not going to be any simple answer: "Yes, we support MS-Access/ODBC" or "No we don't". But as far as I am concerned HANA does support ODBC, so what I am doing actually works and works very well. My whole reason for this post is the fear that a wonderful tool that really supports business needs will be taken away by an IT department that just does not understand what the business needs.
Thanks for engaging in this conversation and please do not take my comments about IT negatively. I am just trying to clearly express how I see things from a business perspective.
Ira