esjewett.com - Latest Comments

Re: SAP's HANA and "the Overall Confusion"

Mike Bestvina — Tue, 31 May 2011 09:57:39 -0000

My last discussion with SAP (tech people, not marketing) was that they are all (or at least most) POC's with no near future plans to go production. I can only assume that means there are some minor bugs or pitfalls that aren't being publicly being disclosed. (disclosure - that's my opinion, not SAP's) What surprises me, and probably the rest of the community, is that SAP marketing can start touting numbers and quotes from these customers.

Re: SAP's HANA and "the Overall Confusion"

esjewett — Sun, 29 May 2011 05:42:09 -0000

SAP is certainly pushing HANA as a revolution in computing itself, at least with regards to data processing. This is certainly not true, as you point out.

For companies that limit themselves to the SAP ecosystem when considering their IT landscape, HANA may be revolutionary, but I think (or perhaps I just hope) that not many IT and business groups limit themselves in this way :-) I have definitely seen some shops moving towards analytical DBMSes for their reporting datamarts, whether those are dedicated MOLAP stores (MS Analysis Services), columnar MPP disk-based ADBMSes like Vertica or Sybase IQ, or in-memory columnar MPP stores like Exasol and Paraccel.

If companies are only aware of the technology that SAP is offering and not the overall ecosystem, then they are at a high risk of overestimating the value of the technology on offer and possibly making bad comparative TCO decisions. I guess the question is how best to stay aware of the technology available in the overall ecosystem. Entering a client relationship with a technical analyst firm might be the best option for companies that are trying to start developing this sort of more general view.

Re: SAP's HANA and "the Overall Confusion"

esjewett — Sun, 29 May 2011 05:26:33 -0000

Thanks for the comment Vijay. I think you're right, especially because the existence of that many live systems implies that a significant number of people have become familiar with HANA outside the inner circle of SAP's own HANA experts.

Has SAP shared information on how many customers have gone live (to production) with a HANA-based system? My understanding was that the customers talking at #sapphirenow were discussing prototype systems, but I may have misunderstood that.

Re: SAP's HANA and "the Overall Confusion"

esjewett — Sun, 29 May 2011 05:22:23 -0000

I completely agree. In private and small conversations SAP is very clear about what HANA really is and that it does not directly address the issues you talk about. You can even get people to admit that HANA is not the best option in all situations :-) But in the keynote-style presentations this is not clear, and these presentations are what most people see. I think it's for this reason that questions like the ones answered here are being asked by a lot of customers.

Re: SAP's HANA and "the Overall Confusion"

Mike Bestvina — Mon, 23 May 2011 06:47:50 -0000

"Personally, I don't agree with SAP's rhetoric about HANA being revolutionary or changing the industry. The technologies and approaches used in the ICE are not new, as far as I have seen. As far as changing the industry from a performance or TCO perspective, I'm reserving judgement on that until SAP releases some repeatable benchmarks against competing products."

It's only game changing in the "SAP Industry" (which is massive), and not in technology sector, and perhaps not in enterprise IT. If anything I see HANA as a catch-up to the abilities of the open source community to create "revolutionary" ways of accessing mass data (see MapReduce/Hadoop for example). Such technologies which have existed and have evolve largely over the last 4-5 years with the explosion of mass data. The largest SAP BW systems I've seen are sitting around 30-40 TB. This is peanuts when compared to the petabytes of data the Google's and the Facebooks of the world are sitting on. More importantly they are building tools around analysis in this area.

The problem (and difference) is that this doesn't matter to the SAP community. Companies (read -> high level IT execs) do not want to change the technology they've already engrained themselves in. HANA is seen as revolutionary to them because they are already indoctrinated to the ecosystem.

Re: SAP's HANA and "the Overall Confusion"

Vijay Vijayasankar — Sun, 22 May 2011 11:00:56 -0000

Several of the pressing questions in those links I think have mostly been answered by SAP and others by now, in my opinion. However, there are lot of new questions that have come up now - and will continue to come, till at least a 100 customers go live with HANA.

Re: SAP's HANA and "the Overall Confusion"

Jean-Michel Franco — Sun, 22 May 2011 09:37:29 -0000

In my POV, part of the confusion in that HANA is presented as the center of the universe, rather than as a service in an IT system. Rather than populating the same story but with bigger noise, SAP has to show what this in means in a real world, a world where all Data isn t in a single centralised ERP, where more that one BI Platform is used to leverage data downstream, where database choices are made because of TCO and existing landscape and not only for pure performance...
Documents like the one you share, those by Thomas Durek for example, obviously go in the right directions. But those have to be shared more widely. Today, this is what misses in many SAP communications

Re: Musing about semantics in BI

Arun Bala — Fri, 08 Apr 2011 12:26:53 -0000

I hope you had a few pts on the Metadata Reg. In general, you'd introduced a good pitch on necessity for SEM layers+Data Modelling. This relation does exist in the existing scenarios. A simple example is like jump queries. users view a simple aggreagated view & take restrictions from the 1st view to 2nd query. Like, say a much simple higher aggregation calls for a complex query, which req a pre-calc server. Say if 'profits' are more macro level presentation, 'gross profits' are streamlined/more aligned. You've clearly defined one phase of idea, but SEM as a whole being a mere shared element concept (dataelement>domain), the constructs being more wider prespective doesn't deal on physical/technical level, i.e. instrinsic data's dependency on platforms, system or tables/ODS. In DW, don't we actually try making relations! Like combine a 'char' with obj's 'class'? Say, my 'Company' > My Company's expenses/profits?? SEM being Obj Orieniented; Viz. you combine concept of 'definition' with an obj's 'class'. (Very similar with joins) Viz. the macro level 'profits' combined with obj 'class' = 'My Company'. So 'My company's gross profts' are indeed very to-the-point & specific than just 'profits' (also granular). Very well, so this is just to add on here how we must relate this concept via framework & definition rules. say viz. your definition for 'income' , 'sales' & 'profits' for the obj 'class' My 'company' defines the same concept, although key words if are different?. (This must be very well managed with custom rules). 2nd pt to be taken care is rg. buckets for 'heterogeneity' cases. esp restrictions more complex like "fisical net profits of my company in the range >10K-30K rupees"? (I wanted to give a more complex scenario here..) pt. one you'd have a diverse set for the obj class (chances are keen I may operate >1 company) & different 'buckets' ranges as opposed to regualar conditions/exception groupings. (This case is more resource consuming) But, as said HANA plays a role eliminating the latency for 'aggregating'. As discz earlier posts reg removing star schema/dependency with DW for BI, we can mandate till one extent if it pars with existing options. Data Int. is quite often solved as a 'data', rather than a 'metadata' issue results with the use of 'master' data. The critical aspect is source, from which we map data-targets; but 'data' as a whole is not given due importance, where our *workhorse* comes to do the ETL & cleansing. The key is to make the ILM phases on par with BI. Rg extractors, there must be options to explore based on the different scenarios. FI-SL might be pretty easy pie for accessing via 2nd layer, but others would seriously need more efforts. Nice Diags. Pretty Neat.

Re: A tour of testing with an SAP focus (in the end)

esjewett — Wed, 06 Apr 2011 04:39:54 -0000

Hi Tim,

I haven't moved on exactly - still struggling with many of the same issues :-) I have realized that at least in BI it is a much bigger problem than just SAP software. I try to use these principles as much as possible on my projects, but from both a cultural and tool-set perspective most projects aren't ready to adopt these methods.

What is your experience with trying to push the idea of a Continuous Delivery model for SAP projects?

Ethan

Re: A tour of testing with an SAP focus (in the end)

Tim C — Tue, 05 Apr 2011 14:07:29 -0000

Ethan
This is really interesting: inadequate test harnesses for SAP are a significant barrier to releasing a lot of value in mature SAP builds, as well as in the new build stages.

Have your thoughts/experience moved on?

ideally, I'd like to build a Continuous Delivery model around SAP.

Re: Musing about semantics in BI

esjewett — Wed, 02 Mar 2011 04:28:59 -0000

Greg, thanks for taking the time to provide a thoughtful response. I agree with you on the DW/OLAP/BI mish-mash. I've given up on trying to differentiate the three as most people use them interchangeably, and indeed the concepts to which they refer overlap more or less. Throw "analytics" into the mix and you've got yourself a real confusion :-)

I think what you are getting at in the rest of your comment is a little bit different from the focus of my post. You are talking about specific representations (in this case government mandated formats) of semantically integrated data. Meanwhile, I am talking about the structures we use to implement semantic integrations on top of the formats used to store this data, preferably with decent performance and at a reasonable cost in effort.

Part of the difference is embedded in your use of "syntax" here. I think it is confusing the syntax of programming languages using to generate semantic representations ("syntax errors") and the syntax or rules of a mandated format (XBRL syntax). This is confusing because XBRL provides both a syntax (defined as an XML format, I believe) and a semantics (tagging rules).

To a certain extent I think we can separate questions of semantic and syntactic representations of data. There is always a syntax, or format, but I think usually it is advantageous to conceptualize a semantic representation coming first and then layering syntax on top of that semantic representation, though indeed we often go through multiple iterations of this layering cycle.

Re: Musing about semantics in BI

greg_not_so — Tue, 01 Mar 2011 12:16:40 -0000

Hi Ethan,

Since SDN is down at the moment and your blog has been recommended by my fellow twitter followees, I decided to answer in a bit more detail than a usual casual remark. Besides, your blog goes into more detail than usual, so it deserves a more detailed answer. We share similar interests, but may have different backgrounds. What you call data warehousing I came to know as OLAP since 1998 and after spending some time at Gartner, KPMG, Hyperion, and IBM, but doing mostly SAP all that time, I couldn't avoid forming an opinion of what BI is and what it is supposed to be.

Whenever I see semantics all I have to add is syntax and we have the rudiments of a language, both human and machine, so I usually like to dig deeper when those layers surface in a discussion. I have my own answer to what semantic layer is, but getting the message out is quite a challenge in this world of instant communication. I keep on trying, though.

The short answer, which your blog is actually alluding to, are actually two answers. Everything else seems to be an implication from those original two. Some may disagree, but to me these two are quite basic. So, the state of the art semantic layer is the sec.xbrl.com and irs.gov. Both are US centric, but they influence other global standards like IFRS and statutory and tax laws of the rest of the world. They don't have to be the influencers for ever, but they seem to be for the time being. They both provide detail behind the revenue, the former through a structured process of defining XBRL taxonomy tags (example here http://xbrl.fasb.org/us-gaa... and the latter by putting the law of the land behind the definition of business receipts (you are right, the Internal Revenue Service doesn't define "revenue", but it defines "receipts", for example here: http://www.irs.gov/pub/irs-.... So, we have started a semantic discussion already, haven't we?

So, how does this all relate to RDBMS and BI? The answer is quite simple, it needs to add up. Whichever facts are shown in whichever dimensions, they need to trace back to a single database record. It's the HANA and extractors (and network broadband) that are supposed to make this "adding up" transparent to the end user. That, however, is the ideal world. We all know that ERP and BI in today's corporations is very far removed from the ideal world. We have legacy systems, spreadsheets, people changing jobs, companies merging and filing for bankruptcy, What's more, both government agencies mentioned above don't make it any easier on the business when they are pushing for continuous standard changes that IT providers and their consulting arms are supposed to somehow make instantly available in their software builds. We both know that it's not going to happen without tremendous investment by all the parties concerned and we have to satisfy both the economic need of maximizing the profit and the political need of fairness and equity.

Getting it down to the single entity level, what I can do is to raise the awareness of those issues and fill some of the semantic gaps as I see them, but this can only happen after all my syntax errors are gone and the code is clean and running and what's most important both business and regulatory requirements have been satisfied, including mobile devices, of course.

my 2 cents/3 Groschen,

@greg_not_so (tweet here)

Re: What does SAP mean by "In-memory"?

Prince and Fox — Thu, 28 Oct 2010 11:47:59 -0000

Wery good blog entry...I wonder why SAP is not using memcache - which is almost becoming the WEB standard.

Re: What does SAP mean by "In-memory"?

esjewett — Wed, 27 Oct 2010 16:32:29 -0000

I'll admit to being a bit skeptical about this take. I don't see RAM in its current state becoming a primary storage medium for applications that require guaranteed persistence of writes (ok, 99.99%+ guaranteed persistence :-). In these use-cases, RAM will continue to be a cache for all realistic purposes. Indeed, it may be a cache that gets nearly 100% of the use, but there will continue to be an underlying storage stratum that can provide a better persistence guarantee and writes will still need to be committed to the underlying medium before concluding a write-transaction.

Eventually we'll see a storage medium that can give a persistance guarantee similar to hard-disks with access speeds similar to RAM, but we aren't there yet.

With regards to SAP's solutions, which were the focus of this piece, RAM is most certainly used in the manner described: as a cache - albeit a cache that contains 100% or nearly 100% of the data-set, but still a cache.

Re: What does SAP mean by "In-memory"?

Muz — Wed, 27 Oct 2010 15:48:57 -0000

You made need to re-think your notions of caching: if RAM becomes the primary - disk is the new tape.

Re: How to get Johnson built

esjewett — Fri, 24 Sep 2010 07:06:29 -0000

I really don't recall, but I think I tried it and it did not work.

Re: How to get Johnson built

Abhinavm — Thu, 23 Sep 2010 03:19:39 -0000

Hi Ethan, did you finally build it for windows?

Re: Why in-memory doesn't matter (and why it does)

Dan Weinreb — Tue, 13 Jul 2010 08:08:52 -0000

Two comments. Supporting your point: measurements of RAM speed these days must take into account the L1, L2, and L3 caches. Qualifying your point: in-memory databases clearly don't work for large data warehouses; they are just too big. What in-memory databases can make sense for (these days) is OLTP. Many OLTP databases used in serious production are less than 1TB. For data warehouses, we are getting to the point where the whole thing can be kept in FLASH/SSD, but not main memory.

Re: Elastic lists using Protovis

esjewett — Wed, 07 Jul 2010 08:42:34 -0000

I've now been able to recreate the problem. It was due to Drupal wiping out some JS files (I got too clever by half about where I stored them) and then I couldn't see the problem as the files were still cached on my system. Sorry for the inconvenience, but it should be working now.

Re: Elastic lists using Protovis

esjewett — Sun, 04 Jul 2010 08:08:31 -0000

What version of Safari are you using, and on what operating system? I'm surprised, because I built this demo using Safari 5, and it appears to work fine on the iPhone (iOS 4), so I'm interested in what versions of Safari it is not working on.

Re: Elastic lists using Protovis

Guest — Sun, 04 Jul 2010 07:53:10 -0000

It doesn't seem to work in Safari....I just see a blank screen.

Re: Toward an analysis of datawarehouse and business intelligence challenges - part 1

Mark Ginnebaugh — Mon, 05 Apr 2010 04:55:10 -0000

A High Level Analysis of Data Warehouse and BI challenges - Part 1

Re: Why in-memory doesn't matter (and why it does)

grantdoug — Thu, 11 Feb 2010 17:34:39 -0000

Great article Ethan, thank you for deconstructing the viral oversimplification around in-memory and BI. You are right on target!

Re: Automated testing in SAP systems

Arnaud Leymet — Sat, 23 Jan 2010 13:51:31 -0000

I recently published an article on a similar topic, since it's about automating behavior testing on SAP Enterprise Portal (EP):
http://t4ke.com/2010/01/ent...
Just thought you may be interested in trying out the EPWT workbench!

Re: What's the deal with JAX-RS and Lift?

Paul Sandoz — Mon, 07 Dec 2009 05:30:54 -0000

Yes, you are right to say "does not force it", i always like to use the word "guide". In any API that exposes HTTP GET responses as functions to be implemented by the developer then there is always a possibility that they are implemented with side-effects :-) and, of course, HTTP streaming comet breaks the REST request/response constraint.

Re: Atmosphere. The high-level part of the atmosphere framework supports annotations that you can annotation your JAX-RS resource methods with, such as @Suspend/Broadcast/Resume.

For example one can annotation a @GET method with @Suspend and clients that perform GET to that method will be suspended. Then one can annotate a @POST method with @Broadcast and clients that perform a POST to that method will have the response returned from that method broadcast to all suspended connections.

So it is very much up to the developer to define the HTTP-based interactions for suspending, broadcasting and what representations are broadcast, much in the same way the developer will define "normal" GET/PUT/POST/DELETE interactions. Even though i think there are common patterns of usage (GET for suspending, POST for broadcasting) those patterns are not enforced.