Future of BODI ?

system · February 21, 2008, 1:42pm

Hi,
I am really concerned about the positioning of BODI after the recent SAP acquisition.

I have a few concerns and thought I should raise it. (am a new member and what a way to start!!). I am playing a devils advocate here. I would really be happy to see some optimistic replies. (but a frank straightforward clarification will help me better)

Is BODI being phased out?
SAP targets large customers whose environments involve huge volume of data. BODI is not really great in handling huge volume of data. Will SAP dump it from developing it further?
SAP has its own ETL capabilities in SAP BW. I dont really know much about SAP BW or Netweaver stack, but heard that SAP doesnt really have any reason to fund the development of BODI.
BODI is cheap and is packaged with other BO tools. SAP might not be interested in the cheap tool at all.

My final question is this:
If you are an ETL consultant with expertise on BODI, where will you go from here?

Regards
Thomson

Thomson_ETL (BOB member since 2008-02-21)

system · February 21, 2008, 9:31pm

Hi Thomson,

You raise some interesting points. I don’t have extensive experience with Data Integrator but I posted the other day about an article in the latest “eWeek” about the acquisition.

According to Jon Schwarz, SAP intends to operate BO as a separate division. He didn’t specifically mention DI but if you take him at his word, then DI will continue as a separate product. It would be up to the market to determine whether it would lose out to BW or other tools.

Judy

JMulders (BOB member since 2002-06-20)

system · February 21, 2008, 9:38pm

Thomson,

Further to Judy’s comment, I find it unlikely that DI will disappear in the short to medium term.

As I understand it, in the next release of DI they are also bundelling in the Data Quality application (currently sold as a separate tool).

To answer your final question about where to go, we have a guy working on our team that was experienced in Informatica and had never used DI, it took him a couple of weeks to figure out DI, I suppose the opposite would also be true.

plessiusa (BOB member since 2004-03-22)

system · February 22, 2008, 4:06pm

I dont have too much knowledge of the SAP DI product other than that I’ve had a few clients with SAP that have conisdered it instead of BODI but have chosen BODI for numerous reasons, in particular its connectivity to other sources.

I would be amazed if they dumped BODI but that’s just my opinion, Werner’s opinon is what will carry most weight but I’m not sure if SAP will have exaplained the product roadmap to BO employees until the decision has been made.

ScoobyDoo (BOB member since 2007-05-10)

system · February 25, 2008, 8:15am

All,
Thanks for the replies. It helps me a lot.

By the way, would Werner be able to comment on this with his views?

The changes might take quite some time… but for a guy who recently spent a lot of time (and money) to learn something and to know that it has a chance of being scrapped… is disturbing!!

Sorry, it this distracts others from the original question, are there any resources on the web for freelancing consultants to keep in touch with Market dynamics, recent changes, discusses billing rates, etc. Would help me a lot.

By the way, sorry for distracting everyone. My original question is to discuss the future of BODI.

thanks to all.

Thomson.

Thomson_ETL (BOB member since 2008-02-21)

system · February 25, 2008, 10:12am

By coincidence, I did not respond to this thread on Friday because of integration meetings with SAP in Walldorf. You have put me into a difficult position as roadmap discussions are not communicated to the outside per company policy and if, certainly not by me. But a couple of facts:

The SAP products in the ETL space are all tailored for SAP BW as target. And mostly SAP R/3 as a source. Therefore SAP partnered with other ETL tool vendors like Infa, Ascential and BO to close that gab. That partnership was not too successful revenue wise, mostly because companies using BW had R/3 as their main source anyway and the BW options for reading non-R3 data was good enough. Good enough to no spend extra money. But now SAP has an offering for all sources/targets.

When a customer wants to build a data warehouse, he has three options. Build all by himself with or without an ETL tool, use our RapidMarts or use BW with its prepackaged business content. So I was interested to see if our SAP RapidMart was going to get discontinued in favor of BW as that would indicate the future and the priorities. To my surprise the SAP team themselves suggested to sell Rapid Marts more agressively. They view Rapid Marts as the quick and simple, yet powerful, Data Mart. And that’s what it is: A Rapid Mart is build for one source system and one reporting language. BW is seen as the Enterprise Data Warehouse, where you consolidate multiple R/3 systems and other ERPs. What we need to figure out is how to embed DI into the BW solution for getting better access to none-R3 ERPs and a migration path from Rapid Marts to BW.

BO is too big to get rid of the none SAP market. If SAP would make the move and focus on SAP access for all products only, we would quickly lose the majority of the market. And that’s why SAP said right from the beginning, BO will remain a company like it is today, with a very good SAP connectivity obviously.

SAP BW is not the only area requiring Data Integration capabilities. Think about ERP migrations to R/3 - the MOVE initiative, or Master Data Management. Every team I have spoken with at SAP is actually very excited to no longer rely on external companies and have build features together. Before they had to build a standard interface and ask other ETL vendors to use it. But now, we can develop the connectivity together, for DI it would be excellent if SAP would have this or that feature…and we simply get it. BW is just a nice example. Today every ETL tool has to load into the staging area of BW and the actual integration of the data into the cubes will happen inside BW. But if you own the ETL tool, wouldn’t you be willing to open up that a bit? Just for your own toolset?

Another interesting thing is, SAP teams related to BI are being transferred to Business Objects. That was communicated at the announcement of the acquisition and is now happening. So you could view it as if BO accqired the SAP BI team, not BO being accquired by SAP. Obviously SAP, as the owner of all BO shares, has quite some influence and even there, John Schwarz is still heading BO. Isn’t that quite a signal to the market? I would have never thought something like that to happen…

Werner Daehn (BOB member since 2004-12-17)

system · February 25, 2008, 9:41pm

Interesting comments Werner, thanks for the little snippets of information.

I think this may help some people rest easy, not only in the DI space, but also within BO product suite.

plessiusa (BOB member since 2004-03-22)

system · February 26, 2008, 10:04am

Thanks Werner.
That helped a lot. And it excites me to know that SAP might be looking at the capabilities of BODI.

That settles my question on BODI being phased out.

The part I have yet to receive answers is this: is there any thread (i didnt find any) here that discusses market dynamics (like ETL, EII, EAI changes in market, roadmap of vendors, job opportunities and opportunity planning for consultants, etc)?

If not here, can I take the liberty to ask for external resources on the web?

(Sorry, I am somewhat new to DW. And have a free period next month which I plan to use for some training to strengthen my skillset.
My apologies if this is the wrong place for such questions.)

As a parting thought, this is the only board that seems to be providing good answers. Thank you all for the wonderful help.

Thomson.

Thomson_ETL (BOB member since 2008-02-21)

system · March 5, 2008, 10:12pm

I posted some information links here but then moved them across to the Good DW/BI informational web sites thread.

vincent.mcburney (BOB member since 2007-04-14)

system · March 5, 2008, 10:29pm

My two cents on the future of ODI - in data integration suites the ETL tool tends to be the star of the show. Every company does batch data integration somewhere in the organisation and often it’s the dominant method of data exchange. By comparison not all companies are doing SOA or data quality or EII. Informatica and IBM data integration suites are built around a successful ETL product with additional products becoming addons.

In recent times ETL is growing even bigger as it becomes added to MDM, DW and BI super packages. The question for SAP and Business Objects is how to put ODI into more packages and how to make more money with addons and SOA capabilities. They would be crazy to get rid of it as ETL is the most popular type of data integration tool.

vincent.mcburney (BOB member since 2007-04-14)

system · March 17, 2008, 9:52pm

Some interesting views here.

http://www.it-director.com/technology/data_mgmt/content.php?cid=10334&extcmp=salesflash_global_2008_1703

regards
Prem

biexplorer (BOB member since 2007-06-21)

system · March 19, 2008, 5:39pm

All,
Thank you for the replies.

From above, it is apparent that things are not that bad for BODI.

Thx

Thomson_ETL (BOB member since 2008-02-21)

system · April 3, 2008, 3:53pm

SAP R/3 to SAP BW is well-covered by other “tools” as Werner mentioned, but even those approaches are more work IMHO than it should be. The end result supports one primary method of access - BW (ok, and variants of other SAP tools). Populating BW outside of SAP’s R/3 toolset is not fun, is time consuming, and relies on loosely coupled methods to ensure you perform the ETL processing correctly. Out of the box business functions in BW are tightly coupled, custom work is not.
When I was with BOBJ GSO, I had to work in an Extraction from SAP BW project to augment an Oracle DW, it was truly awful. BW offers very little in terms of tight sychronization of metadata export, etc. w/o other tools. I ended up having to use text exports from SAP with file layouts, etc. If we had been allowed to access the base data directly from SAP instead, it would have been a significantly lower level of effort with much higher quality and lower ongoing maintenance, effort for enhancements, etc.
Having extensively (ok, 2 years) used DI against SAP for extraction to an Enterprise DW based on Oracle, it is an excellent complimentary product for SAP. They do not have DI’s capabilities to integrate SAP data back out of the systems into a DW.
DI has an excellent and dynamic metadata relationship with SAP R/3, hands-down the best I’ve worked with. The DI SAP datastores give you very good insight into application database tables, functions, etc. Creating efficient R/3 transforms and managing the balance of work in the SAP environment vs. how much work you perform external to SAP in your RDBMS is your main design challenge.
Areas of improvement - use of SAP R/3 Functions and Custom Code, invoking already existing SAP code (again usually functions with a wrapper) is a little more complex than it needs to be. In most cases the bulk of the programs wrapper had to be written as static ABAP code for SAP to allow us efficient access to SAP ABAP (I’m excluding iDocs, another subject).
DI Relevance Post-SAP/BOBJ - If you are early in your career and wish to add some very significant value to your resume, I strongly suggest seeking out work using DI against SAP. Rapid Mart projects are a good starting place. I expect this to be a skill in demand, more so than Oracle-to-Oracle DW work, for example. As a specialized skill it should equate to better compensation, too.

I’ve been in the DW space for over 15 years and have seen the gamut of ETL coding approaches, early and now-departed ETL tools, and the rise of our current ETL tools. I work at many levels from Strategy & Planning to hands-on development. Having spent the last two years putting DI through it’s paces against SAP, the combined entity I hope will recognize this is one of the best value-added products in their portfolio to extend and add value, esp. when used with SAP R/3. Nothing else in the general ETL tool space is even close.

I’d say to the DI PG, “Keep up the good work, and make it even better”

Steve

Stracy (BOB member since 2006-07-14)

system · September 4, 2008, 12:40pm

All,
Any updates on this front?

I am particularly interested in 2 things:

Is there a roadmap for EIM tools (BODI, BODQ, etc) in the SAP world yet? Will they be integrated / enhanced / dropped / blah / blah…? Is there any visibility as to how SAP-BW and BODI/BODS will coexist?
Is the market warm to the idea of a SAP - BOBJ EIM tool suite? Or are they treading this path with caution, waiting for the dust to settle down fully? Have there been enough sales on this front?

Regards,
Prem

biexplorer (BOB member since 2007-06-21)

system · September 4, 2008, 2:04pm

Both worlds remain completely independent, there will be just more interfaces between the two. BW using DQ. DI reading/loading BW better. And SAP ERP interfaces as well obviously.

The only thing that really changed for us is that we have an easier way into SAP accounts, so quite a few customers look at DI these days and figure how much easier it is to use compared to Infa, Ascl etc. The question “How do I migrate an Infa job to BODI” comes up quite often.

My impression is, the next version of DI will have 10% of the budget spent on SAP related feature, 20% on standard housekeeping tasks (performance optimizations and the such) and the rest is for new features for the pure play ETL - not SAP related.

my two cents

Werner Daehn (BOB member since 2004-12-17)

system · September 10, 2008, 3:31am

If you go on the SAP website you can find product roadmaps for all the BO products.
DI is definitely here to stay and will continue to provide both SAP and non-SAP support. With the latest release of Data Services, the tight integration of both DI and DQ give the Data Services product a real edge over it’s rivals.
SAP are also looking to integrate the DQ component into its Master Data Management (MDM) offering as well.
SAP used to OEM and partner with INFA for ETL but no longer and would also work with Trillium for any Data Quality work. Data Services will now be pushed by SAP to fill these gaps so overall, DI and now Data Services, will be around for a long time.
Also, the whole end-to-end BI sales pitch is used extensively by BO which is also a key driver in sales situations and getting rid of DI would drive a big whole in the overall product capability of the company.

chesl73 (BOB member since 2008-05-15)

system · December 11, 2008, 2:43am

I genuinely post these questions with the utmost respect, but I think they deserve answers.

Question 1:
How long do you think that SAP will allow BOBJ to continue to operate as an independent business unit? Those days are numbered. Eventually SAP will merge the parts and pieces (and people) of BOBJ that they want to keep into SAP-proper (20%) and they will get rid of the rest (80%). This happens all the time (see ORCL-PSFT, ORCL-JDE, SAP-TopTier, SAP-A2i, etc, etc, etc). All software companies do this.

The BOBJ sales reps must know that their days are numbered and they need to make hay while the sun is shining - which is exactly why they are pushing BODI in all SAP accounts. They don’t care that Informatica was chosen by SAP to be embedded within SAP MDM just a few short months before SAP acquired BOBJ and that now the SAP sales reps have done a complete 180* turnaround and are pushing BODI for their own financial gain.

Question 2:
Please use your analytical thought processes when looking at SAP’s roadmap for BODI. If you really look at it, SAP is promoting you to use Netweaver ETL-like capabilities for R/3 sources and BODI for non R/3 sources. Why in the world would they think that an IT organization would want to support two DI/ETL tools just because they both have an SAP logo on them?

Further to this point, I asked this question to a long-time SAPer in Product Management (no offense Werner, but the BOBJ perspective doesn’t hold much water to me), and the response was that SAP is waiting to see if BODI is a financially surviveable product or not before they make the decision to keep it or axe it. They were not as cautious with BOBJ reporting/dashboarding/EPM because those technologies were a clear market leader while BODI has always been a niche/challenger (per Gartner) and had low market share (~3% of the entire ETL market).

Werner even notes that only 10% of the R&D spend for BODI will be SAP-related. Isn’t that proof enough that SAP is still waiting to see the viability of BODI in the marketplace? Werner, you seem like a good man, but I’d freshen up your resume if I was you.

Question/Point 3:
Notice the difference between Werner’s posts in Feb vs. September. In Feb immediately after the acquisition was official, he was optimistic that BODI was going to be integrated into BI (circumventing SAP’s API layer), that Rapidmarts were going to magically get migrated into BI-proper, that BODI would be a part of ERP migrations and MDM, etc. Then look at his perspective in Sept…a very light level of integration (only 10% of BODI R&D spend) with most of the “integration” between BOBJ and SAP being at the sales rep level.

Also, if you read the whitepapers on BODI for ERP migrations and MDM, they are almost word-for-word copy-paste from Informatica’s website and IBM-Ascential’s website. It looks like copyright infringement, truthfully.

Furthermore, SAP is NOT looking to dive head first into the ERP/data migration business. SAP quickly and quitely killed their “Safe Passage” program (ERP migration with partners such as Informatica and IBM) after TomorrowNow got them tied up in a collossal lawsuit with Oracle. SAP does not want to get back into the ERP migration business and risk further litigation. Larry Ellison already has SAP right where he wants them.

My assertion is that BOBJ is acting independently of SAP on BODI. By continuing to fund BODI and keeping it off of SAP’s radar, John Schwartz is acting in his own best interest, not the long term best interest of SAP’s customers. Do not bet on BODI’s survival long term.

Lastly, using FirstLogic inside of SAP MDM for DQ is comical. FirstLogic has and always will be a customer-centric DQ tool (despite the UCM efforts, which are like putting lipstick on a pig). SAP MDM is a product master/PIM tool. How in the world is it that SAP plans to take a customer-centric DQ tool and use it to cleanse master data in a product-centric MDM tool (A2i)? Not to mention the fact that while SAP MDM sports a laundry list of customers who have purchased it, there are VERY FEW live implementations of SAP MDM - probably zero that were delivered on time and on budget because of bugs in MDM. A SAP consulting exec told me that SAP MDM is the hands-down worst product SAP has ever sold…and now they are going to repurpose FirstLogic to do product-centric data quality?

Don’t drink John Schwartz’ kool aid. Reason for yourself.

JeffT (BOB member since 2008-12-11)

system · December 11, 2008, 10:07am

Interesting

But let’s go through your points by priority…

I do light housekeeping tasks, would answer the door bell, I am generally nice to my employer, liked by everybody and I don’t ly in a cv
When answering questions like those I have to be very careful. On the one hand, I’d like to tell all SAP customers BODI is the only ETL tool and soon will be integrated perfectly, the none-SAP customers that we are still a generic ETL with light SAP integration but all our development is focusing on pure play ETL. I would love to show you the list of things we are working on, on the other hand I am not allowed to say anything from a company policy perspective, a competitive reason and legal reasons (SEC rules). To cut a long story short, I am walking on ice here.
Let me divide the DI development team into three areas. SAP development, pure-play-ETL and DQ. SAP customer benefit from the SAP work we do and participate a little bit on ETL and DQ developments, e.g. if the engine is 10% faster it helps SAP accounts as well, when DQ is more powerful and used, it would help an SAP customer also. For a pure-play-ETL customer all the SAP related work we do is of no interest to them.
So the big question is, what the ration of developers on SAP related tasks vs. pure-play-ETL is. We doubled that ratio two months ago, from 4% to 8% of the developers working on SAP-only features. And it will remain this way. So this is actually bad news for the SAP customers, that the amount of resources we put into SAP features is so little. On the plus side though, we always have been strong on SAP and with little work here we can do a lot. Right now we streamline the BW load better, we will support more interfaces SAP provides.
I have spent quite some time at SAP and we compared the SAP NW ETL with the DI ETL capabilities, methods and functionalities. It was very obvious that DI is much more mature, starting from pushdowns, over Data Validation, partitioning support, bulkloader support, parallel processing, engine overhead. NetWeaver is completely built on top of the ABAP stack and you are limited there. As a result, we have three implementation options, keep DI a 3rd party tool to SAP with its own GUI; keep the NetWeaver GUI but replace the backend from ABAP to DI engine; merge the GUI so that BW dataflows and DI dataflows use the same GUI.
What development is working on is the first, the 3rd party level. SAP wanted to get to the second level and use the DI engine, you know, BW generating ATL and we execute it. But we quickly figured, to make sense you would need to add the DI transforms to the BW GUI and reinvent the wheel, so we will probably do level 2 (DI engine used in BW) and level 3 (merge GUI) in one step. (The backend will not get merged!)
And what would a none-SAP customer get? He will still have the DI Designer pretty much as it is today. But in addition he will get modelling help, e.g. if the DWH should support multiple languages, you have to build the star-schema that way yourself. In BW the datamodel is fixed, it is a Snowflake of some kind. So as soon as you say “Need multiple languages” and “I want a star schema”, the DI Designer will let you chose to either have n-text fields, one per chosen language. Or you can say “support all languages” and DI Designer will create the text-lookup table for you as a template table or …
If you tell DI Designer you column QTY is actually a Quantitiy measure with its Unit-of-Measure being the UOM column, you could ask the DI Designer to add the code that converts metric tons to kg, deal with non-additivity of “pieces” and “gallons” and “kg”.
If you have monetary measures, the code to convert it to one or two common measures inside your fact table in addition can optionally get added for your.
If the source has a new column, you do not have to start SQL*Plus, add that column yourself, reimport the table, add it in all queries of the dataflow - the Designer can do that automatically or semi-automatic at least.
We merge the Data Integration tasks with Data Modelling and other options SAP BW brings in but remain open. Do not force users into the BW model, but offer their modelling techniques plus Star-Schema design techniques.

And finally a few thoughts on your questions
“How long do you think that SAP will allow BOBJ to continue to operate as an independent business unit?”

SAP had a deficit on BI visualizationand on loading none-SAP data into SAP BW. How much money would you have spent to solve that?
a) No money at all, Informatica, Ascential, BODI all can load BW and there even is an OEM partnership with Infa. And for the BI both, BO and Cognos do very well visualizing off SAP BI.
b) SAP wants the revenue itself. So spend a few millions on one of the few ETL companies like Kettle and you get BI as well. Or even better, jump on the boat with one of the OpenSource vendors and toot that message.
c) For some reason, SAP decided to spend not 10 millions but more than a billion. And what do they get? Okay a more mature ETL and BI tool, a brand name, a huge organization to integrate and a tool that supports the none-SAP market. By your statement they do not need the brand name, not the none-SAP market, not the many people BO has in sales. So spend 100 times more money for a more mature tool which is bigger and more difficult to integrate? That does not make sense.

“SAP is promoting you to use Netweaver ETL-like capabilities for R/3 sources and BODI for non R/3 sources. Why in the world would they think that an IT organization would want to support two DI/ETL tools”
That’s the point where I failed. I wanted to have one BODI tool with level 3 integration as stated above. But that would be a too expensive feature to be done at once, need to go step by step, level by level.
And to be frank, these are two different people. The BW person knows all about ABAP and R/3. He might not even know what a relational database really is or what kind of SQL statements one can create or what data modelling is!

“BODI has always been a niche/challenger (per Gartner) and had low market share.”
Gartner just added us into the market leader quadrant so that there is Infa, IBM and us. They did that because of the progress we made with the tool, with sales alignment, vision and because of what DI customers reported to them.

“Notice the difference between Werner’s posts in Feb vs. September.”
Yeap, reality did bite me.
MDM: MDM doesn’t have any APIs other than Java. Using the DI Java Adapter SDK for millions of rows, in a end-user facing environment (User clicks on save-new-customer-record and MDM will popup with a list “Isn’t that a duplicate with this record?”), with low latency? No, certainly not with the Adapter SDK. So we had to stick with a light integration only. MDM has to provide a useful interface first, before we can start the tight integration - in progress.
RapidMart: On track, but we need to be able in DI to consume the same DataSources like BW does - development is in progress on the SAP side. Once that is done we can change the RapidMarts. And for now, ABAP generation DI does is not liked by SAP but that’s the only issue with them. Actually, we see a quite lot of RapidMart deals these days…
Migration: DI brings the flexible mapping into the equation but the key issue is SAP ERP has no high performance batch interface to load the data. Every record has to be checked by SAP ERP, it impacts multiple SAP tables, lots of exception cases depending on what kind of record loaded etc. And for Enterprise Integration via a Message Bus, there is SAP PI product which is well suited for that.

Aynthing else?

Werner Daehn (BOB member since 2004-12-17)

system · February 25, 2009, 7:04am

Hi Werner

Thanks for the very informative update. Since your last post, three month are gone, are there any updates you are allowed to communicate on the integration road map?

Thanks, Roger

rogerm (BOB member since 2008-05-09)

system · February 25, 2009, 8:02am

If only programming the ideas I have would be equally fast…

Werner Daehn (BOB member since 2004-12-17)