Best Practice for Data Services

Is there such a thing? What about policy regarding use/design. We are probably opting for the central repository. Multiple users (2 in each country maybe).

I stumbled upon this which is really helpful

Any ideas?

I fogot to mention that I am going to be using 4.0


Decisys :de: (BOB member since 2004-05-21)

Use this SAP Wiki Link and this should be really helpful

https://wiki.sdn.sap.com/wiki/display/BOBJ/ETL+Project+Guidelines

Central Repo is good to have in a multi-user environment. Otherwise you would have to dig the grave.

Post your questions/clarifications and we are there to help you out with answers.


ganeshxp :us: (BOB member since 2008-07-17)

Although Werner’s elaboration is helpful, I am looking for something on a higher level in terms of global corporate policy pervading “Best Practice” for Data Services 4.0.

Where to begin? How to govern? etc.

There is also another issue worth pointing out. We are going to use Data Services for loading a normalised model in an ODS not Kimball’s DW.


Decisys :de: (BOB member since 2004-05-21)

For some reason (unknown) it appears that this post has been moved to this location. I don’t think that it belongs here because this is a general question regarding DS best practice and has little to do with jobs or design issues.

On another note, I am currently working on this theme at the moment. I may decide to post it later for critiquing.


Decisys :de: (BOB member since 2004-05-21)

In the consulting practice for my company I’m the author/keeper of our Best Practices for Data Services document. Having seen many mistakes made in how teams use Data Services and their approach to ETL in general we created a 40 page document that we provide to our consulting clients. It provides not only a guide of what not to do but what you should do.

Using a Central Repository is covered in the document. In my opinion if you have more than one person working on a project where their work will be deployed to the same target repository then a central repository is a very good idea. Even if a project staffed by a single developer the central repository can come in quite handy for version control. Implementing version control outside of Data Services is challenging.

Can you elaborate on this point? Will your ETL be doing complete loads of data from the OLTP system or will it be doing incremental loads? How different will the target data model be from the OLTP data model? If you can explain your high level design we may be able to provide advice.


eganjp :us: (BOB member since 2007-09-12)

The current project is to replace pl/sql data loads from various sources (databases, text files, abap docs.) to various ODSs. There is a reporting framework developed on the top of this (the ODSs :crazy_face: ). The client is recultant to develop a dimensional DW for obvious reasons; change, money, ownership, maintenance, etc. I could do a Business Case focussing on lower TCO and higher ROI proposing a proper EIM infrastructure but I think that I would scare them off. SO I’ve decided to go along with them and subtly propose various DMs in the first instance.

Wow, although it would be cheating even at the thought of asking you for a gander at your document, it would be really useful to me but perhaps when I’m done with mine we could compare notes and you could critique my work? I’ve seen some notes somewhere about what not to do in a DS project and if I stumble on it again I’ll post the link here.

To go back to your question (and I remember previous post of yours about setting the location of jobservers on the target database server, which we may implement :+1: ) I think that there will be one initial load and the rest would be delta loads. The ODSs in my opinion are inadequate as far as data sources go. There are tables within there without any relation to each other. The relations between tables (ER) are done in SAPBO designer. :blue:


Decisys :de: (BOB member since 2004-05-21)

Sometimes you have to deal with the hand you’re dealt. When a client is paying for a project it is their right to say what they want done. As a consultant it’s my right to document the problems with their approach (this would be my Cover Your Butt document).

The one “nice” thing about your project is that you don’t really have to come up with a new design. Table “A” in the source is probably going to look the same in the ODS. Probably 30% to 50% of my project time is spent coming up with a new data model for a dimensionalized data warehouse.

Where you can improve on their design is the type of tables (IOT vs heap in Oracle) and the type of indexes (btree vs bitmap), table properties (PCTFREE, compression, etc). Through the use of these you can often greatly decrease storage requirements while improving query execution time. Data Warehouses have a lot fewer limitations than OLTP systems but to make a Data Warehouse work well you have to stop thinking like an OLTP developer. That’s hard for many people.


eganjp :us: (BOB member since 2007-09-12)

Yes, but my opinion of diplomacy is that it has a tendancy toward evasion and is simlpy boring. Anywho, this is consultancy, right? And we stand by diplomacy like tango and cash. :roll_eyes: But that’s off topic.

Getting back to the point, my document appears to be growing rather large and I hate huge, unmanageable and pointless documents. The thing with lot’s of words is that no one really cares (or has the time) to read them. I was wondering if we could set out a table of contents for “Best Practice for DS”? I must say that I have seen some interesting links and perhaps I should include them: Sudheer’s link was of particular interest. There’s also Werner’s ETL Project Guidelines. Stuff I wish to include in my document are the “What not’s” and a testing strategy/testing best practice. In one of the links provided, someone has included content for a BP doc. I think that it’s reletively comprehensive.

I have finally finished, so if anyone wants to ask me questions, I’ll be happy to address them.


Decisys :de: (BOB member since 2004-05-21)

Tango and Cash diplomacy are you the bad cop or the badder cop :wink:


JonnySpec :us: (BOB member since 2008-08-01)