DI SAPR/3 extract Initial and Daily load jobs

dataservices job design
1

Hi All,
I’m working on a proposal for one of our client. As per the requirement, need to extract data from SAP R/3 and generate the Flat files. There will be 2 set of jobs one is Initial load and other one is daily load.

I’m new to Data Integrator, Can someone please explain me is it possible with Data Integrator. If yes guide me how to do initial and daily load jobs.

Thanks
Suresh

isurya (BOB member since 2009-05-14)

2

There are multiple options and variations but in general

You have two dataflows, one bulkloads the initial load data, the other reads the delta and inserts/updates the changes into the target table. Using a global variable you define if the load is initial or delta run. And via a Conditional object you call either the one or the other dataflow.

https://www.sdn.sap.com/irj/scn/wiki?path=/display/BOBJ/ETL+Project+Guidelines

Werner Daehn :de: (BOB member since 2004-12-17)

3

Thanks Wdaehn!

I have gone through couple of DI technical documentation in last couple of days and now I came to know that DI is capable of extracting initial and delta (With or without date field on Sap table) extracts from SAP.

Right now the proposal we are working on has 115 SAP tables for which there should be a initial and delta extract job. Looking at the tables list I’m thinking they all will fall in 3 or 4 patterns of job design.

My question is can we create a template for each and every pattern and use the template to speed up the job development??

Thanks
Suresh

isurya (BOB member since 2009-05-14)

4

As the structure (columns) of each table are different, no. But what you can do often is using the same dataflow for both initial and delta. Obviously not for the large fact tables but for all the small one with just a couple of 100’000rows or less. Identifying the delta - if possible even - would take longer than just initial-load them. Or so.

Look at the link carefully, it is full of tricks to ease up development like using embedded dataflows if the extraction is the same but the load different etc.

Werner Daehn :de: (BOB member since 2004-12-17)