Open Hub extraction from BW is slow

I have a requirement to extract from very large BW DSOs into Data Services 4.0 and further to HANA. For the HANA table I tried to enable bulk loading with a few parallel loaders.

I went by the book and created Open Hub destinations, Process Chains etc. The performance was surprisingly bad, though:

  1. The load to Data Services didn’t start until Open Hub DTP completed. I guess it’s reasonable in order to avoid partial load of the data in case the DTP fails in the middle.

  2. The load from Data Services to HANA didn’t start until all data has been read from BW – I checked that using the Batch Job Monitor and polling number of records in HANA. Now, that’s quite a surprise, and I wonder if the reason is the same as in #1. Having two points of “all or nothing” doesn’t make sense.

Have I missed to make some extra specific tunings? 4 Mio records in 4 hours is soooo 2000-s :wave:

Also, a side thought – with introduction of ODP, all this Open Hub-based extraction kinda loses the point: if ODP was able to work not only with ECC BusinessContent Extractors, but with BW Export DataSources (including customer-specific), wouldn’t it be faster and more convenient?

Codehopper (BOB member since 2010-05-14)

I wonder what sort of a problem this is, that an OSS message for it stays open for three months (0000070312 – in case an SAP person checks out this message).

P.S. To moder: sorry for bumping this up, I would have not done it if any other channel hadn’t been exhausted.

Codehopper (BOB member since 2010-05-14)

The handshake from BW to DataServices is sequentially. BW first writes all the data into the OpenHub Destination table, then DataServices reads from it. But while it is reading it is streaming the data into the target. Otherwise it would pile up in the al_engine memory, that doesn’t make sense.

Werner Daehn :de: (BOB member since 2004-12-17)

Slow it is indeed.
My customer tries to extract APO (planning) data using the Openhub interface. In the APO-embedded BW they’ve got a table with approx. 350 million records (including zero-only records). Out of those, 90 million records contain non-zero values to be extracted thru the OpenHub.
The elapsed time of the full extraction job (from DSO thru PSA and Openhub to DS) is more than a day. Annoying when you want to run it at daily frequency, isn’t it?
Any recommendations to improve the performance are gratefully appreciated.

klo3010 :belgium: (BOB member since 2009-08-03)

Is this still an ongoing issue?

BW writing data into Open Hub Destination tables is ok.

But the next step is painfully slow : reading & streaming data into target.

We are using BODS 14.2.5

Ttekcirt :belgium: (BOB member since 2009-06-04)

Best option would be to directly call DSO/InfoProvider through data services using SAP BW Application datastore of Datastore type SAP BW Source

Antremons (BOB member since 2006-03-21)