BusinessObjects Board

Performance 50% less on DS 4.1 than on DS 4.0

dataservices performance
#1

Hi all,

i’ve detected an issue with one of our huge jobs after having them migrated from DS 4.0 to DS 4.1.

In DS 4.0 the Job was running between 7 and 8 Hours.

In DS 4.1 the job is running between 10 and 12 hours.

So i started to analyze were this bad performance can come from…

The first huge dataflow takes 3:28 hours instead of 1:41 hours!
So it needs 50% more in this Dataflow only.

The submitted SQLs against SAP R/3 are the same (validated via “Displayed optimized SQL”) especially the WHERE clauses. So it’s not extracting / caching more data…

So i found out that in DS 4.0 the Monitor log file is around 66MB, in DS 4.1 it’s around 172MB!!

Could it be that this slows down the processing about 50% ?
Additionally to that it ignores the refresh period of 5 seconds.
If i set it to 120 seconds it’s also refreshing every second!!

We read via old RFC read table method some SAP table with join conditions against Oracle tables.
Why it ignores the refresh period?

Is there a possibility to disable the monitor on Job level?

Thanks in advance!

barthodo :de: (BOB member since 2012-04-18)

#2

Update:

i’ve found in the technical manual how to deactivate the monitor log file.
Just setting the monitoring sampling rate to 0.

I can see that the job is much more faster now (what i’ve seen so far, not finished yet!)

But how i can deactive the monitoring sampling rate in a schedule?
There is no option where i can set the monitoring sampling rate… (to deactivate it)

barthodo :de: (BOB member since 2012-04-18)

#3

setting it to zero definitely disables to monitor, but for all of my batch work I want to get the monitor result, just not populate some pointless log entry.

Im thinking setting it to 50K will do the trick. :stuck_out_tongue:

jlynn73 :us: (BOB member since 2009-10-27)

#4

But where in the “add schedule” section i can change the monitor rate?
Haven’t found this there… :frowning:

Additionally to that changing the monitor sampling rate for old RFC Table read method doesn’t affect anything.
It refresh every second…if i do 5 seconds, 120 seconds, or more…
So i want to completly deactivate it because it writes 172MB monitoring file (or more) and seems to cost 4 hours additional run time.

Any ideas?

barthodo :de: (BOB member since 2012-04-18)

#5

In DS 4.x the monitor sample rate is a value in seconds, not number of rows. The default is 5 seconds. If you want to change that you have to do it before you schedule the job in the Management Console or before using the Export Execution Command. It has to be set in the properties of the job itself.

eganjp :us: (BOB member since 2007-09-12)

#6

Hi eganjb,

thanks.
I will try to re-schedule the job after i’ve changed the sampling rate to 0 in the Job Properties.

My problem is that some dataflows are totally ignoring the sampling rate.
If i set 50000 seconds it also refreshs every second!

For example i have a very simple dataflow:

XLS → Query → Table

This dataflow refreshs every seconds and take ~ 13 Minutes in DS 4.1 instead of 16 seconds in DS 4.0.

If i disable the monitor then in takes ~ 30 seconds in DS 4.1.
So it’s definalty a performance killer in case the monitor refreshs every second.

I don’t know how to avoid this other than switching it off.

I think i must create an OSS for that :reallymad:

barthodo :de: (BOB member since 2012-04-18)

#7

How big is that Excel sheet? Anything complicated going on?

I certainly haven’t seen any such problems with 4.1 although most of our clients have now moved to 4.2 SP1/SP2 (largely due to the MS SQL 2012 and Windows Server 2012 support).

ErikR :new_zealand: (BOB member since 2007-01-10)

#8

this is a bug, and happens only for Excel reader in both DS 4.1 and 4.2, instead of writing the rows every n sec (n is the value of monitor timer rate) the monitor log is written for every row causing performance issue

manoj_d (BOB member since 2009-01-02)

#9

Is there a fix available?

eganjp :us: (BOB member since 2007-09-12)

#10

yes, DS 4.1 SP3 Patch 4 and DS 4.2 SP2 Patch 2

manoj_d (BOB member since 2009-01-02)