BusinessObjects Board

parallel dataflow execution order

dataservices performance
#1

Hi there

We have a workflow containing over 50 dataflows each extracting a separate table from our source system. These were originally all linked up in series but we’ve now discovered the enormous performance gains from removing all links and letting them run in parallel as it is able to load up to 7 tables at a time. For further optimisation we would like to force the order in which tables are loaded (essentially ordered from biggest through to smallest). It seems this is determined by the order in which dataflows are added to the workflow (presumably some sort of sequence behind the scenes). Deleting them all and adding them one at a time is hard work :crazy_face: so does anyone know an easy way of changing the order in which these dataflows are executed.

Many thanks!!!

bkcoward :uk: (BOB member since 2010-08-16)

#2

Nope no easy way that I know of.

I’ve asked for this in the enhancements requests thread.

https://bobj-board.org/t/148144

Upto SAP now to see if they like the idea…and decide to give me a royalty
:smiley:

ScoobyDoo :uk: (BOB member since 2007-05-10)

#3

Maybe try: create a bunch of workflows. String together clumps of dataflows (3-5 of them) in serial in each workflow, putting the largest ones first. Then run all the workflows in parallel in the job.

dnewton :us: (BOB member since 2004-01-30)