BusinessObjects Board

parallel load for hash partitioned tables in DS 3.2

dataservices performance
#1

Hi all,

We load a big flat file into hash partitioned table. It seems in DS 3.2 parallel load is not working for hash type as “enable partition” option can not be set in table properties.

Can someone advise some ways of performance improvement in this situation. Current bulk rate is 2m records per minute. It is a poor load from flat file into a table. No queries or transformations.

job server runs on hp-ux, itanium 8 Processors :hb:

Thanks in advance for your replies,

vato (BOB member since 2011-02-18)

#2

Did you try setting the number of loaders > 1?

If you use bitmap indexes on the partitioned table you may find that with Loaders > 1 you get deadlocks on the bitmap indexes.

eganjp :us: (BOB member since 2007-09-12)

#3

hi,

Thanks for update. We are not going to use bitmaps on this table.

vato (BOB member since 2011-02-18)

#4

The “Enable partitioning” is disabled because DS doesn’t know how to handle hash partitions. List partitions are easy and range partitions just a little more work. DS doesn’t look at the the Oracle table’s metadata at runtime to figure out what partitions the table has configured. It uses whatever was available when the table was imported.

I’m guessing that DS doesn’t know which partition a row is going go into when the partition method is hash.

eganjp :us: (BOB member since 2007-09-12)