Table comparison

dataservices job design
1

Hi,

I have to perform a table comparison for a table which has composite key. The composite key has almost all the columns of the table except lastmod_dt and lastmod_nm columns. How can i handle this?

Since the composite key has lot of columns ,the table comparison is taking lot of time.

Any work arounds for this?

suma_ys (BOB member since 2009-06-12)

2

Any inputs reg. this?

suma_ys (BOB member since 2009-06-12)

3

the amount of columns in the primary key has no impact on the TC performance.

Werner Daehn :de: (BOB member since 2004-12-17)

4

Thanks Werner.

I am adding a sort operation based on composite key before the TC to perform TC by sorted primary key.

WIll that have any impact?

suma_ys (BOB member since 2009-06-12)

5

Well, sorting is expensive if the table is very large.

What are the number of rows going into the TC transform, how many go out and how large is the comparison table? Just round about numbers.

Werner Daehn :de: (BOB member since 2004-12-17)

6

Hi Werner,

Table has around 1.8M million records. None of the rows should go out of TC since no data has been changed in TC. The comparison and the target table are the same.

Even though there is no change in source data, the rows are going out of TC.

Any idea in what scenario this can happen?

suma_ys (BOB member since 2009-06-12)

7

1,8 million rows? That’s not too much. I would go for caching and use pageable cache. Have you tried that already?

Werner Daehn :de: (BOB member since 2004-12-17)