Target based CDC

system · July 7, 2011, 1:32am

Hi Guys,

I am currently proposing a staging database for a client because of the CDC requirement. Since the source system does not have timestamps which can be used for source based CDC, i am looking at the target based CDC technique.

I would just like to know if there is any documentation for target based CDC that I can use to justify my proposal.

Thanks,

R.A. (BOB member since 2008-10-28)

system · July 7, 2011, 2:15am

Shouldn’t the fact that the source system is not equipped to provide insight as to what data has been modified / added be sufficient reason to go for a target CDC solution?

ErikR (BOB member since 2007-01-10)

system · July 7, 2011, 3:11am

Yes… That’s what I’m also saying but the other consultant wants a document from SAP regarding this which is I don’t know why, maybe for formality because there will be an extra server for the staging area.

Thanks!

R.A. (BOB member since 2008-10-28)

system · July 8, 2011, 9:31pm

When you say “target based CDC” do you mean an ODS that is used to compare the entire source to data that has already been processed? Or do you mean something like the CDC that Oracle provides?

The Oracle CDC is entirely source based. So I’m not sure what you are proposing.

eganjp (BOB member since 2007-09-12)

system · July 9, 2011, 12:33am

Yes! The first one that you said.

But I think the SAP consultant is now digesting what I’m saying… And with this I think I don’t need to provide any documentation. Hehehe!

Thanks!

R.A. (BOB member since 2008-10-28)

system · July 11, 2011, 3:07pm

It doesn’t have to be an extra server that your ODS is deployed to. In fact, you may find that processing data from the ODS into the DW is faster if the ODS and DW are co-located.

eganjp (BOB member since 2007-09-12)

system · July 12, 2011, 12:52am

Yes. But I think they won’t allow it. I’m thinking of using the same server as Data Services for the staging database but I am worried about the performance.

It’s hard to get an approval to get another server.

R.A. (BOB member since 2008-10-28)

system · July 13, 2011, 2:13am

Going back to the original post, Oracle CDC is source based and has no requirement for timestamps. It operates off of the archive logs. It is excellent for quickly implementing a method to capture the changed rows. However, the administrative issues need to be understood before you go “all in” with it.

The DBA group at my current client cannot guarantee that there will be no interruption in the CDC flow or loss of data (and there have been gaps). The way the situation was described to me is that Oracle CDC is built on Oracle Streams. By using CDC you give up the finer granularity of control that streams gives you. I’m not an operations DBA and my expericence with CDC has primarly been as a consumer/subscriber of published data, so I could be a bit off in my description.

eganjp (BOB member since 2007-09-12)