BusinessObjects Board

Dynamic values to substitution parameters

dataservices data quality
#1

Is it possible to assign values dynamically to substitution parameters?

Thanks,

urk :us: (BOB member since 2005-11-29)

#2

That’s what global variables are about. Substitution variables are repo constants, global variables are job variables.

Werner Daehn :de: (BOB member since 2004-12-17)

#3

Thanks werner for replying.

The problem is DS not allowing to use the global variables inside the DQ transforms(Wierd). We have one DQ process which processes different client files and the option values are different for each client. Is there a way that I can assign the Option values at run time?

Thanks,

urk :us: (BOB member since 2005-11-29)

#4

Hi Werner,

Thanks for replying…

Actually we are using NCOA processing and we need to pass the values of NCOA related parameters inside the Address Cleanse transform which would be different for each of the PAF ID’s.
And we cannot use global variables inside the transform.

How can we achieve this in one job itself.

Currently we are using two jobs,one job to call the main NCOA job using exec command and passing the substitution parameter values in that script.

Thanks for your help.

Shivbaba :us: (BOB member since 2010-06-01)

#5

I am leaving the ground where I feel secure, just so you know.

First of all, I assume we are talking about the options settings of the transform, not input. Otherwise you would simply add a column to the upstream query with the variable mapped as constant.

If it is the options tab, yes, then you need two transforms. I would not create two jobs but either two dataflows or one dataflow with a case transform routing the rows into two different DQ transforms. And each DQ transform has either the different settings hardcoded or two differently named substitution variables are used.

(The other thing I don’t get - but it’s early morning - is how two jobs help. A job cannot set substitution parameters either!?)

Werner Daehn :de: (BOB member since 2004-12-17)

#6

Hi Werner,

Firstly…I really thank you for replying…
Yes…you are correct I am talking about the options settings of the transform.

The approach you have suggested does not seem to work in my case because we are passing those values at run time. And for NCOA processing, we need to fill all the PAF related information in the options of the transform. Also for each of the PAF ID, we are picking its related information from the database.

We are calling JB1 (which does the NCOA processing) from JB2 which just has a script to call JB1.
Note that we have already taken the GUID command of JB2 from the management console by exporting its execution command.
Now inside the script of JB2, we can use exec command to call JB1 using al_engine exe. In al_engine, we can pass substitution parameters to the other job using some - CSV.

Thanks,

Shivbaba :us: (BOB member since 2010-06-01)

#7

I’m wondering if you ever found a way to make the DQ Substitution parameters dynamic? I’d like to load them from Excel, thus allowing external customer to “tweak” the match / no match paramaters. Many thanks for any help.

Don

donpowell58 :us: (BOB member since 2006-05-18)

#8

You can do it, it just takes some work.

if you export the execution command adding in the substitution variables you need… all you have to do is programatically tweak the export and launch it. Piece of cake. :crazy_face:

jlynn73 :us: (BOB member since 2009-10-27)

#9

I am trying to do this, but when I export the command I notice that all the substitution parameters I add come out with encrypted values.

DanHosler :us: (BOB member since 2013-06-19)

#10

use the -GI switch when you need to add non encrypted values:

 -PLocaleUTF8  -Sdsvcs1 -NsecEnterprise -Q"genRepo1" -UAdministrator -PY2RzMURRWEk -G"2f66966a_6cd0_4f9f_8252_2a7a84119610" -T14 -no_use_cache_stats  -no_audit  -xCR -LocaleGV -GI -GV"$GV_InputFileName=&&FileName;" -CSV"$$CustListRetrn=&&CustListRetrn;$$CustPAF=&&CustPAF" -CSV";$$CustTitleofPersonSigning=&&CustTitleofPersonSigning;$$CustParent" -CSV"Name=&&CustParentName;$$CustCompanyAddr=&&CustCompanyAddr;$$" -CSV"CustAlternateName=&&CustAlternateName;$$CustDateSignedby=&&CustDateSignedby;$$CustCompanyPhone=&&CustCompanyPhone;$$CustNam" -CSV"eofPersonSigning=&&CustNameofPersonSigning;$$CustListRecd=&&CustListRecd;$$CustCompanyName=&&CustCompanyName;$$CustListFreq=&&CustListFreq;$$CustNAICSCode=&&CustNAICSCode;$$CustCom" -CSV"panyPostCode2=&&CustCompanyPostCode2;$$CustCompanyPostCode1=&&CustCompanyPostCode1;$$CustDateSignedbyLicensee=&&CustDateSignedbyLicensee;$" -CSV"$CustCompanyRegion=&&CustCompanyRegion;$$CustCompanyLocality" -CSV"=&&CustCompanyLocality;" -CtBatch -CmDSVCS1 -CaAdministrator -CjDSVCS1 -Cp3502

you can also call the encryption util from outside of DS :wink:

jlynn73 :us: (BOB member since 2009-10-27)

#11

I had to laugh at that. Sarcasm understood. :+1:

eganjp :us: (BOB member since 2007-09-12)

#12

oh … you have no idea~

for this to work in our 3rd party scheduler, the jobs and job streams are dynamically added, scheduled and submitted via command line.

but at that point you can view at a glance all the jobs in the system, and can prioritize/hold add more virtual resources … bla bla to your hearts content.

so ya, as I said … piece of cake. :cookie:

jlynn73 :us: (BOB member since 2009-10-27)

#13

It sounds like you are further along on the pain factor than I was willing to go. I looked at how to construct the command line dynamically and I was able to do it but not to the point where I was comfortable with releasing it for produciton. I ended up running jobs using the Web Service call method. I’m still not happy with it but it works - most of the time.

It’s complicated, I agree. It would be nice if there was an easier way to submit the global variables and all the other “stuff” on the command line. I would also like a batch method to generate the Exprt Execution Command.

eganjp :us: (BOB member since 2007-09-12)

#14

Ive set it up using web services, but since the scheduler cant make the call it takes yet another wrapper.

The really bad thing about setting up automation this way is that unless you’ve got backup staff with some programming experience … if you happen to get hit by a bus on the way home … its pretty tough for regular Joe’s to debug when theres a problem.

and have fun every time you migrate. :frowning:

jlynn73 :us: (BOB member since 2009-10-27)

#15

I never got the two job system working correctly. So my solution was to write a Python script to generate a transform XML file for each client on the PAF list. Now all I need to do is import the transforms as I need them.

DanHosler :us: (BOB member since 2013-06-19)