BusinessObjects Board

Calling profiling as a webservice.

dataservices administration
#7

let me check that, how are you calling the profiling task ? from SoapUI or any other similar tool

or by creating a WebService Adapter Datastore and importing the WSDL profiling function through that data store and calling the function from batch job ?

if you are using Adapter approach, then enable the trace for the Adapter and see what is the request that is passed to function, the SOAP request will be printed in the Adapter trace

if there is a problem with the request then check if there is any error message in the %LINK_DIR%\log webservice.log file

manoj_d (BOB member since 2009-01-02)

#8

Hi,

I have successfully called the webservice using both methods you have stated (SOAP UI and Data Integrator). A valid Task and Run ID is passed back.

There is no problem with the request, or the response.
The problem is the profile, after entering a running state eventually fails! with a ‘cancelled’ message.

I need to understand why after starting successfully the profiler task eventually fails when called from the submit_profiling_task webservice, but runs fine when using the normal method.

buckshot (BOB member since 2007-08-10)

#9

if you are comfortable using al_engine command line you can try the following since you are able to create the Profiler task

check the server_event log file for the Profiler Job server, when you submit a profiling task it will execute al_engine with the taskId, and print the complete command in the log file

copy that command to notepad
remove -l, -m -z arguments
move -U argument to the begining
add -D at the end

open a command prompt and pass all these arguments to al_engine from the command line and run, you get some information on the output, attach the output I can check what going wrong

manoj_d (BOB member since 2009-01-02)

#10

Again thanks for your time manoj_d,

I ran this on the jobserver, there was no output of any significance, maybe I have a parameter wrong ?

F:\Program Files\Business Objects\Data Integrator 11.7\bin>al_engine.exe -Udi_pr
of -CtBatch -CmMSODS1 -CaAdministrator -CjMSODS1 -Cp3500 -SODSP.WORLD -NOracle
-P;49FBD39ECC3E7C7B711195143901BA24D45FF370D9B94635BA792773FEC5ED61 -ServerGroup
null -Kapid4 -Kaprid4 -Kapt"ODS_CITIPAC.WEBSERVICE_TEST" -KapC00=All -KapC01=2 -
KapC02=1000 -KapC03=500000 -KapC04=10 -KapC05=10 -KapC06=1 -KapC07=90 -KapC08=2
-KapC09=1 -KapC10=500000 -KapC11=10000 -w"F:\Program Files\Business Objects\Data
Integrator 11.7/log/DI_PROD/odsp.world__di_prof/monitor_05_01_2009_13_27_13_272
__.txt" -Dt05_01_2009_13_27_13_272 -D

returned
(11.7) 05-01-09 13:51:08 (5472:9364) ORA_LIB: Loaded Oracle Client library <OCI
.DLL> is in the path: <C:\Apps\Oracle920\bin\OCI.DLL>.

F:\Program Files\Business Objects\Data Integrator 11.7\bin>

buckshot (BOB member since 2007-08-10)

#11

There is some output now when I run the command … but its getting late maybe i’ve been playing too long today…

F:\Program Files\Business Objects\Data Integrator 11.7\bin>al_engine.exe -Udi_prof -CtBatch -CmMSODS1 -CaAdministrator -CjMSODS1 -Cp3500 -SO
DSP.WORLD -NOracle -P;49FBD39ECC3E7C7B711195143901BA24D45FF370D9B94635BA792773FEC5ED61 -ServerGroupnull -Kapid4 -Kaprid4 -Kapt"ODS_CITIPAC.W
EBSERVICE_TEST" -KapC00=All -KapC01=2 -KapC02=1000 -KapC03=500000 -KapC04=10 -KapC05=10 -KapC06=1 -KapC07=90 -KapC08=2 -KapC09=1 -KapC10=500
000 -KapC11=10000 -w"F:\Program Files\Business Objects\Data Integrator 11.7/log/DI_PROD/odsp.world__di_prof/monitor_05_01_2009_13_27_13_272_
_.txt" -Dt05_01_2009_13_27_13_272 -D
(11.7) 05-01-09 16:52:03 (2128:4952) ORA_LIB: Loaded Oracle Client library <OCI.DLL> is in the path: <C:\Apps\Oracle920\bin\OCI.DLL>.
(11.7) 05-01-09 16:52:03 (E) (2128:4952) REP-100118: Initialization
Repository <PF_TEXT> is missing the language text associated with object id <PF Task Ob
ject: 4>. Notify Customer Support.
F:\Program Files\Business Objects\Data Integrator 11.7\bin>

buckshot (BOB member since 2007-08-10)

#12

this is not the problem that you are having, this error means the task id is not there in profiling repository, check the PF_TASK table in profiling repository for what is the ID, you might have deleted the profiling task from Admin Console

I tried to run the task from SoapUI and finally could run the profiling by calling it from SoapUI, following was the problem not sure if its the same case for you

from SaopUI the elements in requests will be prefixed with namespace, remove the prefixes from all the tags
pf:PF_Task_Request
do a find replace in the SoapUI request editor for <pf: and replace wih < and for </pf: and replace with </

I am attaching sample request that I used, one table, one column, the physical table name element is optional so I removed it, you can try passing that also

pass the datastore name using following convention
Database:::
Database::Oracle:ORCL

I am not sure if you will be able to call this from DI web service Adapter, since that also adds the namespace prefix to the elements, you will need to pass the request as plain XML message inside the soap body
submit_profiling_task.txt (4.0 KB)

manoj_d (BOB member since 2009-01-02)

#13

Thanks for your time manoj_d I really appreciate it.

Obviously I was hoping to call the profiling webservice from within DI. And It so nearly worked! but in the end a waste of time on my part flogging a dead donkey. Its strange that DI is not compatible with its own webservice ? Is this going to be fixed up in a later version ?
Maybe I should raise a support call with SAP…but its such a horrible portal !!!

Just like the ‘maximum open cursors exceeded’ errors after spending an age setting up Metadata Integrator, yes I know this is fixed in a new patch but applying these patches costs time and money and every iteration reduces the softwares image with management.

Trying to set up these ‘features’ costs time and money (Submit Profiling as a Webservice, Metadata Integrator) maybe if I could charge back my time to SAP they would have more of an incentive to get their product working properly in the first place.

buckshot (BOB member since 2007-08-10)

#14

let me check this, I am not very sure if it was intended to expose the profiling operations as webservice to the end user, the feature is mainly intended for internal use by other BO products and with in DI, hence you will not see any documentation for it

you can still call the web service from batch job using a simple java web service client which can send the XML request

manoj_d (BOB member since 2009-01-02)

#15

You’ve done a great job on chasing this down. I have been of on other things and come back to see all the legwork has been done.

Disappointing that it can’t be called from within DI, as that would be providing an end to end solution.

We are moving everthing onto DataServices so will have to pick this up on there.

Will post if we get any success.

pxbaker :uk: (BOB member since 2007-01-09)

#16

have created a bug for this for DS XI 3.2

for previous releases create a case with support

manoj_d (BOB member since 2009-01-02)

#17

Thats good news!

Thanks for all your help.

buckshot (BOB member since 2007-08-10)

#18

Hi Guys,

Has anyone tried this in 3.2 ? Has the bug been fixed ?

Unfortunately we installed 3.1 on Tuesday as we could not find 3.2 on the SAP download (since found out it was available elsewhere on SAP)

I can arrange through our ITIL to move to 3.2 but I would not be in such a rush if it is not fixed.

(yes I really really need to get the Profiler webservice running from within a DI job) I also need to get a solution for scheduling Profiles (easy if the webservice bug is fixed)

Thanks for any feedback.

buckshot (BOB member since 2007-08-10)

#19

no this issue is not fixed in XI 3.2, I filed it in 3.2 version, its not yet scheduled for any release

have you tried writing a simple Java based webService client to invoke this service, the current issue using WebService Datastore is its not passing it as plain XML but qualifying it with namespace

you call the Java program in a batch job using exec

manoj_d (BOB member since 2009-01-02)

#20

I was just thinking about this the other day… and here I see others are looking for the same feature.

The big drawback for me, with DS’s profiling feature, is that it never has the results instantly ready to use “when I want them”. You always have to kick off a profiling step, and wait, and wait, and it interrupts your stream of thought.

So I’d really like to be able to set up a job that says: Do a detailed (or summary) profile on all columns on all tables in a certain datastore… And schedule it to run every Saturday night, when no one is on the system.

Then relatively-fresh results would be already available, when I needed them.

dnewton :us: (BOB member since 2004-01-30)

#21

Hi manoj_d,

I don’t have very much exposure to Java. I can hack around the CMC, read and parse files and stuff, but calling web services might be a bit stretching it!
I have had a look and there are loads of implementations but I assumed I need the Axis jars etc …
I found a web service example on the Axis site and messed around with it until it built without any errors, and modified it with a bit of guess work!

All I need to do is call one service, say the Logon or even the Ping successfully then I am off and running!
I may have a simple problem or maybe I am way off the mark! I have attached my source and result after running it can you help ?

SOURCE:
[i]import org.apache.axis.client.Call;
import org.apache.axis.client.Service;
import javax.xml.namespace.QName;

public class TestClient {
public static void main(String [] args) {
try {
String endpoint = “http://SERVER:28080/diAdmin/servlet/webservices&wsdlxml”;
Service service = new Service();
Call call = (Call) service.createCall();

System.out.println("buck1");

call.setTargetEndpointAddress( new java.net.URL(endpoint) );
System.out.println(“buck2”);
call.setOperationName(new QName(“Connetction_Operations”, “Logon”));
//call.setOperationName(new QName("", “Connection_Operations.Ping”));
System.out.println(“buck3”);
System.out.println("taregetEndpointAddress " + call.getTargetEndpointAddress());
System.out.println("operationName " + call.getOperationName().toString());
System.out.println("encodingStyle " + call.getEncodingStyle());
String ret = (String) call.invoke( new Object[] { “DummyUser”,“DummyPassword” } );

   System.out.println(call.getResponseMessage().getSOAPPartAsString());

   
   System.out.println("Sent Logon, got '" + ret + "'");
 } catch (Exception e) {
   System.err.println("***Error***   " +  e.toString());
 }

}
}[/i]
OUTPUT:
[i]--------------------Configuration: dataIntegratorProfiler - j2sdk1.4.2_08 - --------------------
buck1
buck2
buck3
taregetEndpointAddress http://SERVER:28080/diAdmin/servlet/webservices&wsdlxml
operationName {Connetction_Operations}Logon
encodingStyle http://schemas.xmlsoap.org/soap/encoding/
Error (404)/diAdmin/servlet/webservices&wsdlxml

Process completed.[/i]

buckshot (BOB member since 2007-08-10)

#22

if you are OK to try using exec and calling the java program, then the code for invoking the profiling service already exists in DI WebService Adapter, sorry didn’t look into that earlier

you don’t have to build your own WS Client, you can use the Profiler test client from the WebServiceAdapter jar file

use the attached script to call the profiling service and pass the following arguments to it
ProfilerWSClient.bat -Alocalhost -p28080 -SSubmit_Profiling_Task -Xtest.xml

only thing that you will have to do is create a XML file with the input message, let me know if it works

make sure that the LINK_DIR is set and all the jar files referred in the bat file are there in the system
ProfilerWSClient.zip (0.0 KB)

manoj_d (BOB member since 2009-01-02)

#23

Bad News…

Just upgraded onto DS3.1 and the profiling options have been completely removed from the WSDL!!!

Shot through the foot springs to mind. Is this a case of fixing the above mentioned DI inconsistency by removing the lot.

Dont know where to go with this. It was close to getting a scheduled profiling solution.

I have jobs which build the profiler output into a couple of stars so they can be easily reported on through BO, but it looks like we need a trained monkey to submit the tables for profiling.

pxbaker :uk: (BOB member since 2007-01-09)

#24

let me check this, I don’t think its removed, by default the profiling operations are not published in the WSDL, there is option in admin.xml to enable this

go to %LINK_DIR%\conf folder on the machine where Webserver is installed and open Admin.xml in any text editor, check for
FALSE it will be either missing or the value will be FALSE, set that value to TRUE and restart your WebServer

manoj_d (BOB member since 2009-01-02)

#25

Where did you find 3.2 on SAP? I’ve been looking all over the place but could only find 3.1!

jmcilvaine :us: (BOB member since 2007-06-14)

#26

There is a thread on it:

Cheers

buckshot (BOB member since 2007-08-10)