Web service - using WS-Security and policy.xml with DS 4.0

system · June 22, 2012, 5:48pm

can you run the job from command line by passing -D -np as additional arguments, this will create a axis2_log.txt file in %LINK_DIR%\log folder, post that file or last 20 - 30 lines of that file

manoj_d (BOB member since 2009-01-02)

system · July 9, 2012, 2:06pm

Here is what we got in axis2_log

[Fri Jul 06 17:24:21 2012] [debug] …\src\core\engine\phase.c(121) axis2_handler_t *request_uri_based_dispatcher added to the index 0 of the phase Transport
[Fri Jul 06 17:24:21 2012] [debug] …\src\core\engine\phase.c(121) axis2_handler_t *addressing_based_dispatcher added to the index 1 of the phase Transport
[Fri Jul 06 17:24:21 2012] [debug] …\src\core\engine\phase.c(121) axis2_handler_t *rest_dispatcher added to the index 0 of the phase Dispatch
[Fri Jul 06 17:24:21 2012] [debug] …\src\core\engine\phase.c(121) axis2_handler_t *soap_message_body_based_dispatcher added to the index 1 of the phase Dispatch
[Fri Jul 06 17:24:21 2012] [debug] …\src\core\engine\phase.c(121) axis2_handler_t *soap_action_based_dispatcher added to the index 2 of the phase Dispatch
[Fri Jul 06 17:24:21 2012] [debug] …\src\core\engine\phase.c(121) axis2_handler_t *dispatch_post_conditions_evaluator added to the index 0 of the phase PostDispatch
[Fri Jul 06 17:24:21 2012] [debug] …\src\core\engine\phase.c(121) axis2_handler_t *context_handler added to the index 1 of the phase PostDispatch
[Fri Jul 06 17:24:21 2012] [debug] …\src\core\deployment\conf_builder.c(233) No custom dispatching order found. Continue with the default dispatching order
[Fri Jul 06 17:24:21 2012] [debug] …\util\src\class_loader.c(140) D:\Program Files (x86)\Business Objects\BusinessObjects Data Services/ext/webservice-c/lib/axis2_http_sender.dll shared lib loaded successfully
[Fri Jul 06 17:24:21 2012] [debug] …\util\src\class_loader.c(140) D:\Program Files (x86)\Business Objects\BusinessObjects Data Services/ext/webservice-c/lib/axis2_http_sender.dll shared lib loaded successfully
[Fri Jul 06 17:24:21 2012] [debug] …\util\src\class_loader.c(140) D:\Program Files (x86)\Business Objects\BusinessObjects Data Services/ext/webservice-c/lib/axis2_http_receiver.dll shared lib loaded successfully
[Fri Jul 06 17:24:21 2012] [debug] …\util\src\class_loader.c(140) D:\Program Files (x86)\Business Objects\BusinessObjects Data Services/ext/webservice-c/lib/axis2_http_receiver.dll shared lib loaded successfully
[Fri Jul 06 17:24:21 2012] [info] No files in the path D:\Program Files (x86)\Business Objects\BusinessObjects Data Services/ext/webservice-c/services.
[Fri Jul 06 17:24:21 2012] [debug] …\src\core\deployment\dep_engine.c(1283) axis2_dep_engine_load_module_dll: DLL path is : D:\Program Files (x86)\Business Objects\BusinessObjects Data Services/ext/webservice-c/modules/addressing/axis2_mod_addr.dll
[Fri Jul 06 17:24:21 2012] [debug] …\util\src\class_loader.c(140) D:\Program Files (x86)\Business Objects\BusinessObjects Data Services/ext/webservice-c/modules/addressing/axis2_mod_addr.dll shared lib loaded successfully
[Fri Jul 06 17:24:21 2012] [debug] …\src\core\deployment\dep_engine.c(1283) axis2_dep_engine_load_module_dll: DLL path is : D:\Program Files (x86)\Business Objects\BusinessObjects Data Services/ext/webservice-c/modules/logging/axis2_mod_log.dll
[Fri Jul 06 17:24:21 2012] [debug] …\util\src\class_loader.c(140) D:\Program Files (x86)\Business Objects\BusinessObjects Data Services/ext/webservice-c/modules/logging/axis2_mod_log.dll shared lib loaded successfully
[Fri Jul 06 17:24:21 2012] [debug] …\src\core\deployment\dep_engine.c(1283) axis2_dep_engine_load_module_dll: DLL path is : D:\Program Files (x86)\Business Objects\BusinessObjects Data Services/ext/webservice-c/modules/rampart/mod_rampart.dll
[Fri Jul 06 17:24:21 2012] [debug] …\util\src\class_loader.c(140) D:\Program Files (x86)\Business Objects\BusinessObjects Data Services/ext/webservice-c/modules/rampart/mod_rampart.dll shared lib loaded successfully
[Fri Jul 06 17:24:21 2012] [debug] …\src\core\deployment\dep_engine.c(1042) No modules configured
[Fri Jul 06 17:24:21 2012] [info] [rampart][rampart_mod] rampart_mod initialized
[Fri Jul 06 17:24:21 2012] [debug] …\src\core\phaseresolver\phase_resolver.c(139) Service name is : ANONYMOUS_SERVICE
[Fri Jul 06 17:24:21 2012] [debug] …\src\core\phaseresolver\phase_resolver.c(377) Module rampart will be engaged to ANONYMOUS_SERVICE
[Fri Jul 06 17:24:21 2012] [debug] …\src\core\phaseresolver\phase_resolver.c(139) Service name is : ANONYMOUS_SERVICE
[Fri Jul 06 17:24:21 2012] [debug] …\src\core\phaseresolver\phase_holder.c(139) Add handler RampartInHandler to phase Security
[Fri Jul 06 17:24:21 2012] [debug] …\src\core\phaseresolver\phase_holder.c(139) Add handler RampartOutHandler to phase Security
[Fri Jul 06 17:24:21 2012] [debug] …\src\core\phaseresolver\phase_holder.c(139) Add handler RampartOutHandler to phase MessageOut
[Fri Jul 06 17:24:21 2012] [debug] …\src\core\phaseresolver\phase_holder.c(139) Add handler RampartInHandler to phase Security
[Fri Jul 06 17:24:21 2012] [debug] …\src\core\phaseresolver\phase_holder.c(139) Add handler RampartOutHandler to phase Security
[Fri Jul 06 17:24:21 2012] [debug] …\src\core\phaseresolver\phase_holder.c(139) Add handler RampartOutHandler to phase MessageOut
[Fri Jul 06 17:24:21 2012] [debug] …\src\core\phaseresolver\phase_holder.c(139) Add handler RampartInHandler to phase Security
[Fri Jul 06 17:24:21 2012] [debug] …\src\core\phaseresolver\phase_holder.c(139) Add handler RampartOutHandler to phase Security
[Fri Jul 06 17:24:21 2012] [debug] …\src\core\phaseresolver\phase_holder.c(139) Add handler RampartOutHandler to phase MessageOut
[Fri Jul 06 17:24:21 2012] [debug] …\src\core\clientapi\op_client.c(888) Start:axis2_op_client_infer_transport
[Fri Jul 06 17:24:21 2012] [debug] …\src\core\clientapi\op_client.c(954) End:axis2_op_client_infer_transport
[Fri Jul 06 17:24:21 2012] [debug] …\src\core\engine\phase.c(210) Invoke the handler RampartOutHandler within the phase Security
[Fri Jul 06 17:24:21 2012] [debug] …\src\util\rampart_sec_header_builder.c(500) [rampart][shb] Building UsernmaeToken
[Fri Jul 06 17:24:21 2012] [debug] …\src\util\rampart_sec_header_builder.c(597) [rampart][shb] Asymmetric Binding.
[Fri Jul 06 17:24:21 2012] [debug] …\src\util\rampart_context.c(2225) [rampart][rampart_context] Nothing to sign outside Secyrity header.
[Fri Jul 06 17:24:21 2012] [info] [rampart][rampart_signature] No parts specified or specified parts can’t be found for Signature.
[Fri Jul 06 17:24:21 2012] [debug] …\src\util\rampart_context.c(2229) [rampart][rampart_context] Nothing to encrypt outside Secyrity header.
[Fri Jul 06 17:24:21 2012] [debug] …\src\util\rampart_encryption.c(800) [rampart][rampart_encryption] No parts specified or specified parts can’t be found for encryprion.
[Fri Jul 06 17:24:21 2012] [debug] …\src\omxmlsec\axiom.c(129) [rampart]Checking node Security for Signature
[Fri Jul 06 17:24:21 2012] [debug] …\src\omxmlsec\axiom.c(129) [rampart]Checking node UsernameToken for Signature
[Fri Jul 06 17:24:21 2012] [debug] …\src\omxmlsec\axiom.c(129) [rampart]Checking node Username for Signature
[Fri Jul 06 17:24:21 2012] [debug] …\src\omxmlsec\axiom.c(129) [rampart]Checking node Password for Signature
[Fri Jul 06 17:24:21 2012] [debug] …\src\omxmlsec\axiom.c(129) [rampart]Checking node Security for EncryptedKey
[Fri Jul 06 17:24:21 2012] [debug] …\src\omxmlsec\axiom.c(129) [rampart]Checking node UsernameToken for EncryptedKey
[Fri Jul 06 17:24:21 2012] [debug] …\src\omxmlsec\axiom.c(129) [rampart]Checking node Username for EncryptedKey
[Fri Jul 06 17:24:21 2012] [debug] …\src\omxmlsec\axiom.c(129) [rampart]Checking node Password for EncryptedKey
[Fri Jul 06 17:24:21 2012] [debug] …\src\core\transport\http\sender\http_transport_sender.c(246) ctx_epr:https://soadev.nih.gov:443/NEDPerson/NEDPersonOPSv2/WSDLNEDPersonOP-service.serviceagent/PortTypeEndpoint0
[Fri Jul 06 17:24:21 2012] [debug] …\src\core\transport\http\sender\http_transport_sender.c(805) using axis2 native http sender.
[Fri Jul 06 17:24:21 2012] [debug] …\src\core\transport\http\sender\http_sender.c(416) msg_ctx_id:urn:uuid:8595a289-4ef6-4b4a-8cbe-7ae9ed641115
[Fri Jul 06 17:24:21 2012] [info] [ssl client] CA certificate not specified
[Fri Jul 06 17:24:21 2012] [error] …\src\core\transport\http\sender\ssl\ssl_utils.c(51) Cannot find certificates
[Fri Jul 06 17:24:21 2012] [error] …\src\core\transport\http\sender\ssl\ssl_stream.c(101) Error occurred in SSL engine
[Fri Jul 06 17:24:21 2012] [error] …\src\core\transport\http\sender\http_client.c(287) Data stream creation failed for Host soadev.nih.gov and 443 port
[Fri Jul 06 17:24:21 2012] [error] …\src\core\transport\http\sender\http_client.c(544) client data stream null or socket error for host soadev.nih.gov and 443 port
[Fri Jul 06 17:24:21 2012] [error] …\src\core\transport\http\sender\http_client.c(548) A read attempt(HTTP) for the reply without sending the request
[Fri Jul 06 17:24:21 2012] [error] …\src\core\transport\http\sender\http_sender.c(1381) status_code < 0
[Fri Jul 06 17:24:21 2012] [error] …\src\core\engine\engine.c(179) Transport sender invoke failed
[Fri Jul 06 17:24:21 2012] [info] [rampart][rampart_mod] rampart_mod shutdown

On strange thing, I have two configuration set in this job, one to run from soa.nih.gov and one from soadev.nih.gov. When I run the job from command line. it will also use soadev.nih.gov url no matter what I set from BO designer.

vickeychen (BOB member since 2010-05-03)

system · July 9, 2012, 11:58pm

looking at the log you are not setting the SERVER_CERTIFICATE parameter correctly in %LINK_DIR%/ext/webservice-c/axis2.xml file, get the server certificate in base 64 encoded format and save to disk and use that location on axis2.xml

what is the version of DS ? there is a bug in DS 12.2 for webService datastore, multiple configuration will not work, this is fixed in DS 12.2.3.5

following is the error in the log file

manoj_d (BOB member since 2009-01-02)

system · July 10, 2012, 1:30pm

we are currently on 12.2.3.4. Do you recommend patch it 12.2.3.5

vickeychen (BOB member since 2010-05-03)

system · July 10, 2012, 1:37pm

sorry, we only have one web service and using default location.

vickeychen (BOB member since 2010-05-03)

system · July 11, 2012, 5:05pm

the problem is with certificate check the following post for getting the certificate and saving it in correct format

manoj_d (BOB member since 2009-01-02)

system · July 25, 2012, 7:29pm

I finially resloved the certification error. our axis2.xml was set up incorrectly.

HTTP/1.1 path to cert

paremter name SERVER_CERT need to be outside of transportSender block.

However, we are still having error

[Wed Jul 25 15:19:07 2012] [error] …\src\core\transport\http\sender\http_sender.c(1411) Error occurred in transport
[Wed Jul 25 15:19:07 2012] [error] …\src\core\transport\http\sender\http_sender.c(1442) Error occurred in transport
[Wed Jul 25 15:19:07 2012] [error] …\src\core\engine\engine.c(179) Transport sender invoke failed

Not sure how to fix this error. please advise

vickeychen (BOB member since 2010-05-03)

system · October 2, 2012, 12:56pm

hi i have same problem, but i configure the axis2.xml file successfully, but i don’t know how to create a policy.xml file, wsdl of my web service

" http://212.154.168.58:8080/egz2emf_v2/services/Egz2Emf?wsdl ",
please help with policy.xml …

el_paso (BOB member since 2012-09-27)

system · October 2, 2012, 10:24pm

the WSDL URL is not accessible, can you post the WSDL ? you can save the WSDL to a file from Internet Explorer by opening the WSDL in IE , right click and select view source, save the file to disk

zip and attach to the post

manoj_d (BOB member since 2009-01-02)

system · October 3, 2012, 2:52am

i saved my wsdl

and else check please my datastore configuration and axis2.xml

maybe you’ll find out some mistakes …
datastore_configure_and_axis2.xml.zip (60.0 KB)
wsdl.zip (1.0 KB)

el_paso (BOB member since 2012-09-27)

system · October 3, 2012, 3:59am

do the following
don’t set the keystore path in Datastore configuration, the endPoint is not SSL enabled it’s http not https

don’t set the WS-Policy path, set the vlaue to empty
set the axis2 configuration file path to as below
/u01/BOBJ_DID/dataservices/ext/eg2emf

save the Datastore, go to command line and cd to $LINK_DIR/ext directory and make a copy of webservice-c directory (cp -R webservice-c eg2emf)

edit the aixs2.xml, comment out the HTTPS sender and receiver sections that you have uncommented, the endPoint is not SSL, if you have a copy of original file use that

uncomment only the rampart module ref and security tags in inflow and outflow phase order

use the same policy.xml that is attached to one of the posts

other thing is, your WSDL doesn’t have complete definition for input and output schema, you will have to get that also

before using this is DS have you tested this WSDL using SoapUI or any other WebService Client ?

if you are still not able to make this work, file a case with support and let me know the case #

manoj_d (BOB member since 2009-01-02)

system · October 3, 2012, 4:55am

checked datastore configuration
copied original axis2.xml and uncomment only rampart
got a policy.xml from attached post file
sent request ( we have generated correct soap message that has been generated from other working system ) with soap ui, web service works

after error from DS :
<AL_ERROR_NUM>3</AL_ERROR_NUM>
<AL_ERROR_MSG><S:Fault xmlns:ns4="http://www.w3.org/2003/05/soap-envelope"><faultcode>S:MustUnderstand</faultcode><faultstring>MustUnderstand headers:[{http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd}Security] are not understood</fault</AL_ERROR_MSG>

so what about path of SERVER_CERT and KEY_FILE. where do we have to configure settings of out certificates and key.

/u01/BOBJ_DID/dataservices/ext/webservice-c/mikk/sert_base64.cer
/u01/BOBJ_DID/dataservices/ext/webservice-c/mikk/p8gostkey.p8

Then how can we point our Encryption algorithm to sign the body of message
(it’s SIGN_METHOD_GOST = "xmldsig-more namespace)
datastore_axis2_policy.zip (71.0 KB)

el_paso (BOB member since 2012-09-27)

system · October 3, 2012, 5:43am

check the following link for policy.xml files for differen scenarios

can you run the job from command line using additional args -D -np, this create the aixs2_log.txt file, attach that log

manoj_d (BOB member since 2009-01-02)

system · October 3, 2012, 6:48am

i don’t know how run the job from command line, please explain how it works, i found out AL_RWJobLauncher and …

i exported job from management_console and run it …
[root@devds bin]# ./AL_RWJobLauncher.sh -D -np “/u01/BOBJ_DID/dataservices/log/JOB_DS_DEV/” -w “inet:devds:3500” " -PLocaleUTF8 -R"repo.txt" -G"2786caaf_a656_4814_a6bd_5b7350a59ed7" -r1000 -T14 -Kspdevelopment -LocaleGV -CtBatch -Cmdevds.emf.minfin.kz -CaAdministrator -Cjdevds -Cp3500 "
./AL_RWJobLauncher.sh: line 9: 12739: Abort

job is completed successfully but already axis2_log.txt file is empty …

launch like this ?
version of product : SAP BusinessObjects Data Services 4.0 (14.0.1) SP2 Patch 4

-check please soap request, i must send request like this in Data Services

-what else ?
this situation enrages me …
soap_ui_request.zip (2.0 KB)

el_paso (BOB member since 2012-09-27)

system · October 3, 2012, 8:02pm

add the -D -np after -Cp3500 not after AL_RWJobLauncher.sh

something as below
./AL_RWJobLauncher.sh “/u01/BOBJ_DID/dataservices/log/JOB_DS_DEV/” -w “inet:devds:3500” " -PLocaleUTF8 -R"repo.txt" -G"2786caaf_a656_4814_a6bd_5b7350a59ed7" -r1000 -T14 -Kspdevelopment -LocaleGV -CtBatch -Cmdevds.emf.minfin.kz -CaAdministrator -Cjdevds -Cp3500 -D -np "

before you run this, add the logging module also, add it after addressing and before rampart in the axis2.xml
the module list will look like something as below

check if you see the axis2_log.txt file, remove any information from the log that you don’t want to share and attach the log

copy the attached policy.xml file to /u01/BOB_DID/dataservices/ext/eg2emf , rename the existing policy.xml to something else or mv out of this directory

update following in the policy.xml as per your env
rampc:ReceiverCertificateC:\cacert.pem</rampc:ReceiverCertificate>
rampc:CertificateC:\mycert.pem</rampc:Certificate>
rampc:PrivateKeyC:\mykey.pem</rampc:PrivateKey>
policy.zip (0.0 KB)

manoj_d (BOB member since 2009-01-02)

system · October 4, 2012, 4:05am

-thanks a lot Manoj D, finally i got it this axis2_log !

-so, next, please check it out this file axis2_log

-what kind of cert or key i have to write in this section
( server cert or some root certs and keys ??? )

rampc:ReceiverCertificateC:\cacert.pem ? </rampc:ReceiverCertificate>
rampc:CertificateC:\mycert.pem ? </rampc:Certificate>
rampc:PrivateKeyC:\mykey.pem ? </rampc:PrivateKey>
axis2_log.zip (1.0 KB)

el_paso (BOB member since 2012-09-27)

system · October 4, 2012, 4:41am

from the log the call is failing when trying to load the certificate I think you have not updated the location correctly

rampc:ReceiverCertificate give the location of the server certificate
rampc:Certificate client certificate
rampc:PrivateKey key for the client certificate

I think you were entering this values in HTTPS section earlier, also how are you giving this information in SoapUI ?

manoj_d (BOB member since 2009-01-02)

system · October 4, 2012, 6:32am

thanks for your help and advises Mr. Manoj D.
soap ui request is example which generated by other system and signed by java code in the mapping of other PI Sap system
location of files is correct, i checked and again run the job but had the same errors from rampc section (couldn’t load private key from file )
web service need just signed body of message
we have key storage ( key_gost.p12 )
there is a private key inside of our key_store, which is used for encrypt and sign message by GOST algorithm
ECGOST3410 ( “xmldsig-more namespace” )
i think that problem is in that format of keys and in rampart cannot understand these keys and cipher algorythm
check please axis2_log, new log, security phase error …
dear Manoj D, today i got new RSA keys, then prepared these keys and certs in right form (by using openssl)
next,made some experiments with job, and got a results:
first security phase error
second after read the log, comment out the security phase
and had a log which attached below
now i really have two questions :
what’s wrong with my configuration
how can i see a soap message after it has been generated by DS (axis, rampart, etc ) before sending to web service

axis2_log.zip (3.0 KB)
axis2_log_2.zip (2.0 KB)

el_paso (BOB member since 2012-09-27)

system · October 7, 2012, 12:13am

use TCPMon, I think it’s available on Linux also, to capture the message that going from DS and coming back to DS

can you also, add Security phase to the InFault

manoj_d (BOB member since 2009-01-02)

system · October 8, 2012, 11:30am

Dear Manoj D could you sent me right axis2.xml or your own file which real working on system, i want to compare with my axis ?

el_paso (BOB member since 2012-09-27)