Web service - using WS-Security and policy.xml with DS 4.0

system · May 7, 2012, 12:34pm

We’re trying to use Data Services 4.0 (Designer version 14.0.1.142) to consume an external web service with SSL and WS-Security. We are using a datastore of the web service type.

Importing the wdsl is no problem, and setting up the SSL has also been completed. I’ve removed the comment in axis2.xml for the rampart line, according to the Integrators Manual. I’ve also created a policy.xml, but I’m not sure how it should look. Im able to run a job moving the data to a local xml file, but all it says is:

-<XML_out> <AL_ERROR_NUM>3</AL_ERROR_NUM> <AL_ERROR_MSG><S:Fault xmlns:ns4="http://www.w3.org/2003/05/soap-envelope"><faultcode xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd">wsse:InvalidSecurity</faultcode><faultstring>Security Requirements not met - No Security</AL_ERROR_MSG> </XML_out>

Here is a link to the wdsl: https://crm.upsales.com/wsapi/client?wsdl

A correct call (that works in soapUI on the machine) is:

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:wsap="http://wsapi.upsales.com/">
<soapenv:Header>
     <wsse:Security xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">
          <wsse:UsernameToken wsu:Id="randomrandom-12312">
              <wsse:Username>xxxx</wsse:Username>
              <wsse:Password Type="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#PasswordText">xxxx</wsse:Password>
          </wsse:UsernameToken>
      </wsse:Security>
 </soapenv:Header>
   <soapenv:Body>
      <wsap:getClient>
         <clientId>42874</clientId>
      </wsap:getClient>
   </soapenv:Body>
</soapenv:Envelope>

So that is the Security header Im trying to create, and right now the policy.xml looks like this:

<wsp:Policy xmlns:wsp="http://schemas.xmlsoap.org/ws/2004/09/policy">
    <wsp:ExactlyOne>
        <wsp:All>
            <sp:SignedSupportingTokens xmlns:sp="http://schemas.xmlsoap.org/ws/2005/07/securitypolicy">
            <wsp:Policy>
                <sp:UsernameToken sp:IncludeToken="http://schemas.xmlsoap.org/ws/2005/07/securitypolicy/IncludeToken/AlwaysToRecipient"/>
          <wsp:Policy>
            <sp:WssUsernameToken10 /> 
          </wsp:Policy>
            </wsp:Policy>
            </sp:SignedSupportingTokens>
        </wsp:All>
    </wsp:ExactlyOne>
</wsp:Policy>

Can anyone help me with how the policy.xml should look in order to create the correct header?

stefanenroth (BOB member since 2012-04-20)

system · May 7, 2012, 5:26pm

use the attached policy.xml and also make sure the Security phase tag is uncommented in the axis2.xml for outflow, this will be towards the end of the file

it should be something like below


    <phaseOrder type="outflow">
        <!-- User defined phases could be added here -->
        <!--phase name="userphase1"/-->
        <!--system predefined phase-->
        <phase name="MessageOut"/>
        <phase name="Security"/>
    </phaseOrder>

other thing, try not to modify the default webservice-c\axis2.xml, since this is also used internally by DS, instead make a copy the webservice-c folder (compelete folder) and rename it as webservice-, for ex:- webservice-upsales, you can give any name this just and example
Copy the attched policy.xml to new folder
Open the webservice datastore and in the adavanced properties, use this new folder location as the axis2c config file path (the path should be upto folder name, for ex:- C:\Program Files\BusinessObjects\Data Services\ext\webservice-upsales
enter the WSS Secuirty username, password and set the WSS Security password type to textPassword
don’t enter the suername and password for the basic auth
Policy.zip (0.0 KB)

manoj_d (BOB member since 2009-01-02)

system · May 8, 2012, 11:52am

Thank you! Worked like a charm, I just had to add the username to the line

<rampc:User></rampc:User>

It also worked commenting that line and specifying the username in the datastore properties.

Again, thank you for your help!

stefanenroth (BOB member since 2012-04-20)

system · May 24, 2012, 9:54pm

Hi Manoj/stefanenroth,

I just exactly followed all the steps explained in the previous posts on this thread. However, i still do not get my Web Service to work during runtime.

The error i get is " 14549180 2828 RUN-248007 5/24/2012 9:51:13 PM Unable to create WS-security policy. Ensure that the policy file {1} exists and is valid."

The policy.xml is the file which i downloaded from this post.

Please let me know how to proceed further and making this connectivity work.

arunnura17 (BOB member since 2007-03-30)

system · May 25, 2012, 6:44am

Have you opened the policy.xml to see that its a valid xml file? And have you made sure the policy.xml file path in the data store configuration is correct?

stefanenroth (BOB member since 2012-04-20)

system · May 25, 2012, 10:25am

Yes. We ahve updated the Policy.xml file in the DataStore setting and also the xml file is a valid one. Still same error.

arunnura17 (BOB member since 2007-03-30)

system · May 25, 2012, 1:11pm

Another idea is to comment the rampc:User line and enter wss username/password in the datastore config.

Manoj is probably alot better than me helping, but I can try at least

stefanenroth (BOB member since 2012-04-20)

system · May 25, 2012, 7:09pm

there is no option in the Datastore to specify the Policy.xml, you can speficy the folder lcoation of axis2.xml in datastore, make sure you have done the following
1 - Set the WSS Username, password, password type parameters in the Datastore
2 - Copied the policy.xml file to %LINK_DIR%\ext\webservice-c (assuming that you have not specified any other folder location for datastore in for axis2 config) if it’s different then copy this file in that folder
3 - axis2.xml if update to engage rampartc and security tag is uncommented for outflow

can you post your axis2.xml ?

manoj_d (BOB member since 2009-01-02)

system · May 28, 2012, 4:13am

@stefanenroth,
We have commented the user name in policy.xml and then gave the WSS userid and password and password type in data store. But still getting same error.

@Manoj,

We ahve a new folder created as WebService-XATA. This folder has both Policy.xml and Axis2.xml file.
DataStore setting has WSS-Userid and Pwd and Pwd type set and also Axis2/c config path file set to this new folder.
3.When we removed the Policy file path setting from Data Store and ran the job, we got the below error.
There is no response for the web service . Ensure that the network, web server, and service are running properly. Also
ensure that the service client call time out is set properly.

Find attached the policy.xml and Axis2.xml file.
XATA.zip (2.0 KB)

arunnura17 (BOB member since 2007-03-30)

system · May 29, 2012, 5:43pm

the axis2.xml and policy.xml looks fine, is the endpoint SSL (HTTPS) or HTTP ? if it’s HTTPS then you will have to modify the axis2.xml to enable HTTPS

can you do the following
run the job from command line with -D -np options, this will create a file named axis2_log.txt in %LINK_DIR%\log folder, remove any information you consider sensitive (server url etc, replace is with XXXX) from this log and post it

manoj_d (BOB member since 2009-01-02)

system · May 30, 2012, 3:07am

Hi Manoj,

The web service end point is just http and not https.
The execution with -D -np created an empty axis2_log file.

Please advise on any other troubleshooting steps. Please help ASAP.

Thanks
Arun

arunnura17 (BOB member since 2007-03-30)

system · May 30, 2012, 7:13pm

if the axis2_log.txt is empty then it’s possible that the job is crashing do you see error reply from webservice function in the error log (%LINK_DIR%\log\errorlog.txt)?

this will require some investigation, please file a case with support and give me the incident #

manoj_d (BOB member since 2009-01-02)

system · May 30, 2012, 10:29pm

Thanks Manoj,

I saw the errorlog.txt and it does not have any entry for 5/29 date. We have a case open already. I shall update the case and send you an email with the details.

Thanks,
Arun

arunnura17 (BOB member since 2007-03-30)

system · June 20, 2012, 10:05pm

Hi, did u ever got the no response issue resloved? We are have similiar issue

vickeychen (BOB member since 2010-05-03)

system · June 22, 2012, 5:48pm

can you run the job from command line by passing -D -np as additional arguments, this will create a axis2_log.txt file in %LINK_DIR%\log folder, post that file or last 20 - 30 lines of that file

manoj_d (BOB member since 2009-01-02)

system · July 9, 2012, 2:06pm

Here is what we got in axis2_log

[Fri Jul 06 17:24:21 2012] [debug] …\src\core\engine\phase.c(121) axis2_handler_t *request_uri_based_dispatcher added to the index 0 of the phase Transport
[Fri Jul 06 17:24:21 2012] [debug] …\src\core\engine\phase.c(121) axis2_handler_t *addressing_based_dispatcher added to the index 1 of the phase Transport
[Fri Jul 06 17:24:21 2012] [debug] …\src\core\engine\phase.c(121) axis2_handler_t *rest_dispatcher added to the index 0 of the phase Dispatch
[Fri Jul 06 17:24:21 2012] [debug] …\src\core\engine\phase.c(121) axis2_handler_t *soap_message_body_based_dispatcher added to the index 1 of the phase Dispatch
[Fri Jul 06 17:24:21 2012] [debug] …\src\core\engine\phase.c(121) axis2_handler_t *soap_action_based_dispatcher added to the index 2 of the phase Dispatch
[Fri Jul 06 17:24:21 2012] [debug] …\src\core\engine\phase.c(121) axis2_handler_t *dispatch_post_conditions_evaluator added to the index 0 of the phase PostDispatch
[Fri Jul 06 17:24:21 2012] [debug] …\src\core\engine\phase.c(121) axis2_handler_t *context_handler added to the index 1 of the phase PostDispatch
[Fri Jul 06 17:24:21 2012] [debug] …\src\core\deployment\conf_builder.c(233) No custom dispatching order found. Continue with the default dispatching order
[Fri Jul 06 17:24:21 2012] [debug] …\util\src\class_loader.c(140) D:\Program Files (x86)\Business Objects\BusinessObjects Data Services/ext/webservice-c/lib/axis2_http_sender.dll shared lib loaded successfully
[Fri Jul 06 17:24:21 2012] [debug] …\util\src\class_loader.c(140) D:\Program Files (x86)\Business Objects\BusinessObjects Data Services/ext/webservice-c/lib/axis2_http_sender.dll shared lib loaded successfully
[Fri Jul 06 17:24:21 2012] [debug] …\util\src\class_loader.c(140) D:\Program Files (x86)\Business Objects\BusinessObjects Data Services/ext/webservice-c/lib/axis2_http_receiver.dll shared lib loaded successfully
[Fri Jul 06 17:24:21 2012] [debug] …\util\src\class_loader.c(140) D:\Program Files (x86)\Business Objects\BusinessObjects Data Services/ext/webservice-c/lib/axis2_http_receiver.dll shared lib loaded successfully
[Fri Jul 06 17:24:21 2012] [info] No files in the path D:\Program Files (x86)\Business Objects\BusinessObjects Data Services/ext/webservice-c/services.
[Fri Jul 06 17:24:21 2012] [debug] …\src\core\deployment\dep_engine.c(1283) axis2_dep_engine_load_module_dll: DLL path is : D:\Program Files (x86)\Business Objects\BusinessObjects Data Services/ext/webservice-c/modules/addressing/axis2_mod_addr.dll
[Fri Jul 06 17:24:21 2012] [debug] …\util\src\class_loader.c(140) D:\Program Files (x86)\Business Objects\BusinessObjects Data Services/ext/webservice-c/modules/addressing/axis2_mod_addr.dll shared lib loaded successfully
[Fri Jul 06 17:24:21 2012] [debug] …\src\core\deployment\dep_engine.c(1283) axis2_dep_engine_load_module_dll: DLL path is : D:\Program Files (x86)\Business Objects\BusinessObjects Data Services/ext/webservice-c/modules/logging/axis2_mod_log.dll
[Fri Jul 06 17:24:21 2012] [debug] …\util\src\class_loader.c(140) D:\Program Files (x86)\Business Objects\BusinessObjects Data Services/ext/webservice-c/modules/logging/axis2_mod_log.dll shared lib loaded successfully
[Fri Jul 06 17:24:21 2012] [debug] …\src\core\deployment\dep_engine.c(1283) axis2_dep_engine_load_module_dll: DLL path is : D:\Program Files (x86)\Business Objects\BusinessObjects Data Services/ext/webservice-c/modules/rampart/mod_rampart.dll
[Fri Jul 06 17:24:21 2012] [debug] …\util\src\class_loader.c(140) D:\Program Files (x86)\Business Objects\BusinessObjects Data Services/ext/webservice-c/modules/rampart/mod_rampart.dll shared lib loaded successfully
[Fri Jul 06 17:24:21 2012] [debug] …\src\core\deployment\dep_engine.c(1042) No modules configured
[Fri Jul 06 17:24:21 2012] [info] [rampart][rampart_mod] rampart_mod initialized
[Fri Jul 06 17:24:21 2012] [debug] …\src\core\phaseresolver\phase_resolver.c(139) Service name is : ANONYMOUS_SERVICE
[Fri Jul 06 17:24:21 2012] [debug] …\src\core\phaseresolver\phase_resolver.c(377) Module rampart will be engaged to ANONYMOUS_SERVICE
[Fri Jul 06 17:24:21 2012] [debug] …\src\core\phaseresolver\phase_resolver.c(139) Service name is : ANONYMOUS_SERVICE
[Fri Jul 06 17:24:21 2012] [debug] …\src\core\phaseresolver\phase_holder.c(139) Add handler RampartInHandler to phase Security
[Fri Jul 06 17:24:21 2012] [debug] …\src\core\phaseresolver\phase_holder.c(139) Add handler RampartOutHandler to phase Security
[Fri Jul 06 17:24:21 2012] [debug] …\src\core\phaseresolver\phase_holder.c(139) Add handler RampartOutHandler to phase MessageOut
[Fri Jul 06 17:24:21 2012] [debug] …\src\core\phaseresolver\phase_holder.c(139) Add handler RampartInHandler to phase Security
[Fri Jul 06 17:24:21 2012] [debug] …\src\core\phaseresolver\phase_holder.c(139) Add handler RampartOutHandler to phase Security
[Fri Jul 06 17:24:21 2012] [debug] …\src\core\phaseresolver\phase_holder.c(139) Add handler RampartOutHandler to phase MessageOut
[Fri Jul 06 17:24:21 2012] [debug] …\src\core\phaseresolver\phase_holder.c(139) Add handler RampartInHandler to phase Security
[Fri Jul 06 17:24:21 2012] [debug] …\src\core\phaseresolver\phase_holder.c(139) Add handler RampartOutHandler to phase Security
[Fri Jul 06 17:24:21 2012] [debug] …\src\core\phaseresolver\phase_holder.c(139) Add handler RampartOutHandler to phase MessageOut
[Fri Jul 06 17:24:21 2012] [debug] …\src\core\clientapi\op_client.c(888) Start:axis2_op_client_infer_transport
[Fri Jul 06 17:24:21 2012] [debug] …\src\core\clientapi\op_client.c(954) End:axis2_op_client_infer_transport
[Fri Jul 06 17:24:21 2012] [debug] …\src\core\engine\phase.c(210) Invoke the handler RampartOutHandler within the phase Security
[Fri Jul 06 17:24:21 2012] [debug] …\src\util\rampart_sec_header_builder.c(500) [rampart][shb] Building UsernmaeToken
[Fri Jul 06 17:24:21 2012] [debug] …\src\util\rampart_sec_header_builder.c(597) [rampart][shb] Asymmetric Binding.
[Fri Jul 06 17:24:21 2012] [debug] …\src\util\rampart_context.c(2225) [rampart][rampart_context] Nothing to sign outside Secyrity header.
[Fri Jul 06 17:24:21 2012] [info] [rampart][rampart_signature] No parts specified or specified parts can’t be found for Signature.
[Fri Jul 06 17:24:21 2012] [debug] …\src\util\rampart_context.c(2229) [rampart][rampart_context] Nothing to encrypt outside Secyrity header.
[Fri Jul 06 17:24:21 2012] [debug] …\src\util\rampart_encryption.c(800) [rampart][rampart_encryption] No parts specified or specified parts can’t be found for encryprion.
[Fri Jul 06 17:24:21 2012] [debug] …\src\omxmlsec\axiom.c(129) [rampart]Checking node Security for Signature
[Fri Jul 06 17:24:21 2012] [debug] …\src\omxmlsec\axiom.c(129) [rampart]Checking node UsernameToken for Signature
[Fri Jul 06 17:24:21 2012] [debug] …\src\omxmlsec\axiom.c(129) [rampart]Checking node Username for Signature
[Fri Jul 06 17:24:21 2012] [debug] …\src\omxmlsec\axiom.c(129) [rampart]Checking node Password for Signature
[Fri Jul 06 17:24:21 2012] [debug] …\src\omxmlsec\axiom.c(129) [rampart]Checking node Security for EncryptedKey
[Fri Jul 06 17:24:21 2012] [debug] …\src\omxmlsec\axiom.c(129) [rampart]Checking node UsernameToken for EncryptedKey
[Fri Jul 06 17:24:21 2012] [debug] …\src\omxmlsec\axiom.c(129) [rampart]Checking node Username for EncryptedKey
[Fri Jul 06 17:24:21 2012] [debug] …\src\omxmlsec\axiom.c(129) [rampart]Checking node Password for EncryptedKey
[Fri Jul 06 17:24:21 2012] [debug] …\src\core\transport\http\sender\http_transport_sender.c(246) ctx_epr:https://soadev.nih.gov:443/NEDPerson/NEDPersonOPSv2/WSDLNEDPersonOP-service.serviceagent/PortTypeEndpoint0
[Fri Jul 06 17:24:21 2012] [debug] …\src\core\transport\http\sender\http_transport_sender.c(805) using axis2 native http sender.
[Fri Jul 06 17:24:21 2012] [debug] …\src\core\transport\http\sender\http_sender.c(416) msg_ctx_id:urn:uuid:8595a289-4ef6-4b4a-8cbe-7ae9ed641115
[Fri Jul 06 17:24:21 2012] [info] [ssl client] CA certificate not specified
[Fri Jul 06 17:24:21 2012] [error] …\src\core\transport\http\sender\ssl\ssl_utils.c(51) Cannot find certificates
[Fri Jul 06 17:24:21 2012] [error] …\src\core\transport\http\sender\ssl\ssl_stream.c(101) Error occurred in SSL engine
[Fri Jul 06 17:24:21 2012] [error] …\src\core\transport\http\sender\http_client.c(287) Data stream creation failed for Host soadev.nih.gov and 443 port
[Fri Jul 06 17:24:21 2012] [error] …\src\core\transport\http\sender\http_client.c(544) client data stream null or socket error for host soadev.nih.gov and 443 port
[Fri Jul 06 17:24:21 2012] [error] …\src\core\transport\http\sender\http_client.c(548) A read attempt(HTTP) for the reply without sending the request
[Fri Jul 06 17:24:21 2012] [error] …\src\core\transport\http\sender\http_sender.c(1381) status_code < 0
[Fri Jul 06 17:24:21 2012] [error] …\src\core\engine\engine.c(179) Transport sender invoke failed
[Fri Jul 06 17:24:21 2012] [info] [rampart][rampart_mod] rampart_mod shutdown

On strange thing, I have two configuration set in this job, one to run from soa.nih.gov and one from soadev.nih.gov. When I run the job from command line. it will also use soadev.nih.gov url no matter what I set from BO designer.

vickeychen (BOB member since 2010-05-03)

system · July 9, 2012, 11:58pm

looking at the log you are not setting the SERVER_CERTIFICATE parameter correctly in %LINK_DIR%/ext/webservice-c/axis2.xml file, get the server certificate in base 64 encoded format and save to disk and use that location on axis2.xml

what is the version of DS ? there is a bug in DS 12.2 for webService datastore, multiple configuration will not work, this is fixed in DS 12.2.3.5

following is the error in the log file

manoj_d (BOB member since 2009-01-02)

system · July 10, 2012, 1:30pm

we are currently on 12.2.3.4. Do you recommend patch it 12.2.3.5

vickeychen (BOB member since 2010-05-03)

system · July 10, 2012, 1:37pm

sorry, we only have one web service and using default location.

vickeychen (BOB member since 2010-05-03)

system · July 11, 2012, 5:05pm

the problem is with certificate check the following post for getting the certificate and saving it in correct format

manoj_d (BOB member since 2009-01-02)