BusinessObjects Board

RealTime Services Slower With Twice The Instances

Hi there good people of the BOB community. :smiley:

I have a Data Services Installation (Version: 14.2.9.1549) on an AIX 7.2 server. I’m running two JMS Adapter instances to process xml messages received on an ActiveMQ message queue - placing the response on another ActiveMQ message queue.

Adapter Instance One

Initially ran two instance operations of “Get: Request/Reply and Request/Acknowledge using queues” that call a realtime service

First Adapter Instance operation Settings:

Polling interval = 1,000 msec
Thread count = 3
Operation Retry count = 10
Operation retry interval = 24,000 msec
Enable = false
Source Queue = Request_Inbound
Service = RTJ_Process_Request_1
Timeout = 120,000 msec
Continue after Error = true
Default Resposne Queue = blank
Undelivered Queue = Request_Errors
Enable Transaction = no

The second adapter instance operation settings are identical - except that the Service = RTJ_Process_Request_2

Adapter Instance Two

This runs one instance operation of “Put: Request/Acknowledge using queues” configured thus:

Operation Instance = PUT_REQUEST_RESPONSE
Thread count = 1
Operation retry count = 5
Operation retry interval = 15,000
Display name = UT_REQUEST_RESPONSE
Description = places response to request on outbound queue
Enable = false
Destination Queue = Response_Outbound
Request Format = […]/Response.xsd
Request XML Root Element = RequestResponse

This is utilised by the realtime services “RTJ_Process_Request_1” and “RTJ_Process_Request_2” to respond to each inbound request. The realtime service is configured as follows:

Service name = RTJ_Process_Request_1 (and RTJ_Process_Request_2)
Job name = RTJ_Process_Request
Repository name = Repo_DS42
Processing retry count max = 2
Enable job tracing = blank
System configuration = Live
Startup timeout = 500
Queuing timeout = 1,200
Processing timeout = 1,800
Recycle request count max = 10,000
Enable = “ticked”
Job Server = DS42:3250
Max Instances = 1
Min Instances = 1
Enabled = Yes

.
1 The initial configuration worked as follows:

6 queue connections seen in ActiveMQ to the “Request_Inbound” queue.

We placed 12 Identical test messages on the “Request_Inbound” queue and these were processed almost instantaniously.

Next we placed 1,000 Identical test messages on the “Request_Inbound” queue and these were processed in 46 seconds.

Next we placed 10,000 Identical test messages on the “Request_Inbound” queue, these were processed as follows:

Minute Volume TPS Comments
1 1392 23.2
2 2673 21.35 both realtime service instances dropped towards the end of the minute.
3 2766 1.55 One realtime service instance recovered towards the end of the minute.
4 3783 16.95 The other realtime service recovered about halfway through the minute.
5 5148 22.75
6 6500 22.53
7 7880 23
8 9257 22.95
9 9996 12.32 We had 4 messages on the “Request_Errors” queue.

.
2 So, we thought we’d try doubling up on the Adapter Operation Instances and the Realtime Services.

12 queue connections seen in ActiveMQ to the “Request_Inbound” queue.

We placed 24 Identical test messages on the “Request_Inbound” queue and these were processed almost instantaniously.
We placed 1,000 Identical messages on the “Request_Inbound” queue and Oh my word - the performance was abysmal! :face_with_symbols_over_mouth:

Here is the first few minutes:

Minute Volume TPS Comments
1 269 4.48
2 533 4.4

The only positive was that all four of the realtime service instances kept running. Huh? :confounded:

.
3 So, we thought we’d revert to the previous two instances configuration. … Performance remained at the same abysmal level! Huh? How? :disappointed:

I’m hoping someone with the right expertise is about that may be able to help me. I will be most grateful. :pray: