Automatically Disable Proxy Service to avoid overloading OSB

A very frequently asked question of using Oracle Service Bus (OSB) is how to avoid OSB to be overloaded once an endpoint becomes temporarily unreachable. The same question is equally asked for an endpoint doesn't respond as quickly as expected and, in worse case, the situation deteriorates gradually. The concern is raised for the potential risk of overwhelming OSB if the relevant proxy service continues receiving requests from callers.

From the design practice point of view, it is always recommended to make your endpoint highly available. With the help of either load balancer or OSB's failover capability across multiple endpoints, you should be able to make your OSB safer and less vulnerable from endpoint errors. In addition, you'd better implement OSB best practice by configuring a Work
Manager attached to Proxy Service and enabling throttling control on
Business Service. The details of these two features will not be covered
in this blog. You can find the details of using work manager with OSB here and How to enable and use throttling control here in the documentation.

However, this is not the whole picture of the story. Once an unexpected problem occurs on endpoint, you certainly want to be noticed for the incident and take some appropriate action to cope with it especially in the case where you have only one endpoint or the endpoint systems reside externally and out of your control.

Now, let's see how can you get notified when the following two common problems occur at endpoint:

• the endpoint becomes unreachable
• the endpoint still alive but the response becomes slow and results in SLA violation

OSB provides a nice feature to offline the unresponsive endpoint URI due to communication error. A Service Level Agreement (SLA) rule can be defined to trigger the alert once endpoint URI is marked as offline. You can also define a SLA regarding response time. If the response time is longer than the defined value, the alert can be fired automatically by OSB.

Once alerted, it is up to you to decide what are the appropriate actions to take. If the problem on the endpoint persists and cannot be resolved
quickly, a possible action you might want to take is to stop the Proxy Service for not receiving requests anymore until the problem is sorted out. You can manually disable the Proxy Service via OSB console or WLST to achieve this.  This blog shows an alternative approach which is to disable Proxy Service automatically via OSB's Java API when SLA is violated due to endpoint abnormalities.

The following steps highlight how to implement the solution:

1. create a SLA rule which triggers alert if the response time is longer than the expected one or the endpoint is unreachable
2. configure alert to be sent to a designated JMS queue
3. create an "administrative" Proxy Service to listen on the JMS queue
4. make a Java callout within the "administrative" Proxy Service to disable the relevant Proxy Service once the alert is received from JMS queue.  

 The details are as follows:

Create an alert destination to send alert to a designated JMS queue.

 Create the Business Service and enable the offline endpoint URI. If the endpoint URI is unreachable, OSB changes its status to offline automatically.