The scenario is:
So currently the approach is the block using java Futures and return the results when they come in. My concern is this is quite thread intensive and I'm worried we will run out of resources on the server. I was thinking of going async in server A by using
DeferredResult but since there is a thread for each request to server B, that doesn't seem to carry much benefit.
Alternatively, we could do
This has the advantage of being much much simpler in code (no complicated futures work) at the cost of being more complicated architecturally. It also introduces more layers (memcache and JMS) for failure. It also has the advantage of being much less resource intensive.
This is a high use system. Calls to system B can take anywhere from 1 to several seconds.
I think your original plan sounds better than the 'alternative' you described. KISS. Your current approach is very simple.
Your analysis that its 'thread intensive' is sort of right, but maybe not totally. And I think you may have mis-analyzed where they thread intensive part is. It has nothing todo with your calls from server A to server B. It's due to how your server A is servicing the web service calls. If your web service engine in server A is written to specially handle DeferredResult objects (I doubt this but you can check) - then your plan to use DeferredResult would work well. But most likely, your webservice just will force a GET call on the result of the websevice call, and will block the webservice call handling thread while the work completes whereever it comes from.
You did hint at the possibility of another strategy. Depending on the nature of your calls from ServerA to ServerB, perhaps those could be sped up or cached. Using something like redis/memchached (or just an in memory cache inside your ServerA) MIGHT be an effective way to make your overall throughput much faster, but that depends ENTIRELY on the cachability of calls from ServerA to ServerB (which you said nothing about). If you want to experiement with this caching approach, use a HashMap, and some abstraction to count hits/misses, to see how effective the approach is, before trying to scale up to something more complex like memcached.
Also, if you make MULTIPLE calls from ServiceA to ServiceB, and they are independent, you can make them using ASYNC calls, so that they all happen at the same time (for the cache misses if you try caching).