Below are a few examples of routing topologies.
Of course you can design routing topologies to suit your application using elements from any of the models below.
AMQP can be used to create a remote procedure call system, where remote method calls are sent as AMQP messages. These messages are routed to appropriate “request queues” on which the handlers are listening. The requester needs no knowledge of the number or the location of these handlers, so the requester is decoupled from the handlers, and the handlers can be reconfigured with no changes to the requesting service.
A disadvantage is that the requester can be kept waiting if the handler does not put a result into the result queue, perhaps because it has hung or crashed, etc. The requester would need appropriate timeouts, etc.
Availability and load sharing can be achieved simply by adding more handlers, as decribed in the Task Distribution pattern below.
Suggested Settings:
We can modify the RPC model such that instead of a job going to one particular worker, it instead goes to many workers that each have a partial data set or particular responsibility for part of a task.
When a job is submitted it is received by each one of a number of shards. Each shard computes a result, which is put into a result queue. A reducer service can combine these results.
If necessary, the combined result might be put into another queue to be returned to the original requester.
Suggested Settings:
With this configuration the developer must choose how to deal with missing workers.
If the problem involves intensive tasks we can spread them over a cluster of machines. This allows machines to be added to the cluster elastically to deal with load spikes.
Often we don’t need to send results back to the original producer: it may be sufficient to simply record them to a database or filesystem to be retrieved at a later time.
Suggesting settings would be similar to the map-reduce pattern described above.
The Publish-Subscribe (Pub-Sub) pattern can be used to loosely couple arbitrary services. Perhaps we have a number of services that generate events, and a variety of consumers that need to receive some subset of those events to stay in sync or to take some other action.
The pub-sub pattern is useful because the consumer is responsible for registering to receive the events it is interested in. The publisher is responsible just for making events available.
Suggested settings: