Thought of sharing a relatively simple gist of Retry akka pattern I came up recently. It could be useful in case Client actor sends request to another Service provider actor and request may have to be retried due to different reasons (network failure in distributed environment, service down or other...). To increase resilience Client attempts to re-try operation until either response is received or after number of retries exceeded Client simply gives up.
The pattern is quite generic and I thought it would be cool to have a pluggable re-tries for akka actors.
The implementation assumes following:
1) Service provider actor suppose to respond to a Client actor. (it is not just fire and forget scenario)
2) Client attempts to send multiple requests: Each request is given an individual Timeout and requests are retried after Retry interval
3) Each request completes as Time out (service did not respond) or actual response message from Service provider. Time outs are simply logged. While response propagated to an original sender actor and Retry proxy is stopped.
4) In case of total failure - after retries exceeded exception sent to an original sender actor.
If you need to implement retries here is how you could do that.
Example 1. Existing code using tell:
worker ! someWorkCan be modified to include re-tries as:
context.actorOf(ReTry.props(tries = 2, retryTimeOut = 1000 millis, retryInterval = 100 millis, worker)) ! SomeWorkUsually response will be listened in receive loop
Exmaple 2. Or using akka ask pattern:
val futureResult = worker ? someWorkCould be modified to include re-tries as:
val futureResult = context.actorOf(ReTry.props(tries = 10, retryTimeOut = 1000 millis, retryInterval = 100 millis, worker)) ? SomeWork
Above will be re-tried 10 times with individual time out of 1 sec per try and with 100 milliseconds interval between tries.
And several test cases here...