Timeout Pattern — Resilient Microservice Design With Spring Boot
In this article, I would like to show you the very simple design pattern — Timeout Pattern — for designing resilient microservice. Timeout pattern is one of the simplest techniques. We would be discussing other techniques as well later.
Need For Resiliency:
MicroServices are distributed in nature. It has more components and moving parts. In the distributed architecture, dealing with any unexpected failure is one of the biggest challenges to solve. It could be a hardware failure, network failure etc. Ability of the system to recover from the failure and remain functional makes the system more resilient. It also avoids any cascading failures.
Why Timeout?:
We do experience intermittent application slowness once in a while for no obvious reasons. It could have happened to all of us even for applications like google.com. In microservice architecture, one service (A) depends on the other service (B), sometimes due to some network issue, Service B might not respond as expected. This slowness could affect Service A as well as A is waiting for the response from B to proceed further. As it is not uncommon issue, It is better to take this service unavailability issue into consideration while designing your microservices. So that we could have the core services working as expected even when the dependent services are NOT available.
The advantages of including the timeout would be
- Make the core services work always even when the dependent services are not available
- We do not want to wait indefinitely
- We do not want to block any threads
- To handle network related issues and make the system remain functional using some cached responses.
Sample Application:
Lets consider this simple application to explain this timeout pattern.
- We have multiple microservices as shown above
- Product service acts as product catalog and responsible for providing product information.
- We have a separate service for product reviews and ratings
- Whenever we look at the product details, product service sends the request to the rating service to get the reviews for the product.
- We have other services like account-service, order-service and payment-service etc which is not relevant to this article discussion.
- Product service is a core service without which the user can not place any order.
- Product service depends on the rating service.
This sample project code is available in GitHub here. It is a multi-module maven project.
Common Util:
As we have few services which are going to share the DTOs among them, we can have a module to share the DTOs. Lets call common-util.
The project structure is as shown below.
ReviewsDTO:
public class ReviewsDTO {
private String userFirstname;
private String userLastname;
private int rating;
private int productId;
private String comment;
// Getters & Setters
}ProductRatingDTO:
public class ProductRatingDTO {
private double avgRating;
private List<ReviewsDTO> reviews;
// Getters & Setters
}ProductDTO:
public class ProductDTO {
private int productId;
private String description;
private double price;
private ProductRatingDTO productRating;
//Getters & Setters
}Rating MicroService:
The project structure is as shown below.
RatingService:
RatingController:
@RestController
@RequestMapping("v1")
public class RatingController {
@Autowired
private RatingService ratingService;
@GetMapping("/ratings/{prodId}")
public ProductRatingDTO getRating(@PathVariable int prodId){
return this.ratingService.getRatingForProduct(prodId);
}
}Product MicroService:
The project for product service is as shown below.
RestConfig:
@Configuration
public class RestConfig {
@Bean
public RestTemplate restTemplate(){
return new RestTemplate();
}
}RatingService:
As ProductService needs to fetch information about product ratings and reviews, It needs to make a call to Rating service. Lets define an interface for that.
public interface RatingService {
ProductRatingDTO getRatings(int productId);
}
@Service
public class RatingServiceImpl implements RatingService {
@Value("${rating.service.url}")
private String ratingServiceUrl;
@Autowired
private RestTemplate restTemplate;
@Override
public ProductRatingDTO getRatings(int productId) {
String url = this.ratingServiceUrl + "/" + productId;
return this.restTemplate.getForObject(url, ProductRatingDTO.class);
}
}The above rating.service.url property is inject via application.yaml as shown below.
rating:
service:
url: http://localhost:8081/v1/ratingsProductService:
Lets define an interface for ProductService.
ProductController:
@RestController
@RequestMapping("v1")
public class ProductController {
@Autowired
private ProductService productService;
@GetMapping("/product/{id}")
public ProductDTO getProduct(@PathVariable int id){
return this.productService.getProduct(id);
}
}Now when we run these services together, If I send any request to the product service to fetch product details, I get the proper response as shown below.
//http://localhost:8080/v1/product/2
{
"productId":2,
"description":"Blood on the dance floor",
"price":12.45,
"productRating":{
"avgRating":4.5,
"reviews":[
{
"userFirstname":"slim",
"userLastname":"shady",
"rating":5,
"productId":2,
"comment":"best"
},
{
"userFirstname":"snoop",
"userLastname":"dogg",
"rating":4,
"productId":2,
"comment":"great"
}
]
}
}Lets assume that our Rating service is notorious for having some kind of performance issues or some intermittent network issues. Lets simulate that by adding hard coded sleep of 30 seconds in the RatingController class.
Now restart the rating service. Resend the same request to the product service for the id 2. Now we can see that product service is simply hanging for 30 seconds to respond as it depends on the rating service to respond so that it can build its response object. Performance/Network issues related to 1 service will cascade to other services as well.
Introducing Time Out:
As Product service depends on the Rating service, Lets introduce some timeout when we make a call to RatingService. That is — any request sent to RatingService should get the response within 3 seconds. If not, We just throw timeout exception and sends the response to the ProductService request. We are not going to wait indefinitely for the response from the RatingService.
So, I modify the RestConfig class as shown here to introduce timeout in the RestTemplate.
@Configuration
public class RestConfig {
@Bean
public RestTemplate restTemplate(RestTemplateBuilder restTemplateBuilder){
return restTemplateBuilder
.setConnectTimeout(Duration.ofSeconds(3))
.setReadTimeout(Duration.ofSeconds(3))
.build();
}
}RatingServiceImpl is also modified to handle the timeout exception. In case of timeout, we simply return the empty object.
Now when we run these services with above timeout changes, I get the response as shown below. Here the rating and reviews are empty. But that is ok as it is not critical. But the product itself is not available then we will have very bad user experience and could impact the revenue as well.
//http://localhost:8080/v1/product/2
{
"productId":2,
"description":"Blood on the dance floor",
"price":12.45,
"productRating":{
"avgRating":0.0,
"reviews":[
]
}
}Pros:
- In this approach, we do not block any threads in the product service
- Any unexpected events during network call will get timed out within 3 seconds.
- Core services are not affected because of the poor performance of the dependent services.
Cons:
Source Code:
This project source code is available here.
Summary:
Timeout Pattern is one of the patterns for designing resilient microservices. Introducing timeout solves the network related issues partially. Not fully. There are other design patterns which could handle this better along with timeout pattern. Please take a look at these articles.
- Resilient MicroService Design — Retry Pattern
- Resilient MicroService Design — Circuit Breaker Pattern
- Resilient MicroService Design — Bulkhead Pattern
- Resilient MicroService Design — Rate Limiter Pattern
Happy learning 🙂
Originally published at https://www.vinsguru.com.