Load balancing is the process of distributing network traffic between multiple servers,
used to improve the performance and reliability of websites, applications, databases and other services.
Using a centralized load balancer is the most traditional approach for this, but client-side load balancing still has some advantages and is also quite common.
In this tutorial, we are going to look at how to implement client-side load balancing in gRPC Java client.
1. Multiple server instances
We will start by creating multiple server instances serving the same gRPC service, which we will use to demonstrate load balancing.
First we define simple service in a proto file:
Then we implement this service so that it returns the server name passed to it in the constructor and the message received:
To simplify the example, we will start all servers from one main method in separate threads:
Running the main method, we will see that the servers are up and ready to receive requests:
The two main components responsible for client-side request distribution are the name resolver and the load balancing policy.
2.1. Name Resolver
Before starting interaction with the server, the client needs to obtain all its IP addresses using a name resolver.
By default, gRPC uses DNS as its name-system, i.e. each A record of the DNS entry will be used as a possible endpoint.
But in case we want to use a service registry, such as Eureka or Consul, or specify several addresses ourselves, we will need to implement a custom name resolver.
In this example, we create a simple name resolver that will accept multiple addresses as an argument:
2.2. Load Balancing
In the case of the load balancing policy, we do not need to implement anything ourselves,
we can choose one of the policies presented in the library, e.g. grpclb, pick_first or round_robin.
Although the first two policies do not suit us, a Round Robin will work just fine.
Let’s create an instance of the name resolver with the addresses of previously started servers
and use it along with the chosen load balancing policy to create a client connection channel:
Finally, we use the channel to create a stub and make several calls, the result of which we output to the console:
We can see that responses are coming back from different servers:
We looked at how to enable client-side load balancing using our custom name resolver and round-robin algorithm.
Although this does not require writing a lot of code, it may take some time to figure out how the different parts fit together.