Golang. CRUD with gRPC (plus REST API transcoding).
The current chapter is the continuation of Golang related series started with the previous ones:
The input for the current chapter is here. The project has gRPC framework to support CRUD operations for simple Brand entity. The protobuf message from the same scheme are used to persist Brand repository into file system. From the very beginning, project exposes same CRUD for Brand via REST API (with the help of gorilla mux). While such multi-protocol API access is useful on its own, after gRPC is added to project there is no longer a need for gorilla mux as gRPC can also expose HTTP endpoints, which allows to fully expose REST API for Brand entity with it. This is a plan for the current chapter.
Since I use protoc as a compiler to produce Go code from proto schema file, I am going to install a plugin to protoc called gRPC-Gateway. Please read through the initial page and tutorial, but diagram from there tells everything:
So, gRPC-Gateway uses same proto schema file and generates code that serves REST API calls and this is really handy: one place proto messages and proto services definitions but two protocols (gRPC and REST API) are supported in the result.
First, install gRPC-Gateway plugin locally:
Next, extend existing proto schema file with custom options to instruct new plugin how to generate reverse proxy, which is mainly adding google.api.http annotation to the service definition.
Importing that annotation (and one more) into the current proto-schema:
Important! It seems to be the only way to satisfy protoc compiler to work is to have all imported proto files locally, so need to physically add it into project, like below (see ~/google/api folder):
A slight modification in ProtoBrand message itself:
#5..#6: customize JSON name for fields and make it required.
Each CRUD operation in gRPC service in proto-schema keeps same signature, but receives extra annotation:
#4: tell gRPC-Gateway this method needs to be created in reverse-proxy and be available as HTTP call.
#5: it needs to be mapped to POST method to “api/brands” path.
#6: it needs to pass whole POST body to operation parameter (casting it along the way to ProtoBrandRepo.ProtoBrand struct.
Similar transformation for GetList operation:
#4: interesting gotcha of plugin: since method accepts one Int64Value, the endpoint’s path ends with {value} which does the trick.
Since UpdateRequest proto message has two fields (ID and Brand),
#6: the suffix of URL is sent to ID part of message by indicating {ID}.
#7: whole PUT method body is sent to the root of the message.
And, finally, the last operation:
#6: same trick as in case of Update method to pass Brand ID to delete to operation by specifying {value}.
The amended schema file can be found here. It is time to generate Go wrapper code for it:
#3: pointing gRPC-Gateway to output to the same protobuf subfolder.
Due to specifying package as ‘golang_protobuf_brand’ at the top of schema file, the generated Go files are structured like this:
brand_grpc.pb.go: gRPC service for CRUD.
brand.pb.go: structures and operation related to messages defined in schema file.
brand.pb.gw.go: new file generated by gRPC-Gateway with server and client for accessing CRUD as REST API.
As of now, the current version of main.go contains code to run two servers: REST API ‘old style’ with gorilla mux (IP port 6000) and gRPC (IP port 6001) using code from auto-generated brand_grpc.pb.go. The next step is to remove REST API ‘old style’ and use its IP port to start auto-generated server from brand.pb.gw.go.
Please read through gRPC-Gateway tutorial page for more context why my new function looks like following:
#3: start mux-server implementation from gRPC-Gateway (importing it with “github.com/grpc-ecosystem/grpc-gateway/v2/runtime”).
#4: gateway server going to reverse-proxy all incoming requests to gRPC sibling server (AppConfig.RPCPort is equal “6001”) which is run as separate go routine (see my previous chapter for details).
#8..#11: create a traditional HTTP server, passing custom Mux. This server listens to port AppConfig.Port, which is equal “6000” and previously was used by gorilla mux, and going to be decommissioned.
#14: start new HTTP server, which will be transcoded and reverse-proxy to gRPC server.
Armed with a new function, the existing main function going to look like following:
#1..#8: no changes. Load config JSON, create Brand repository and start gRPC server in goroutine.
#10..#11: call create and run gateway REST API server in another gouroutine.
#12..#21: comment REST API ‘old style’ server implemented with gorilla mux.
#22..#25: as both active servers are run in goroutines by now, need to keep main thread idle until SIGTERM to prevent program termination.
Time to decommission (not just comment out) REST API ‘old style’, because for now both communication protocols are supported by gRPC server and gRPC-Gateway generated reverse-proxy.
So, main.go become smaller as it has no more commented code, function RegisterBrandRoutes(…) and corresponding imports. As later had used generic router, and this was the only place it has been in use, file generic-router.go should also be removed.
Let’s run the updated project, see both are run on needed ports and test:
In the previous chapter, where gRPC was added, I use Postman to test gRPC:
For REST API I use VS code plugin and CRUD operations defined in brand.rest which can be used again (worth mentioning that no modification needed for this file):
Creating a new Brand via REST API (it receives ID equals to 4):
And getting it via gRPC protocol:
The final version of the project is here.
gRPC-Gateway is one of plugin to extend core functionality on gRPC. It helps in the situation when different protocols (legacy clients etc.) need to be still supported and the developer does not want to maintain too much duplication code and considers proto schema file as a single source of truth. gRPC ecosystem offers a wide range of plugins that I am planning to explore.
