Go gRPC servers with Bazel in one command

Follow @popovicu94

In this article we’ll look at how to build gRPC servers in Go using Bazel. Bazel is a multi-language build system built and open sourced by Google. The workflow described below is heavily inspired by Google’s best build practices (at the moment of writing, I am a Google software engineer).

Table of contents

Background

Along with this article, I am silently releasing another article on building and deploying HTTP servers with Go and Bazel, using no tooling or external dependencies. The concept here is the same, we’ll simply use Bazel for which you do not need to install any tooling whatsoever on your machine. You also do not need Go tooling installed on your machine to fully reproduce the guide below. gRPC is also not necessary, Bazel will figure everything out.

Please consider reading this article for an in-depth introduction to Bazel. It describes the build methodology as used by Google and that keeps everything “hermetic”, meaning it doesn’t depend on anything being installed on your machine, and it doesn’t “pollute” your system as well.

Note: We won’t be discussing why you should use gRPC in this text.

GitHub

You can go ahead and clone the repository from GitHub in order to follow this exercise more easily. I will be referring to the code from that repo below.

Bazel modules

Long story short, for the purposes of this exercise, Bazel module functionality is the latest and most modern way to work with external dependencies in Bazel. More details can be found here, but to those familiar with the traditional WORKSPACE-based dependency management system — this is a much better and concise way to do it, and takes care of your transitive dependencies much more easily.

WORKSPACE file

The WORKSPACE file can be empty. It will serve only as the “checkpoint” for Bazel to know where the repository begins.

.bazelrc file

We’ll put 2 lines in this file to make sure the module functionality is always used, even if not explicitly listed on the command line:

build --enable_bzlmod
query --enable_bzlmod

MODULE.bazel file

In this file we will declare dependencies on 2 things:

1. Go rules and toolchain
2. Protocol Buffer rules (including the proto compiler)

The first one should be obvious — we want to use Go in this project, and we’ll even pin the exact version of the toolchain that we want to use. Bazel will download the toolchain as a part of the build flow, so the only thing for us here is to declare what we want to use.

Protocol Buffer rules will give us a way to build protos using Bazel, and that will include pulling the proto compiler (it will be built from source as a part of your build flow; it is a bit of a heavy build, but nothing too bad).

bazel_dep(name = "rules_go", version = "0.46.0")
bazel_dep(name = "rules_proto", version = "6.0.0-rc3")

go_sdk = use_extension("@rules_go//go:extensions.bzl", "go_sdk")
go_sdk.download(version = "1.22.0")
go_sdk.host()

Protocol Buffer definition

The BUILD file should be straightforward:

load("@rules_proto//proto:defs.bzl", "proto_library")
load("@rules_go//proto:def.bzl", "go_grpc_library")

proto_library(
    name = "foo_proto",
    srcs = ["foo.proto"],
)

go_grpc_library(
    name = "foo_go_proto",
    proto = ":foo_proto",
    importpath = "popovicu/foo",
    visibility = [
        "//visibility:public",
    ],
)

And the protocol can be super simple (we’ll create an RPC to add up 2 numbers, as a part of the DoFoo RPC):

syntax = "proto3";

package popovicu_foo;

service EchoService {
  rpc DoFoo(FooRequest) returns (FooResponse) {}
}

message FooRequest {
  int32 a = 1;
  int32 b = 2;
}

message FooResponse {
  int32 c = 1;
}

The ugly part — gRPC library

This is probably the worst part, honestly. We’ll need a few things from the gRPC library. Now, when you’re defining your go_grpc_library, the Go-based Protocol Buffer compiler is downloaded, built and invoked, which includes building the gRPC library.

If you explicitly define your own dependency on the gRPC library, Bazel gets thrown off and starts reporting some strange cyclical dependency errors. I guess the module system is not perfect? The way around this (at least from what I could figure out) is to use whatever is already used under the hood.

To make sure the solution below doesn’t look like magic, this is how I got to it. I ran the following:

bazel query "deps(//proto:foo_go_proto)"

This gave output which looks something like this:

//proto:foo.proto
//proto:foo_go_proto
//proto:foo_proto
@bazel_tools//src/conditions:host_windows
@bazel_tools//src/conditions:host_windows_arm64_constraint
@bazel_tools//src/conditions:host_windows_x64_constraint
@bazel_tools//src/conditions:remote
@bazel_tools//third_party/def_parser:def_parser
@bazel_tools//third_party/def_parser:def_parser.cc
@bazel_tools//third_party/def_parser:def_parser.h
@bazel_tools//third_party/def_parser:def_parser_lib
@bazel_tools//third_party/def_parser:def_parser_main.cc
@bazel_tools//tools/allowlists/function_transition_allowlist:function_transition_allowlist
@bazel_tools//tools/build_defs/build_info:cc_build_info
@bazel_tools//tools/build_defs/build_info/templates:non_volatile_file.h.template
@bazel_tools//tools/build_defs/build_info/templates:redacted_file.h.template
@bazel_tools//tools/build_defs/build_info/templates:volatile_file.h.template
@bazel_tools//tools/build_defs/cc/whitelists/parse_headers_and_layering_check:disabling_parse_headers_and_layering_check_allowed
@bazel_tools//tools/cpp:build_interface_so
@bazel_tools//tools/cpp:compiler
@bazel_tools//tools/cpp:current_cc_toolchain
@bazel_tools//tools/cpp:empty_lib
@bazel_tools//tools/cpp:interface_library_builder
@bazel_tools//tools/cpp:link_dynamic_library
@bazel_tools//tools/cpp:link_dynamic_library.sh
@bazel_tools//tools/cpp:link_extra_lib
@bazel_tools//tools/cpp:link_extra_libs
@bazel_tools//tools/cpp:malloc
@bazel_tools//tools/cpp:optional_current_cc_toolchain
@bazel_tools//tools/cpp:toolchain
@bazel_tools//tools/cpp:toolchain_type
@bazel_tools//tools/def_parser:def_parser
@bazel_tools//tools/def_parser:def_parser.exe
@bazel_tools//tools/def_parser:def_parser_windows
@bazel_tools//tools/def_parser:no_op.bat
@bazel_tools//tools/genrule:genrule-setup.sh
@bazel_tools//tools/objc:host_xcodes
@bazel_tools//tools/osx:current_xcode_config
@bazel_tools//tools/proto:protoc
@@bazel_tools~cc_configure_extension~local_config_cc//:builtin_include_directory_paths
@@bazel_tools~cc_configure_extension~local_config_cc//:cc-compiler-armeabi-v7a
@@bazel_tools~cc_configure_extension~local_config_cc//:cc-compiler-darwin_arm64
@@bazel_tools~cc_configure_extension~local_config_cc//:cc_wrapper
@@bazel_tools~cc_configure_extension~local_config_cc//:cc_wrapper.sh
@@bazel_tools~cc_configure_extension~local_config_cc//:compiler_deps
@@bazel_tools~cc_configure_extension~local_config_cc//:empty
@@bazel_tools~cc_configure_extension~local_config_cc//:local
@@bazel_tools~cc_configure_extension~local_config_cc//:stub_armeabi-v7a
@@bazel_tools~cc_configure_extension~local_config_cc//:toolchain
@@gazelle~~go_deps~com_github_golang_protobuf//proto:buffer.go
@@gazelle~~go_deps~com_github_golang_protobuf//proto:defaults.go
@@gazelle~~go_deps~com_github_golang_protobuf//proto:deprecated.go
...

It’s a pretty big output. However, the part that really matters here is the following:

@@gazelle~~go_deps~org_golang_google_grpc//:backoff.go
@@gazelle~~go_deps~org_golang_google_grpc//:balancer_conn_wrappers.go
@@gazelle~~go_deps~org_golang_google_grpc//:call.go
@@gazelle~~go_deps~org_golang_google_grpc//:clientconn.go
@@gazelle~~go_deps~org_golang_google_grpc//:codec.go
@@gazelle~~go_deps~org_golang_google_grpc//:dialoptions.go
@@gazelle~~go_deps~org_golang_google_grpc//:doc.go
@@gazelle~~go_deps~org_golang_google_grpc//:go_default_library
@@gazelle~~go_deps~org_golang_google_grpc//:grpc
@@gazelle~~go_deps~org_golang_google_grpc//:interceptor.go
@@gazelle~~go_deps~org_golang_google_grpc//:picker_wrapper.go
@@gazelle~~go_deps~org_golang_google_grpc//:pickfirst.go
@@gazelle~~go_deps~org_golang_google_grpc//:preloader.go
@@gazelle~~go_deps~org_golang_google_grpc//:resolver_conn_wrapper.go
@@gazelle~~go_deps~org_golang_google_grpc//:rpc_util.go
@@gazelle~~go_deps~org_golang_google_grpc//:server.go
@@gazelle~~go_deps~org_golang_google_grpc//:service_config.go
@@gazelle~~go_deps~org_golang_google_grpc//:stream.go
@@gazelle~~go_deps~org_golang_google_grpc//:trace.go
@@gazelle~~go_deps~org_golang_google_grpc//:version.go
@@gazelle~~go_deps~org_golang_google_grpc//attributes:attributes
@@gazelle~~go_deps~org_golang_google_grpc//attributes:attributes.go
...

OK, so Bazel injected this virtual workspace under the name of @@gazelle~~go_deps~org_golang_google_grpc. For those who are not familiar, Gazelle is a tool to “Bazefily” existing projects. If you’re working on, for example, a Go project, without Bazel and with standard tooling only, you can invoke Gazelle to generate your BUILD files and so on to make it work with Bazel. This comes in handy when you’re building a project with Bazel and you’d like to depend on other projects as if they were a part of your Bazel monorepo. Gazelle would be invoked on the fly to “Bazelify” those projects, giving you the appropriate *_library targets that you can depend on, and so on.

In order to contain this mess, we can create a BUILD file under a package we can call //remote, and simply alias those targets we’re interested in. That way, if we use those targets repeatedly later, we have a central file to update, rather than a bunch of them. This BUILD file looks like this:

alias(
    name = "grpc_go",
    actual = "@@gazelle~~go_deps~org_golang_google_grpc//:go_default_library",
    visibility = [
        "//visibility:public",
    ],
)

alias(
    name = "grpc_credentials_insecure",
    actual = "@@gazelle~~go_deps~org_golang_google_grpc//credentials/insecure",
    visibility = [
        "//visibility:public",
    ],
)

We’ll see later how we use those targets.

The proper solution?

I think the right way to overcome this is to declare your own dependency on the core gRPC library explicitly, and then tell Gazelle to use your dependency instead of figuring out one on its own. There is some documentation here. Go proto rules will use Gazelle under the hood, that’s why I’m confident this would work, but I wanted to keep this exercise simple.

The server

This is all straightforward now. The BUILD file is like this:

load("@rules_go//go:def.bzl", "go_binary")

go_binary(
    name = "server",
    srcs = ["server.go"],
    deps = [
        "//proto:foo_go_proto",
        "//remote:grpc_go",
    ],
)

And the Go code is:

package main

import (
	"context"
	"flag"
	"fmt"
	"log"
	"net"
	"popovicu/foo"

	grpc "google.golang.org/grpc"
)

type sampleServer struct{}

func (s sampleServer) DoFoo(ctx context.Context, req *foo.FooRequest) (*foo.FooResponse, error) {
	return &foo.FooResponse{
		C: req.A + req.B,
	}, nil
}

func main() {
	flag.Parse()
	lis, err := net.Listen("tcp", fmt.Sprintf("localhost:9876"))

	if err != nil {
		log.Fatalf("failed to listen: %v", err)
	}

	var opts []grpc.ServerOption
	grpcServer := grpc.NewServer(opts...)
	foo.RegisterEchoServiceServer(grpcServer, sampleServer{})

	fmt.Println("Starting")
	grpcServer.Serve(lis)

	fmt.Println("Hello!")
}

The DoFoo method matches the signature of the DoFoo RPC from the Protocol Buffer file, and this is really all it takes to implement the functionality of the service. The implementation is simple, it just adds up 2 numbers.

The core gRPC library was needed for the grpc.NewServer() call, that’s why we went through all the trouble to bring it in.

The client

Similarly, the BUILD file is like this:

load("@rules_go//go:def.bzl", "go_binary")

go_binary(
    name = "client",
    srcs = ["client.go"],
    deps = [
        "//proto:foo_go_proto",
        "//remote:grpc_go",
	    "//remote:grpc_credentials_insecure",
    ],
)

And the Go code is the following:

package main

import (
	"context"
	"fmt"
	"log"
	"popovicu/foo"

	grpc "google.golang.org/grpc"
	insecure "google.golang.org/grpc/credentials/insecure"
)

func main() {
	conn, err := grpc.Dial("localhost:9876", grpc.WithTransportCredentials(insecure.NewCredentials()))

	if err != nil {
		log.Fatalf("failed to listen: %v", err)
	}

	defer conn.Close()

	client := foo.NewEchoServiceClient(conn)

	request := &foo.FooRequest{
		A: 10,
		B: 20,
	}
	resp, err := client.DoFoo(context.Background(), request)

	if err != nil {
		log.Fatalf("failed to listen: %v", err)
	}

	fmt.Printf("Response: %v\n", resp)
}

The client type is autogenerated, and you simply need to create a connection object to pass to the factory. We’re not using any authentication here for simplicity (insecure). Once that is set up, you just need to create the request object and send it over to the gRPC server: the response will be a proto.

Running it all

And that’s pretty much it! Let’s run the server first:

bazel run //server

As promised, one command! This will pull your Go toolchain, proto compiler, build everything, then build your server with the generated proto code and run it. Similarly, one command runs the client:

bazel run //client

The output is:

Response: c:30

Which is great! Our client sends a request with 10 and 20, the result is 30.

Conclusion

I think the part that could trip people is getting to the core gRPC library without the confusing cyclical dependency errors. What’s presented here should be OK, and I will spend some time in the future to try out what I think would be the ultimate and clean way to do it (using Gazelle overrides).

That said, it’s still a pretty minimal amount of code to write in order to build powerful servers with gRPC. I hope this end to end solution is helpful for your project!

Please consider following on Twitter/X and LinkedIn to stay updated.