How to mock OAuth 2.0 in Go

As you might know seriesci is a GitHub application. GitHub uses the standard OAuth 2.0 flow to obtain an authorization code and then exchange it for a token. Our users depend on GitHub to be able to login and also to be able to make requests to their API. This is all great in production but makes development on our own machines harder. Especially problematic is the OAuth callback as GitHub must have access to your server. During development our servers run on localhost and GitHub cannot access them. At the beginning we used ngrok and serveo to open tunnels to our machines. We were assigned random URLs and GitHub could access those. The callback request and also webhooks were then forwarded to our servers running on localhost. However this whole setup is a bit messy, can be expensive and also unreliable. We wanted something easier to we looked at the whole flow very carefully.

At first we will explain how the standard OAuth 2.0 sequence works with GitHub. Afterwards we will describe our own solution to get rid of this dependency and also local tunnels. So let us have a look at the standard implementation.

1. User visits /login
2. In our case the client renders the login template
3. User clicks on Login with GitHub and makes a GET request to /authorize

4. On our server we do a few things

   • create some random state
   • store the state in a cookie
   • generate the auth URL for GitHub
   • redirect to the generated URL
5. The redirect triggers a GET request to github.com/login/oauth/authorize?...
6. GitHub generates some random code
7. GitHub then redirects and triggers a GET request to /callback

8. Back on our server we do a few more things

   • compare state from GitHub with our own state from cookie
   • extract code from query parameters
   • call the exchange method
9. Our server does a POST request to github.com/login/oauth/access_token
10. GitHub validates the code and responds with a real token
11. We put this token into a cookie so only you have access
12. We direct the user to /
13. Our client side web application starts
14. The web app makes a request to our own API route /api/repos

15. On the server we do a few things

    • extract the token from the cookie
    • create an authenticated client for GitHub
16. Use this client to make requests to the GitHub API, e.g. list all repositories
17. GitHub returns a list of repositories to our server
18. Our server enhances and returns the list to the user
19. Our client web app rerenders and shows you your repositories

Phew! That is a lot to grasp. In pure text it is a bit hard to understand so here is an image.

    
User ->> User: GET /login
activate User
note left of User
    - render login template
end note
User ->> -App: GET /authorize
activate App 
note left of App
    - create new state
    - store state in cookie
    - call AuthCodeURL()
    - redirect to generated URL
end note
App ->> -GitHub: GET github.com/login/oauth/authorize\n?client_id=...\n&response_type=code\n&scope=...\n&state=...
activate GitHub 
note right of GitHub
    - check client id
    - generate code
end note
GitHub ->> -App: GET /callback\n?code=...\n&state=...
activate App 
note left of App 
    - compare state with cookie
    - use code to request token
    - code will expire after 10 mins
    - call Exchange()
end note
App ->> -GitHub: POST github.com/login/oauth/access_token
activate GitHub
note right of GitHub: - generate token
GitHub ->> -App: response with:\naccess_token=...&token_type=bearer
activate App 
note left of App: - store token in cookie
App ->> -User: redirect to /
activate User
note left of User
    - load index.html
    - download JavaScript
    - start web app
end note
User ->> -App: GET /api/repos
activate App
note left of App 
    - get token from cookie
    - create authenticated client
    - use client for GitHub API
end note
App ->> -GitHub: client.Repositories.List()
activate GitHub 
note right of GitHub 
    - check token
end note
GitHub ->> -App: List of Repositories
App ->> User: List of Repositories

Our goal is to get rid of the right lane and eliminate the GitHub dependency during development. Generally speaking we have to mock OAuth and also mock the API requests. We are a heavy Go user so we will show you how to do it in Go but it should be the same for other languages. The important packages are github.com/golang/oauth2 and github.com/google/go-github.

oauth2 is responsible for generating a URL to OAuth 2.0 provider's consent page that asks for permissions for the required scopes. It also exchanges the code for a real token. Using this package is pretty straightforward. You just have to create a new instance of oauth2.Config.

import (
    "golang.org/x/oauth2"
    oauth2GitHub "golang.org/x/oauth2/github"
)

oauthConfig := &oauth2.Config{
    ClientID:     "your own client id here",
    ClientSecret: "your own client secret here",
    Endpoint:     oauth2GitHub.Endpoint,
    RedirectURL:  "",
    Scopes:       []string{"user:email"},
}

Our whole backend application roughly follows Mat Ryer's style How I write Go HTTP services after seven years. We attach the *oauth2.Config instance to our main application struct. That way we can access it within our handlers.

Mock the /login/oauth/authorize request

Here is our source code for the first request to GitHub. As you can see we are doing exactly what is described in the sequence diagram.

1. Create a new random state
2. Store the state in a cookie
3. Call AuthCodeURL() method to generate the URL
4. Redirect user to URL

// Authorize handles requests to /authorize.
func (app *Application) Authorize(w http.ResponseWriter, r *http.Request) error {
	state := uuid.NewV4().String()

	session, err := app.SessionStore.Get(r, sessionName)
	if err != nil {
		return fmt.Errorf("session get: %w", err)
	}
	session.Values["state"] = state
	if err := session.Save(r, w); err != nil {
		return fmt.Errorf("session save: %w", err)
	}

    // mock the next call to return a URL pointing to our own server
	url := app.OAuthConfig.AuthCodeURL(state)
	http.Redirect(w, r, url, http.StatusTemporaryRedirect)
	return nil
}

Since we do not want to change this behavior during development on localhost we have to mock the AuthCodeURL() method. We do not redirect the user to GitHub but back to our own server.

type OAuth2Mock struct{}

// AuthCodeURL redirects to our own server.
// A handler which is only available in development handles the request.
func (o *OAuth2Mock) AuthCodeURL(state string, opts ...oauth2.AuthCodeOption) string {
	u := url.URL{
		Scheme: "http",
		Host:   "localhost",
		Path:   "login/oauth/authorize",
	}

	v := url.Values{}
	v.Set("state", state)

	u.RawQuery = v.Encode()
	return u.String()
}

Then we use a dummy handler to return some code and redirect to our /callback handler. It acts as a substitution for github.com/login/oauth/authorize. Make sure this handler is only available during development.

// DevOAuthAuthorize handles requests to /login/oauth/authorize.
func (app *Application) DevOAuthAuthorize(w http.ResponseWriter, r *http.Request) error {
	state := r.FormValue("state")

	u, err := url.Parse("http://localhost/callback")
	if err != nil {
		return err
	}

	v := url.Values{}
	v.Set("code", "code")
	v.Set("state", state)
	u.RawQuery = v.Encode()

	http.Redirect(w, r, u.String(), http.StatusTemporaryRedirect)

	return nil
}

Well done, we have already mocked the first round trip to GitHub.

Mock the /access_token request

Back in our application inside the /callback handler everything still works.

// Callback handles requests to /callback.
func (app *Application) Callback(w http.ResponseWriter, r *http.Request) error {
	session, err := app.SessionStore.Get(r, sessionName)
	if err != nil {
		return fmt.Errorf("session get: %w", err)
	}

	if r.FormValue("state") != session.Values["state"] {
		return fmt.Errorf("invalid state: %s", r.FormValue("state"))
	}

    // ++ we have to mock this line ++
	token, err := app.OAuthConfig.Exchange(context.Background(), r.FormValue("code"))
	if err != nil {
		return fmt.Errorf("oauth exchange: %w", err)
	}

	if !token.Valid() {
		return errors.New("invalid token")
	}

	// save username to session
	session.Values["token"] = token

	if err := session.Save(r, w); err != nil {
		return fmt.Errorf("session save: %w", err)
	}

	http.Redirect(w, r, "/", http.StatusSeeOther)
	return nil
}

Compare the returned state with the state from the cookie. Then call Exchange() to exchange the code for a real token. The method simply does a POST request to github.com/login/oauth/access_token. github.com/golang/oauth2/blob/.../internal/token.go#L169-L172. Our mock returns a dummy token. We must set AccessToken and Expiry as token.Valid() checks these two fields.

// Exchange takes the code and returns a real token.
func (o *OAuth2Mock) Exchange(ctx context.Context, code string, opts ...oauth2.AuthCodeOption) (*oauth2.Token, error) {
	return &oauth2.Token{
		AccessToken: "AccessToken",
		Expiry:      time.Now().Add(1 * time.Hour),
	}, nil
}

Now that we have a valid token we continue in our sequence diagram. We store the token in a cookie and redirect the user to our index page. We managed to mock the second request to GitHub. The OAuth 2.0 part is done here. All OAuth related requests are handled by us and none of them reaches the GitHub servers. The final step is all about mocking real data request as an authenticated user.

Mock requests to the OAuth 2.0 provider

We have our dummy token and would like to do an authenticated request to GitHub. Although our token is valid, GitHub complains that it is not a real token and we do not have access to their data. So we have to pretend that we are doing an actual request to GitHub and mock this call. The github.NewClient method returns a struct of type *github.Client. We cannot mock this struct so we have to create our own interface that has the same methods. This is common in Go and follows the rule "Accept interfaces, return structs".

Here is a starting point from GitHub issues https://github.com/google/go-github/issues/113. First of all we have to create our GitHub interface.

// RepositoriesService handles communication with the repository related methods
// of the GitHub API.
// https://godoc.org/github.com/google/go-github/github#RepositoriesService
type RepositoriesService interface {
	Get(context.Context, string, string) (*github.Repository, *github.Response, error)
	// ...
}

// UsersService handles communication with the user related methods
// of the GitHub API.
// https://godoc.org/github.com/google/go-github/github#UsersService
type UsersService interface {
	Get(context.Context, string) (*github.User, *github.Response, error)
	// ...
}

// GitHubClient manages communication with the GitHub API.
// https://github.com/google/go-github/issues/113
type GitHubClient struct {
	Repositories RepositoriesService
	Users        UsersService
}

// GitHubInterface defines all necessary methods.
// https://godoc.org/github.com/google/go-github/github#NewClient
type GitHubInterface interface {
	NewClient(httpClient *http.Client) GitHubClient
}

Now we need a real type that implements the above mentioned interface GitHubInterface. For the production environment where we do want to communicate with GitHub we pick the interfaces from the client and attach them to our own instance.

// GitHubCreator implements GitHubInterface.
type GitHubCreator struct{}

// NewClient returns a new GitHubInterface instance.
func (g *GitHubCreator) NewClient(httpClient *http.Client) GitHubClient {
	client := github.NewClient(httpClient)
	return GitHubClient{
		Repositories: client.Repositories,
		Users:        client.Users,
	}
}

During development we simply return dummy values without making any network requests.

// RepositoriesMock mocks RepositoriesService
type RepositoriesMock struct {
	RepositoriesService
}

// Get returns a repository.
func (r *RepositoriesMock) Get(context.Context, string, string) (*github.Repository, *github.Response, error) {
	return &github.Repository{
		ID:              github.Int64(185409993),
		Name:            github.String("wayne"),
		Description:     github.String("some description"),
		Language:        github.String("JavaScript"),
		StargazersCount: github.Int(3141),
		HTMLURL:         github.String("https://www.foo.com"),
		FullName:        github.String("john/wayne"),
	}, nil, nil
}

// UsersMock mocks UsersService
type UsersMock struct{
	UsersService
}

// Get returns a user.
func (u *UsersMock) Get(context.Context, string) (*github.User, *github.Response, error) {
	return &github.User{
		Login: github.String("john"),
	}, nil, nil
}

// GitHubMock implements GitHubInterface.
type GitHubMock struct{}

// NewClient something
func (g *GitHubMock) NewClient(httpClient *http.Client) GitHubClient {
	return GitHubClient{
		Repositories: &RepositoriesMock{},
		Users:        &UsersMock{},
	}
}

Create an authenticated GitHub client

So where does the *http.Client for NewClient() come from? The Client(ctx context.Context, t *oauth2.Token) *http.Client method from the oauth2 package takes a context and a token and returns an HTTP client. "The go-github library does not directly handle authentication. Instead, when creating a new client, pass an http.Client that can handle authentication for you" https://godoc.org/github.com/google/go-github/github#hdr-Authentication. Use the token from our previous Exchange() method as input.

// ...
// use token from Exchange() method
httpClient := app.OAuthConfig.Client(context.Background(), token)
client := app.GitHub.NewClient(httpClient)

// continue with an authenticated client
repo, res, err := client.Repositories.GetByID(context.Background(), 1)
if err != nil {
	return nil, err
}
// ...

Voilà! We're done with the second part and are able to make and mock authenticated requests to the GitHub API.

OAuth interface summary

So our final OAuth 2.0 interface looks like this.

type OAuth2ConfigInterface interface {
	AuthCodeURL(state string, opts ...oauth2.AuthCodeOption) string
	Exchange(ctx context.Context, code string, opts ...oauth2.AuthCodeOption) (*oauth2.Token, error)
    Client(ctx context.Context, t *oauth2.Token) *http.Client
}

Here is the complete mock again that implements this interface.

type OAuth2Mock struct{}

// AuthCodeURL redirects to our own server.
func (o *OAuth2Mock) AuthCodeURL(state string, opts ...oauth2.AuthCodeOption) string {
	u := url.URL{
		Scheme: "http",
		Host:   "localhost",
		Path:   "login/oauth/authorize",
	}

	v := url.Values{}
	v.Set("state", state)

	u.RawQuery = v.Encode()
	return u.String()
}

// Exchange takes the code and returns a real token.
func (o *OAuth2Mock) Exchange(ctx context.Context, code string, opts ...oauth2.AuthCodeOption) (*oauth2.Token, error) {
	return &oauth2.Token{
		AccessToken: "AccessToken",
		Expiry:      time.Now().Add(1 * time.Hour),
	}, nil
}

// Client returns a new http.Client.
func (o *OAuth2Mock) Client(ctx context.Context, t *oauth2.Token) *http.Client {
	return &http.Client{}
}

Here is our simplified application code. We define the main Application struct and attach our http handlers to it.

type Application struct {
	OAuthConfig  OAuth2ConfigInterface
	GitHub       GitHubInterface
}

r.Methods("GET").Path("/callback").Handler(app.Callback)
r.Methods("GET").Path("/authorize").Handler(app.Login)

// route only needed during development
if config.Development {
	r.Methods("GET").Path("/login/oauth/authorize").Handler(app.DevOAuthAuthorize)
}

Sequence diagram with mocks in place

Here is the same sequence diagram again. This time we only have two lanes as GitHub is completely mocked away. We achieved our goal, are able to login via OAuth and make API requests without changing the business logic of our code. This provides us with the flexibility we need.

    
User ->> User: GET /login
activate User
note left of User
    - render login template
end note
User ->> -App: GET /authorize
activate App 
note left of App
    - create new state
    - store state in cookie
    - call AuthCodeURL()
    - redirect to generated URL
end note
# App ->> -GitHub: GET github.com/login/oauth/authorize\n?client_id=...\n&response_type=code\n&scope=...\n&state=...
# deactivate App
App ->> App: GET localhost:8080/.../...
# deactivate App
note right of App
    - generate code
end note
# deactivate App
App ->> App: GET /callback\n?code=...\n&state=...
# activate App 
note left of App 
    - compare state with cookie
    - use code to request token
    - code will expire after 10 mins
    - call Exchange()
end note
App ->> App: POST localhost:8080/login/oauth/access_token
# activate GitHub
note right of App: - generate token
App ->> App: response with:\naccess_token=...&token_type=bearer
# activate App 
note left of App: - store token in cookie
App ->> -User: redirect to /
activate User
note left of User
    - load index.html
    - download JavaScript
    - start web app
end note
User ->> -App: GET /api/repos
activate App
note left of App 
    - get token from cookie
    - create authenticated client
    - use client for GitHub API
end note
App ->> App: client.Repositories.List()
# activate GitHub 
note right of App
    - check token
end note
App ->> App: List of Repositories
App ->> User: List of Repositories
    

Conclusion

Using all those interfaces and mocks might seem a bit complicated. However having the mock in place during development is great. We are able to write code offline on a train or even on a plane. We do not have to set up complicated tools to open a tunnel from our development machine to the outside world. You usually have to do this in order to receive the real OAuth 2.0 callback. The whole architecture also makes testing very easy. Within our tests we can use the existing interfaces and provide dummy implementations. This way we can even test all error paths that are often untested as the OAuth provider usually works. In addition the setup has a superb user experience. Without any network requests response times are super fast. Everything is available as soon as you click on any link.

We hope you enjoyed this deep dive into Go interfaces and dependency injection. We've been using this setup for about half year now and are happy we took the time to add all those abstractions.