The detour

When we started working on the new Xbus, gnatsd (the NATS server) provided no authorization system (none that we knew of): any connected client could eavesdrop all the communications, and send fake messages to mess with the bus.

So we decided the add our own encryption layer on top of NATS: each message would be encrypted with PGP, allowing exchanges privacy and verification of the sender.

After handling it by hand in the various Xbus components, we wrote natscrypto to implement this layer in the most transparent way we could.

When it started working pretty well, we discovered that gnatsd 0.9 had an exciting new feature: Multi-user and authorization.

After a small hesitation, we decided to drop all the PGP encryption layer and base Xbus security on gnatsd authorization features.

Authentication

We need our server (which embed gnatsd) to handle the accounts itself, and we do not want to have to restart gnatsd each time we add or modify and account.

It means using the configuration file to define all the accounts is not an option. We need to provide our own authentifier on top of the Xbus account storage.

Fortunately, gnatsd authentication is delegated to any type implementing the Auth interface. And since it also provides 3 different authentifiers, implementing our own was pretty straightforward… until we wanted to use TLS certificates and not a token or a password.

TLS authentication

We wanted a PKI-based authentication for the Xbus clients (we already did with the PGP based encryption layer). Since gnatsd and nats clients can use TLS, it was only natural to use TLS for the authentication too.

The problem with the Auth interface is that it originally provided no sensible way to access the underlying connection, even less TLS connection, which would be required to get the TLS certificate that was verified.

Here is the Auth interface:

1
2
3
4

// Auth is an interface for implementing authentication
type Auth interface {
    // Check if a client is authorized to connect
    Check(c ClientAuth) bool }

‘ClientAuth’ is an interface, which provides informations about the client (that it got from the ‘CONNECT’ phase of the nats protocol), and allow the authentifier to register a user and its permissions into the server.

There is its definition:

1
2
3
4
5
6

// ClientAuth is an interface for client authentication
type ClientAuth interface {
    // Get options associated with a client
    GetOpts() *clientOpts
    // Optionally map a user after auth.
    RegisterUser(*User) }

So, to enable TLS authentication, we need ClientAuth to provide directly or indirectly an access to the TLS certificate.

In the normal case (when we have access to the tls connection), we would call the ConnectionState function and check the ‘VerifiedChain’ attribute.

Since giving access to the connection itself is a little overkill (and a pain to test), we settled on simply adding a GetTLSConnectionState function to the ClientAuth interface:

1
2

// If TLS is enabled, TLS ConnectionState, nil otherwise
GetTLSConnectionState() *tls.ConnectionState

The implementation is pretty straightforward, and the pull request was merged into gnatsd.

Authorizations

Basics

NATS authorizations are pretty simple. Basicaly each user has 2 sets of permissions, Publish and Subscribe, listing on which subjects it can publish and subscribe.

For example, to allow an account to use our registration api, and all the control apis, we would give it the following permissions:

1
2

Publish: []string{"xbus.registration", "xbus.control.*"},
Subscribe: []string{"_INBOX.>"},

Simple isn’t it?

So now we can allow specific accounts to access services. This is nice, but not enough. We also need to control who sends the messages.

Control sender

A NATS message is very simple (which is a very good thing) and we have no way to know which account sent it. So we need to somehow add this information in a non-falsifiable way. Adding cryptography in the message would be tedious and costly.

So we use the permissions system, by reserving namespaces to each account. This way, we can assert that a message incoming on a given subject can only come from a specific account.

For an emitter named “emitter1”, which needs to send messages to the Xbus services, the permissions would become:

1
2

Publish: []string{"xbus.msgbox.emitter1"},
Subscribe: []string{"_INBOX.>"},

The corresponding service will subscribe to xbus.msgbox.*, and will be able to verify the sender by parsing the subject.

Secure replies

All this is great, but we still have a big security issue: reply eavesdroping and faking.

Indeed, because we often use the request pattern, most if not all the accounts have permissions to publish and subscribe to _INBOX.>. It allows a malicious account to read replies intended for other accounts, and in some case reply faster than the callee and mess the whole system.

To solve this, we use namespaced inboxes. Each account has a dedicated inbox namespace, in which it has two subjects: one for replying, one for listening to reply:

1
2

Publish: []string{"_INBOX.emitter1.out.*"},
Subscribe: []string{"_INBOX.emitter1.in.*"},

This scheme is not directly usable by the go-nats client, which generates one-shot subjects directly in the _INBOX. namespace. So we wrappednats.EncodedConn (which we use instead of nats.Conn) and completed its API with functions that gives more control on the reply subjects.

On the client side, the Request function generates reply subjects with a custom prefix that we initialize with the ‘in’ reply namespace of the account.

In the Xbus server, we always know which account we target and pass the adequate prefix to the RequestFor function.