Subway Rate Limit Distributor
Turnstile retrieves its rate limits configuration from a Redis database. It also listens on a "control" channel for commands ordering it to reload that configuration. There is some capability to spread this reload across a time interval, but in deployments with large numbers of Turnstile instances, this could still result in a large number of queries to a single Redis server.
Subway provides a means to mitigate this problem. Subway listens to the "control" channel on a single "master" server; when it receives a command to reload the limits, it distributes the limits configuration to a set of slave servers and forwards the reload command to the "control" channel on those slave servers. This can allow the limits to be fanned out to a large number of Redis servers, avoiding the single-bottleneck described above.
Configuring Subway
Subway requires a single command-line argument, which specifies the
configuration file. The configuration file is an INI-style file. The
basic configuration for Subway is in the [config]
section; all
values here are optional, and the recognized options are:
- channel
- Specifies the control channel used by Turnstile. This will be used for both the master and all slaves. It defaults to "control".
- limits_key
- Specifies the key in which the rate limits are stored. This will be used for both the master and all slaves. It defaults to "limits".
- reload_spread
- Normally, when a reload occurs, the limits are reloaded from the
database immediately. This configuration option allows the
declaration of a time interval; the reload will be scheduled at a
random time within the specified interval, after receipt of a
reload command. It is possible for the reload command to override
this value; see Turnstile's
setup_limits
command for more information. - shard_hint
- Subway calls the
pubsub()
method ofStrictRedis
, which takes an optionalshard_hint
argument, which could be used by the underlying client for sharding. This configuration option allows thisshard_hint
to be specified.
The remaining sections--[master]
and all the [slave:*]
sections--specify the connection to the master or slave Redis servers.
The recognized options for these sections are as follows:
- db
- The database number on the Redis server to connect to. Must be an integer. Optional.
- host
- The host name of the Redis server to connect to. Either this or
unix_socket_path
must be provided. - password
- The required authentication password for the Redis server. Optional.
- port
- The port number of the Redis server to connect to. Must be an integer. Optional.
- socket_timeout
- Allows specification of an alternate socket timeout, in seconds. Must be an integer. Optional.
- unix_socket_path
- The path to a UNIX socket for the Redis server. Either this or
host
must be provided.
The [master]
section is required, and specifies the master Redis
server for the limits data. Slaves are specified using
[slave:<name>]
, where <name>
may be any string; this string is
not used by Subway and exists solely to allow specification of slaves.
At least one slave must be specified.
Running Subway
Subway requires a single argument, the configuration file described
above. Following is a usage summary for the subway
daemon:
usage: subway [-h] config Run the Subway limits configuration synchronization daemon. positional arguments: config Configuration file for subway. optional arguments: -h, --help show this help message and exit